WO2021193121A1 - Solid-state imaging device and image processing method - Google Patents

Abstract

[Problem] To provide a solid-state imaging device and an image processing device that are capable of reducing the amount of time required for generating a reduced image and performing image processing. [Solution] A solid-state imaging device according to the present disclosure includes: a pixel array that performs photoelectric conversion and that contains a plurality of pixels; a plurality of AD conversion units that are provided so as to correspond to the plurality of pixels and that digitally convert pixel signals coming from the pixels; a plurality of first memory units that are provided so as to correspond to the plurality of AD conversion units and that store the pixel signals digitally converted by the AD conversion units; and a plurality of second memory units, each of which is provided for each of N (N is an integer that is equal to or greater than 2) pixels or N AD conversion units.

Description

Solid-state image sensor and image processing method

The present disclosure relates to a solid-state image sensor and an image processing method.

In image processing techniques such as object detection and tracking, a reduced image obtained by hierarchically reducing the original image is used.
However, in order to generate a reduced image, the original image is transferred to a frame memory (for example, DRAM (Dynamic Random Access Memory) or the like) outside the pixel array, and the reduced image is generated from the original image using the frame memory. I needed it. In this case, since the reduced image is generated after transferring the original image having a large amount of data, it takes time to generate the reduced image and perform image processing such as object detection.

International Patent Publication No. 2018/047618 International Patent Publication No. 2016/136448

Provided is a solid-state image sensor and an image processing method capable of shortening the time for generating a reduced image and image processing.

The solid-state image sensor on one aspect of the present disclosure includes a pixel array including a plurality of pixels that perform photoelectric conversion, and a plurality of AD conversion units that are provided corresponding to each of the plurality of pixels and digitally convert a pixel signal from the pixels. A plurality of first memory units provided corresponding to each of the plurality of AD conversion units and storing digitally converted pixel signals by the AD conversion unit, and N pixels (N is an integer of 2 or more) or A plurality of second memory units provided for each of N AD conversion units are provided.

A signal processing circuit that arithmetically processes a signal from the first or second memory unit, and a selection circuit that receives an output signal from the signal processing circuit and transfers the output signal to either the first or second memory unit. May be further provided.

The signal processing circuit generates one first reduced signal from a plurality of pixel signals from the first and second memory units, and the selection circuit stores the first reduced signal in either the first or second memory unit. You may.

One second memory unit may be provided for two pixels.

The signal processing circuit compresses a plurality of pixel signals stored in a plurality of first memory units to a quarter to generate a plurality of first reduced signals, and a part of the plurality of second memory units is used. Each first reduction signal may be stored.

The signal processing circuit compresses a plurality of first reduced signals stored in a plurality of second memory units into a quarter to generate a plurality of second reduced signals, and the plurality of first memory units have a plurality of pixels. While holding the signal, the other second memory unit of the second memory unit may store each second reduction signal.

After transferring the plurality of first reduced signals or the plurality of second reduced signals to the outside of the second memory unit, the pixel signal may be transferred to the outside of the first memory unit.

The signal processing circuit further compresses the plurality of second reduced signals stored in the plurality of second memory units to a quarter to generate a plurality of third reduced signals, and the plurality of first memory units have a plurality of first memory units. While holding the pixel signal and holding the second reduced signal by the plurality of second memory units, the other second memory unit stores each third reduced signal, and the signal processing circuit has a plurality of second reduced signals. A plurality of nth (n ≧ 3) reduction signals stored in the second memory unit are further compressed to a quarter to generate a plurality of n + 1 reduction signals, and the other second memory units are each n + 1th. A reduced signal may be stored.

One second memory unit may be provided for every three pixels.

The signal processing circuit compresses a plurality of pixel signals stored in a plurality of first memory units to a quarter to generate a plurality of first reduced signals, and a part of the plurality of second memory units is used. Each first reduction signal may be stored.

The signal processing circuit compresses a plurality of first reduced signals stored in a plurality of second memory units into a quarter to generate a plurality of second reduced signals, and the plurality of first memory units have a plurality of pixels. While holding the signal, the other second memory unit of the second memory unit may store each second reduction signal.

After transferring the plurality of first reduced signals or the plurality of second reduced signals to the outside of the second memory unit, the pixel signal may be transferred to the outside of the first memory unit.

The signal processing circuit further compresses the plurality of second reduced signals stored in the plurality of first memory units into a quarter to generate a plurality of third reduced signals, and the plurality of first memory units have a plurality of first memory units. While holding the pixel signal and storing the second reduced signal in the plurality of second memory units, the other second memory unit stores each third reduced signal, and the signal processing circuit has a plurality of thirds. A plurality of nth (n ≧ 3) reduction signals stored in the second memory unit are further compressed to a quarter to generate a plurality of n + 1 reduction signals, and the other second memory units are each n + 1th. A reduced signal may be stored.

One second memory unit may be provided for every four pixels.

The signal processing circuit compresses a plurality of pixel signals stored in a plurality of first memory units to a quarter to generate a plurality of first reduction signals, and the plurality of second memory units each perform the first reduction. The signal may be stored.

The signal processing circuit compresses a plurality of first reduced signals stored in a plurality of second memory units to a quarter to generate a plurality of second reduced signals, and the first memory unit generates each second reduced signal. The signal may be stored.

After transferring the pixel signals stored in the plurality of first memory units to the outside of the first memory unit, the plurality of first memory units may store the second reduced signals.

The signal processing circuit further compresses the plurality of second reduced signals stored in the plurality of first memory units to a quarter to generate a plurality of third reduced signals, and the first or second memory unit may generate a plurality of third reduced signals. Each third reduced signal is stored, and the signal processing circuit further compresses the plurality of nth (n ≧ 3) reduced signals stored in the plurality of first or second memory units into a plurality of quarters. The n + 1 reduction signal may be generated, and the first or second memory unit may store each n + 1 reduction signal.

The image processing method of one aspect of the present disclosure includes a pixel array including a plurality of pixels for photoelectric conversion, a plurality of AD conversion units provided corresponding to each of the plurality of pixels, and a plurality of AD conversion units, respectively. A plurality of first memory units provided corresponding to, and a plurality of second memory units provided one by one for N pixels (N is an integer of 2 or more) or N AD conversion units. It is an image processing method using a solid-state imaging device equipped with One first reduction signal is generated from a plurality of pixel signals from one memory unit, and the plurality of first reduction signals are stored in either the first or second memory unit.

The plurality of first reduction signals may be signals generated by compressing a plurality of pixel signals into a quarter.

The block diagram which shows one configuration example of the image pickup apparatus in the 1st Embodiment of this technique. The block diagram which shows one configuration example of the solid-state image sensor in 1st Embodiment of this technique. The schematic diagram which shows the ADC part of the pixel array part and the peripheral circuit part. The schematic diagram which shows the structural example of a part of the ADC part. The figure which shows the structure of a part of the ADC part in more detail. The conceptual diagram which shows the pixel signal or the reduction signal stored in the 1st and 2nd memory part by 1st Embodiment. The conceptual diagram which shows the pixel signal or the reduction signal stored in the 1st and 2nd memory part by 1st Embodiment. The conceptual diagram which shows the pixel signal or the reduction signal stored in the 1st and 2nd memory part by 1st Embodiment. The conceptual diagram which shows the pixel signal or the reduction signal stored in the 1st and 2nd memory part by 1st Embodiment. The conceptual diagram which shows the pixel signal or the reduction signal stored in the 1st and 2nd memory part by 1st Embodiment. A conceptual diagram showing an original pixel and a reduced image. A graph showing the transfer time of the original image and the reduced image. The conceptual diagram which shows the pixel signal or the reduction signal stored in the 1st and 2nd memory part by 2nd Embodiment. The conceptual diagram which shows the pixel signal or the reduction signal stored in the 1st and 2nd memory part by 2nd Embodiment. The conceptual diagram which shows the pixel signal or the reduction signal stored in the 1st and 2nd memory part by 2nd Embodiment. The conceptual diagram which shows the pixel signal or the reduction signal stored in the 1st and 2nd memory part by 2nd Embodiment. The conceptual diagram which shows the pixel signal or the reduction signal stored in the 1st and 2nd memory part by 2nd Embodiment. The conceptual diagram which shows the pixel signal or the reduction signal stored in the 1st and 2nd memory part by 3rd Embodiment. The conceptual diagram which shows the pixel signal or the reduction signal stored in the 1st and 2nd memory part by 3rd Embodiment. The conceptual diagram which shows the pixel signal or the reduction signal stored in the 1st and 2nd memory part by 3rd Embodiment. The conceptual diagram which shows the pixel signal or the reduction signal stored in the 1st and 2nd memory part by 3rd Embodiment. The conceptual diagram which shows the pixel signal or the reduction signal stored in the 1st and 2nd memory part by 3rd Embodiment. A graph showing the transfer time of the original image and the reduced image.

Hereinafter, specific embodiments to which the present technology is applied will be described in detail with reference to the drawings. The drawings are schematic or conceptual, and the ratio of each part is not always the same as the actual one. In the specification and the drawings, the same elements as those described above with respect to the existing drawings are designated by the same reference numerals, and detailed description thereof will be omitted as appropriate.

(First Embodiment)

FIG. 1 is a perspective view showing a configuration example of a solid-state image sensor according to the first embodiment of the present technology. The solid-state image sensor 100 is configured by laminating a semiconductor chip 511 and a semiconductor chip 512. The semiconductor chip 511 includes a pixel array unit 240 formed on the semiconductor substrate. The semiconductor chip 512 includes an ADC unit 280 formed on another semiconductor substrate and other peripheral circuits. The semiconductor chips 511 and 512 are formed as separate semiconductor chips, and are subsequently laminated.

Elements such as each pixel of the pixel array unit 240 of the semiconductor chip 511 and the ADC unit 280 of the semiconductor chip 512 are, for example, TSVs (Through Silicon Vias) provided in the via region around the pixel array unit 240 and the ADC unit 280. It is electrically connected using a through electrode or the like. Alternatively, both semiconductor chips may be bonded together so that the wiring of the semiconductor chip 511 and the wiring of the semiconductor chip 511 are in contact with each other (Cu-Cu bonding). Further, the pixel array unit 240 and the ADC unit 280 may be configured as one semiconductor chip, and the other peripheral circuits may be configured as another semiconductor chip. Further, the pixel array unit 240, the ADC unit 280, and other peripheral circuits may all be configured as one semiconductor chip. The number of laminated semiconductor chips may be three or more.

FIG. 2 is a block diagram showing a configuration example of a solid-state image sensor according to the first embodiment of the present technology. A plurality of pixel circuits 250 are arranged in a two-dimensional lattice pattern on the pixel array unit 240 on the semiconductor chip 511. The pixel circuit 250 has a photoelectric conversion element (not shown), receives power from a voltage source 180, and photoelectrically converts incident light to generate an electric charge.
A plurality of ADC circuits 290 are arranged in a two-dimensional lattice pattern in the ADC section 280 on the semiconductor chip 512. Each of the ADC circuits 290 is provided corresponding to the pixel circuit 250, and converts the pixel signal (analog signal) from the pixel circuit 250 into a digital signal. In the present technology, since the ADC circuit 290 is provided corresponding to each of the pixel circuits 250, the pixel signals simultaneously detected by all the pixel circuits 250 in the pixel array unit 240 are immediately and simultaneously AD-converted. Can be done.
The digital signal AD-converted by the ADC circuit 290 is subjected to signal processing such as CDS (Correlated Double Sampling) processing. After that, the digital signal is transferred to the frame memory 115 via the transfer circuit 260, and is detected and processed by the DSP (Digital Signal Processing) circuit 120 provided inside the semiconductor chip 512. Alternatively, the digital signal may be transferred to an external DSP circuit (not shown) of the semiconductor chips 511 and 512 for detection processing. As described above, the digital signal may be detected and processed in the same semiconductor chip after the CDS processing, or may be detected and processed in another semiconductor chip.

FIG. 3 is a schematic diagram showing a pixel array unit and an ADC (Analog-to-Digital Converter) unit. As described above, the pixel array unit 240 includes a plurality of pixel circuits 250 that perform photoelectric conversion. The pixel circuit 250 as a pixel is two-dimensionally arranged on the semiconductor substrate.

The ADC unit 280 includes a plurality of ADC circuits 290 that digitally convert the pixel signal from the pixel circuit 250. The ADC circuit 290 as an AD conversion unit is two-dimensionally arranged on another semiconductor substrate. Further, the ADC circuit 290 is provided corresponding to each of the pixel circuits 250. In FIG. 3, the ADC circuit 290 is provided in the lateral direction over two pixel circuits 250 adjacent to each other. Further, two ADC circuits 290 are arranged in the vertical direction with respect to one pixel circuit 250. Therefore, in the present disclosure, the ADC circuit 290 is provided on a one-to-one basis with respect to the pixel circuit 250.

FIG. 4 is a schematic view showing a partial configuration example of the ADC section. The ADC unit 280 includes a plurality of ADC circuits 290 provided corresponding to the plurality of pixel circuits 250, and a second memory unit 292. As shown in FIG. 5, the plurality of ADC circuits 290 include a first memory unit 291 in addition to a comparison circuit, a selector, and the like.

The first memory unit 291 is provided in the ADC circuit 290, and is provided corresponding to each of the ADC circuits 290. The first memory unit 291 stores the pixel signals that have been AD-converted by the corresponding ADC circuits 290.

The second memory unit 292 is an additional memory provided separately from the first memory unit 291 of the ADC circuit 290, and in the present embodiment, the second memory unit 292 is provided for two ADC circuits 290 (that is, two pixels). It is provided one by one. The second memory unit 292 is provided to store one reduced signal generated by using the plurality of pixel signals stored in the plurality of first memory units 291. For example, the first memory unit 291 stores pixel signals constituting a high-resolution original image. The second memory unit 292 stores the signal (reduced signal) of the reduced image generated by compressing the original image, respectively. As described above, the solid-state image sensor 100 according to the present disclosure does not output the pixel signal of the original image generated by the ADC circuit 290 as it is to the outside of the semiconductor chips 511 and 512, but for each pixel circuit 250 (that is, pixels). It is compressed in the ADC circuit 290 provided (for each) to make a reduced image. As a result, the original image can be compressed into a reduced image in a short time. A more detailed description of the image processing method will be described later.

[ADC configuration example]
FIG. 5 is a diagram showing a part of the configuration of the ADC unit in more detail. The ADC section 280 includes an ADC circuit 290a_1, 290a_2, 290b_1, 290b_2, 290c_1, 290c_2 ..., a second memory section 292a, 292b, 292c ..., and a signal processing circuit 293a, 293b, 293c ... The demultiplexer 294a, 294b, 294c ...

The ADC circuits 290a_1, 290a_2, 290b_1, 290b_2, 290c_1, 290c_2, and the like each have the same internal configuration. Therefore, for convenience, the internal configuration of the ADC circuit 290a_1 will be described, and the description of the internal configuration of other ADC circuits will be omitted.

The ADC circuit 290a_1 includes a comparison circuit 295, a selector 296, and a first memory unit 291a_1. The comparison circuit 295 compares the pixel signal from the corresponding pixel circuit 250 with the reference signal from the DAC 231. The comparison circuit 295 measures the time from the start of the voltage change of the reference signal to the crossing of the voltage level of the pixel signal. As a result, the pixel signal is AD-converted and output as a digital value.

The selector 296 receives the digital value from the comparison circuit 295 and the output signal from the demultiplexer 294a, and selectively transfers any of them to the first memory unit 291a_1.

The first memory unit 291a_1 temporarily stores the signal from the demultiplexer 294a. First, when the comparison circuit 295 receives the pixel signal of the original image from the corresponding pixel circuit 250, the demultiplexer 294a sends the pixel signal of the original image to the first memory unit 291a_1, and the first memory unit 291a_1 Stores pixel signals.

At this time, the other first memory units 291a_2 and the like also store the pixel signals of the original image from the corresponding pixel circuits 250, similarly to the first memory unit 291a_1. Therefore, the ADC unit 280 stores the pixel signal of the original image as a whole. The second memory unit 292a and the like have not yet stored a specific signal.

The first and second memory units 291a_1 to 291c_1, 291a_2 to 291c_2, and 292a to 292c may be latch circuits having the same configuration.

The signal processing circuit 293a receives a plurality of pixel signals from the first memory unit 291a_1, 291a_2, 291b_1, 291b_2 and the second memory unit 292a, and performs these arithmetic processing. The arithmetic processing is filtering processing of a plurality of pixel signals, thinning out a certain pixel signal from a plurality of pixel signals, calculating the average of a plurality of pixel signals, or multiplying a plurality of pixel signals by a coefficient and adding them. It may be a process to be performed. By such arithmetic processing, those pixel signals can be combined to generate one signal (for example, a signal of a reduced image). For example, when the signal processing circuit 293a synthesizes two pixel signals from the first memory units 291a_1 and 291a_2, the signal processing circuit 293a synthesizes the two pixel signals from the first memory units 291b_1 and 291b_2 adjacent to them in consideration of the pixel signals. Output as one signal (reduced signal). Since the signal of the second memory unit 292a or the like is undefined when synthesizing the pixel signal of the original image, the signal processing circuit 293a does not use the signal of the second memory unit 292a.

The signal processing circuits 293b and 293c also function in the same manner as the signal processing circuits 293a. For example, the signal processing circuit 293b receives a plurality of pixel signals from the first memory unit 291b_1, 291b_2, 291c_1, 291c_2, and the second memory unit 292b, and performs these arithmetic processing. The signal processing circuit 293c receives a plurality of pixel signals from the first memory unit 291c_1, 291c_2, 291d_1, 291d_2, and the second memory unit 292c, and performs these arithmetic processing. The signal processing circuits 293a to 293c may be digital logic circuits having the same configuration.

The demultiplexer 294a receives the output signal from the signal processing circuit 293a and selectively transfers the output signal to either the ADC circuit 290a_1 or the second memory unit 292a. For example, in the present embodiment, the demultiplexer 294a transfers an output signal to the second memory unit 292a and stores the output signal (that is, a reduced signal).

The demultiplexer 294b, 294c also functions in the same manner as the demultiplexer 294a. Therefore, the demultiplexer 294b receives the output signal from the signal processing circuit 293b and selectively transfers the output signal to either the ADC circuit 290b_1 or the second memory unit 292b. The demultiplexer 294c receives the output signal from the signal processing circuit 293c and selectively transfers the output signal to either the ADC circuit 290c_1 or the second memory unit 292c.

For example, the second memory units 292a to 292c each store an output signal. In this case, the output signal is a reduced signal obtained by synthesizing a plurality of pixel signals. That is, the second memory units 292a to 292c each store the signal of the reduced image (first reduced signal) obtained by compressing the original image.

Although the wiring is not shown in FIG. 5, the signal processing circuits 293a to 293c further synthesize at least two first reduction signals among the plurality of first reduction signals stored in the second memory units 292a to 292c. , The signal of the reduced image obtained by further compressing the reduced image (second reduced signal) may be stored respectively. At this time, the pixel signal of the original image may be further added.

In this way, the ADC unit 280 can create and hold a reduced image of the original image before transferring the original image to the outside of the pixel array unit 240.

In FIG. 5, the second memory unit 292a is provided one by one for each of the two ADC circuits 290a_1 and 290a_2. Similarly, the other second memory units 292b and 292c are provided one for each of the two ADC circuits 290b_1, 290b_2, and 290c_1 and 290c_2, respectively. That is, one second memory unit 292 is provided for each of the two pixel circuits 250. The number of memories (latches) provided in the ADC unit 280 is 1.5 times the number of the first memory units 291 including the first and second memory units 291 and 292. As a result, the second memory unit 292 can hold the reduced signal while the first memory unit 291 maintains the pixel signal of the original image. The relationship between the number of second memory units 292 and the amount of image data will be described in detail later.

In the present embodiment, the ratio of the number of the first memory unit 291 to the number of the second memory unit 292 is 2: 1, but the number thereof is not particularly limited. The number of each of the ADC circuit 290, the second memory unit 292, the signal processing circuit 293, and the demultiplexer 294 is also not particularly limited.

Further, in FIG. 5, demultiplexers 294a to 294c are provided. However, the demultiplexers 294a to 294c may be omitted, and the signal processing circuits 293a to 293c may transfer the output signal to both the first memory units 291a_1 to 291c_1 and the second memory units 292a to 292c. In this case, although not shown, a write enable signal may be input to the first memory units 291a_1 to 291c_1 and the second memory units 292a to 292c, and a memory for storing the output signal may be selected by the write enable signal. For example, when the write enable signal is the first logic, the first memory units 291a_1 to 291c_1 store the output signals from the signal processing circuits 293a to 293c, respectively. When the write enable signal is the second logic, the second memory units 292a to 292c store the output signals from the signal processing circuits 293a to 293c, respectively. With this configuration, the output signal can be stored in one of the first memory units 291a to 291c and the second memory units 292a to 292c. Further, since the demultiplexers 294a to 294c are not required, the area of the ADC unit 280 is reduced.

6A to 6E are conceptual diagrams showing pixel signals or reduced signals stored in the first and second memory units according to the first embodiment. FIG. 7 is a conceptual diagram showing an original pixel and a reduced image.

In the ADC unit 280, the first memory unit 291 and the second memory unit 292 in the ADC circuit 290 are arranged. As shown in FIGS. 4 and 6A, the plurality of first memory units 291 are arranged in the X direction, and the plurality of second memory units 292 are also arranged in the X direction. One second memory unit 292 is provided for each of the two first memory units 291. The first memory unit 291 is provided for each ADC circuit 290, and the ADC circuit 290 is provided for each pixel circuit 250. Therefore, the first memory unit 291 is provided in the pixel circuit 250 in a one-to-one correspondence, and the second memory unit 292 is provided one for every two pixels. The rows of the first memory unit 291 and the rows of the second memory unit 292 are alternately arranged in the Y direction orthogonal to the X direction.

The black-painted portion B of FIGS. 6A to 6D is an area in which the ADC circuit 290 (first memory unit 291) and the second memory unit 292 are not provided. The other white parts indicate that the ADC circuit 290 (first memory unit 291) or the second memory unit 292 is provided, and each of them can store one pixel signal or a reduced signal.

In FIG. 6A, the first and second memory units 291 and 292 are in the initial state and do not yet store the pixel signal. The plurality of pixel circuits 250 send the pixel signals of the captured image to the ADC unit 280.

As shown in FIG. 6B, the pixel signal S0 of the original image from the pixel circuit 250 is AD-converted and then stored in the first memory unit 291 corresponding to each of the pixel circuits 250. At this time, the pixel signal S0 stored in the first memory unit 291 has not been compressed yet.

Next, as shown in FIG. 6C, the pixel signal S0 stored in the first memory unit 291 is combined to generate the first reduction signal S1. The first reduced signal S1 is generated in the signal processing circuit 293 as described with reference to FIG. The signal processing circuit 293 compresses the pixel signal S0 to a quarter to generate the first reduced signal S1. The first reduction signal S1 is stored in a part of the second memory unit 292. At this time, the first memory unit 291 holds the pixel signal of the original image. The original image composed of the pixel signal S0 of FIG. 7 is compressed to a quarter of the reduced image composed of the first reduced signal S1. Each point in FIG. 7 indicates one pixel signal or one reduced signal.

Next, as shown in FIG. 6D, the first reduction signal S1 stored in the second memory unit 292 is combined to generate the second reduction signal S2. The signal processing circuit 293 compresses the first reduced signal S1 to a quarter to generate the second reduced signal S2. The second reduction signal S2 is stored in another second memory unit 292 that does not store the first reduction signal S1. At this time, the first memory unit 291 holds the pixel signal of the original image. Further, a part of the second memory unit 292 holds the first reduction signal S1. The reduced image composed of the first reduced signal S1 in FIG. 7 is further compressed to a quarter of the reduced image composed of the second reduced signal S2.

Next, as shown in FIG. 6E, the second reduced signal S2 is combined to generate the third reduced signal S3. The signal processing circuit 293 compresses the second reduced signal S2 to a quarter to generate the third reduced signal S3. The third reduced signal S3 is stored in yet another second memory unit 292 that does not store the first and second reduced signals S1 and S2. At this time, the first memory unit 291 holds the pixel signal of the original image. Further, the second memory unit 292 holds the first and second reduction signals S1 and S2. The reduced image composed of the second reduced signal S2 in FIG. 7 is further compressed to a quarter of the reduced image composed of the third reduced signal S3.

Further, the signal processing circuit 293 may compress the third reduced signal S3 to a quarter to generate a fourth reduced signal (not shown). In this way, the signal processing circuit 293 repeats the compression processing of the original image to generate a reduced image. That is, the signal processing circuit 293 compresses the nth (n ≧ 3) reduced signal to a quarter to generate the n + 1 reduced signal, and the second memory unit 292 further stores the n + 1 reduced signal. The number of repetitions of the compression process may be k (k is an integer of 1 or more). That is, the signal processing circuit 293 compresses the k-1 reduced signal to a quarter to generate the k reduced signal. Note that k = 0 is a pixel signal of the original image. In the present embodiment, the amount of pixel signal data is reduced by a quarter each time the compression process is repeated. Therefore, assuming that the data amount of the original image is D0, the total data amount of the original image and the first to kth reduced signals is D0 + (1/4) D0 + (1/4 ² ) D0 + (1/4 ³ ) D0 +. ..., and the maximum (4/3) * D0. That is, the total data amount of the original image and the first to kth reduced signals is 1.333 ... times the data amount D0 of the original image. Therefore, if the total number of the first and second memory units 291 and 292 is 1.5 times the number of the first memory units 291 that store the original image (data amount D0), the first and second memory units 291 and 292 Can sufficiently store the original image and all of the first to k-th reduction signals. For this reason, in the present embodiment, one second memory unit 292 is provided for each of the two first memory units 291.

Further, if any one of the second to fourth reduced signals S2 to S4 is to be directly generated from the pixel signal S0 of the original image, the calculation load of the signal processing circuit 293 increases. Therefore, in the present embodiment, the first reduction signal S1 is generated from the pixel signal S0, the second reduction signal S2 is generated from the first reduction signal S1, and the third reduction signal S3 is generated from the second reduction signal S2. There is. Thereby, the load of the signal processing circuit 293 can be reduced.

FIG. 8 is a graph showing the transfer time of the original image and the reduced image. The horizontal axis represents time t. In the present embodiment, the first and second memory units 291 and 292 store the original image and all the reduced images. Therefore, after the image compression processing is completed, the images can be transferred in the order of detection processing, that is, in ascending order from the reduced image having the smallest amount of data, and the original image can be transferred last.

Normally, in object detection, the object to be detected exists or the rough coordinates of the object are detected using a reduced low-resolution image. This identifies a detection area with an object that should be detected in detail. Next, the detection area is cut out from the high-resolution image, and the object detection process is performed in the detection area of the high-resolution image. In motion detection, a reduced low-resolution image is used to detect the rough motion of an object. This identifies the area of motion that should be detected in detail. Next, a detection area is cut out from the high-resolution image, and motion detection processing is performed in the detection area of the high-resolution image. By hierarchically detecting from a low-resolution image using a high-resolution image in this way, it is possible to improve the accuracy and detection speed of motion detection while suppressing the amount of data to be processed.

Therefore, according to the present embodiment, in order to hierarchically detect the low-resolution image to the high-resolution image using the high-resolution image, the transfer circuit 260 orders the larger image from the reduced image with the smallest amount of data (the order of detection processing). ) Is transferred to the frame memory 115 outside the signal processing circuit 293. For example, in FIG. 8, it is assumed that the amount of image data of the third reduction signal S3 is the minimum. In this case, the transfer circuit 260 first transfers the third reduced signal S3. Since the data amount D3 of the third reduced signal S3 is the minimum, the transfer time t_t3 is the shortest time among t_t0 to t_t3. The detection time t_d3 is the time required for object detection, motion detection, and the like using the third reduction signal S3. During the detection process using the third reduced signal S3, the transfer process of the second reduced signal S2 is started.

Since the amount of image data of the second reduced signal S2 is the second smallest after the third reduced signal S3 and is detected and processed next to the third reduced signal S3, the transfer circuit 260 is next to the third reduced signal S3. 2 The reduced signal S2 is transferred. The transfer time t_t2 is longer than the transfer time t_t3, but is relatively short. The detection time t_d2 is the time required for object detection, motion detection, and the like using the second reduction signal S2. During the detection process using the second reduced signal S2, the transfer process of the first reduced signal S1 is started.

Since the amount of image data of the first reduced signal S1 is the second smallest after the second reduced signal S2 and the detection process is performed next to the second reduced signal S2, the transfer circuit 260 is next to the second reduced signal S2. 1 Transfer the reduced signal S1. The transfer time t_t1 is longer than the transfer time t_t2, but shorter than the transfer time of the original image. The detection time t_d1 is the time required for object detection, motion detection, and the like using the first reduction signal S1. During the detection process using the first reduction signal S1, the transfer process of the pixel signal S0 of the original image is started.

Since the amount of data in the image of the pixel signal S0 of the original image is the largest and the detection process is performed last, the transfer circuit 260 transfers the pixel signal S0 last. The transfer time t_t0 is the longest among t_t0 to t_t3. The detection time t_d0 is the time required for object detection, motion detection, and the like using the pixel signal S0. If this compression process and transfer process are generalized, when the k-th reduction signal is generated, the transfer circuit 260 detects the pixel signal and the first to k-th reduction signals in the order of detection processing (after the generation of the k-reduction signal). Here, the data is transferred to the frame memory 115 outside the signal processing circuit 293 in ascending order of the amount of data). That is, after the image is compressed, the transfer circuit 260 frames the k-th reduced signal, the k-1 reduced signal, ... The third reduced signal, the second reduced signal, the first reduced signal, and the pixel signal of the original image in this order. Transfer to memory 115.

In this way, the transfer circuit 260 transfers the reduced images in the order of detection processing, that is, in ascending order from the reduced image having the smallest amount of data, and finally transfers the original image to the frame memory 115 outside the signal processing circuit 293. .. As a result, the DSP circuit 120 can immediately start the detection process from an image having a small amount of data without delay. Further, the detection process and the transfer process of the next reduced signal or pixel signal can be executed in parallel.

If the second memory unit 292 is not provided and only the original image stored in the first memory unit 291 is transferred to the frame memory 115, the DSP circuit 120 transfers the pixel signal S0 of the original image and then the DSP circuit 120. A reduced image is generated using the pixel signal S0. Therefore, the detection process is started after the pixel signal S0 of the original image is transferred and the reduced image is generated. Therefore, the timing of starting the detection process is delayed. Furthermore, the transfer process and the detection process cannot be executed in parallel.

On the other hand, according to the present embodiment, in the ADC section 280 of the signal processing circuit 293, the signal processing circuit 293 generates reduced signals S1 to Sk using the original image of the first memory section 291. 2 Stored in the memory unit 292. Since the reduced image is generated inside the ADC unit 280 in this way, the signal processing circuit 293 can perform the image compression processing without waiting for the long transfer time t_t0 of the original image. Further, the ADC unit 280 can transfer the reduced image and the original image in ascending order from the reduced image having the smallest amount of data in the order of detection processing without waiting for the transfer time t_t0 of the original image. The transfer time for images with a small amount of data is relatively short. Therefore, the DSP circuit 120 can immediately start the detection process without delay.

Further, when the DSP circuit 120 executes the detection process, the image compression process has already been completed. Therefore, while the DSP circuit 120 is executing the detection process, the transfer circuit 260 can transfer the next reduced signal or pixel signal to the frame memory 115 in parallel. As a result, the end time of the detection process can be accelerated.

(Second Embodiment)
9A to 9E are conceptual diagrams showing pixel signals or reduced signals stored in the first and second memory units according to the second embodiment. In the second embodiment, the second memory unit 292 is provided one by one for each of the three first memory units 291. That is, one second memory unit 292 is provided for each of the three pixel circuits 250. In this case, the number of memories (latches) provided in the ADC unit 280 is 4/3 (1.3333 ...) Of the number of the first memory units 291 including the first and second memory units 291 and 292.・) It has doubled. As a result, the second memory unit 292 can hold the reduced signal while the first memory unit 291 maintains the pixel signal of the original image.

As described above, the total amount of data of the original image and the first to kth reduced signals is the maximum (4/3) * D0. That is, it is 1.333 ... times the data amount D0 of the original image. Therefore, according to the second embodiment, the capacity of the second memory unit 292 is equal to the total amount of data of the first to kth reduction signals. As a result, the first and second memory units 291 and 292 can store the entire original image and the first to k-th reduction signals, and the extra second memory unit 292 is eliminated. That is, waste of the first and second memory units 291 and 292 can be suppressed.

For example, in FIG. 9A, the first and second memory units 291 and 292 are in the initial state and do not yet store the pixel signal. The plurality of pixel circuits 250 send the pixel signals of the captured image to the ADC unit 280.

As shown in FIG. 9B, the pixel signal S0 of the original image from the pixel circuit 250 is AD-converted and then stored in the first memory unit 291 corresponding to each of the pixel circuits 250. At this time, the pixel signal S0 stored in the first memory unit 291 has not been compressed yet.

Next, as shown in FIG. 9C, the signal processing circuit 293 synthesizes the pixel signal S0 stored in the first memory unit 291 to generate the first reduction signal S1. That is, the signal processing circuit 293 compresses the pixel signal S0 to a quarter to generate the first reduced signal S1. The first reduction signal S1 is stored in a part of the second memory unit 292. At this time, the first memory unit 291 holds the pixel signal of the original image.

Since three first reduction signals S1 are generated for the twelve pixel signals S0, the second memory unit 292 refers to the twelve first memory units 291 (12 first reduction signals S1). Three are used. For example, the frame C1 of FIG. 9C includes twelve first memory units 291, and three second memory units 292 in the frame C1 store the first reduction signal S1. The pattern of the frame C1 is repeated in the X direction and the Y direction.

Next, as shown in FIG. 9D, the first reduction signal S1 stored in the second memory unit 292 is combined to generate the second reduction signal S2. The signal processing circuit 293 compresses the first reduced signal S1 to a quarter to generate the second reduced signal S2. The second reduction signal S2 is stored in another second memory unit 292 that does not store the first reduction signal S1. At this time, the first memory unit 291 holds the pixel signal of the original image. Further, a part of the second memory unit 292 holds the first reduction signal S1.

Since three second reduction signals S2 are generated for the twelve first reduction signals S1, the second memory unit 292 is used three more for the twelve first reduction signals S1. For example, the frame C2 of FIG. 9D includes twelve second memory units 292 that store the first reduction signal S1, and the other three second memory units 292 in the frame C1 store the second reduction signal S2. .. The pattern of the frame C2 is repeated in the X direction and the Y direction.

Next, as shown in FIG. 9E, the second reduced signal S2 is combined to generate the third reduced signal S3. The signal processing circuit 293 compresses the second reduced signal S2 to a quarter to generate the third reduced signal S3. The third reduced signal S3 is stored in yet another second memory unit 292 that does not store the first and second reduced signals S1 and S2. At this time, the first memory unit 291 holds the pixel signal of the original image. Further, the second memory unit 292 holds the first and second reduction signals S1 and S2.

Three third reduction signals S3 are generated for twelve second reduction signals S2. Three more second memory units 292 are used for the twelve second reduction signals S2. Although not shown in FIG. 9E, the other three free second memory units 292 store the third reduced signal S3 with respect to the twelve second memory units 292 that store the second reduced signal S2. do.

Further, the signal processing circuit 293 may similarly compress the third reduced signal S3 to a quarter to generate a fourth reduced signal (not shown). In this way, the signal processing circuit 293 repeats the compression processing of the original image to generate a reduced image. That is, the signal processing circuit 293 compresses the nth (n ≧ 3) reduced signal to a quarter to generate the n + 1 reduced signal, and the second memory unit 292 further stores the n + 1 reduced signal.

Other configurations and operations of the second embodiment may be the same as the corresponding configurations and operations of the first embodiment. Therefore, the second embodiment can obtain the same effect as the first embodiment.

Further, according to the second embodiment, the capacity of the second memory unit 292 is equal to the total amount of data of the first to kth reduced signals. As a result, the first and second memory units 291 and 292 can store the entire original image and the first to k-th reduction signals, and the extra second memory unit 292 is eliminated. As a result, waste of the first and second memory units 291 and 292 can be suppressed, and memory resources can be fully utilized.

(Third Embodiment)
10A to 10E are conceptual diagrams showing pixel signals or reduced signals stored in the first and second memory units according to the third embodiment. In the third embodiment, the second memory unit 292 is provided one by one for each of the four first memory units 291. That is, a second memory unit 292 is provided for each of the four pixel circuits 250. In this case, the number of memories (latches) provided in the ADC unit 280 is 1.25 times the number of the first memory units 291 including the first and second memory units 291 and 292. Therefore, the capacities of the first and second memory units 291 and 292 are smaller than the total data amount (4/3) * D0) of the original image and the first to kth reduced signals. Therefore, in the third embodiment, the signal processing circuit 293 needs to execute the compression operation after transferring the original image or the reduced image before compression.

According to the third embodiment, the first and second memory units 291 and 292 cannot store the entire original image and the first to kth reduced signals. However, the image compression process is executed inside the ADC unit 280. Therefore, the transfer circuit 260 can transfer the pixel signal or the compressed signal used for the compression operation to the frame memory 115 while executing the compression operation. As a result, the compression operation can be executed almost simultaneously with the transfer process. As a result, the transfer time of the original image and the first to kth reduced signals is shortened, and the timing of starting the detection process is shortened.

For example, in FIG. 10A, the first and second memory units 291 and 292 are in the initial state and do not yet store the pixel signal. The plurality of pixel circuits 250 send the pixel signals of the captured image to the ADC unit 280.

As shown in FIG. 10B, the pixel signal S0 of the original image from the pixel circuit 250 is AD-converted and then stored in the first memory unit 291 corresponding to each of the pixel circuits 250. At this time, the pixel signal S0 stored in the first memory unit 291 has not been compressed yet.

Next, as shown in FIG. 10C, the signal processing circuit 293 synthesizes the pixel signal S0 stored in the first memory unit 291 to generate the first reduction signal S1. That is, the signal processing circuit 293 compresses the pixel signal S0 to a quarter to generate the first reduced signal S1. The first reduction signal S1 is stored in the second memory unit 292.

Next, the transfer circuit 260 transfers the pixel signal S0 of the original image of the first memory unit 291 to the frame memory 115. As a result, the first memory unit 291 is free and can store other signals. After the pixel signal S0 is transferred, the data in the first memory unit 291 may be overwritten with the next signal without erasing.
Further, while the signal processing circuit 293 executes the compression operation of the pixel signal S0 of the original image, the transfer circuit 260 may transfer the pixel signal S0 whose compression operation has been completed to the frame memory 115 in parallel with the compression operation. Alternatively, the transfer circuit 260 may transfer the pixel signal S0 while the first memory unit 291 holds the pixel signal S0, and then the signal processing circuit 293 may execute a compression operation of the pixel signal S0 of the original image.

Next, as shown in FIG. 10D, the first reduction signal S1 stored in the second memory unit 292 is combined to generate the second reduction signal S2. The signal processing circuit 293 compresses the first reduced signal S1 to a quarter to generate the second reduced signal S2. The second reduced signal S2 is stored in the first memory unit 291 that does not store the first reduced signal S1.

Next, the transfer circuit 260 transfers the first reduction signal S1 of the second memory unit 292 to the frame memory 115. As a result, the second memory unit 292 is free and can store other signals. After the transfer of the first reduction signal S1, the data in the second memory unit 292 may be overwritten with the next signal without erasing.

Next, as shown in FIG. 10E, the second reduced signal S2 is combined to generate the third reduced signal S3. The signal processing circuit 293 compresses the second reduced signal S2 to a quarter to generate the third reduced signal S3. The third reduced signal S3 is stored in the second memory unit 292 that does not store the first and second reduced signals S1 and S2.

Next, the transfer circuit 260 transfers the second reduction signal S2 of the first memory unit 291 to the frame memory 115. As a result, the first memory unit 291 is free and can store other signals. After the transfer of the second reduced signal S2, the data in the first memory unit 291 may be overwritten with the next signal without erasing.

Further, the signal processing circuit 293 may similarly compress the third reduced signal S3 to a quarter to generate a fourth reduced signal (not shown). In this way, the signal processing circuit 293 repeats the compression processing of the original image to generate a reduced image. That is, the signal processing circuit 293 compresses the nth (n ≧ 3) reduced signal to a quarter to generate the n + 1 reduced signal, and the first or second memory units 291 and 292 are the n + 1 reduced signals. To store.

Other configurations and operations of the third embodiment may be the same as the corresponding configurations and operations of the first embodiment.

FIG. 11 is a graph showing the transfer time of the original image and the reduced image. The horizontal axis represents time t. In the third embodiment, the transfer circuit 260 executes the compression operation and the transfer process in parallel with each other. After the third reduced signal S3 having the smallest amount of data is transferred, the DSP circuit 120 starts the detection process from the third reduced signal S3.

For example, the signal processing circuit 293 executes the compression operation of the pixel signal S0 of the original image, and the transfer circuit 260 transfers the pixel signal S0 for which the compression operation has been completed to the frame memory 115 in parallel. The time t_c1 is the compression time of the pixel signal S0 and also the generation time of the first reduction signal S1. The transfer process of the pixel signal S0 can be executed substantially simultaneously with the compression operation of the pixel signal S0. Therefore, the compression time t_c1 of the pixel signal S0 can be substantially overlapped with the transfer time t_t0 of the pixel signal S0.

Next, the signal processing circuit 293 executes the compression operation of the first reduction signal S1, and the transfer circuit 260 transfers the first reduction signal S1 for which the compression operation has been completed to the frame memory 115 in parallel. The time t_c2 is the compression time of the first reduction signal S1 and also the generation time of the second reduction signal S2. The transfer process of the first reduced signal S1 can be executed substantially simultaneously with the compression operation of the first reduced signal S1. Therefore, the compression time t_c2 of the first reduced signal S1 can be substantially overlapped with the transfer time t_t1 of the first reduced signal S1.

Next, the signal processing circuit 293 executes the compression operation of the second reduction signal S2, and the transfer circuit 260 transfers the second reduction signal S2, which has completed the compression operation, to the frame memory 115 in parallel. The time t_c3 is the compression time of the second reduced signal S2 and also the generation time of the third reduced signal S3. The transfer process of the second reduced signal S2 can be executed substantially simultaneously with the compression operation of the second reduced signal S2. Therefore, the compression time t_c3 of the second reduced signal S2 can be substantially overlapped with the transfer time t_t2 of the second reduced signal S2.

Next, the transfer circuit 260 transfers the third reduced signal S3, which has completed the compression operation, to the frame memory 115. The transfer time of the third reduced signal S3 is t_t3.

After that, the DSP circuit 120 executes the detection process using the third reduction signal S3. Next, the DSP circuit 120 executes the detection process using the second reduced signal S2, executes the detection process using the first reduced signal S1, and executes the detection process using the pixel signal S0.

According to the third embodiment, the image compression process is executed inside the ADC unit 280. Therefore, the transfer circuit 260 can transfer the pixel signal or the compressed signal used in the compression operation to the frame memory 115 while executing the compression operation. As a result, the compression operation can be executed in parallel with the transfer process, and the start timing of the detection process is accelerated. As a result, the solid-state image sensor 100 can reduce the time required for generating a reduced image and processing an image.

The embodiment according to the present technology is not limited to the above embodiment, and various changes can be made without departing from the gist of the present technology.
(1)
A pixel array containing multiple pixels that perform photoelectric conversion,
A plurality of AD conversion units provided corresponding to each of the plurality of pixels and digitally converting a pixel signal from the pixels.
A plurality of first memory units provided corresponding to each of the plurality of AD conversion units and storing the pixel signals digitally converted by the AD conversion unit, and a plurality of first memory units.
A solid-state image sensor including N (N is an integer of 2 or more) of the pixels or a plurality of second memory units provided for each of the N AD conversion units.
(2)
A signal processing circuit that arithmetically processes the signal from the first or second memory unit, and
The solid-state image sensor according to (1), further comprising a selection circuit that receives an output signal from the signal processing circuit and transfers the output signal to either the first or second memory unit.
(3)
The signal processing circuit generates one first reduced signal from the plurality of pixel signals from the first and second memory units.
The solid-state image sensor according to (2), wherein the selection circuit stores the first reduction signal in either the first or second memory unit.
(4)
The solid-state image sensor according to any one of (1) to (3), wherein one second memory unit is provided for two pixels.
(5)
The signal processing circuit compresses a plurality of the pixel signals stored in the plurality of first memory units to a quarter to generate a plurality of the first reduced signals.
The solid-state image sensor according to (4), wherein a part of the plurality of second memory units stores each first reduction signal.
(6)
The signal processing circuit compresses the plurality of first reduced signals stored in the plurality of second memory units to a quarter to generate a plurality of second reduced signals.
The solid according to (5), wherein the other second memory unit of the second memory unit stores each second reduced signal while the plurality of first memory units hold the plurality of pixel signals. Image sensor. (7)
The pixel signal is transferred to the outside of the first memory unit after the plurality of first reduced signals or the plurality of second reduced signals are transferred to the outside of the second memory unit, according to (6). Solid-state image sensor.
(8)
The signal processing circuit further compresses the plurality of second reduced signals stored in the plurality of second memory units to a quarter to generate a plurality of third reduced signals.
The plurality of first memory units hold a plurality of the pixel signals, the plurality of second memory units hold the second reduced signal, and the other second memory units are each a third. Stores the reduced signal,
The signal processing circuit further compresses the plurality of nth (n ≧ 3) reduction signals stored in the plurality of second memory units to a quarter to generate a plurality of n + 1 reduction signals.
The solid-state image sensor according to (6) or (7), wherein the second memory unit stores each n + 1 reduction signal.
(9)
The solid-state image sensor according to any one of (1) to (3), wherein one second memory unit is provided for each of three pixels.
(10)
The signal processing circuit compresses a plurality of the pixel signals stored in the plurality of first memory units to a quarter to generate a plurality of the first reduced signals.
The solid-state image sensor according to (9), wherein a part of the plurality of second memory units stores each first reduction signal.
(11)
The signal processing circuit compresses the plurality of first reduced signals stored in the plurality of second memory units to a quarter to generate a plurality of second reduced signals.
The solid according to (10), wherein the other second memory unit of the second memory unit stores each second reduced signal while the plurality of first memory units hold the plurality of pixel signals. Image sensor.
(12)
(11). The pixel signal is transferred to the outside of the first memory unit after the plurality of first reduced signals or the plurality of second reduced signals are transferred to the outside of the second memory unit. Solid-state image sensor.
(13)
The signal processing circuit further compresses the plurality of second reduced signals stored in the plurality of first memory units to a quarter to generate a plurality of third reduced signals.
The plurality of first memory units hold the plurality of the pixel signals, the plurality of second memory units store the second reduction signal, and the other second memory units are each of the third. Stores the reduced signal,
The signal processing circuit further compresses the plurality of nth (n ≧ 3) reduction signals stored in the plurality of second memory units to a quarter to generate a plurality of n + 1 reduction signals.
The solid-state image sensor according to (11) or (12), wherein the second memory unit stores each n + 1 reduction signal.
(14)
The solid-state image sensor according to any one of (1) to (3), wherein one second memory unit is provided for each of four pixels.
(15)
The signal processing circuit compresses a plurality of the pixel signals stored in the plurality of first memory units to a quarter to generate a plurality of the first reduced signals.
The solid-state image sensor according to (14), wherein the plurality of second memory units store each first reduction signal.
(16)
The signal processing circuit compresses the plurality of first reduced signals stored in the plurality of second memory units to a quarter to generate a plurality of second reduced signals.
The solid-state image sensor according to (15), wherein the first memory unit stores each second reduction signal.
(17)
(16), wherein after transferring the pixel signal stored in the plurality of first memory units to the outside of the first memory unit, the plurality of first memory units store each second reduced signal. Solid-state image sensor.
(18)
The signal processing circuit further compresses the plurality of second reduced signals stored in the plurality of first memory units to a quarter to generate a plurality of third reduced signals.
The first or second memory unit stores each third reduction signal and stores the third reduction signal.
The signal processing circuit further compresses the plurality of k (k ≧ 3) reduced signals stored in the plurality of first or second memory units to a quarter to generate a plurality of k + 1 reduced signals. death,
The solid-state image sensor according to (16) or (17), wherein the first or second memory unit stores each k + 1 reduction signal.
(19)
A pixel array including a plurality of pixels for photoelectric conversion, a plurality of AD conversion units provided corresponding to each of the plurality of pixels, and a plurality of AD conversion units provided corresponding to each of the plurality of AD conversion units. A solid-state image sensor including a first memory unit and a plurality of second memory units provided for each of the N pixels (N is an integer of 2 or more) or the N AD conversion units. The image processing method used,
The pixel signal from the pixel is digitally converted and
The pixel signal digitally converted by the AD conversion unit is stored in the plurality of first memory units.
One first reduction signal is generated from the plurality of pixel signals from the plurality of first memory units, and the plurality of first reduction signals are stored in either the first or second memory unit. Image processing method to be performed.
(20)
The method according to (19), wherein the plurality of first reduction signals are signals generated by compressing the plurality of pixel signals to a quarter.

Note that the present disclosure is not limited to the above-described embodiment, and various changes can be made without departing from the gist of the present disclosure. Further, the effects described in the present specification are merely examples and are not limited, and other effects may be obtained.

100 solid-state imaging device, 240 pixel array unit, 260 transfer circuit, 115 frame memory, 120 DSP circuit, 250 pixel circuit, 280 ADC unit, 290 ADC circuit, 291 first memory unit, 292 second memory unit, 293 signal processing circuit. 294 demultiplexer, 295 comparison circuit, 296 selector

Claims (20)

1. A pixel array containing multiple pixels that perform photoelectric conversion,
   A plurality of AD (Analogue-to-Digital) converters provided corresponding to each of the plurality of pixels and digitally converting a pixel signal from the pixels.
   A plurality of first memory units provided corresponding to each of the plurality of AD conversion units and storing the pixel signals digitally converted by the AD conversion unit, and a plurality of first memory units.
   A solid-state image sensor including N (N is an integer of 2 or more) of the pixels or a plurality of second memory units provided for each of the N AD conversion units.
2. A signal processing circuit that arithmetically processes the signal from the first or second memory unit, and
   The solid-state imaging device according to claim 1, further comprising a selection circuit that receives an output signal from the signal processing circuit and transfers the output signal to either the first or second memory unit.
3. The signal processing circuit generates one first reduced signal from the plurality of pixel signals from the first and second memory units.
   The solid-state image sensor according to claim 2, wherein the selection circuit stores the first reduction signal in either the first or second memory unit.
4. The solid-state image sensor according to claim 1, wherein one second memory unit is provided for two pixels.
5. The signal processing circuit compresses a plurality of the pixel signals stored in the plurality of first memory units to a quarter to generate a plurality of the first reduced signals.
   The solid-state image sensor according to claim 4, wherein a part of the plurality of second memory units stores each first reduction signal.
6. The signal processing circuit compresses the plurality of first reduced signals stored in the plurality of second memory units to a quarter to generate a plurality of second reduced signals.
   The solid according to claim 5, wherein the other second memory unit of the second memory unit stores each second reduced signal while the plurality of first memory units hold the plurality of pixel signals. Image sensor.
7. The sixth aspect of claim 6, wherein the pixel signal is transferred to the outside of the first memory unit after the plurality of first reduced signals or the plurality of second reduced signals are transferred to the outside of the second memory unit. Solid-state image sensor.
8. The signal processing circuit further compresses the plurality of second reduced signals stored in the plurality of second memory units to a quarter to generate a plurality of third reduced signals.
   The plurality of first memory units hold a plurality of the pixel signals, the plurality of second memory units hold the second reduced signal, and the other second memory units are each a third. Stores the reduced signal,
   The signal processing circuit further compresses the plurality of nth (n ≧ 3) reduction signals stored in the plurality of second memory units to a quarter to generate a plurality of n + 1 reduction signals.
   The solid-state image sensor according to claim 6, wherein the second memory unit stores each n + 1 reduction signal.
9. The solid-state image sensor according to claim 1, wherein one second memory unit is provided for each of three pixels.
10. The signal processing circuit compresses a plurality of the pixel signals stored in the plurality of first memory units to a quarter to generate a plurality of the first reduced signals.
    The solid-state image sensor according to claim 9, wherein a part of the plurality of second memory units stores each first reduction signal.
11. The signal processing circuit compresses the plurality of first reduced signals stored in the plurality of second memory units to a quarter to generate a plurality of second reduced signals.
    The solid according to claim 10, wherein the other second memory unit of the second memory unit stores each second reduced signal while the plurality of first memory units hold the plurality of pixel signals. Image sensor.
12. The eleventh aspect of claim 11, wherein the pixel signal is transferred to the outside of the first memory unit after the plurality of first reduced signals or the plurality of second reduced signals are transferred to the outside of the second memory unit. Solid-state image sensor.
13. The signal processing circuit further compresses the plurality of second reduced signals stored in the plurality of first memory units to a quarter to generate a plurality of third reduced signals.
    The plurality of first memory units hold the plurality of the pixel signals, the plurality of second memory units store the second reduction signal, and the other second memory units are each of the third. Stores the reduced signal,
    The signal processing circuit further compresses the plurality of nth (n ≧ 3) reduction signals stored in the plurality of second memory units to a quarter to generate a plurality of n + 1 reduction signals.
    The solid-state image sensor according to claim 11, wherein the second memory unit stores each n + 1 reduction signal.
14. The solid-state image sensor according to claim 1, wherein one second memory unit is provided for each of four pixels.
15. The signal processing circuit compresses a plurality of the pixel signals stored in the plurality of first memory units to a quarter to generate a plurality of the first reduced signals.
    The solid-state image sensor according to claim 14, wherein the plurality of second memory units store each first reduction signal.
16. The signal processing circuit compresses the plurality of first reduced signals stored in the plurality of second memory units to a quarter to generate a plurality of second reduced signals.
    The solid-state image sensor according to claim 15, wherein the first memory unit stores each second reduction signal.
17. 16. The 16. Solid-state image sensor.
18. The signal processing circuit further compresses the plurality of second reduced signals stored in the plurality of first memory units to a quarter to generate a plurality of third reduced signals.
    The first or second memory unit stores each third reduction signal and stores the third reduction signal.
    The signal processing circuit further compresses the plurality of nth (n ≧ 3) reduced signals stored in the plurality of first or second memory units to a quarter to generate a plurality of n + 1 reduced signals. death,
    The solid-state image sensor according to claim 16, wherein the first or second memory unit stores each n + 1 reduction signal.
19. A pixel array including a plurality of pixels for photoelectric conversion, a plurality of AD conversion units provided corresponding to each of the plurality of pixels, and a plurality of AD conversion units provided corresponding to each of the plurality of AD conversion units. A solid-state image sensor including a first memory unit and a plurality of second memory units provided for each of the N pixels (N is an integer of 2 or more) or the N AD conversion units. The image processing method used,
    The pixel signal from the pixel is digitally converted and
    The pixel signal digitally converted by the AD conversion unit is stored in the plurality of first memory units.
    One first reduction signal is generated from the plurality of pixel signals from the plurality of first memory units, and the plurality of first reduction signals are stored in either the first or second memory unit. Image processing method to be performed.
20. The method according to claim 19, wherein the plurality of first reduction signals are signals generated by compressing the plurality of pixel signals to a quarter.

Images