JP2001326857A - Image pickup element provided with arithmetic function - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an excellent image pickup element provided with an arithmetic function that can realize a sufficient number of pixels and a small pixel size. SOLUTION: The image pickup element has an optical area consisting of pixels with the same area as that of a conventional image acquisition image pickup element and is provided with an arithmetic processing circuit in common to pixels so as to avoid the sensitivity from being deteriorated by simultaneously utilizing signals of the pixels so as to supplement the lightness when the lightness of an object is dark or a time used to apply photoelectric conversion to the lightness of the object is short. As a result, the arithmetic operation on the basis of the lightness of the object can correctly be executed without receiving the effect of a noise.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a small and lightweight imaging device, and more particularly, to a CMOS (Complementarity).
The present invention relates to an imaging device with an arithmetic function realized by using a semiconductor manufacturing technology such as a metal-oxide semiconductor (complementary metal oxide semiconductor).

More specifically, the present invention relates to an image pickup device having an arithmetic function which realizes a sufficient number of pixels and a small pixel size, and more particularly to an image pickup device capable of taking a color image by using an on-chip color filter. .

[0003]

2. Description of the Related Art Along with recent rapid advances in semiconductor manufacturing technology, relatively inexpensive imaging devices have become available. As a result, mobile phones and PDAs (Personal Digital A
For portable terminals such as ssistants, devices equipped with or equipped with a small camera have been developed and have begun to be distributed on the market. However, these portable terminals are characterized by small size and light weight, so that a camera to be mounted must be small and light. In addition, since a portable device is generally a battery-driven device, it is required that the device itself and its attached / mounted components have low power consumption.

[0004] Generally speaking, a camera is a CCD (Charge).
I can imagine one using a Coupled Device (charge coupled device) sensor. CCD means MOS (Me
talOxide Semiconductor) An integrated circuit consisting of electrodes arranged like a chain. Image data captured using the function of sequentially transferring the charge on the semiconductor surface from one electrode to the next Is output. However, it is hard to say that the CCD sensor is suitable for the above-mentioned applications because it requires a plurality of power supply voltages and consumes relatively large power.

On the other hand, as a camera or an image pickup device, C
MOS (Complementary Metal-Oxide Semiconductor:
(Complementary metal oxide semiconductor) image sensors are attracting attention as next-generation image sensors. In particular, a so-called “smart sensor” having an arithmetic function for performing image processing on a sensor is expected to be widely used from games to security.

An image sensor of this type implemented using CMOS technology can satisfy specifications such as miniaturization and weight reduction and low power consumption. Further, various circuit components that can be realized by CMOS technology can be integrated on the same chip as the image sensor. In particular, a CMOS image having a function of converting a photodiode output from each pixel on a sensor after noise removal and gain correction, converting an analog value to a digital value, and further performing image processing as a digital signal.・ Some reports on sensors have been made.

[0007] Special articles on CMOS image sensors include, for example, "CMOS Image Sensor with Digital Image Processing Function" (Journal of the Institute of Image Information and Television Engineers).
Vol.53, No.2, pp.172-177, 1999).

Further, as a specific example of an image pickup device, "A
n Artificial Retina Chip with Current-Mode Focal P
lane Image Processing Functions "(Eiichi Funatsu,
et al, IEEE Trans. Electron Devices, Vol.44, No.1
0, Oct. 1997) and Fujimoto et al., CM with Motion Detection Function
"Development of OS Imager" (IPU99-62, 1999), Ishiwatari et al., "CMOS Image Sensor for 3D Gesture Recognition"
(Technical Report of the Institute of Image Information and Television Engineers, Vol.23, No.30, 199
9).

These smart sensors have a storage unit and a calculation unit for each pixel (pixel-parallel type) or for each column in which the pixels are arranged (column-parallel type), and perform some calculation while receiving the brightness of the subject. Has the common feature that desired processing is completed at high speed.

However, in many of the prior arts of the present application, in order to provide the smart sensor with an arithmetic processing function, there is a problem that a complicated circuit is realized in a pixel and the number of pixels is reduced as a result. there were.

On the other hand, many of the column-parallel imaging devices in which the arithmetic functions are arranged for each column have limited functions (for example, only an AD converter operation, etc.) even though the number of pixels is equivalent to that of a normal image sensor. However, there is a disadvantage that a sufficient calculation cannot always be performed.

Further, in terms of colorization of the image pickup device, the latter column-parallel type image pickup device has a small pixel size and can be realized by an on-chip color filter (OCCF), but the functions are limited to the last. . Also,
In the former pixel-parallel imaging device, the number of pixels is small in the first place, and further reduction in resolution due to colorization is a concern.

[0013]

An object of the present invention is to provide a CM
An object of the present invention is to provide an excellent imaging device having an arithmetic function, which is formed small and lightweight by using a semiconductor manufacturing technology such as an OS (Complementary Metal-Oxide Semiconductor).

A further object of the present invention is to provide an excellent image pickup device having an arithmetic function, which realizes a sufficient number of pixels and a small pixel size.

A further object of the present invention is to provide an excellent image pickup device having an arithmetic function capable of taking a color image by using an on-chip color filter.

[0016]

SUMMARY OF THE INVENTION The present invention has been made in consideration of the above problems, and a first aspect of the present invention relates to a light receiving unit for converting the brightness of a subject into an electric signal and an output of the light receiving unit. A plurality of pixels comprising an in-pixel amplifying unit to be amplified and a reading unit for reading a signal from the in-pixel amplifying unit to outside the pixel,
An extra-pixel amplifying unit that amplifies a signal read from each pixel, a plurality of storage units that store signals read from each pixel, and a calculation unit that calculates signals read from one or more storage units A digital memory that stores the operation result of the operation unit for each pixel, an output unit that outputs a digital memory output to the outside, and a drive control unit that controls driving of each unit. The image pickup device with an arithmetic function, characterized in that the unit has a drive control mode in which the number of pixels from which a signal is read out and the number of storage units that store the read out signal are made variable in accordance with the content of operation in the arithmetic unit is there.

[0017] In the imaging device with an arithmetic function according to the first aspect of the present invention, the arithmetic unit and / or the extra-pixel amplifying unit may be shared by a plurality of pixels.

Further, the drive control section may determine the content of the calculation in the calculation section based on a combination of drive pulses supplied to the calculation section and the timing thereof.

The drive control section may have a drive control mode in which two or more pixel readout signals detected at the same timing are stored in one storage section.

The drive control section stores the readout signal of each pixel detected at each timing in a separate storage section, and reads out a pixel in the same pixel based on the signal read out from each storage section. A first drive control mode for causing the arithmetic unit to perform a comparison process along a time axis of a signal may be provided. As a result, the imaging device with the calculation function can calculate the change in the brightness of the subject and output the change to the outside.

Further, the drive control section further stores the two or more pixel read signals detected at each timing in the same storage section, and based on the signals read from each storage section, A second drive control mode may be provided in which the arithmetic unit performs the comparison process of the pixel readout signals of the pixels along the time axis.

In the second drive control mode, the brightness of the subject is too low, or the sampling interval for photoelectrically converting the brightness of the subject is too short, and the output of a single pixel is small and the influence of noise is reduced. Even in a case where the signal is easily received, by combining and processing the signals of a plurality of adjacent pixels into one, it is possible to compensate for a low output and avoid a decrease in sensitivity. As a result, the imaging device with an arithmetic function can correctly execute an arithmetic operation without being affected by noise. On the other hand, when sufficient brightness of the subject can be obtained, the above-described first drive control mode is used to operate each pixel independently so as to perform arithmetic processing, thereby reducing the resolution of the image sensor. Can be avoided.

The drive control section stores a pixel readout signal corresponding to a reference signal level of a certain pixel in one storage section, and stores a pixel readout signal corresponding to the brightness of a subject of the pixel in another storage section. And a driving control mode in which the arithmetic unit compares the reference signal level with the brightness of the subject based on the pixel read signal read from each storage unit. In such an operation mode, the imaging device with the arithmetic function can externally output a captured image obtained by converting the brightness of a subject, which is an analog amount, into a digital amount.

Further, for each pixel of the image pickup device having an arithmetic function, for example, on-chip color filters of each color such as M (magenta), C (cyan), G (green), and Y (yellow) are alternately provided. It may be arranged. In such a case, the drive control unit stores the pixel readout signal corresponding to the reference signal level of a certain pixel in one storage unit and stores the pixel readout signal corresponding to the brightness of the subject of the pixel in another storage unit. Unit, and the reference signal level and the brightness of the subject are compared in the calculation unit based on the pixel read signal read from each storage unit. Can be externally output.

Further, an on-chip lens may be arranged for each pixel of the image pickup device having the arithmetic function. In such a case, the degree of condensing light in each pixel can be increased, so that the sensitivity of the imaging device with an arithmetic function can be improved regardless of which drive control mode is operated.

According to a second aspect of the present invention, there is provided a light receiving unit for converting the brightness of an object into an electric signal, an in-pixel amplifying unit for amplifying the output of the light receiving unit, and a signal from the in-pixel amplifying unit to outside the pixel. A pixel area in which a plurality of pixels including a read-out unit to be read are arranged, an analog storage area in which two or more storage units are arranged for each pixel, and an extra-pixel amplifying unit that amplifies a signal read from each pixel A calculation area in which a calculation unit for calculating a signal read from one or more storage units is arranged; and a digital storage area in which a plurality of digital memories for storing calculation results by the calculation unit for each pixel are arranged. And an output area in which a plurality of output units for outputting digital memory outputs to the outside are provided.

For example, CMOS (Complementary Metal-
By using a semiconductor manufacturing technique such as Oxide Semiconductor (complementary metal oxide semiconductor), each of the above areas can be mounted on a single circuit chip.

In the image pickup device with an arithmetic function according to the second aspect of the present invention, a predetermined number of adjacent pixels may be driven as a basic operation unit in the pixel area.

The arithmetic unit and / or the extra-pixel amplifying unit may be shared by a plurality of pixels.

[0030] The image processing apparatus may further include a drive control unit that determines the content of the calculation in the calculation unit based on the combination and timing of the drive pulses supplied to the calculation unit. This drive control unit can have, for example, a drive control mode in which two or more pixel readout signals detected at the same timing are stored in one storage unit.

The drive control section stores the readout signal of each pixel detected at each timing in a separate storage section, and reads out a pixel in the same pixel based on the signal read out from each storage section. A first drive control mode for causing the arithmetic unit to perform a comparison process along a time axis of a signal may be provided. In such an operation mode, the imaging device with an arithmetic function can calculate a change in the brightness of the subject and output it to the outside.

The drive control section stores the two or more pixel read signals detected at each timing in the same storage section, respectively, and based on the signals read from each storage section, A second drive control mode may be provided in which the arithmetic unit performs a comparison process along a time axis of a pixel readout signal of a pixel. Under this operation mode, there are cases where the brightness of the subject is too low, or the time interval for photoelectrically converting the brightness of the subject is short, and a single pixel has a small output and is easily affected by noise. Also, by processing the signals of a plurality of adjacent pixels as one, it is possible to compensate for a low output and avoid a decrease in sensitivity. As a result, the imaging device with an arithmetic function can correctly execute an arithmetic operation without being affected by noise.

The drive control section stores a pixel readout signal corresponding to a reference signal level of a certain pixel in one storage section, and stores a pixel readout signal corresponding to the brightness of a subject of the pixel in another storage section. And a driving control mode in which the arithmetic unit compares the reference signal level with the brightness of the subject based on the pixel read signal read from each storage unit. In this operation mode, the imaging device with the arithmetic function can externally output a captured image obtained by converting the brightness of a subject, which is an analog amount, into a digital amount.

Further, for each pixel of the image pickup device having an arithmetic function according to the second aspect of the present invention, for example, M, C, G,
On-chip color filters of each color such as Y may be arranged alternately. In such a case, the drive control unit stores the pixel readout signal corresponding to the reference signal level of a certain pixel in one storage unit and stores the pixel readout signal corresponding to the brightness of the subject of the pixel in another storage unit. Unit, and the reference signal level and the brightness of the subject are compared in the calculation unit based on the pixel read signal read from each storage unit. Can be externally output. Also, a set of adjacent pixels on which the M, C, G, and Y color filters are mounted may be treated as a basic operation unit in the pixel area.

Further, an on-chip lens may be arranged for each pixel of the image pickup device having an arithmetic function. In such a case, the degree of condensing light in each pixel can be increased, so that the sensitivity of the imaging device with an arithmetic function can be improved regardless of which drive control mode is operated.

[0036]

The image pickup device according to the present invention has an optical area composed of pixels having the same area as that of a normal image pickup image pickup device. However, by providing a plurality of pixels with a common arithmetic processing circuit, In the case where the brightness of the subject is low or the time for photoelectrically converting the brightness of the subject is short, a reduction in sensitivity can be avoided by simultaneously using the signals of a plurality of pixels to compensate for the brightness. As a result,
The calculation based on the brightness of the subject can be correctly executed without being affected by noise.

On the other hand, when sufficient brightness is obtained from the subject, it is possible to avoid a decrease in resolution by using each pixel independently for calculation.

Since the image pickup device according to the present invention has a sufficiently large number of pixels, colorization can be easily achieved by employing an on-chip color filter. In addition, since the image sensor according to the present embodiment has a small pixel size, it is possible to increase the light condensing degree by using an on-chip lens,
Sensitivity can be increased.

Further, in the image pickup device according to the present invention, the operation content in the operation section can be freely changed according to a combination of drive pulses supplied from the outside and the timing thereof. Therefore, various functions that cannot be performed by an imaging device having only a single arithmetic function can be realized on a single imaging device.

Still other objects, features and advantages of the present invention are:
It will become apparent from the following more detailed description based on the embodiments of the present invention and the accompanying drawings.

[0041]

Embodiments of the present invention will be described below in detail with reference to the drawings.

FIG. 1 schematically shows the structure of an image sensor according to an embodiment of the present invention. As shown in the figure, the image sensor includes a pixel area, an analog storage area, a calculation area, a digital storage area, and an output area. For example, CMOS (Complementary Metal-Oxide
These functional modules can be mounted on a single circuit chip by using a semiconductor manufacturing technology such as a semiconductor (complementary metal oxide semiconductor). Hereinafter, each unit in the image sensor will be described.

In the pixel area 10, M pixels 11 in a horizontal direction and N pixels 11 in a vertical direction are two-dimensionally arranged. As shown in the figure, the output signals of two columns of pixels adjacent in the horizontal direction are 1
The vertical signal lines 101 are shared, and an extra-pixel amplifier 31 is connected to the end of each vertical signal line 101.

In the analog storage area 20, M / 2 storage units 21 in the horizontal direction and N / 2 storage units 21 in the vertical direction are two-dimensionally arranged. As shown in the figure, N / 2 storage units 21 arranged vertically share one vertical amplified signal line 102, and the corresponding extra-pixel amplifying unit 31 described above is provided at the end of each vertical amplified signal line 102. It is connected. Also, the vertical amplification signal line 1
02 is also connected to the calculation unit 32 in the calculation area 30. However, one storage unit 21 stores a plurality of analog
It is assumed that a memory is built-in.

In the operation area 30, an extra-pixel amplifying unit 31 is provided.
And M / 2 arithmetic units 32 are arranged.
A corresponding vertical signal line 101 from the pixel area 10 is connected to an input of each extra-pixel amplifying unit 31.
Further, the output of each extra-pixel amplifying unit 31 is connected to the vertical amplification signal line 1
02, the corresponding arithmetic unit 32 and the storage unit 21 in the analog storage area 20 are connected. On the other hand, the output of the arithmetic unit 32 is shared as the arithmetic output signal 103 by the plurality of digital memories 41 in the digital storage area 40.

In the digital storage area 40, M × N digital memories 41 are two-dimensionally arranged corresponding to the pixels 11. The operation output signal 103 output from each operation unit 32 acts as a write enable signal for the corresponding digital memory 41. A digital signal (to be described later) supplied from the inside or outside of the image sensor is connected to an input of the digital memory 41, and is written according to the above-described write enable signal. The outputs of the N digital memories 41 arranged in one column in the vertical direction share the memory bus 104 which is a digital signal line having a width corresponding to the number of bits. Each memory bus 104 is connected to a corresponding output unit 51 in the output area 50.

In the output area 50, M output units 51 are arranged side by side. A memory bus 104 which is an output of the corresponding digital memory 41 in the digital storage area 40 is connected to an input of each output unit 51. The output of each output unit 51 is a digital string output 105. The data width of the digital column output 105 does not necessarily need to be the same as the data width of the memory bus 104, and can be a bit width according to the processing content of the output unit 51.

In the image pickup device shown in FIG. 1, M × N pixels 11 are collectively arranged in the pixel area 10, and the pixel 11 has no special arithmetic circuit inside. . By adopting such a circuit structure, the pixel size can be reduced by a normal image sensor,
It can be made the same as a CD or the like. As a result, it is possible to realize an image sensor having the same number of pixels (resolution) and also having an arithmetic function.

FIG. 2 illustrates a basic operation unit extracted from the image pickup device structure shown in FIG. The basic operation unit referred to here is four pixels, one vertical signal line,
It comprises one extra-pixel amplifying unit, one storage unit, one vertical amplified signal line, one arithmetic unit, four digital memories, and two output units. .

FIG. 3 shows a configuration example of a unit pixel, and FIG. 4 shows a configuration example of an analog storage unit corresponding to the unit pixel.

In the examples shown in FIGS. 2 to 4, one application when the number of pixels in the basic operation unit is four is because it is assumed that a color filter of a single-chip color image sensor is used. is there. That is, for example, when a complementary color filter is used, the four pixels correspond to magenta (M), green (G), cyan (C), and yellow (Y) color filters, respectively. Or,
When the primary color filters are used, the four pixels may correspond to red (R), green (G1), blue (B), and green (G2) color filters, respectively.

Further, as another application when the number of pixels as a basic operation unit is four (or a plurality), a structure in which a plurality of pixels can be collectively processed as described above is used, for example, in the first place. Even if the brightness of the subject is low or the light accumulation time is short and the signal obtained from one pixel is weak, the signal level can be prevented from lowering by simultaneously using multiple pixel signals. It becomes possible.

Hereinafter, referring to FIGS. 3 and 4 as needed,
The structure and operation characteristics of the basic operation unit shown in FIG. 2 will be described in detail.

4 assigned to each color filter
The pixels 11 are connected to a common vertical signal line 101. As shown in FIG. 3, in each pixel, after the incident light corresponding to the brightness of the subject is photoelectrically converted by the light receiving unit 12 during the accumulation time, the light passes through the in-pixel amplifying unit 13 and the reading unit 14 and The signals are sequentially output to the vertical signal line 101.

The operation of each pixel 11 is based on the reset pulse 112 (φRST) supplied to the in-pixel amplifier 13,
The transfer pulse 113 (φPT
X), the pixel signal readout pulses 114 to 117 (φRD_G, φRD_M, φRD_
C, φRD_Y. However, other than φRD_G is not shown in FIG. 3). Further, a reset voltage 111 (VRST) for setting the reset level is supplied to the in-pixel amplifier 13.

In this embodiment, the reset pulse 112
(ΦRST) and the transfer pulse 113 (φPTX) are common to the four pixels which are the basic operation units, so that the light accumulation timings of the four pixels coincide.
That is, among the four pixels that are the basic operation units, each of the in-pixel amplifiers 13 outputs the reset pulse 112 (φ
RST), the signal is reset to the reset level determined by the reset voltage 111 (VRST) at the same time, and then the transfer pulse 113 (φPTX) is applied. To be transferred simultaneously. The time interval from one transfer to the next transfer is the light accumulation time in the pixel.

The signal transferred inside the pixel 11 includes a pixel signal readout pulse (for example, φRD for a G pixel).
_G) is applied to the outside of the pixel 11, more specifically, to the extra-pixel amplification unit 31 through the vertical signal line 101.

Returning to FIG. 2, the description will be continued. Vertical signal line 1
01 has its terminal connected to the extra-pixel amplifying unit 31. The extra-pixel amplifying unit 31 includes a pixel 1 serving as a basic operation unit.
1 is amplified to a level required for the subsequent processing. The amplifier read-out pulse 137 (φCRD) is given to the extra-pixel amplifier 31.
The aforementioned four pixel signal readout pulses 114 to 117 (φ
The output signal from each pixel 11 is sent to the extra-pixel amplifying unit 31 and is amplified in synchronization with (RD_G, φRD_M, φRD_C, φRD_Y). Out-of-pixel amplifier 3
1 is connected to the vertical amplified signal line 102,
The signal amplified here is output to the storage unit 21 and the arithmetic unit 32.

In the storage unit 21, two pixels are stored for each pixel.
, That is, a total of eight analog memories 22 in the basic operation unit. Each analog memory 22
Share the vertical amplification signal line 102, and can store the amplified signal and read the stored signal in accordance with the rules of the control signal described later.

Here, referring to FIG.
The memory 22 will be described.

Each of the eight analog memories 22 corresponds to one pixel. That is, G, M, C,
The analog memories 1G and 2G correspond to each pixel of Y.
G, 1M and 2M, 1C and 2C, 1Y and 2Y are arranged. Further, each analog memory 22 has analog memory read pulses 121 to 128 (φAMR_
1G, φAMR_2G, φAMR_1M, φAMR_2
M, φAMR_1C, φAMR_2C, φAMR_1Y,
φAMR_2Y) and analog memory write pulses 129 to 136 (φAMW_1G, φAMW_2
G, φAMW_1M, φAMW_2M, φAMW_1C,
φAMW_2C, φAMW_1Y, φAMW_2Y) are given, respectively.

From the analog memory 22 to which the analog memory read pulse has been applied, the vertical amplified signal line 102
The signal is read out via. The signal appearing on the vertical amplification signal line 102 can be stored in the analog memory 22 to which the analog memory write pulse has been applied.

Returning to FIG. 2, the description will be continued.

The vertical amplification signal line 102 is connected not only to the analog memory 22 but also to the operation unit 32. The signal stored or read in the analog memory 22 can be transmitted to the arithmetic unit 32 according to a control signal described later.

The operation section 32 operates a read signal input from the analog memory 22 and outputs the operation result as an operation output signal 103. The output signal is written in the digital memory 41 corresponding to each pixel. -Acts as an enable signal. The operation unit 32 includes an operation pulse 138 (φOP) and an operation read pulse 139 (φ
PRD). The operation unit 32 compares the magnitudes of the two signals input during the period in which the operation pulse 138 (φOP) is applied, and compares the comparison result while the operation read pulse 139 (φPRD) is applied. For example, a low or high level operation output signal 10
3 is output.

In this embodiment, one digital memory 41 is prepared for each pixel. The number of bits of each digital memory 41 is set to an optimal number (for example, 16 bits) for a signal to be processed, and the digital data input signal 141 (DATA_I) having a bit width corresponding to the number of bits is set.
N) are connected. The digital data input signal 141 may be a signal generated by a digital circuit inside the image sensor or a signal supplied from outside the image sensor.

In the digital memory 41 corresponding to each pixel, chip select signals 142 to 145 (φCS_G, φC
S_M, φCS_C, φCS_Y. However, except for φCS_G, which is not shown in FIG. 2), only the digital memory 41 selected by the chip select signal can perform the write or read operation. The output of the digital memory 41 is a digital signal line having a bit width corresponding to the number of memory bits, that is, a memory signal.
It is connected to the output unit 51 via the bus 104.

Since the operation output signal 103 from the operation unit 32 is a write enable signal, when the write enable signal is applied, only the digital memory selected by the chip selection signal is supplied to the image sensor. A digital data input signal 141 (DATA_IN) supplied from inside or outside is stored. On the other hand, when the write enable signal to the digital memory 41 is not applied, the signal stored in the digital memory 41 to which the chip select signal is applied is read out to the memory bus 104, and the output unit 51
To be transmitted to

The output unit 51 outputs a digital output for each pixel to the outside of the image sensor in synchronization with a control signal described later. That is, as shown in FIG.
Numerals 1 are juxtaposed by the number of basic units of vertically arranged pixels. Each output unit 51 is sequentially provided with a digital memory output pulse 151 (φPOUT), and is read out from the corresponding digital memory 41 via the memory bus 104 in synchronization with the input. When the signal is output
1 and output as a digital column signal 105 to the outside of the image sensor as a digital pixel signal.

Next, the operation timing in the basic operation unit of FIG. 2 will be described with reference to FIG.

Generally, in order to calculate the temporal change of the light intensity as fast as possible, the light accumulation time should be as short as possible. However, as the accumulation time becomes shorter,
Naturally, the photoelectrically converted signal obtained from one pixel used in one operation is weakened, so that the operation process is more likely to be affected by noise. According to the imaging device according to the present embodiment, when performing the calculation in such a short accumulation time, as described later, the output signals of a plurality of pixels (that is, a basic operation unit) are simultaneously used. By doing so, it is possible to increase the signal amount and reduce the influence of noise as compared with the case where only the signal of a single pixel is used.

The image sensor according to the present embodiment can be driven in four operation modes from mode A to mode D. These four modes of operation are based on the analog memory 22
Sequentially perform a signal storage operation and a read operation to
As a result, it is possible to continuously perform a kind of differential processing along the time axis. Hereinafter, each operation mode will be described.

Mode A: In the mode A, first, signals are read from the pixels M and C by applying the pixel read pulses φRD_M (115) and φRD_C (116), and the analog memory write pulse φAMW_
By applying 1M (131), these read signals are stored in the analog memory 1M corresponding to the pixel M.

Subsequently, the pixel readout pulses φRD_G (114) and φRD_Y (11
7), the signals are read out from the pixels G and Y, and the analog memory write pulse φAM
By applying W_1G (129), these read signals are stored in the analog memory 1G corresponding to the pixel G.

In the above operation, the amplifier reading pulse 137 (φCRD) is changed to the pixel reading pulses 114 to 1.
17 (φRD_G, φRD_M, φRD_C, φRD_Y)
At the same time, the signal amplified by the extra-pixel amplification unit 31 is stored in the corresponding analog memory 22 via the vertical amplification signal line 102.

Subsequently, analog memory read pulses φAMR_2M (124), φAMR_1C (12
5), φAMR_2G (122), φAMR_1Y (12
7), the analog memory 2M for the pixel M, the analog memory 1C for the pixel C,
The signals stored in the analog memory 2G for the pixel G and the signal stored in the analog memory 1Y for the pixel Y are simultaneously read, and the operation pulse φOP (138)
, The signals are collectively input to the arithmetic unit 32.

Then, the analog memory read pulses φAMR_1M (123) and φAMR_2C (12
6), φAMR_1G (121), φAMR_2Y (12
8), the analog memory 1M for the pixel M, the analog memory 2C for the pixel C,
The signals stored in the analog memory 1G for the pixel G and the signal stored in the analog memory 2Y for the pixel Y are read out at the same time, and these are collectively input to the arithmetic unit 32.

At this time, the operation section 32 compares the result of comparing the sum of the four signals input at once with the sum of the four signals input at one time later with the operation read pulse φPRD (139). The calculation output signal 103 is output as a write enable signal for the digital memory 41 during the period in which the signal is applied.

Since φCS_M (143) is applied as a chip selection signal for selecting the digital memory 41, the result of the above operation is the digital memory 4 corresponding to the pixel M in accordance with the write enable signal.
1, the digital data input signal 141 is stored.

Mode B: In mode B, first, signals are read from the pixels M and C by applying the pixel read pulses φRD_M (115) and φRD_C (116), and further, the analog memory write pulse φAMW_
By applying 2M (132), these read signals are stored in the analog memory 2M corresponding to the pixel M.

Subsequently, the pixel readout pulses φRD_G (114) and φRD_Y (11
7) to read out the signals from the pixels G and Y, and further apply the analog memory write pulse φA
By applying MW_2G (130), these read signals are stored in the analog memory 2G corresponding to the pixel G.

Even in this operation mode, the amplifier read pulse 137 (φCRD) is supplied to the pixel read pulses 114 to 1.
17 (φRD_G, φRD_M, φRD_C, φRD_Y)
At the same time, the signal amplified by the extra-pixel amplifying unit 31 is stored in the corresponding analog memory 22 via the vertical amplified signal line 102.

Subsequently, analog memory read pulses φAMR_1C (125) and φAMR_2C (12)
6), φAMR_1Y (127), φAMR_2Y (12
8), the analog memory 1C for the pixel C, the analog memory 2C, the analog memory 1Y for the pixel Y, and the signal stored in the analog memory 2Y are simultaneously read out, and the operation pulse φO
By applying P (138), those signals are collectively input to the arithmetic unit 32.

Then, the analog memory read pulses φAMR_1M (123) and φAMR_2M (12
4), φAMR_1G (121), φAMR_2G (12
2), the analog memory 1M and the analog memory 2M for the pixel M, the analog memory 1G and the analog memory 2G for the pixel G
Are read out at the same time, and these are collectively input to the arithmetic unit 32.

At this time, the operation section 32 compares the result of comparing the sum of the four signals input collectively at the beginning with the sum of the four signals input collectively later, with the operation read pulse φPRD (139). The calculation output signal 103 is output as a write enable signal for the digital memory 41 during the period in which the signal is applied.

Since φCS_M (143) is also applied as a chip selection signal for selecting the digital memory 41, the digital memory corresponding to the pixel M is output as the operation result in accordance with the write enable signal. At 41, a digital data input signal 141 is stored.

Mode C: In mode C, first, signals are read from the pixels M and C by applying the pixel read pulses φRD_M (115) and φRD_C (116), and the analog memory write pulse φAMW_
By applying 1C (133), these read signals are stored in the analog memory 1C corresponding to the pixel C.

Subsequently, the pixel readout pulses φRD_G (114) and φRD_Y (11
7) read out signals from the pixels G and Y by applying
By applying AMW_1Y (135), these read signals are converted to the analog memory 1Y corresponding to the pixel Y.
To memorize.

Even in this operation mode, the amplifier read pulse 137 (φCRD) is supplied to the pixel read pulses 114 to 1.
17 (φRD_G, φRD_M, φRD_C, φRD_Y)
At the same time, the signal amplified by the extra-pixel amplifying unit 31 is stored in the corresponding analog memory 22 via the vertical amplified signal line 102.

Subsequently, analog memory read pulses φAMR_1M (123) and φAMR_2C (12
6), φAMR_1G (121), φAMR_2Y (12
8), the analog memory 1M for the pixel M, the analog memory 2C for the pixel C,
The signals stored in the analog memory 1G for the pixel G and the signal stored in the analog memory 2Y for the pixel Y are read out at the same time, and by applying an operation pulse φOP (138), the signals are collectively transmitted to the operation unit 32. input.

Then, the analog memory read pulses φAMR_2M (124) and φAMR_1C (12
5), φAMR_2G (122), φAMR_1Y (12
7) by applying an analog
The signals stored in the memory 2M, the analog memory 1C for the pixel C, the analog memory 2G for the pixel G, and the signal stored in the analog memory 1Y for the pixel Y are simultaneously read and input to the arithmetic unit 32 at a time.

At this time, the arithmetic unit 32 compares the sum of the four signals input collectively at the beginning with the sum of the four signals input collectively later and calculates the operation read pulse φPRD (139). The calculation output signal 103 is output as a write enable signal for the digital memory 41 during the period in which the signal is applied.

Then, by applying the chip selection signal 143 (φCS_M) based on the above operation result,
In response to the write enable signal, the digital data input signal 141 is stored in the digital memory 41 corresponding to the pixel M.

Mode D: In mode D, first, signals are read out from the pixels M and C by the pixel readout pulses φRD_M (115) and φRD_C (116), and the analog memory write pulse φAMW_2C (134) is applied. The read signal is stored in the analog memory 2C corresponding to the pixel C.

Subsequently, the pixel read pulses φRD_G (114) and φRD_Y (11
7), the signals are read from the pixels G and Y, and the analog memory write pulse φAMW_2Y (1
36), these readout signals are stored in the analog memory 2Y corresponding to the pixel Y.

Even in this operation mode, the amplifier read pulse 137 (φCRD) is supplied to the pixel read pulses 114-1.
17 (φRD_G, φRD_M, φRD_C, φRD_Y)
At the same time, the signal amplified by the extra-pixel amplifying unit 31 is stored in the corresponding analog memory 22 via the vertical amplified signal line 102.

Subsequently, analog memory read pulses φAMR_1M (123) and φAMR_2M (12
4), φAMR_1G (121), φAMR_2G (12
2), the analog memory 1M and the analog memory 2M for the pixel M, the analog memory 1G and the analog memory 2G for the pixel G
Are read out at the same time, and by applying an operation pulse φOP (138), the signals are collectively input to the operation unit 32.

Then, analog memory read pulses φAMR_1C (125) and φAMR_2C (12
6), φAMR_1Y (127), φAMR_2Y (12
8) to apply an analog signal to pixel C
The signals stored in the memory 1C and the analog memory 2C, and the signals stored in the analog memory 1Y and the analog memory 2Y for the pixel Y are simultaneously read out and input to the arithmetic unit 32 at a time.

At this time, the operation section 32 compares the sum of the four signals input at once with the sum of the four signals input at a later time by the operation read pulse φPRD (139). The calculation output signal 103 is output as a write enable signal for the digital memory 41 during the period in which the signal is applied.

By applying the chip select signal φCS_M (143) of the digital memory 41 to the pixel M, the digital data input signal 141 at that time is stored only in the digital memory 41. I have.

In modes A to D described above, reset pulse 112 (φRST) and transfer pulse 113 (φPTX) are both pixel readout pulses 114 to 117 (φRD_G, φRD_M, φRD_C,
(RD_Y), a newly photoelectrically converted signal is used for calculation every time the mode is switched.

As described above, when the four operation modes from mode A to mode D are repeatedly executed, the total signal level of every two pixels photoelectrically converted in one mode period is represented by f (k). In this case, it is possible to perform an arithmetic process corresponding to the differentiation of the signal along the time axis as shown in the following equation (where k is an index indicating time sampling).

[0103]

F (f) + f (k−1) − {f (k−2) + f (k−3)} (Equation 1)

This is tabulated below.

[0105]

[Table 1]

In the above table, 1M and 2G are described as follows.
It means signals stored in the analog memory 1M of the pixel M and the analog memory 2G of the pixel G, respectively.
For example, at sampling time k-4, time k-4,
The operation in mode A can be performed as shown in the following equation using the signals of the respective pixels at successive sampling times k-5, k-6, and k-7.

[0107]

(2) (1M + 1G + 2C + 2Y) − (1C + 1Y + 2M + 2G) (Equation 2)

Similarly, at the next sampling time k-3, the operation in mode B can be performed as shown in the following equation.

[0109]

(2M + 2G + 1M + 1G)-(2C + 2Y + 1C + 1Y) (Equation 3)

By continuing the calculation with k-2, k-1,... In the above order, the calculation of (Equation 1) is performed as a result.

Further, in this case, since the photoelectrically converted signals are read out in the four pixels, it is possible to obtain the temporal change of the signal with respect to the weak light having a low signal level by using only one pixel output. it can. Of course, if the number of pixels to be processed simultaneously is increased from six to eight instead of four pixels, arithmetic processing for weaker light becomes possible.

Although the timing of reading from digital memory 41 is not shown in the operation timing chart shown in FIG. 5, modes A, B, and
The processing is sequentially performed in the order of C, D, A, B, C, D,.
(ΦPOUT) is sequentially generated by the number of digital memories 41 used for storage, and at the same time, a chip select signal (φCS_M in this example) is applied to the digital memory 41 to be read, and the memory bus 104 , A digital string output 105 can be obtained via the output unit 51.

Next, the light intensity (brightness) received by the pixel
An operation timing when converting into a digital signal and outputting the signal will be described.

In the case of the operation of the image sensor according to this embodiment, 1
Although the accumulation time is short, the signals read from the pixels are sequentially added, so that a decrease in the signal level can be avoided (described above). Therefore, each pixel can be used independently for calculation, and a decrease in resolution can be avoided. In addition, when an on-chip color filter is employed, color output is possible as a one-chip color image sensor.

First, the principle of converting the brightness, which is an analog amount, into a digital amount in the unit pixel shown in FIG. 3 will be described with reference to FIG.

VFD is the signal voltage level of the in-pixel amplifier 13 in the pixel 11 shown in FIG. In a normal operation, the reset pulse 112 (φRS
By applying T), VFD is set to the reset level determined by the reset voltage 111 (VRST) set to the power supply voltage.

When the pixel 11 is irradiated with light in this state,
Electrons generated by photoelectric conversion are accumulated in the light receiving unit 12. Then, the transfer pulse 113 (φPT
When the stored electrons are transferred to the in-pixel amplifier 13 by applying X), the signal voltage level VFD decreases from the previous reset level.

The degree to which the voltage of the signal voltage level VFD decreases is proportional to the amount of transferred electrons, that is, the intensity of light incident on the light receiving section 12. Therefore, the lower the signal voltage level VFD is, the steeper the light is, the lower the VFD is.

By utilizing such properties, for example, FIG.
In the graph shown in FIG. 5, bright light indicated by line H,
The respective brightness levels VH, VM, VL of the light of the middle brightness of the line M and the dark light of the line L can be expressed by the following formula based on the similarity of the triangle. However, TH, TM, and TL respectively represent the times when the lines H, M, and L representing the brightness cross the reference level determined by the reset voltage 111 (VRST) set as the reference voltage.

[0120]

VH = TS · ΔVR / TH (Equation 4) VM = TS · ΔVR / TM (Equation 5) VL = TS · ΔVR / TL (Equation 6)

Accordingly, light accumulation starts from time zero and the instant when the signal voltage level VFD, which decreases every time the transfer pulse 113 (φPTX) is applied, reaches the reference level, is detected. Can be expressed as a function of time.

At this time, the transfer pulse 113 (φPTX)
Is counted as a time interval of the sampling period ΔT, the time at which the signal voltage level VFD reaches the reference level can be obtained as a digital amount of brightness.

Also, instead of directly comparing the value of the signal voltage level VFD with the reference level, the signal level read from the pixel when VFD is at the reference level is stored, and when the light is irradiated. To transfer pulse 113
By comparing the value of the VFD that changes every moment with the signal level read from the pixel 11 every time (φPTX) is given, the brightness can be detected similarly. It will be easy to understand.

In this way, it is possible to convert an analog quantity called brightness into a digital quantity quantized by time information sampled at intervals of time. The number of bits of the digital amount depends on the fineness of the sampling of the time information.

FIG. 7 shows an operation of storing a pixel output signal corresponding to a reference level serving as a reference of brightness in the analog memory 22. Hereinafter, the operation will be described with reference to FIG.

First, a transfer pulse 113 (φPTX) is applied to sweep out the photoelectrically converted signal from the light receiving unit 12 to the in-pixel amplifier 13. However, since the output signal at this time is not necessary for the subsequent processing, the reset pulse 11
2 (φRST) is applied, and the in-pixel amplifier 13 is reset by a reset voltage 111 (VRST) set to a reference voltage lower than the power supply voltage.

Then, the pixel M, the pixel C, the pixel G, and the pixel Y
, The pixel read pulse φRD_M (115), φR
D_C (116), φRD_G (114), φRD_Y
At the same time as applying (117), the read pulse 1
37 (φCRD) and analog memory write pulses φAMW_1M (131), φAMW_1C (133),
By applying φAMW_1G (129) and φAMW_1Y (135), the signals are output from the pixels corresponding to each of the analog memory 1M, the analog memory 1C, the analog memory 1G, and the analog memory 1Y, and are output by the extra-pixel amplification unit 31. A signal corresponding to the amplified reference level is stored.

In this period, since the arithmetic section 32 and the digital memory 41 do not need to operate, no drive pulse is generated for them.

FIG. 8 is a timing chart showing a series of operations for comparing the reference level signal stored in accordance with the operation timing shown in FIG. 7 with the photoelectrically converted signal for each pixel and storing the result in a digital memory.・ It is a chart

First, in the first storage period, each pixel M,
Each analog memory 2 corresponding to C, G, Y
M, 2C, 2G, and 2Y store the signals photoelectrically converted after the previous reference signal storage period.

First, the reset pulse 112 (φRS
T) and reset voltage 1 set to the power supply voltage
11 (VRST) (not shown) resets the in-pixel amplifier 13 in each pixel.

Immediately thereafter, the transfer pulse 113 (φPTX)
To transfer the signal photoelectrically converted by the light receiving unit 12 to the in-pixel amplifying unit 13.

Then, the pixels M, C, G, Y
, The pixel read pulse φRD_M (115), φR
D_C (116), φRD_G (114), φRD_Y
(117) is applied, and at the same time
37 (φCRD) and analog memory write pulses φAMW_2M (132), φAMW_2C (134),
By applying φAMW_2G (130) and φAMW_2Y (136), the analog memories 2M, 2C,
In each of 2G and 2Y, a signal read from a corresponding pixel and amplified by the extra-pixel amplifying unit 31 is stored.

Subsequently, pixel M, pixel C, pixel G, pixel Y
, The reference level signals stored in the analog memories 1M, 1C, 1G, and 1Y and the analog memories 2M, 2M,
The pixel signals stored in C, 2G, and 2Y are compared.

First, while the operation pulse 138 (φOP) is being applied, first, the analog memory read pulse 1
24 (φAMR_2M) to read the signal of the pixel M stored in the analog memory 2M, and then read the analog memory read pulse 123 (φ
AMR_1M), the reference level signal of the pixel M stored in the analog memory 1M is read out to the arithmetic unit 32.

Then, by comparing the signal levels of the two, the digital memory 41 corresponding to the pixel M is selected by the chip select signal 143 (φCS_M) while the operation read pulse 139 (φPRD) is being applied. For example,
If the reference level signal is smaller than the read pixel signal, the digital data input signal 141 (D
ATA_IN) is written to the digital memory 41 corresponding to the pixel M.

When the above processing is completed, the operation pulse 13
8 (φOP) is once returned to the low level, and while the pulse is being applied again, the analog memory read pulse 126 (φAMR_2C) is applied to the pixel C stored in the analog memory 2C. Read the signal and then the analog memory read pulse 125
By applying (φAMR_1C), the reference level signal of the pixel C stored in the analog memory 1C is read out to the arithmetic unit 32.

Then, by comparing the signal levels of the two, the digital memory 41 corresponding to the pixel C is selected by the chip select signal 144 (φCS_C) while the operation read pulse 139 (φPRD) is being applied. For example,
If the reference signal is smaller than the read pixel signal, the digital data input signal 141 (DATA_
IN) is written into the digital memory 41 corresponding to the pixel C.

By performing the same processing as described above on the pixels G and Y, the comparison operation on the four pixels can be performed independently. By performing such processing sequentially on all the pixels sharing the vertical signal line in a similar manner, arithmetic processing can be performed on all the pixels on the image sensor.

By the way, in the above-described operation, only the process for the pixel signal which has been photoelectrically converted once can be performed. Therefore, it is necessary to repeat the above-described arithmetic operation.

If a reset operation is performed when reading a photoelectrically converted signal, the previously read signal is reset. Therefore, in the second and subsequent processes, only the reset pulse φRST (112) is not applied, and the other drive pulses may be applied in exactly the same manner.

In this manner, the signal which has been photoelectrically converted once is added to the signal level once read out after the photoelectric conversion, and the above processing is performed until the value reaches the reference level signal. By repeating, it is possible to obtain a result in which brightness is changed to time information.

As described above, the arithmetic function performed on the image sensor is 2
Although the above examples have been introduced, these are only some of the functions of the image sensor according to the present invention. According to the present invention,
Various calculations can be performed according to the application depending on the timing and combination of driving pulses supplied from the outside.

In the image pickup device according to the present invention, the parts for realizing such multi-function operation, that is, the analog storage area 20, the operation area 30, and the digital storage area 4
0, the output area 50 is provided separately from the pixel area 10 in which a large number of pixels 11 are arranged two-dimensionally (see FIG. 1). Therefore, it is possible to realize the arithmetic processing without reducing the number of pixels (resolution).

[Supplement] The present invention has been described in detail with reference to the specific embodiments. However, it is obvious that those skilled in the art can modify or substitute the embodiment without departing from the spirit of the present invention. That is, the present invention has been disclosed by way of example, and should not be construed as limiting. In order to determine the gist of the present invention, the claims described at the beginning should be considered.

[0146]

As described above in detail, according to the present invention,
CMOS (Complementary Metal-Oxide Semiconducto)
(r: Complementary metal oxide semiconductor)), it is possible to provide an excellent imaging device with an arithmetic function, which is configured to be small and lightweight by using a semiconductor manufacturing technology such as a semiconductor.

Further, according to the present invention, it is possible to provide an excellent image pickup device having an arithmetic function, which realizes a sufficient number of pixels and a small pixel size.

Further, according to the present invention, it is possible to provide an excellent image pickup device having an arithmetic function capable of taking a color image by using an on-chip color filter.

Although the image pickup device according to the present invention has an optical area composed of pixels having the same area as that of a normal image pickup image pickup device, by further providing a plurality of pixels with a common arithmetic processing circuit, When the brightness of the subject is low, or when the sampling interval for photoelectrically converting the brightness of the subject is short, a decrease in sensitivity can be avoided by simultaneously using the signals of a plurality of pixels to compensate for the brightness. As a result, the imaging device can correctly execute the calculation based on the brightness of the subject without being affected by noise.

On the other hand, when sufficient brightness can be obtained from the subject, it is possible to avoid a decrease in resolution by using each pixel independently for calculation.

Since the image sensor according to the present invention has a sufficiently large number of pixels, colorization can be easily achieved by employing an on-chip color filter. In addition, since the image sensor according to the present embodiment has a small pixel size, it is possible to increase the light condensing degree by using an on-chip lens,
Sensitivity can be increased.

Further, in the image pickup device according to the present invention, the operation content in the operation section can be freely changed in accordance with the combination of externally applied drive pulses and their timing. Therefore, various functions that cannot be performed by an imaging device having only a single arithmetic function can be realized on a single imaging device.

[0153]

[Brief description of the drawings]

FIG. 1 is a block diagram schematically showing a configuration of an image sensor according to an embodiment of the present invention.

FIG. 2 is a diagram showing a basic operation unit extracted from the imaging device structure shown in FIG. 1;

FIG. 3 is a diagram illustrating a configuration example of a unit pixel.

FIG. 4 is a diagram illustrating a configuration example of an analog storage unit;

FIG. 5 is an operation timing in a basic operation unit of a pixel;
It is a chart.

FIG. 6 is a principle diagram of converting brightness, which is an analog amount, into a digital amount in a unit pixel.

7 is a timing chart showing an operation of storing a pixel output signal corresponding to a reference level serving as a reference of brightness in an analog memory 22. FIG.

8 is a timing chart showing a series of operations for comparing a reference level signal stored in accordance with the operation timing shown in FIG. 7 with a photoelectrically converted signal for each pixel and storing the result in a digital memory; .

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 10 ... Pixel area 11 ... Pixel 12 ... Light receiving part 13 ... In-pixel amplifying part 14 ... Read-out part 20 ... Analog storage area 30 ... Calculation area 31 ... External storage part 40 ... Digital storage area 41 ... Digital memory 50 ... Output area 51 ... output unit

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) H04N 1/028 H04N 9/07 C 9/07 H01L 27/14 AD Term (Reference) 4M118 AA10 AB01 BA14 DD09 FA06 FA40 FA42 GC09 GD04 GD07 5B047 AB04 BB04 BC01 5C024 CX37 CX41 CY14 DX01 EX42 GX03 GY31 HX17 5C051 AA01 BA03 DA06 DB01 DB11 DB16 DB18 DC03 DE12 DE17 DE19 EA01 5C065 AA01 BB22 BB42 CC BB22 BB42 CC

Claims (20)

[Claims] A plurality of pixels each comprising a light receiving unit for converting the brightness of an object into an electric signal, an in-pixel amplifying unit for amplifying an output of the light receiving unit, and a reading unit for reading a signal from the in-pixel amplifying unit to outside the pixel. An extra-pixel amplifying unit that amplifies a signal read from each pixel; a plurality of storage units that store signals read from each pixel; and a signal read from one or more storage units An operation unit, a digital memory that stores the operation result of the operation unit for each pixel, an output unit that outputs a digital memory output to the outside, and a drive control unit that controls driving of the units. The drive control unit has a drive control mode in which the number of pixels from which signals are read and the number of storage units that store read signals are variable in accordance with the content of the calculation in the calculation unit. element. 2. The imaging device with an arithmetic function according to claim 1, wherein the arithmetic unit and / or the extra-pixel amplifying unit are shared among a plurality of pixels. 3. The imaging device with an arithmetic function according to claim 1, wherein the drive control unit determines the content of the arithmetic operation in the arithmetic unit based on a combination of drive pulses supplied to the arithmetic unit and its timing. . 4. The apparatus according to claim 1, wherein the drive control unit has a drive control mode in which two or more pixel readout signals detected at the same timing are stored in one storage unit. Imaging device. 5. The drive control section stores two or more pixel read signals detected at each timing in the same storage section, and based on the signals read from each storage section, stores the two or more pixel read signals. 2. The imaging device with an arithmetic function according to claim 1, further comprising a drive control mode that causes the arithmetic unit to perform a comparison process along a time axis of a pixel read signal from the pixel. 3. 6. The drive control section stores a read signal of each pixel detected at each timing in a separate storage section, and reads a pixel in the same pixel based on a signal read from each storage section. A first drive control mode for causing the arithmetic unit to perform a comparison process along the time axis of signals, and storing two or more pixel readout signals detected at each timing in the same storage unit, and A second drive control mode for causing the arithmetic unit to perform a comparison process along a time axis of pixel readout signals of the two or more pixels based on the readout signals. 2. The imaging device with an arithmetic function according to 1. 7. The drive control section stores a pixel readout signal corresponding to a reference signal level of a certain pixel in one storage section, and stores a pixel readout signal corresponding to the brightness of a subject of the pixel in another storage section. A drive control mode in which the arithmetic unit compares the reference signal level with the brightness of the subject based on the pixel read signal read from each storage unit and outputs the brightness of the subject with respect to the reference signal level. The imaging device with an arithmetic function according to claim 1, comprising: 8. Each pixel is provided with an on-chip color filter of each color, and the drive control section stores a pixel read signal corresponding to a reference signal level of a certain pixel in one storage section, The pixel readout signal corresponding to the brightness of the subject is stored in another storage unit, and the reference signal level and the brightness of the subject are compared in the arithmetic unit based on the pixel readout signal read out from each storage unit. The imaging device with an arithmetic function according to claim 1, further comprising a drive control mode for outputting a color signal at the pixel. 9. An on-chip lens is mounted on each pixel, and the drive control section stores a pixel read signal corresponding to a reference signal level of a certain pixel in one storage section, The pixel readout signal corresponding to the brightness is stored in another storage unit, and the reference signal level and the brightness of the subject are compared in the arithmetic unit based on the pixel readout signal read out from each storage unit. The imaging device with an arithmetic function according to claim 1, further comprising a drive control mode for outputting the brightness of the subject with respect to the level. 10. A plurality of pixels each comprising a light receiving unit for converting the brightness of a subject into an electric signal, an in-pixel amplifying unit for amplifying the output of the light receiving unit, and a reading unit for reading a signal from the in-pixel amplifying unit to outside the pixel , An analog storage area in which two or more storage units are arranged for each pixel, an extra-pixel amplification unit that amplifies a signal read from each pixel, and a readout from one or more storage units. An operation area in which an operation unit for operating the calculated signal is arranged; a digital storage area in which a plurality of digital memories for storing the operation results of the operation unit for each pixel are provided; An image pickup device with an arithmetic function, comprising: an output area in which a plurality of output units for outputting are arranged. 11. The imaging device with an arithmetic function according to claim 10, wherein a predetermined number of adjacent pixels are driven as a basic operation unit in said pixel area. 12. The apparatus according to claim 1, wherein the arithmetic unit and / or the extra-pixel amplifying unit are shared by a plurality of pixels.
0. The imaging device with an arithmetic function according to 0. 13. The imaging device with an arithmetic function according to claim 10, further comprising a drive control unit that determines the content of operation in said arithmetic unit based on a combination of drive pulses supplied to said arithmetic unit and its timing. element. 14. The drive control unit has a drive control mode in which the number of pixels from which a signal is read out and the number of storage units for storing the read out signal are made variable in accordance with the operation content of the operation unit. The imaging device with an arithmetic function according to claim 13. 15. The arithmetic function according to claim 13, wherein the drive control unit has a drive control mode for storing two or more pixel readout signals detected at the same timing in one storage unit. Imaging device. 16. The drive control unit stores two or more pixel read signals detected at each timing in the same storage unit, and based on the signals read from each storage unit, stores the two or more pixel read signals. The imaging device with an arithmetic function according to claim 13, further comprising a drive control mode for causing the arithmetic unit to perform a comparison process along a time axis of a pixel read signal from the pixel. 17. The drive control section stores a read signal of each pixel detected at each timing in a separate storage section, and reads a pixel in the same pixel based on a signal read from each storage section. A first drive control mode for causing the arithmetic unit to perform a comparison process along the time axis of signals, and storing two or more pixel readout signals detected at each timing in the same storage unit, and A second drive control mode for causing the arithmetic unit to perform a comparison process along a time axis of pixel readout signals of the two or more pixels based on the readout signals. An imaging device with an arithmetic function according to claim 13. 18. The drive control section stores a pixel readout signal corresponding to a reference signal level of a certain pixel in one storage section and stores a pixel readout signal corresponding to the brightness of a subject of the pixel in another storage section. A drive control mode in which the arithmetic unit compares the reference signal level with the brightness of the subject based on the pixel read signal read from each storage unit and outputs the brightness of the subject with respect to the reference signal level. 14. The method according to claim 13, wherein
An imaging device with an arithmetic function according to item 1. 19. An on-chip color filter for each color is mounted on each pixel, and the drive control section stores a pixel read signal corresponding to a reference signal level of a certain pixel in one storage section, The pixel readout signal corresponding to the brightness of the subject is stored in another storage unit, and the reference signal level and the brightness of the subject are compared in the arithmetic unit based on the pixel readout signal read out from each storage unit. The imaging device with an arithmetic function according to claim 13, further comprising a drive control mode for outputting a color signal at the pixel. 20. Each pixel is provided with an on-chip lens of each color, and the drive control unit stores a pixel read signal corresponding to a reference signal level of a certain pixel in one storage unit, and The pixel readout signal corresponding to the brightness of the subject is stored in another storage unit, and the reference signal level and the brightness of the subject are compared in the arithmetic unit based on the pixel readout signal read out from each storage unit. The imaging device with an arithmetic function according to claim 13, further comprising a drive control mode for outputting the brightness of the subject with respect to the reference signal level.