JP2005354568A - Physical information acquisition method, physical information acquisition device and semiconductor device for detecting physical value distribution - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To make it possible to output a normal image preparation and arithmetic processing in an appropriate format without causing incompatibility in an imaging apparatus having an arithmetic function. <P>SOLUTION: The imaging apparatus is provided with: a first vertical scanning circuit 14a and a first horizontal scanning circuit 12a which perform slow sweep processing to a unit pixel 3; a column processing part 26 which performs signal processing concerning the generation of normal image based on a pixel signal obtained by the slow sweep processing; a second vertical scanning circuit 14b and a second horizontal scanning circuit 12b which perform fast sweep processing to the same unit pixel 3; and an arithmetic processing part 27 which performs the arithmetic processing for executing a predetermined application based on a pixel signal obtained by the fast sweep processing. Since acquisition of unit signals required for performing each signal processing is independently performed, respectively, the normal image preparation and the arithmetic processing are outputted without causing incompatibility or in the appropriate format in the case of being provided with the arithmetic function such as, for example, a dynamic range extension function and an ID recognition function. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a physical information acquisition method, a physical information acquisition device, and a semiconductor device for physical quantity distribution detection in which a plurality of unit components are arranged. More specifically, for example, a plurality of unit components that are sensitive to electromagnetic waves input from the outside such as light and radiation are arranged, and the physical quantity distribution converted into an electric signal by the unit components is converted into an electric signal. The present invention relates to an arithmetic function technique for acquiring information for a predetermined purpose (various applications) based on a plurality of pieces of image information, which is suitable when using a semiconductor device with physical quantity distribution detection, such as a solid-state imaging device.

  Physical quantity distribution formed by arranging multiple unit components (for example, pixels) that are sensitive to changes in physical quantity such as electromagnetic waves or pressure (contact, etc.) input from outside such as light and radiation, in a line or matrix form. Sensing semiconductor devices are used in various fields.

  For example, in the field of video equipment, a CCD (Charge Coupled Device) type, a MOS (Metal Oxide Semiconductor) type, or a CMOS (Complementary Metal-oxide Semiconductor) type solid that detects changes in light (an example of an electromagnetic wave) that is an example of a physical quantity. An imaging device is used. In the field of computer equipment, fingerprint authentication devices that detect fingerprint images based on changes in electrical characteristics based on pressure and changes in optical characteristics are used. These read out, as an electrical signal, a physical quantity distribution converted into an electrical signal by a unit component (a pixel in a solid-state imaging device).

  For example, the solid-state imaging device reads signal charges (photoelectrons) accumulated in a photodiode that is a photoelectric conversion device as image information. The analog pixel signal read from the pixel unit is converted into digital data by an analog-digital converter (AD converter; Analog Digital Converter) as necessary.

  On the other hand, various arithmetic processes are performed on the pixel signals output from the pixels in order to generate high-quality images and use special applications.

  For example, in recent years, simple camera systems such as digital still cameras and camera-equipped mobile phones are rapidly spreading due to downsizing and cost reduction of solid-state imaging devices represented by CCD and CMOS image sensors. In particular, CMOS image sensors are attracting attention as a replacement for CCDs in the future because they can be manufactured with lower power consumption and lower cost than CCDs. Furthermore, since CMOS image sensors are made based on the normal MOS process, it is easy to mount logic circuits and analog signal processing circuits on the sensors. By using this feature, images are simply acquired. In addition, various attempts have been made to add new functions by processing received light signals inside the sensor.

  For example, in a video camera or an electronic still camera, in a scene that requires a wide dynamic range, such as shooting in a backlight where a person on the window is photographed from inside the room, the target person is crushed black when performing general exposure control. Appropriate exposure is performed on the scenery outside the window. On the other hand, if you perform backlight correction processing to prevent blackout of the subject in the room and correct the video signal of the target person so that it will not be blackened, the outdoor scenery that was the optimal exposure before backlight correction will be It will fly white. Alternatively, if the image is taken at a compromise between both indoor and outdoor exposure, the image is not satisfactory both indoors and outdoors.

  As a technique for solving the problem of shooting a scene that requires such a wide dynamic range, the same subject was acquired under different detection times (for example, signal charge accumulation time based on exposure time). A mechanism is conceived in which a long dynamic exposure suitable for indoor images and a short exposure suitable for outdoor exposure are continuously performed using a plurality of signals to acquire an image with a wide dynamic range (see, for example, Patent Document 1). ).

JP 2003-163831 A

  For example, by synthesizing an image with a synthesis circuit outside the device (off-chip) using two outputs 1 and 2 having different accumulation times, a signal output that is less likely to saturate with respect to a wider amount of incident light is obtained. The dynamic range can be expanded by preventing the saturation of the high-luminance portion while preventing the S / N (Signal (to) Noise ratio) of the luminance portion from decreasing. For example, an indoor image uses a portion with a good long-time exposure S / N, while a short-exposure image can be used in an outdoor area that is whitened by long-time exposure. Can be realized.

  On the other hand, on the premise of an image sensor having such a new function, new systems and applications that have not been realized so far have been proposed. For example, a data communication system, a data transmission device, and a data reception device that transfer data in real space are also known.

  For example, a data communication system that transfers data directly from an object without using a communication medium such as a wired or wireless network, such as information related to an object in the real world such as a device ID, a network address, a host name, and a URL, data A transmitting device and a data receiving device have been proposed (see Non-Patent Document 1, for example).

Nobuyuki Matsushita et al., “ID Cam: Image sensor that can acquire scene and ID simultaneously”, Interaction 2002; Paper Session 1 “Real World”

  The mechanism described in Non-Patent Document 1 is a mechanism that can transfer data robustly in distance from an object in front of an object to a relatively distant object, and is not connected through a communication medium, for example. A technique for recognizing a spatial position of an object while obtaining information such as an ID from an object in the real world has been proposed.

  FIG. 18 is a diagram showing an overview of the mechanism described in Non-Patent Document 1. In Non-Patent Document 1, as a camera system, a normal image is acquired, and a blinking pattern such as an LED light source encoded by some information is identified. If an object is set, an image of the object and ID information associated therewith can be acquired by detecting a blinking pattern and performing edge detection and sampling processing.

  Various applications are conceivable for such a system. For example, a means for establishing a communication connection between an object and a terminal equipped with a camera can be realized by transmitting a network ID as information. It can also be used for navigation systems that provide position information, advertising means, and the like.

  On the other hand, in order to realize such an application, it is impossible with a normal image sensor. A normal image sensor such as a CCD or CMOS image sensor acquires an image by frame scanning of about 30 fps (frame per second). Some special CMOS image sensors can achieve several hundreds of fps, but such a frame rate is the limit for obtaining a normal image.

  However, when the application described in Non-Patent Document 1 is realized, shortening the blinking interval of the LED leads to an increase in the amount of transmitted information. Therefore, a sensor that detects the information also requires high-speed frame scanning. . For example, when the LED light source is blinking at several hundred fps, the sensor side needs high-speed operation of about several kfps in order to sample it. Therefore, in order to realize the above-mentioned application, it is necessary that the image sensor to be used has two functions of acquiring a normal image and detecting a blinking timing of light blinking at high speed by high-speed frame scanning. There is. As a sensor that realizes such a function, for example, there is a mechanism described in Patent Document 2.

JP 2003-169251 A

  FIG. 19 is a plan view showing an overall outline of the solid-state imaging device described in Patent Document 2. The solid-state imaging device 901 includes a pixel unit (pixel array unit) 910, a normal image acquisition vertical scanning circuit (pixel V scanner unit) 914a, a horizontal scanning circuit (pixel H scanner unit) 912a, and current / voltage (IV). ) A conversion circuit unit 922, a CDS processing unit 926, a current mirror circuit unit 946, an analog memory array unit 947, a vertical scanning circuit (memory V scanner unit) 914c for vertically scanning the analog memory array unit 947, a comparator / latch unit 950, and A horizontal scanning circuit (memory H scanner unit) 912b that performs horizontal scanning for acquiring ID information is provided. Although not shown, for example, a bias circuit unit is provided in addition to these functional units.

  The solid-state imaging device 901 has two operation modes: a normal image capturing mode for acquiring a normal image and an ID detection mode for detecting blinking of an LED. In each mode, both the optical signals are received by the pixel portion 910. In the pixel unit 910, signal lines are wired in the vertical direction, and in the normal image capturing mode, signals are read out to the current / voltage conversion circuit unit 922 and the CDS processing unit 926 above the pixel unit 910, and the normal CMOS The same processing as that of the image sensor is performed and output to the outside of the sensor.

  On the other hand, in the ID detection mode, the signal is read out to the current mirror circuit unit 946 below the pixel unit 910. A comparator / latch unit 950 and the analog memory array unit 947 are arranged below the analog memory array unit 947. In the analog memory array unit 947, the signal of the previous frame is recorded, and in the signal reading operation from the pixel unit 910, the pixel signal is once stored in the analog memory array unit 947, and then the further lower comparator / latch. Part 950 compares the signal strength between the previous frame and the new frame.

  By this comparison scan, the rise or fall of pulse blinking is determined, and these data are read out outside the sensor as binary data. The read data is subjected to data code decoding by a processor outside the sensor, and ID information is extracted.

  However, according to such a solid-state imaging device 901, the normal image acquisition mode and the ID detection mode are completely separated, and these are switched every video frame scan (about 1/30 sec) at the fastest. This is because, in the normal image capturing mode, an accumulation time of about 30 fps is required to maintain sufficient light sensitivity, whereas in the ID detection mode, scanning of several kfps or more is required. This is because the operation cannot be performed in both modes.

  On the other hand, in the application of Non-Patent Document 1, it is possible to clearly separate the case where only images are continuously taken and the case where only ID recognition is continuously carried out, but in many cases, images and ID information are acquired simultaneously. Thus, it is assumed that the ID information is associated with the object in real time. In this case, when both the normal image and the ID image are output, the normal image capturing mode and the ID detection mode are alternately repeated. Therefore, the normal image and the ID image are combined every other frame. Are output at a low frame rate. When attention is paid only to a normal image, there is a problem that the frame rate deteriorates to half (or 1/3), and a sense of incongruity occurs when a fast-moving subject is imaged.

  Further, in the mechanism of Patent Document 2, since both modes are switched in about 1/30 sec to 1/60 sec, it can be considered that simultaneous acquisition is performed substantially in real time, but mode switching is performed. When attention is paid only to the normal image, there is a problem that the frame rate is deteriorated to half (or 1/3), and when a fast-moving subject is imaged, there is a sense of incongruity.

  From the viewpoint of the system developer, there is a demand for handling image signals in the same manner as a normal CMOS image sensor, and a format in which a normal image or an ID image is output every other frame is not preferred. .

  The present invention has been made in view of the above circumstances, and in having arithmetic functions such as a dynamic range expansion function and an ID recognition function, normal signal processing (for example, generation of a normal image) and arithmetic processing can be performed without a sense of incongruity. Or to provide a mechanism that can be executed in an appropriate format.

  In the physical information acquisition method according to the present invention, independent drive control is performed on the unit components constituting the semiconductor device, and based on the unit signals output from the unit components, independent signals are obtained. It was made to process.

  The physical information acquisition device according to the present invention is a device suitable for carrying out the physical information acquisition method according to the present invention, and is a first drive that performs first drive control on a unit component. A control unit, a first signal processing unit that performs signal processing based on a unit signal output from the unit component under the control of the first drive control unit, and a second drive control for the unit component And a second signal processing unit that performs signal processing based on the unit signal output from the unit component under the control of the second drive control unit.

  The semiconductor device according to the present invention is a device suitable for use in the physical information acquisition device according to the present invention.

  Further, the invention described in the dependent claims defines further advantageous specific examples of the physical information acquisition method, the physical information acquisition device, or the semiconductor device according to the present invention.

  For example, if each performs independent drive control, at a certain timing, both drive controls may compete (collision), that is, a situation may occur in which the same unit component is accessed at a certain point in time. Alternatively, when a simple read process is performed, physical information detected by the unit component by one drive control disappears, and the read process by the other drive control may be adversely affected.

  Therefore, for example, drive control is performed so as to avoid competition between the two sweep processes for the same unit component, or in one drive control, after the unit signal is output from the unit component, Drive control may be performed so that a unit signal in which a change in physical quantity is detected is held in a unit component.

  For example, in the case where the unit component has a floating node that holds a signal charge, in one drive control that performs high-speed sweep processing, reading is performed every predetermined charge accumulation period without initializing the floating node. In the other drive control in which low-speed sweep processing is performed, the floating node may be initialized and read out every predetermined charge accumulation period.

  According to the present invention, independent drive control is performed for each unit component constituting the semiconductor device, and thereby independent signal processing is performed based on each unit signal output from the unit component. I made it.

  Since unit signals required for performing each signal processing can be acquired independently, for example, in order to have a calculation function such as a dynamic range expansion function and an ID recognition function, normal signal processing (for example, normal Image generation) and calculation processing can be executed without any sense of incongruity or in an appropriate format.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the following, a case where a CMOS image sensor, which is an example of an XY address type solid-state imaging device, is used as a device will be described as an example. The CMOS image sensor will be described on the assumption that all pixels are made of NMOS or PMOS.

  However, this is merely an example, and the target device is not limited to a MOS imaging device. All the semiconductor device for physical quantity distribution detection in which a plurality of unit components that are sensitive to electromagnetic waves input from outside such as light and radiation are arranged in a line or matrix form, and all implementations described later. Forms are applicable as well.

<Configuration of solid-state imaging device>
FIG. 1 is a schematic configuration diagram of a CMOS solid-state imaging device (CMOS image sensor) which is an embodiment of a semiconductor device according to the present invention. This CMOS solid-state imaging device is also an aspect of an electronic apparatus according to the present invention.

  The solid-state imaging device 1 according to the present embodiment achieves high image quality by performing normal image generation processing and calculation processing separately based on pixel signals acquired by pixels (pixels) of an image sensor. In addition, a normal image processing system for performing normal image output processing and calculation processing for generating and outputting calculation information for achieving a predetermined purpose such as a difference image and an added image so as to enable an optimum design for calculation processing It is characterized in that the system is provided independently. Further, the present invention is characterized in that vertical scanning for controlling output processing from the pixel portion of the pixel signal used by each processing system can be performed independently. This will be specifically described below.

  In the solid-state imaging device 1, a plurality of pixels including a photoelectric conversion element (an example of a charge generation unit) such as a photodiode that outputs an electric signal corresponding to an incident light amount is arranged in rows and columns (that is, in a two-dimensional matrix form). ), And the signal output from each pixel is a voltage signal, and data processing such as a CDS (Correlated Double Sampling) processing function unit or a digital conversion unit (ADC) The portions are provided in parallel with each other.

  “The data processing unit is provided in parallel with the column” means that a plurality of CDS processing function units and digital conversion units are provided substantially in parallel with the vertical signal line 19 in the vertical column. . Each of the plurality of functional units is arranged only on one end side in the column direction with respect to the pixel unit 10 (output side arranged on the lower side of the drawing) when the device is viewed in plan view. Or one end side in the column direction (output side arranged on the lower side in the figure) and the other end side (the upper side in the figure) opposite to the pixel unit 10. The thing of the form distributed separately may be sufficient. In the latter case, it is preferable that the horizontal scanning unit that performs readout scanning (horizontal scanning) in the row direction is also arranged separately on each edge side so that each can operate independently.

  For example, as a typical example in which a CDS processing function unit and a digital conversion unit are provided in parallel in a column, a CDS processing function unit and a digital conversion unit are arranged in a vertical column (a column area provided on the output side of the imaging unit). A column type is provided for each column and sequentially read out to the output side. In addition to the column type, a configuration in which one CDS processing function unit or digital conversion unit is assigned to a plurality of adjacent (for example, two) vertical signal lines 19 (vertical columns), or every N (N is It is also possible to adopt a form in which one CDS processing function unit or digital conversion unit is assigned to N vertical signal lines 19 (vertical columns) of a positive integer (with N−1 lines in between).

  Except for the column type, in any form, since a plurality of vertical signal lines 19 (vertical columns) commonly use one CDS processing function unit and digital conversion unit, they are supplied from the pixel unit 10 side. A switching circuit (switch) that supplies pixel signals for a plurality of columns to one CDS processing function unit or digital conversion unit is provided. Depending on the subsequent processing, it is necessary to take measures such as providing a memory for holding the output signal.

  In any case, the signal processing of each pixel signal is read out in units of pixel columns by adopting a form in which one CDS processing function unit or digital conversion unit is assigned to a plurality of vertical signal lines 19 (vertical columns). By performing the processing later, the configuration in each unit pixel can be simplified and the number of pixels of the image sensor can be reduced, the size can be reduced, and the cost can be reduced as compared with the case where the same signal processing is performed in each unit pixel.

  In addition, since a plurality of signal processing units arranged in parallel in a column can simultaneously process pixel signals for one row, one CDS processing function unit or digital conversion unit is provided on the output circuit side or outside the device. Therefore, the signal processing unit can be operated at a low speed as compared with the case where processing is performed, which is advantageous in terms of power consumption, bandwidth performance, noise, and the like. In other words, when the power consumption and bandwidth performance are the same, the entire sensor can be operated at high speed.

  In the case of a column type configuration, it can be operated at a low speed, which is advantageous in terms of power consumption, bandwidth performance, noise, and the like, and has an advantage that a switching circuit (switch) is unnecessary. In the following embodiments, this column type will be described unless otherwise specified.

  As shown in FIG. 1, the solid-state imaging device 1 of the present embodiment includes a pixel unit (imaging unit) 10 in which a plurality of square unit pixels 3 are arranged in rows and columns (that is, in a square lattice pattern), and pixels. Drive control unit 7 provided outside the unit 10, a CDS processing unit (third difference information acquisition unit) provided above the pixel unit 10 in the figure, and a column processing unit (first signal) having a column switch A processing unit) 26 and an output circuit 28. The column processing unit 26 functions as a main part of a normal image processing system for performing signal processing related to normal image generation based on the pixel signal acquired by the pixel unit 10.

  In addition, an AGC (Auto Gain Control) circuit or an AD (Analog to Digital) conversion circuit having a signal amplification function is provided in the same semiconductor region as the column processing unit 26 before or after the column processing unit 26, if necessary. It is also possible to provide it. When AGC is performed before the column processing unit 26, analog amplification is performed. When AGC is performed after the column processing unit 26, digital amplification is performed. If the n-bit digital data is simply amplified, the gradation may be lost. Therefore, it is preferable to perform digital conversion after amplification by analog.

  In addition, the solid-state imaging device 1 of the present embodiment generates an arithmetic processing image such as a difference image and an addition image and other arithmetic information based on the pixel signal acquired by the pixel unit 10 and outputs the arithmetic processing unit (second Signal processing unit) 27 is provided on the lower side of the pixel unit 10 in the figure.

  The arithmetic processing unit 27 obtains difference information between frames (also referred to as high-speed frame difference information) at a high-speed frame rate (high-speed processing unit time) in the arithmetic processing (first difference information acquisition unit) 272. An analog memory array unit 274 that holds the high-speed inter-frame difference information J12 acquired by the difference calculation unit 272, and a bias circuit unit (Offset Generator) 275 that supplies a bias current to the current copier cell in the analog memory array unit 274 And. The analog memory array unit 274 is configured to temporarily store the output from the difference calculation unit 272 in the current copier cell.

  The arithmetic processing unit 27 also compares the difference between the high-speed inter-frame difference information J12a in the previous high-speed frame held in the analog memory array unit 274 and the high-speed inter-frame difference information J12b acquired by the differential arithmetic unit 272 in the current high-speed frame. A comparator unit 276 that is an example of a second difference information acquisition unit that detects (hereinafter also referred to as difference information J14), and a data latch unit 278 that holds digital data indicating the difference information J14 acquired by the comparator unit 276. I have.

  Further, as a component of the drive control unit 7, a control circuit function for sequentially reading out signals from the pixel unit 10 is provided. For example, the drive control unit 7 includes a horizontal scanning circuit (column scanning circuit) 12 that controls column addresses and column scanning, a vertical scanning circuit (row scanning circuit) 14 that controls row addresses and row scanning, and an internal clock. And a communication / timing control unit 20 having a function such as generation. The horizontal scanning circuit 12 has a function of a horizontal drive control unit (particularly a horizontal reading scanning unit) for reading pixel information from the column processing unit 26 and the arithmetic processing unit 27.

  Each element of these drive control units 7 is formed integrally with a pixel unit 10 in a semiconductor region such as single crystal silicon using a technique similar to a semiconductor integrated circuit manufacturing technique, and is a solid-state imaging which is an example of a semiconductor system It is configured as an element (imaging device).

  Here, as a configuration peculiar to the present embodiment, the vertical scanning circuit 14 includes a column processor as a component that supplies the pixel signal from the pixel unit 10 to the column processing unit 26 and the arithmetic processing unit 27 provided independently of each other. A first vertical scanning circuit (image acquisition vertical scanner) 14 a for the processing unit 26, a second vertical scanning circuit (ID detection vertical scanner) 14 b for the arithmetic processing unit 27, and a current copier cell of the analog memory array unit 274 Is scanned in the vertical scanning direction, and a third vertical scanning circuit (memory V scanner unit) 14c for taking out the data in the cell is provided.

  The horizontal scanning circuit 12 is a first horizontal scanning circuit (image acquisition horizontal scanner) that sequentially transfers pixel data output from the column processing unit 26 for generating and outputting normal images in the horizontal scanning direction and supplies the pixel data to the output circuit 28. 12a and a second horizontal scanning circuit (ID detection) for sequentially transferring the data output from the data latch unit 278, which is the output of the arithmetic processing unit 27 for output of calculation information generation, to the data output bus 279 by transferring it in the horizontal scanning direction. Horizontal scanner) 12b.

  In FIG. 1, some of the rows and columns are omitted for the sake of simplicity, but in reality, tens to thousands of unit pixels 3 are arranged in each row and each column. The unit pixel 3 is typically composed of a photodiode as a light receiving element (charge generation unit) and an in-pixel amplifier having an amplifying semiconductor element (for example, a transistor).

  As the intra-pixel amplifier, for example, a floating diffusion amplifier configuration is used. As an example, with respect to the charge generation unit, a read selection transistor that is an example of a charge readout unit (transfer gate unit / read gate unit), a reset transistor that is an example of a reset gate unit, a vertical selection transistor, and a floating diffusion As a CMOS sensor having a source follower-amplifying transistor, which is an example of a detection element for detecting a change in potential, a sensor composed of four general-purpose transistors can be used.

  In the solid-state imaging device 1 of the present embodiment, the pixel unit 10 is adapted for color imaging. That is, color separation composed of a combination of a plurality of color filters for capturing a color image on a light receiving surface on which electromagnetic waves (light in this example) of each charge generation unit (such as a photodiode) in the pixel unit 10 is incident. Any color filter of the filter is provided.

  The illustrated example uses a basic color filter of a so-called Bayer array, and unit pixels 3 arranged in a square lattice are three color colors of red (R), green (G), and blue (B). In order to correspond to the filter, the repeating unit of the color separation filter is arranged by 2 pixels × 2 pixels to constitute the pixel unit 10.

  For example, a first color pixel for sensing a first color (red; R) is arranged in an odd-numbered row and an odd-numbered column, and a second color (green; G; ) Is arranged, and the third color pixel for sensing the third color (blue; B) is arranged in the even-numbered row and the even-numbered column, and is different for each row. Further, two color pixels of R / G or G / B are arranged in a checkered pattern.

  In the color arrangement of the basic color filter in such a Bayer arrangement, two colors of R / G or G / B are repeated every two in both the row direction and the column direction.

  In FIG. 1, three color filters of red (R), green (G), and blue (B) are arranged according to the basic form of the Bayer arrangement for the unit pixels 3 arranged in a square lattice pattern. However, the filter colors and the order of arrangement are not limited to the example shown in FIG. For example, a modified Bayer arrangement can be used, and four complementary color filters of Y (yellow), G (green), Cy (cyan), and M (magenta) or other filter colors can be used.

  For example, instead of the second color pixel for sensing the second color (green; G) arranged in even rows and even columns, the fourth color for sensing the fourth color (emerald; E). Pixels may be arranged.

  Although detailed description of the color signal processing is omitted, when a fourth color pixel is provided, RGB signals close to the human eye are obtained from the video signals of the respective colors photographed in four colors corresponding to the four-color filter. An image processor for performing a matrix operation for generating the three colors is provided at the subsequent stage of the output circuit 28. If an emerald (E) filter is mounted in addition to the red (R), green (G), and blue (B) filters, the difference in color reproduction can be reduced compared to a three-color filter, for example, blue-green And red reproducibility can be improved.

  The unit pixel 3 has a vertical scanning circuit 14 via a row control line 15 for row selection, and a column processing unit 26 for normal image generation output and arithmetic processing for calculation information generation output via a vertical signal line 19. The unit 27 is connected to each. Here, the row control line 15 indicates the entire wiring that enters the pixel from the vertical scanning circuit 14.

  The vertical scanning circuit 14 (14a, 14b, 14c) and the horizontal scanning circuit 12 (12a, 12b) are configured to include a decoder, and control signals CN1 (CN1a, CN1b, CN1c) supplied from the communication / timing control unit 20, In response to CN2 (CN2a, CN2b), reading of the pixel signal to be processed is started. For this reason, the row control line 15 includes various pulse signals (for example, a reset pulse RST, a transfer control pulse TX, a DRN control pulse DRN, a vertical selection pulse SEL, etc.) for driving the unit pixel 3.

  The vertical scanning circuit 14 (14a, 14b) and the communication / timing control unit 20 specify the positions of the plurality of unit pixels 3 to be processed, and perform column processing on the pixel signals from the unit pixels 3 respectively. A unit signal selection control unit (vertical drive control unit) to be input to the unit 26 and the arithmetic processing unit 27 (particularly, the difference calculation unit 272) is configured.

  Although not shown, the communication / timing control unit 20 is a master via a functional block of a timing generator TG (an example of a read address control device) that supplies a clock signal necessary for the operation of each unit and a pulse signal of a predetermined timing, and a terminal 5a. A communication interface functional block that receives the clock CLK0, receives data DATA for instructing an operation mode or the like via the terminal 5b, and outputs data including information of the solid-state imaging device 1;

  For example, the horizontal address signal is output to the horizontal decoder and the vertical address signal is output to the vertical decoder, and each decoder receives it and selects the corresponding row or column.

  At this time, since the unit pixels 3 are arranged in a two-dimensional matrix, analog pixel signals generated by the pixel signal generation unit 5 and output in the column direction via the vertical signal lines 19 are arranged in a row unit (column parallel). (In) Scan (access) to read (vertical) scan, and then access the row direction, which is the arrangement direction of vertical columns, and read out pixel signals (in this example, digitized pixel data) to the output side (horizontal) scan By performing reading, it is preferable to speed up reading of pixel signals and pixel data. Of course, not only scanning reading but also random access for reading out only the information of the necessary unit pixel 3 is possible by directly addressing the unit pixel 3 to be read out.

  Further, in the communication / timing control unit 20 of the present embodiment, the clock CLK1 having the same frequency as the master clock (master clock) CLK0 input via the terminal 5a, a clock obtained by dividing the clock CLK1, or a low speed obtained by further dividing the clock. Is supplied to each part in the device, for example, the horizontal scanning circuit 12, the vertical scanning circuit 14, the column processing unit 26, or the arithmetic processing unit 27. Hereinafter, the clocks divided by two and the clocks having a frequency lower than that are collectively referred to as a low-speed clock CLK2.

  The vertical scanning circuit 14 selects a row of the pixel unit 10 and supplies a necessary pulse to the row. For example, each of the first vertical scanning circuit and the second vertical scanning circuit includes a vertical decoder that defines a vertical readout row (selects a row of the pixel unit 10), and a readout address defined by the vertical decoder ( And a vertical drive circuit that drives the row control line 15 for the unit pixels 3 in the row direction by supplying pulses. Note that the vertical decoder selects a row for electronic shutter in addition to a row from which a signal is read.

  The horizontal scanning circuit 12 sequentially selects the functional units of the column processing unit 26 in synchronization with the low-speed clock CLK2, and guides the signals to the horizontal signal line (horizontal output line) 18 and the data output bus 279. For example, each of the first horizontal scanning circuit 12a and the second horizontal scanning circuit 12ba defines a horizontal readout column (for example, selects an individual CDS processing unit in the column processing unit 26, or the data latch unit 278). The horizontal decoder and a read address defined by the horizontal decoder lead each signal of the column processing unit 26 to the horizontal signal line 18 or each pixel data of the data latch unit 278 to the data output bus And a horizontal drive circuit leading to H.279. If the horizontal signal line 18 and the data output bus 279 are, for example, 10 (= n) bits, which corresponds to the number of bits n (n is a positive integer) handled by the column processing unit 26 or the data latch unit 278, the number of bits. Ten lines are arranged corresponding to the minutes.

  The first vertical scanning circuit 14a and the first horizontal scanning circuit 12a are functional units related to the first drive control unit, and the second vertical scanning circuit 14b and the second horizontal scanning circuit 12b are second drive control. It is a functional part related to the department.

  In the solid-state imaging device 1 having such a configuration, the pixel signal output from the unit pixel 3 is supplied to the CDS processing unit of the column processing unit 26 via the vertical signal line 19 for each vertical column.

  On the signal path between the column processing unit 26 and the horizontal scanning circuit 12, a load transistor unit including a load MOS transistor (not shown) having a drain terminal connected to each vertical signal line 19 is arranged. A load control unit (load MOS controller) for driving and controlling the load MOS transistor is provided (see FIG. 3 described later).

  In the normal image generation output system, the pixel signal from the pixel unit 10 is transmitted to the column processing unit 26 arranged in the upward direction of the pixel unit 10 in the figure. At this time, in the pixel unit 10, all the pixels in the same horizontal direction are simultaneously selected by the first vertical scanning circuit 14a, and pixel signals from the respective vertical columns are simultaneously output in parallel, that is, a column parallel operation is performed. .

  In the CDS processing function unit of the column processing unit 26, the signal level (noise level) immediately after pixel reset and true (according to the amount of received light) with respect to the voltage mode pixel signal input through the vertical signal line 19. CDS processing is performed to perform processing for obtaining a difference from the signal level Vsig. Thereby, it is possible to remove a noise signal component called fixed pattern noise (FPN) or reset noise.

  The pixel signal subjected to the CDS processing in the column processing unit 26 is transmitted to the horizontal signal line 18 via a horizontal selection switch (column switch S1) driven by a horizontal selection signal from the first horizontal scanning circuit 12a. Further, it is input to the output circuit 28. In addition, the processing procedure at the time of normal image output as described above is basically conventionally known (see, for example, ISSCC / 2000 / SESSION6 / CMOS IMAGE SENSORS WITH EMBEDDED PROCESSORS / 6.1 (2000 IEEE International Solid-State Circults Conference)). Therefore, detailed description is omitted.

  On the other hand, in the calculation processing system, the pixel signal from the pixel unit 10 is transmitted to the difference calculation unit 272 of the calculation processing unit 27 arranged in the downward direction of the pixel unit 10 in the figure. At this time, in the pixel unit 10, all the pixels in the same horizontal direction are simultaneously selected by the second vertical scanning circuit 14b, and the pixel signals from the respective vertical columns are simultaneously output in parallel, that is, the column parallel operation is performed. .

  The signal transmitted to the difference calculation unit 272 is subjected to difference processing between each frame of the high-speed frame rate, and high-speed inter-frame difference information J12 obtained by this difference processing is held in the analog memory array unit 274 for each high-speed frame. Thereafter, the comparator section 276 compares the data contents of the high-speed inter-frame difference information J12. The comparison result is converted into digital data by the data latch section 278 and is latched, and then the data is output from the data output bus 279. Is output.

  With such a configuration, from the pixel unit 10 in which the light receiving elements as charge generation units are arranged in a matrix, pixel signals are sequentially sent to the column processing unit 26 and the arithmetic processing unit 27 for each vertical column for each row. The signals are output at a normal frame rate driven by the first vertical scanning circuit 14a and a high-speed frame rate driven by the second vertical scanning circuit 14b.

  An imaging signal output from the output circuit 28 indicating an image for one sheet corresponding to the pixel unit 10 in which light receiving elements (photoelectric conversion elements such as photodiodes) are arranged in a matrix, that is, a normal frame image, is a pixel. This is indicated by a set of pixel signals of the entire unit 10.

  On the other hand, calculation information at a high frame rate that is output from the data output bus 279 and driven by the second vertical scanning circuit 14b is difficult to realize in various IT (Information Technology) devices. It is used to realize various application functions such as image processing and image recognition. As an arithmetic function for executing various applications, for example, a three-dimensional measurement camera that performs distance measurement by acquiring distance information of each point of a subject by the principle of triangulation, an ID (Identification) of a real world object, Light from a light source (beacon) with a blinking pattern representing transmitted data, including network address, host name, URL (Uniform Resource Locator), data / content, program code and other object related information There is a function such as an ID camera that forms an ID recognition system for identifying (collectively referred to as an optical ID).

  Here, a pixel unit 10 that is a main part of an image sensor as an example of a semiconductor device, and a predetermined signal based on a pixel signal output from the drive control unit 7 that controls driving of the pixel unit 10 or the pixel unit 10 A physical information acquisition device (in a narrow sense) having a column processing unit 26 and an arithmetic processing unit 27 for signal processing is arranged on one circuit board or formed on one semiconductor substrate The solid-state imaging device 1 is an example of a physical information acquisition device (in a broad sense). However, this is an example, and various modifications can be employed. For example, the pixel unit 10 and other functional elements may be provided individually. In this case, the drive control unit 7, the column processing unit 26, and the arithmetic processing unit 27 constitute a physical information acquisition device.

  Here, in the solid-state imaging device 1 of the present embodiment, the normal image processing system for generating the normal image and the arithmetic processing system for generating and outputting the calculation information are completely separated by separate circuit blocks, respectively. Since the vertical scanning functional unit for reading out the pixel signal from the pixel unit 10 is also provided independently, the configuration in each pixel is simplified, and the entire apparatus is downsized and the actual image quality is improved. In addition, it is possible to design optimally for arithmetic processing and to scan each function independently. Hereinafter, a specific operation of the solid-state imaging device 1 of the present embodiment will be described in detail using an application example.

<Specific Configuration; First Embodiment>
FIG. 2 is a diagram illustrating an example of a relationship between an effective image region (effective portion) in the pixel unit (imaging unit) 10 and a reference pixel region that gives optical black. As shown in FIG. 2, in the pixel unit 10, in addition to an effective image region (effective unit) 11b that is an effective region for capturing an image, a reference pixel region 11c that gives optical black is an effective image region 11b. It is arranged around.

  As an example, reference pixels for providing optical black for several rows (for example, 1 to 10 rows) are arranged above and below in the vertical column direction, and there are several numbers on the left and right in the horizontal direction including the effective image area (effective portion) 11b. Reference pixels that provide optical black for pixels to several tens of pixels (for example, 3 to 40 pixels) are arranged.

  The reference pixel for providing optical black is shielded on the light receiving surface side so that light does not enter a charge generation unit made of a photodiode or the like. The pixel signal from this reference pixel is used for the black reference of the video signal.

  Specifically, in an effective pixel area of 650 (H; pixel) × 488 (V; line), reference pixels for providing optical black for four lines are arranged above and below in the vertical column direction, respectively, In the horizontal direction including the effective image area (effective portion) 11b, reference pixels for providing optical black of 30 pixels on the left side and 5 pixels on the right side are arranged. The total pixel area including the reference pixel area 11c for setting the reference black level is 685 (H) × 496 (V).

  FIG. 3 is a diagram illustrating a first embodiment of a specific configuration example (internal configuration) of the column processing unit 26 and the arithmetic processing unit 27 focusing on one vertical column in the solid-state imaging device 1 illustrated in FIG. 1. In the first embodiment, the function of the ID camera is realized. For example, a column processing unit 26 is arranged as a circuit for realizing the image acquisition function on the upper side in the figure, and an arithmetic processing unit 27 is arranged as a circuit for realizing the ID detection function on the lower side in the figure.

  The configuration of the unit pixel (pixel cell) 3 in the pixel unit 10 is the same as that of a normal CMOS image sensor. In this embodiment, a general-purpose 4TR configuration is used as the CMOS sensor. Of course, this pixel configuration is an example, and a structure capable of non-destructive reading, that is, a pixel that can hold a pixel signal in the floating diffusion FD or the like without necessarily resetting the charge storage unit such as the floating diffusion FD after reading. As long as the array configuration is a normal CMOS image sensor, any of them can be used.

  The unit pixel 3 having the 4TR configuration shown in FIG. 3 includes a charge generation unit 32 having both a photoelectric conversion function for receiving light and converting it into a charge, and a charge storage function for storing the charge, and a charge generation unit 32. On the other hand, a read selection transistor (transfer transistor) 34 which is an example of a charge readout unit (transfer gate unit / read gate unit), a reset transistor 36 which is an example of a reset gate unit, a vertical selection transistor 40, and a floating diffusion It has an amplifying transistor 42 having a source follower configuration, which is an example of a detecting element that detects a potential change of 38.

  The unit pixel 3 includes a pixel signal generation unit 5 having an FDA (Floating Diffusion Amp) configuration including a floating diffusion 38 which is an example of a charge injection unit having a function of a charge storage unit. The floating diffusion 38 is a diffusion layer having parasitic capacitance.

  The read selection transistor (second transfer unit) 34 is driven by a transfer drive buffer (not shown) via a transfer wiring (read selection line TX) 55. The reset transistor 36 is driven by a reset driving buffer (not shown) via a reset wiring (RST) 56. The vertical selection transistor 40 is driven by a selection drive buffer (not shown) via a vertical selection line (SEL) 52. Each drive buffer can be independently driven by the first vertical scanning circuit 14a or the second vertical scanning circuit 14b.

  The reset transistor 36 in the pixel signal generation unit 5 has a source connected to the floating diffusion 38 and a drain connected to the power supply VDD, and a reset pulse RST is input to the gate (reset gate RG) from the reset drive buffer. The reset transistor 36 has a function of resetting the potential of the floating diffusion 38.

  For example, in the vertical selection transistor 40, the drain is connected to the source of the amplification transistor 42, the source is connected to the pixel line 51, and the gate (in particular, the vertical selection gate SELV) is connected to the vertical selection line 52. The vertical selection transistor 40 is not limited to such a connection configuration, and the drain is connected to the power supply VDD, the source is connected to the drain of the amplification transistor 42, and the gate is connected to the vertical selection line 52. Good.

  A vertical selection signal SEL is applied to the vertical selection line 52. The amplification transistor 42 has a gate connected to the floating diffusion 38, a drain connected to the power supply VDD, a source connected to the pixel line 51 via the drain of the vertical selection transistor 40, and further connected to the vertical signal line 19. It has become.

  In such a configuration, since the floating diffusion 38 is connected to the gate of the amplifying transistor 42, the amplifying transistor 42 outputs a signal corresponding to the potential of the floating diffusion 38 (hereinafter referred to as FD potential) in the voltage mode, and the pixel line. The signal is output to the vertical signal line 19 via 51.

  The reset transistor 36 resets the floating diffusion 38. The read selection transistor (transfer transistor) 34 transfers the signal charge generated by the charge generator 32 to the floating diffusion 38. A large number of pixels are connected to the vertical signal line 19. To select a pixel, the vertical selection transistor 40 is turned on only for the selected pixel. Then, only the selected pixel is connected to the vertical signal line 19, and the signal of the selected pixel is output to the vertical signal line 19.

  Here, in the first embodiment, the first vertical scanning circuit 14a for controlling the normal image generation and the second vertical scanning circuit 14b for controlling the calculation information generation are vertical at independent frame rates, that is, at different speeds. By performing the sweep, each unit pixel 3 of the pixel unit 10 is controlled, and each output signal is output to a different signal line, that is, the pixel signal is read to the column processing unit 26 side or the arithmetic processing unit 27 side, respectively. . In particular, the vertical scanning by the second vertical scanning circuit 14b has a frame rate that is faster (for example, one digit or more) than the vertical scanning by the first vertical scanning circuit 14a.

  For this reason, free reading processing can be performed for each row, and for example, the accumulation time can be set freely. For example, the arithmetic processing unit 27 can freely perform processing for acquiring arithmetic information for detecting a beacon that is blinking at high speed. Since the degree is greatly expanded, it is easy to use when realizing a predetermined application. Further, such a setting is not performed by one vertical scanning circuit 14, but is controlled by providing dedicated vertical scanning circuits 14a and 14b, respectively, so that the control becomes easy.

  However, since each unit pixel 3 is controlled at an independent frame rate and the pixel signal is read out, at a certain timing, both vertical scans compete (collision), that is, at the same time, the same unit pixel 3 is accessed simultaneously. When a simple readout process is performed, the signal charge detected by the unit pixel 3 is extinguished by one (in this example, the high-speed frame rate scanning side) and the other (in this example, the low-speed frame rate scanning side) ) May be adversely affected. In the present embodiment, the first vertical scanning circuit 14a and the second vertical scanning circuit 14b operate so as to eliminate these problems. This point will be described in detail later.

  Note that the row selection scanning of the low-speed frame scanning function by the first vertical scanning circuit 14a may be synchronized with the row selection scanning of the high-speed frame scanning function by the second vertical scanning circuit 14b. In this case, it is possible to avoid contention by adjusting the synchronization phase. However, since a PLL (phase lock loop) circuit or the like is required, for example, the circuit scale increases. In this respect, the mechanism in which the first vertical scanning circuit 14a and the second vertical scanning circuit 14b operate so that each output signal shares one vertical signal line 19 in a time division manner is highly effective.

  A column processing unit 26 connected to the vertical signal line 19 is provided above the unit pixel 3 in the drawing. As an example, the column processing unit 26 has a CDS processing unit 26a having a sample and hold function and a function of transferring the potential sampled and held by the CDS processing unit 26a to the horizontal signal line 18, and selects the CDS processing unit 26a. A column switch (horizontal selection switch) S1 is provided. A load transistor unit 29 including a load MOS transistor 290 is provided on the lower side of the unit pixel 3 in the drawing.

  The amplifying transistor 42 constituting the unit pixel 3 is connected to each vertical signal line 19. The vertical signal line 19 is connected to the drain of the load MOS transistor 290 for each vertical column. A load control signal VL1 from a load control unit (load MOS controller) (not shown) is commonly input to the gate terminal to be driven, and a load connected to each amplification transistor 42 at the time of signal reading. The MOS transistor 290 continues to pass a predetermined constant current.

  For example, in each of the high-speed frame rate scanning function and the low-speed frame rate scanning function, the response speed can be adjusted by changing the bias of the load MOS transistor 290 serving as a load when reading the potential to the vertical signal line 19.

  The CDS processor 26a is controlled by control pulses SH and CLP1 from the first horizontal scanning circuit 12a, and the column switch S1 is also controlled by control pulses SELH from the first horizontal scanning circuit 12a.

  The output side of the column switch S1 is connected to the horizontal signal line 18, and the pixel signal from the CDS processing unit 26a selected when the column switch S1 is turned on is supplied to the output circuit 28 (not shown). ing.

  On the other hand, the arithmetic processing unit 27 arranged on the lower side in the drawing of the unit pixel 3 is a functional element that realizes the function of the ID camera which is an example of the arithmetic processing function, that is, the potential change of the vertical signal line 19. As a circuit for processing as an ID detection signal, a difference calculation unit 272, a set of current copier cells 320 and 340 (two in this example) forming the analog memory array unit 274, and an output signal of the analog memory array unit 274 Are provided with a bias circuit section (bias processing section) 275 for supplying a bias current as an offset component, a comparator section 276, and a data latch section 278.

  On the lower vertical signal line 19 of the pixel unit 10 in the drawing, that is, on the arithmetic processing unit 27 side, a switch S2 for opening and closing both functional units is provided between the difference calculation unit 272 and the analog memory array unit 274. A switch S3 that opens and closes both functional units is provided between the analog memory array unit 274 and the bias circuit unit 275 and the comparator unit 276, and a switch that opens and closes between the data latch unit 278 and the data output bus 279. S4 is provided.

  The difference calculation unit 272 includes a coupling capacitor 302 provided on the vertical signal line 19, an Nch MOS transistor 304 that functions as a clamp switch, and an Nch MOS drive transistor 306 that has a voltage / current conversion function for converting a voltage signal into a current signal. have.

  The gate of the drive transistor 306 is connected to the coupling capacitor 302 on the vertical signal line 19, the source is grounded, the drain is connected to the switch S 2, and functions as an output terminal of the difference calculation unit 272. It has become.

  The transistor 304 has a drain connected to the reference power supply Vb and a source connected to a node N1 that is a connection point between the coupling capacitor 302 on the vertical signal line 19 and the gate of the drive transistor 306. The clamp pulse CLP2 from the second vertical scanning circuit 14b is input to the gate of the transistor 304. When the clamp pulse CLP2 is at the H (high) level, the transistor 304 is turned on. The pixel signal on the vertical signal line 19 is clamped to the reference power supply Vb.

  The analog memory array unit 274 is characterized in that current copier cells (current storage cells) 320 and 340 are used as current sampling units. On the output side of each of the current copier cells 320 and 340, a switch S3 that opens and closes the analog memory array unit 274, the bias circuit unit 275, and the comparator unit 276 is provided.

  The analog memory array unit 274 has a function of receiving the current signal output from the difference calculation unit 272, storing a current signal having a magnitude corresponding to the magnitude of the received current signal, and outputting the current signal at a predetermined timing.

  For example, as illustrated in FIG. 3, one current copier cell 320 includes a transistor 322 that functions as a switch that controls current input and current output to the current copier cell 320, and a drain as an input / output terminal of the transistor 322. A PchMOS transistor 324 connected to the drain and having a source connected to the power supply line VDD, a sampling capacitor 326 connected between the gate of the PchMOS transistor 324 and the power supply line VDD, and a gate of the PchMOS transistor 324 An NchMOS transistor 328 that functions as a switch element connected between the drains is formed.

  That is, first, the output of the difference calculation unit 272, that is, the drain terminal of the drive transistor 306 is connected to the drain terminal of the Pch MOS transistor 324. A sampling capacitive element 326 is connected to the power supply voltage VDD at the gate of the Pch MOS transistor 324, and a transistor 328 as a switching element is inserted between the gate and the drain to constitute a current copier 90. .

  The vertical signal line 19 is connected to the end of the node connecting the drain terminals of the Pch MOS transistor 324 and the transistor 328 via the transistor 322 functioning as a switch element.

  The selection control pulse SM1 is input to the gate of the transistor 322. When the selection control pulse SM1 is at the H (high) level, the transistor 322 is turned on. A write control pulse ME1 is input to the gate of the transistor 328. When the write control pulse ME1 is at the H (high) level, the transistor 328 is turned on.

  With such a configuration, when the selection control pulse SM1 is set to H level to turn on the transistor 322 (on) and the write control pulse ME1 is set to H level to turn on the transistor 328 (on), the current copier is turned on. The cell 320 is in an input phase, that is, a mode for writing a current signal on the vertical signal line 19 on the output side of the difference calculation unit 272.

  On the other hand, when the selection control pulse SM1 is set to H level to turn on the transistor 322 (ON) and the write control pulse ME1 is set to L level to turn off the transistor 328 (OFF), the current copier cell 320 is In the output phase, that is, a mode in which the current signal held in the current copier cell 320 is read out to the subsequent circuit on the vertical signal line 19 (the bias circuit unit 275 and the comparator unit 276).

  The other current copier cell 340 has a transistor 342 functioning as a switch for controlling current input and output to the current copier cell 340, a drain as an input / output terminal connected to the drain of the transistor 342, and a source A PchMOS transistor 344 connected to the power supply line VDD, a sampling capacitor 346 connected between the gate of the PchMOS transistor 344 and the power supply line VDD, and a switch connected between the gate and drain of the PchMOS transistor 344 It is composed of an Nch MOS transistor 348 that functions as an element. That is, the functional member reference number 32 @ in the current copier cell 320 may be replaced with the reference number 34 @.

  The selection control pulse SM2 is input to the gate of the transistor 342. When the selection control pulse SM2 is at the H (high) level, the transistor 342 is turned on. A write control pulse ME2 is input to the gate of the transistor 348. When the write control pulse ME2 is at the H (high) level, the transistor 348 is turned on. Since the operation of the current copier cell 340 is the same as that of the current copier cell 320 described above, a detailed description thereof will be omitted.

  In the example of FIG. 3, since the unit pixel 3 includes an Nch MOS transistor as the amplifying transistor 423, a Pch MOS is used as the transistor 324 of the current copier cell 320. However, in the case where a PchMOS transistor is provided as the amplifying transistor 42, the current copier cell 320 may be formed by inverting the polarity of Nch and Pch of the transistor.

  The bias circuit unit 275 supplies a bias current to the vertical signal line 19 by a bias control signal for signal weighting. Specifically, the bias circuit unit 275 arbitrarily weights the signal from the analog memory array unit 274. It is possible to do.

  For example, as the bias circuit unit 275 of this embodiment, a driver configuration in which a Pch MOS transistor 362 and an Nch MOS transistor 364 are cascade-connected can be used. An active L (low) bias control signal PBIS is supplied to the gate of the transistor 362, and an active H (high) bias control signal NBIS is supplied to the gate of the transistor 364.

  Of course, the present invention is not limited to such a configuration, and for example, a source follower circuit of a Pch MOS transistor biased by a current mirror common to the columns can be used.

  Note that the bias circuit unit 275 sets the potential of the vertical signal line 19 in the previous stage of the comparator unit 276 to the power supply voltage by turning off the switch S3 provided between the bias circuit unit 275 and the analog memory array unit 274 except for the read operation. It also has a bias function.

  The comparator unit 276 is configured to have a chopper type comparator function. For example, the comparator unit 276 includes a capacitor 381, an amplifier 382, and a switch 383 in order to clamp a predetermined signal level in the pixel signal on the vertical signal line 19 after the switch S3 in the comparator unit 276. And a second clamp circuit composed of a capacitor 384, an amplifier 385, and a switch 386, has a two-stage clamp circuit (double clamp circuit) configuration. Clamp pulses CP1 and CP2 are input to the switches 383 and 386 of the comparator unit 276 from the second vertical scanning circuit 14b. The output of the second clamp circuit is input to the data latch unit 278 having a data holding function.

  In such a configuration, the comparator unit 276 performs an initialization operation and a comparison operation on the two clamp circuits by the clamp pulses CP1 and CP2. For example, the comparator unit 276 is initialized when both of the clamp pulses CP1 and CP2 are at the H (high) level, and thereafter, the clamp pulse CP1 is set to the L (low) level while the clamp pulse CP2 is maintained at the H level. Then, the pixel signal (first pixel signal) on the vertical signal line 19 after the switch S3 is clamped, and then the clamp pulse CP2 is set to the L level to set the pixel signal (first pixel signal on the vertical signal line 19 after the switch S3). 2 pixel signal), the magnitudes of the first pixel signal and the second pixel signal are compared.

  That is, the comparator unit 276 is initialized at a predetermined timing in the 1H period at the high-speed frame rate, and then the first pixel signal is read out and sent to the comparator unit 276 to perform the first clamping process. After that, the second pixel signal is read out and sent to the comparator unit 276 to perform the second clamping process, thereby detecting the level at which the digital data is inverted, thereby detecting the image information (beacon in this example). ID information represented by blinking information) is extracted.

  The data latch unit 278 includes a latch 278a. The output data of the latch 278a is transmitted to the data output bus 279 via the switch S4 that opens and closes the data latch unit 278 and the data output bus 279.

  The data latch unit 278 captures and holds “0; L level” or “1; H level” indicating the comparison result in the comparator unit 276 in the latch 278a in synchronization with the capture pulse CKD. Thereafter, the switch S4 controlled by the readout pulse from the second vertical scanning circuit 14b is turned on, so that the comparison result is output as 1-bit digital data on the data output bus 279. In other words, the comparator unit 276 and the data latch unit 278 constitute an AD conversion function unit that acquires calculation information as digital data.

  Further, in the subsequent stage of the column processing unit 26 and the arithmetic processing unit 27, a usage signal for obtaining a usage signal that achieves a predetermined purpose based on the outputs obtained by the column processing unit 26 and the arithmetic processing unit 27, respectively. An acquisition unit (third signal processing unit) 100 is provided.

  Here, as the use signal acquisition unit 100 of the first embodiment, a functional unit that recognizes a blinking pattern of beacons on hardware and transmits it to a personal computer or the like via a predetermined communication interface in order to realize an ID camera. Provided.

  The beacon serving as the blinking light source is, for example, 8 bits as ID information represented by the blinking information, and allows 255 ways of identification. The 8-bit ID information is Manchester encoded with a 4 kHz carrier, and the ID information is transmitted to a personal computer or the like as a 22-bit packet. As a result, even if the beacon is hidden during transmission of a packet due to an obstacle or the like, data can be received in units of packets.

  Although not shown, when realizing the solid-state imaging device 1 having such an ID camera function, in addition to the optical lens, the image sensor, and the drive control unit 7, an optical lens and an integrated circuit for decoding (for example, FPGA (Field Programmable Gate Array) and communication IF unit When executing the ID camera function, the ID information acquired on the arithmetic processing unit 27 side is decoded, and the normal image (obtained on the column processing unit 26 side) (Scene image) and decoded ID information are paired and output via a predetermined communication interface.

  Alternatively, when decoding the blinking data, the arithmetic processing unit 27 repeats sampling at 12 kHz 200 times, decodes 8-bit ID information with a carrier frequency of 4 kHz transmitted by the beacon in all pixels of the pixel unit 10, and 15 fps. To create an ID image. The ID image is ID information obtained by decoding the value of each pixel of the image. By using the normal image and the ID image as a set, the position of the beacon on the normal image and the value of the ID information are synthesized. Is possible.

<Specific Operation; First Embodiment>
FIG. 4 is a timing chart illustrating pixel signal readout timings in the image acquisition function and the ID detection function in the solid-state imaging device 1 according to the first embodiment shown in FIG. Here, FIG. 4A shows the reading operation of the image acquisition function, and FIG. 4B shows the reading operation of the ID detection function. In this example, the row selection scanning of the low-speed frame for the image acquisition function by the first vertical scanning circuit 14a is synchronized with the row selection scanning of the high-speed frame for the ID detection function by the second vertical scanning circuit 14b. However, even in this case, as will be described later, competition is inevitable.

  Note that, during the pixel cell timing for driving the unit pixel 3, the vertical selection pulse SEL, the reset pulse RST, and the transfer control pulse TX that are selected in the image acquisition scanning line are used as the vertical selection pulse SEL (1) and the reset pulse RST, respectively. (1) Described as transfer control pulse TX (1). On the other hand, the signal lines selected in the ID detection line are distinguished from each other by being expressed as a vertical selection pulse SEL (2), a reset pulse RST (2), and a transfer control pulse TX (2).

  In the first embodiment, the first vertical scanning circuit 14a and the first horizontal scanning circuit 12a for image acquisition, and the second vertical scanning circuit 14b and the second horizontal scanning circuit 12b for ID detection are independently provided on the same sensor. The normal image acquisition function that requires low-speed frame scanning and the ID detection function that requires high-speed frame scanning are performed by performing two independent vertical (V) sweep and horizontal (H) sweep scans. Realize at the same time.

  For example, in the read operation of the image acquisition function, as shown in FIG. 4A, the vertical (V) sweep for image acquisition is swept by the first vertical scanning circuit (vertical scanner for image acquisition) 14a by the Vsync_image signal. And reading out of one frame is completed in 1/30 sec. Hereinafter, the frame during the image acquisition operation is referred to as one image frame. The selection of each row is started by a horizontal (H) sweep trigger signal Hsync_image by the first horizontal scanning circuit 12a, and the 1H period is 64 μs.

  The 1H period is divided into a blanking period and a horizontal transfer period. During the blanking period, signals from the pixels are read in a batch and fixed in the CDS processing unit 26a of the column processing unit 26 provided at the upper end of the column. Sample hold is performed after the pattern noise removal processing. The pixel signals sampled and held by the CDS processing unit 26a are sequentially selected in the horizontal direction by the first horizontal scanning circuit (horizontal scanner) 12a in the horizontal transfer period and output via the horizontal signal line 18. A pixel signal is read out to the circuit 28.

  Here, in the readout operation in the unit pixel 3, a specific row is selected by the vertical selection pulse SEL (1) after a fixed delay period td from the horizontal sweep trigger signal Hsync_image, and the corresponding vertical selection transistor 40 is turned on (ON). ) Next, the read selection transistor 34 is turned on by the transfer control pulse TX (1), whereby the charge in the charge generation unit 32 is read to the floating diffusion 38, and the source follower configured by the amplification transistor 42 and the load MOS transistor 290 is read. The voltage of the vertical signal line 19 is determined by the circuit.

  This signal potential is clamped by the clamp pulse CLP1 in the CDS processor 26a. Next, the reset transistor 36 is turned on by the reset pulse RST (1), the floating diffusion 38 is reset, and the reset signal potential is similarly read out to the vertical signal line 19. The CDS processing unit 26a removes the fixed pattern noise for each pixel by subtracting the reset potential and the previous signal potential. Then, the signal potential is sampled and held by a sample hold pulse SH.

  In a normal CMOS sensor, the reset state is read out first and clamped by the CDS processing unit 26a, and then the charge is transferred from the charge generation unit 32 such as a photodiode to read out the signal level. In the operation of the form, the reverse operation is performed.

  Next, the reading operation of the ID detection function will be described with reference to FIG. Note that the period Th of the pixel cell readout portion is shown in accordance with the time range in FIGS. 4 (A) and 4 (B).

  The ID detection function V sweep is performed by the second vertical scanning circuit (ID detection vertical scanner) 14b. The sweep of the first vertical scanning circuit (image acquisition vertical scanner) 14a and the third vertical scanning circuit (memory vertical scanner) 14c is started by the Vsync_ID signal, and the 1V period is 512 μsec, which is about 2 kfps. Hereinafter, the 1V period of the ID detection function is referred to as an ID frame. Reading of each row is started by a horizontal (H) sweep trigger signal Hsync_ID by the second horizontal scanning circuit 12b, and 1H period is 1 μs.

  First, the vertical selection pulse SEL (2) rises by the horizontal (H) sweep trigger signal Hsync_ID, the vertical selection transistor 40 is selected, and the signal determined by the charge already accumulated in the floating diffusion 38 at that time is a vertical signal line. 19 is read out.

  This FD potential corresponds to the potential due to the signal charge transferred during the previous ID frame scan or the reset potential reset by the readout scan of the image acquisition function. The potential of the vertical signal line 19 is determined by the potential division between the amplifying transistor 42 and the load MOS transistor 290, and the signal potential of the vertical signal line 19 is thereby determined by the arithmetic processing unit 27 provided on the lower side of the pixel unit 10 in the figure. In the difference calculation unit 272, the active H of the clamp pulse CLP2 is supplied to the gate of the transistor 304, whereby the node N1 is clamped at the reference potential Vb.

  Next, the charge of the charge generator 32 is read out to the floating diffusion 38 by the transfer control pulse TX. At this time, since the charge before the transfer control pulse TX (2) is applied remains in the floating diffusion 38, the charge read by applying the transfer control pulse TX (2) is added to the previous charge, and the FD The potential is also the sum of the previous signal and the signal from the transfer. This addition signal is read to the difference calculation unit 272 via the vertical signal line 19.

  Since the difference calculation unit 272 is clamped at the signal potential before transfer, a potential corresponding to the difference between the signals before and after transfer appears at the node N1. Thereby, it is possible to extract only the signal potential accumulated in one ID frame period without resetting the unit pixel 3.

  Here, the unit pixel 3 selected by the image acquisition scanning and the unit pixel 3 selected by the ID detection share the same vertical signal line 19 in the same column. To avoid conflicts. Specifically, in the first embodiment, the selection period of the ID selection vertical selection pulse SEL (2) is ended within the delay period td in the above-described image acquisition operation.

  The node N1 is connected to the gate of the drive transistor 306, and draws current from the current copier cells 320 and 340 in the input phase (write mode) constituting the analog memory array unit 274 when the switch S2 is turned on. .

  In the analog memory array unit 274, two cells (for two frames; current copier cells 320 and 340) are set per pixel, and the corresponding current copier cell 320 is set by the third vertical scanning circuit (memory vertical scanner) 14c. , 340 (320 in the figure) is selected by the selection control pulse SM1, and the current copier cell 320 is written by scanning the write control pulse ME1.

  That is, as described above, the selection control pulse SM1 is set to H level to turn on the transistor 322, and the write control pulse ME1 is set to H level to turn on the transistor 328, whereby the current copier cell 320 is turned on. And the current signal on the vertical signal line 19 is written to the current copier cell 320.

  Thereafter, the comparator sequence is entered. First, one of the current copier cells 320 and 340 (340 in the timing in the figure) storing the previous frame data is selected by the selection control pulse SM2, and the current copier cell 340 is selected. Reads out the current information held by the comparator unit 276.

  That is, as described above, the selection control pulse SM2 is set to H level to turn on the transistor 342, and the write control pulse ME2 is set to L level to turn off the transistor 348, whereby the current copier cell 320 is turned on. In the output phase, the current signal held by the current copier cell 340 is read onto the vertical signal line 19.

  At the beginning of this read operation, both the clamp pulses CP1 and CP2 are set to the H level to initialize the chopper type comparator unit 276. Thereafter, the clamp pulse CP1 and the clamp pulse CP2 are sequentially followed by a slight time difference. The pixel signal (that is, the previous frame data) on the vertical signal line 19 after the switch S3 is clamped at the L level.

  Next, with the clamp pulses CP1 and CP2 set to L level, the selection control pulse SM1 is selected from one of the current copier cells 320 and 340 (current copier cell 320 at the timing in the figure) in which the current frame data is stored. By turning on the transistor 322 under the control, the pixel signal on the vertical signal line 19 after the switch S 3, that is, the current frame current information held by the current copier cell 320 is read out to the comparator unit 276. Thus, after the previous frame data and the current frame data are sequentially taken into the comparator unit 276, the comparator unit 276 compares the previous frame data with the current frame data. Thereby, the edge of LED blinking (beacon) can be detected.

  Here, the bias circuit unit 275 has an offset processing function and functions as a load transistor that draws current from the current copier cells 320 and 340, and at the same time weights each data during the comparison operation. This weighting is for fixing the comparator data, which is the comparison result by the comparator unit 276, to a constant value (for example, “0” data) in a steady state where there is no light change.

  That is, in a steady state where there is no light change, the previous frame data and the current frame data are slightly different (the magnitude relationship becomes unstable), so that the comparator data becomes unstable unless an offset is given. By providing an appropriate level of offset, the comparator data becomes stable even in a steady state where there is no light change.

  For example, as shown in FIG. 4, the bias control signal supplied to the gate of the PchMOS transistor 362 is kept constant at active H while the bias control signal NBIS supplied to the gate of the NchMOS transistor 364 is kept constant during the comparison processing in the comparator unit 276. PBIS is set to active L only when reading the current copier cell 340 holding the previous frame data.

  In this way, when the input potential to the comparator unit 276 becomes a certain level higher and the current frame read potential becomes sufficiently higher than that in the previous frame (when LED blinking falls), the output value of the comparator is changed from “0” to “ 1 "changes.

  Thereafter, the comparison result is latched as binary data in the latch 278a of the data latch unit 278 for each column, and is read out to the outside of the sensor via the data output bus 279.

  Although the data output timing is not specified, the data bus width is set so that the output for 1H is made within 1H period (1 μs), and the process from reading to data output is repeated in a cycle of 1 μs. Thereby, in the ID detection function, LED blinking edge detection can be performed.

  In the above ID detection function sequence, the difference calculation unit 272 and the comparator unit 276 perform the difference calculation twice. In other words, in the architecture of the first embodiment, the rising edge and the falling edge of the LED blinking are detected by performing the second-order difference operation on the charges sequentially stored in the floating diffusion 38 for each ID frame. Will be.

  The operation of the image acquisition function and the operation of the ID detection function can be performed simultaneously as functions by scanning each vertical and horizontal scanner independently.

<Concrete action: Processing at the time of competition>
FIG. 5 is a timing chart for explaining a coping method when a conflict occurs when the first vertical scanning circuit 14a and the second vertical scanning circuit 14b perform independent vertical scanning. FIG. 5 shows the relationship between the movement of the selection line of the first vertical scanning circuit (image vertical scanner) 14a and the movement of the selection line of the second vertical scanning circuit (ID detection vertical scanner) 14b with respect to the time axis. it can.

  For example, at the start of the image frame scan, the image acquisition scan and the ID detection scan are started simultaneously from the first line at the lower end of the pixel unit 10 by the Vsyn_image and Vsync_ID pulses, respectively.

  However, since the 1H period is greatly different from 64 μs for the image acquisition function and 1 μs for the ID detection function, the respective selection lines are greatly separated. For example, when the image acquisition scan line is selected as the eighth line, the ID detection scan line has finished scanning for exactly one ID frame, and the ninth line scan of the image acquisition function and the ID detection first line are completed. Scanning is simultaneous. Thus, the ID detection scan sweeps one ID frame every 8 line scans of the image output scan, and this is repeated thereafter.

  At this time, first, when focusing on image acquisition scanning, the non-selected line should originally be accumulated in the charge generation unit 32 such as a photodiode in one image frame period. The charge read operation is always performed from the charge generation unit 32.

  However, in the ID detection scanning that performs a sweep process faster than the sweep process for the normal image generation process, the signal charge read from the unit pixel 3 remains held in the floating diffusion 38 of the unit pixel 3 as it is. In one image frame period on the sweep side, even if the signal charge reading process is performed by the high-speed sweep process, the signal charge detected in the physical quantity (in this example, light) is held in the unit pixel 3 and lost. Absent. That is, non-destructive reading is performed in the drive control of the high-speed sweep process.

  In the image acquisition scanning, as described above, the signal of the floating diffusion 38 is first read out and then the reset signal is read out later. Therefore, the operation is reverse to that of a normal sensor. This is because it is necessary to read before resetting the electric charge accumulated in the. As described above, the ID detection scan disturbs the image acquisition scan, that is, does not adversely affect the reading of the pixel signal for generating the normal image.

  On the other hand, when paying attention to the ID detection scanning, the difference calculation unit 272 always processes the potential of the floating diffusion 38 before and after the charge transfer. Therefore, how the potential state of the floating diffusion 38 was before the line selection does not affect the calculation result. The line on which the image acquisition scan has been performed undergoes a reset operation of the floating diffusion 38, but when the floating diffusion 38 is reset does not affect the ID detection scan.

  In addition, when the V sweep of the image acquisition scan starts (when the Vsync_image pulse is applied) and when the ID detection line passes the image acquisition scan line (circle portion in FIG. 5), the line selection overlaps. However, also in this case, in the operation in the pixel cell, the image acquisition operation and the ID detection operation are divided into time intervals with a delay time td, and therefore, each of the vertical selection pulse SEL, the transfer control pulse TX, and the reset pulse RST. By setting the control pulse to a timing setting in which (1) and (2) are superimposed, it is possible to scan each function independently.

<Specific operation: Modification 1 of the first embodiment>
FIG. 6 is a timing chart illustrating a first modification of the image acquisition function and the ID detection function in the solid-state imaging device 1 according to the first embodiment illustrated in FIG. 3. The basic operation is the same as that shown in FIG. 4, but there is a slight difference in the drive timing of the unit pixel 3.

  Specifically, in Modification 1, TX (1) for image acquisition scanning and TX (2) for ID detection scanning are set at the same timing. Even if the timing is the same, since the vertical selection lines 52 (SEL (1), SEL (2)) remain as they are, the time division sharing of the vertical signal lines 19 does not change.

  That is, in each selected row, charges are simultaneously transferred from the charge generation unit 32 such as a photodiode to the floating diffusion 38, but in the image acquisition row, signal transmission to the vertical signal line 19 is performed on the selection line SEL (1 ) Is selected.

  In this way, there is an advantage that there is a margin in the readout timing of the image acquisition line scan, and the timing can be easily set.

<Specific Operation; Modification 2 of First Embodiment>
FIG. 7 is a timing chart for explaining another modification 2 of the image acquisition function and the ID detection function in the solid-state imaging device 1 of the first embodiment shown in FIG. The basic operation is the same as that shown in FIG. 4, but there is a slight difference in the drive timing of the load MOS transistor 290.

  For example, in FIG. 3, a load MOS transistor 290 is a constant current transistor serving as a load at the time of reading a signal from the vertical signal line 19, and is connected to the vertical signal line 19 in both the image acquisition function and the ID detection function. It is set as a constant current source during reading. Usually, by changing the set current of the load MOS transistor 290, it is possible to adjust the operating point and speed at the time of signal reading.

  In the examples of FIGS. 4 and 6, this setting value is constant during the image acquisition function and the ID detection function. However, the setting value is not limited to a constant value and is set during the image acquisition function and the ID detection function. It is possible to change and optimize. For example, as shown in FIG. 7, the setting may be changed in accordance with the operation timing of the vertical selection pulse SEL (1) and the vertical selection pulse SEL (2).

  As described above, according to the solid-state imaging device 1 of the first embodiment (including the first and second modifications), the arithmetic circuit held for each pixel of the image sensor described in the conventional example is provided for each column. In addition, image output processing and arithmetic processing are completely separated by separate circuit blocks, simplifying the configuration within each pixel, reducing the overall size of the device and improving the quality of actual images In addition, it is possible to design optimally for arithmetic processing.

  In addition, a column processing unit 26 that processes pixel signals for normal images and an arithmetic processing unit 27 that processes pixel signals for ID images are provided independently, and vertical scanning circuits 14a and 14b that perform vertical scanning with respect to each of the column processing units 26 and 14b. Since the first horizontal scanning circuits 12a and 12b for scanning (drive control unit for collectively controlling the driving of the image sensor) are also provided independently, even when both the normal image and the ID image are output in this way, There is no need to alternately repeat the normal image mode and the ID detection mode.

  Also, the blinking interval of the beacon can be shortened without worrying about the frame rate of the normal image, and the arithmetic processing unit 27 that detects it can be operated by high-speed frame scanning to increase the amount of transmitted information. be able to.

  For example, even when the LED light source is blinking at several hundred fps, in order to sample it, the arithmetic processing unit 27 is several kfps, which is faster than the frame rate of the normal image (generally 30 fps or 60 fps). However, in this embodiment, such a setting can be easily realized.

  That is, according to the present embodiment, in order to realize the application of the ID camera, the image sensor to be used acquires a normal image and detects the blinking timing of the light that blinks at high speed by high-speed frame scanning. Can have one function.

  In addition, by operating the vertical scanning of the arithmetic processing unit 27 at a high speed, the blinking timing of the beacon does not need to be synchronized throughout the system, the beacon installation is not limited, and the flexibility of application is very wide. For example, the present invention can be applied even when a plurality of beacons operate with their own clock without being synchronized. This allows multiple independent beacons to exist in the environment at the same time.

  Further, since the sampling processing can be performed at high speed by operating the vertical scanning of the arithmetic processing unit 27 at a high speed, the recognition speed is also high, and for example, it can be used with a moving object. For example, in applications such as autonomous mobile robots and garage control of automobiles, it is necessary to measure the position of a moving object with high accuracy.

  As an application in such a case, by applying the first embodiment, for example, a transmitter that generates blink patterns (light beacons) at a plurality of points whose positions are known on the work space is arranged, and a robot or A receiver is mounted on a vehicle such as an automobile to shoot real space. When a certain number or more of combinations of coordinates on the image frame and blinking points on the work space are known at the same time, the position, tilt, or posture of the receiver, that is, the aircraft, on the work space can be calculated. Of course, a plurality of receivers may be installed instead of only one on a robot or automobile. In such a case, it is possible to search for a transmitter with a wide field of view and measure its position and inclination.

  Alternatively, the receiver may be used in combination with a rotation sensor such as a gyro by applying the first embodiment. In such a case, first determine the absolute coordinates and direction of the receiver, and then track the relative rotational direction with the gyro so that even if the transmitter protrudes from the screen of the receiver, the rotational movement will occur. You can follow and catch the transmitter.

  Further, the form of sweeping (scanning) is not limited to sequential scanning, and random sweeping is also possible. For example, it is possible to perform optimum processing by changing the order of sweeping each pixel in normal image output and arithmetic processing. Further, as the order of pixel sweeping, serial processing in units of one pixel or a small number of pixel blocks and parallel processing by a plurality of signal lines can be used properly.

  For example, serial processing can be performed in units of one pixel at the time of image information output processing, and parallel processing by a plurality of signal lines can be performed at the time of other arithmetic processing, enabling optimization according to the characteristics of each signal processing. Become.

  In addition, the number of pixels simultaneously transmitted to one signal line can be changed. On the image information output processing side, a signal for each pixel is transmitted to one signal line, and the other arithmetic processing side has a plurality of signals. It is also possible to transmit pixel signals simultaneously to one signal line. By doing so, it is possible to optimize signal transmission in accordance with the characteristics of each signal processing.

  Further, by using the number of pixels corresponding to the combination of the color filters as the number of pixels simultaneously transmitted to one signal line at the time of calculation processing, high-precision calculation can be performed.

  In addition, a switch is provided for each of one or more columns of the pixel array, a switch that is turned on at the time of signal readout is selected from these switches, and a column that is input to each differential operation unit 272 is selected. Adopt a method specific to arithmetic processing by combining and adding signals from multiple pixels by simultaneously selecting multiple rows in a pixel array or simultaneously selecting multiple columns, and treating multiple pixels as a single light-receiving pixel unit Can be optimized.

<Specific Configuration; Second Embodiment>
FIG. 8 is a diagram illustrating a second embodiment of a specific configuration example (internal configuration) of the column processing unit 26 and the arithmetic processing unit 27 focusing on one vertical column in the solid-state imaging device 1 illustrated in FIG. 1. The second embodiment is also configured to realize the function of the ID camera as in the first embodiment.

  Here, the solid-state imaging device 1 according to the second embodiment is characterized in that the solid-state imaging device 1 is configured by a single circuit that doubles as a CDS processing function unit for image acquisition and a difference calculation processing function unit for ID detection. Have

  For example, in the configuration of the first embodiment, the column processing unit 26 includes the CDS processing unit 26a for image acquisition and the arithmetic processing unit 27 for ID detection. However, each of these functional units actually performs a difference calculation of two signal differences, and the circuit can be similarly configured.

  In the second embodiment, paying attention to this point, in these two functions, the CDS processing unit 26a and the arithmetic processing unit 27 are used in a single circuit (CDS / difference arithmetic processing unit). .

  The CDS / difference calculation processing unit 400 includes a coupling capacitor 402 corresponding to the coupling capacitor 302 of the first embodiment, a transistor 404 corresponding to the transistor 304, and a drive transistor 406 corresponding to the drive transistor 306.

  The CDS / difference calculation processing unit 400 includes a cascade circuit of an amplifier 410 and an Nch MOS transistor 412 between the coupling capacitor 402 and the drive transistor 406. Further, a cascade circuit of an Nch MOS transistor 416 and a sample hold capacitor 418 is provided in parallel with this cascade circuit.

  The transistor 412 functions as a sample and hold switch, the drain is connected to the output of the amplifier 410, the source is connected to the gate of the drive transistor 406, and the active H (high) sample and hold is connected to the gate. The pulse SH2 is supplied from a second vertical scanning circuit 14b (not shown).

  The transistor 416 also functions as a sample and hold switch, with its drain connected to the output of the amplifier 410 and its source connected to one terminal of the sample and hold capacitor 418 at the node N3. The other terminal of the sample hold capacitor 418 is grounded. Further, an active H (high) sample hold pulse SH1 is supplied to the gate of the transistor 416 from a second vertical scanning circuit 14b (not shown).

  In the CDS / difference calculation processing unit 400 having such a configuration, the node N1 where the difference calculation is performed and the amplifier 410 that amplifies the potential are shared by both the normal image generation processing function and the ID detection function.

  Further, the amplifier 410 and later are branched by transistors 412 and 416 having a sample and hold switch function. For example, after one sample hold switch (transistor 412) controlled by the sample hold pulse SH2, the process of the ID detection function is performed as in the first embodiment.

  On the other hand, after the other sample and hold switch (transistor 416) controlled by the sample and hold pulse SH1, the signal obtained by subtracting the reference level at the node N1 is amplified by the amplifier 410 and applied to the node N3. Retained.

  The signal charges are sequentially transferred to the horizontal signal line 18 by sequentially selecting the column switch S1 by a first vertical scanning circuit (image acquisition horizontal scanner) 14a (not shown) disposed in the vicinity thereof.

  In the second embodiment, unlike the first embodiment, the first horizontal scanning circuit (image acquisition horizontal scanner) 12a (not shown) and the illustrated horizontal signal line 18 are on the same side as the arithmetic processing unit 27 for the ID detection function. Placed in.

<Specific Operation; Second Embodiment>
FIG. 9 is a timing chart illustrating pixel signal readout timings in the image acquisition function and the ID detection function in the solid-state imaging device 1 according to the second embodiment illustrated in FIG. Here, FIG. 9A shows the reading operation of the image acquisition function, and FIG. 9B shows the reading operation of the ID detection function.

  4 differs from the first embodiment shown in FIG. 4 in that the timing of the sample hold switch SH2 is added. The sample hold switch SH2 is turned on for a period from when the signal charge is transferred in the pixel by the transfer wiring TX (2) to when data is written to the analog memory array unit 274 at the timing of the ID detection function. The selection line SEL during the operation of the image acquisition function and others are the same as in the first embodiment.

  The sample hold pulse SH1 is the same as the sample hold pulse SH in the first embodiment. After the signal is sampled and held in the sample hold capacitor 418 by the sample hold switch SH1, the horizontal signal is independent of the ID detection operation. Transfer to line 18 is possible.

  As in the second embodiment, first, the corresponding vertical selection transistor 40 is selected by the vertical selection pulse SEL (2), and the signal potential due to the charge accumulated in the floating diffusion 38 until then is applied to the vertical signal line 19. The read signal potential is clamped at the node N1 by supplying the active H of the clamp pulse CLP2 to the transistor 504.

  In the second embodiment, the sample hold switches SH1 and SH2 are provided in order to clarify the separation of the processing of the image acquisition function and the ID detection function. Of these, the sample hold switch SH2 is not provided. There is no problem. The pixel readout timing at this time can be completely equivalent to that in the first embodiment.

  As described above, according to the solid-state imaging device 1 of the second embodiment, the CDS processing function unit for image acquisition and the difference calculation processing function unit for ID detection are configured by one processing circuit. As a result, a more compact and lower cost structure than the configuration of the first embodiment can be achieved.

<Specific Configuration; Third Embodiment>
FIG. 10 is a diagram illustrating a third embodiment of a specific configuration example (internal configuration) of the column processing unit 26 and the arithmetic processing unit 27 focusing on one vertical column in the solid-state imaging device 1 illustrated in FIG. The third embodiment is configured to realize the function of the ID camera as in the first embodiment.

  Here, in the solid-state imaging device 1 according to the third embodiment, two sets of amplification transistors 42 and two vertical selection transistors 40 for selecting the same exist in the unit pixel 3 constituting one pixel, and each of them is different. Connected to the signal line, two vertical signal lines 19a. It is characterized in that 19b exists. These two sets are distinguished for image acquisition and ID detection.

  For example, in FIG. 12, the unit pixel 3 includes a charge generation unit 32 including a photodiode PD, a read selection transistor 34, a reset transistor 36, two amplification transistors 42a and 42b, and selection transistors 40a and 40b. Yes. The amplifying transistor 42b and the selection transistor 40b constitute a first pixel signal generation unit 5b, and the amplification transistor 42a and the selection transistor 40a constitute a second pixel signal generation unit 5a.

  The charge accumulated in the charge generation unit 32 by the optical signal is transferred to the floating diffusion 38 by the read selection transistor 34. The gate electrodes of the amplifying transistors 42a and 42b are both connected to the floating diffusion 38, and receive the same FD potential signal at the gate electrodes.

  The source electrodes of the amplification transistors 42a and 42b are connected to the selection transistors 40a and 40b, respectively, and the vertical selection transistors 40a and 40b are connected to the vertical signal lines 19a and 19b, respectively. The combinations of the amplifying transistors 42a and 42b, the vertical selection transistors 40a and 40b, and the vertical signal lines 19a and 19b are used separately for the image acquisition function and the ID detection function, and the amplification transistor 42a and the vertical selection transistor 40a. The vertical signal line 19a is used exclusively for image acquisition, and the amplifying transistor 42b, the vertical selection transistor 40b, and the vertical signal line 19b are used exclusively for ID detection.

  Further, since two vertical signal lines 19a and 19b are prepared as the vertical signal line 19, the load transistor section 29 is also a load MOS transistor serving as a constant current source, and the two vertical signal lines 19a and 19b. Prepared for each (load MOS transistors 290a and 290b).

  Signals read to the vertical signal line 19a are simultaneously read above the pixel unit 10 for all columns, and fixed pattern noise for each pixel is removed by the CDS processing unit 26a, and a signal is output to the outside of the pixel unit 10. The

  On the other hand, the ID detection signal is read below the pixel unit 10 along the vertical signal line 19b, and is transmitted to the previous frame through the difference calculation unit 272, the analog memory array unit 274, the comparator unit 276, and the like of the calculation processing unit 27. Comparison processing is performed, and the edge of LED blinking (beacon) is detected.

<Specific Operation; Third Embodiment>
FIG. 11 is a timing chart illustrating pixel signal readout timings in the image acquisition function and the ID detection function in the solid-state imaging device 1 according to the third embodiment illustrated in FIG. Here, FIG. 11A shows the reading operation of the image acquisition function, and FIG. 11B shows the reading operation of the ID detection function. The scanning timing in the V sweep period and the H sweep period is the same as that in the first embodiment shown in FIG.

  In the vertical (V) sweep for image acquisition, the vertical scanner for image acquisition starts sweeping by the Vsync_image signal, and reading of one frame is completed in 1/30 sec (hereinafter, the frame during the image acquisition operation is referred to as one image frame). . Reading of each row is started by a horizontal (H) sweep trigger signal Hsync_image, and the 1H period is 64 μs.

  The 1H period is divided into a blanking period and a horizontal transfer period. During the blanking period, signals from the pixels are read out in a row and sample-held at the column end after processing by the CDS processing unit 26a. Thereafter, the column-switched data S1 is sequentially turned on in the horizontal direction to read the sampled and held data.

  Here, in the reading operation in the pixel unit 10, first, a specific row is selected by the vertical selection pulse SEL (1), and the corresponding vertical selection transistor 40 is turned on. Then, the charge in the charge generator 32 is read to the floating diffusion 38 by the transfer control pulse TX (1), and the signal amplified by the amplifying transistor 42 is supplied to the vertical signal line 19a.

  This signal potential is clamped by the clamp pulse CLP1 in the CDS processing unit 26a arranged at the column end. Next, the floating diffusion 38 is reset by the reset pulse RST (1), and the reset signal potential is similarly read to the CDS processing unit 26a.

  The CDS processing unit 26a generates a signal potential from which fixed pattern noise has been removed by subtracting the reset potential and the previous signal potential, and further uses the signal potential from which fixed pattern noise has been removed for each pixel as a sample hold pulse. Sample hold by SH.

  Next, ID detection scanning will be described with reference to FIG. In the ID detection function V sweep, the second vertical scanning circuit (ID detection vertical scanner) 14b starts sweeping by the Vsync_ID signal, and the 1V period is about 512 μsec, which is about 2 kfps (hereinafter, the ID detection function frame 1 ID frame). For reading out each row, first, the vertical selection transistor 40 is selected by the vertical selection pulse SEL (2), and a signal corresponding to the FD potential determined by the electric charge already accumulated at that time is read out to the vertical signal line 19b. .

  This FD potential corresponds to the potential due to the signal charge transferred during the previous ID frame scan or the reset potential reset by the readout scan of the image acquisition function. Thus, the signal potential of the vertical signal line 19b is clamped to the node N1 by the clamp pulse CLP2 of the difference calculation unit 272 on the lower side of the pixel unit 10 in the drawing.

  Next, the charge of the charge generator 32 is read out to the floating diffusion 38 by the transfer control pulse TX. At this time, since the charge before application of the transfer control pulse TX remains in the floating diffusion 38, the charge read out by applying the transfer control pulse TX (2) is added to the previous charge, and the FD potential is also simultaneously. This is the addition of the previous signal and the signal from the transfer.

  This addition signal is read out to the vertical signal line 19b. Here, since the difference calculation unit 272 is clamped at the signal potential before the transfer, a potential corresponding to the difference between the signals before and after the transfer appears at the node N1. Thereby, it is possible to extract the signal potential accumulated in one ID frame period without the reset operation in the unit pixel 3.

  Node N1 is connected to the gate of load MOS transistor 290 and draws current from analog memory array section 274 formed of current copier cells 320 and 340. In the analog memory array section 274, two cells (for two frames; current copier cells 320 and 340) are set per pixel, and the corresponding current copier cells 320 and 340 are selected and written.

  Thereafter, as in the first embodiment, the data of the previous frame and the data of the current frame are sequentially read out from the current copier cells 320 and 340 and compared in magnitude by the lower comparator unit 276.

  The result is latched as binary data by the data latch unit 278 for each column and read out from the sensor via the data output bus 279. Thereby, in the ID detection function, LED blinking edge detection can be performed.

  The operation of the image acquisition function and the operation of the ID detection function can be performed simultaneously as independent scanning as in the first embodiment. Further, the relationship between the image acquisition scan and the ID detection scan line scan (V scan scan) is the same as in FIG.

  However, in the third embodiment, the timing setting flexibility can be increased by preparing two vertical signal lines 19a and 19b for image acquisition and ID detection. That is, in the first embodiment, since the same signal line 19 is shared by both functions, the timing of line selection (SEL (1)) at the time of image acquisition and line selection for ID detection (SEL (2)) is set. It was necessary to shift. On the other hand, in the third embodiment, as shown in FIG. 13, if only the rising timing of the transfer control pulse TX is set at the same time, it is not necessary to shift both timings. That is, it is possible to set the readout operation for image acquisition within the Th time without being restricted by the timing of the vertical selection pulse SEL (2).

  Note that, using the third embodiment, similarly to the first embodiment, it is possible to set the scanning timing by shifting the timings of the vertical selection pulses SEL1 and SEL2.

  Further, unlike the first embodiment, in the third embodiment, two vertical signal lines 19a and 19b are prepared as the vertical signal line 19, so that two load MOS transistors 290 serving as constant current sources are also provided. The vertical signal lines 19a and 19b are respectively prepared (29a1 and 29a2). Therefore, the constant current value and timing can be freely set for each function as indicated by VL1 and VL2 in FIG.

<Specific Configuration; Fourth Embodiment>
FIG. 12 is a diagram illustrating a fourth embodiment of a specific configuration example (internal configuration) of the column processing unit 26 and the arithmetic processing unit 27 focusing on one vertical column in the solid-state imaging device 1 illustrated in FIG. 1. As in the first embodiment, the fourth embodiment is configured to realize the function of the ID camera.

  Here, the solid-state imaging device 1 according to the fourth embodiment is characterized in that it includes two floating diffusion nodes. For example, a transfer transistor (first transfer unit) 35 is added between the read selection transistor 34 and the reset transistor 36 in the configuration of the first embodiment. The two floating nodes separated and formed by the transfer transistor 35 are defined as floating diffusions 38a and 38b (the reading selection transistor 34 side is the floating diffusion 38b), respectively.

  The gate of the amplification transistor 42a used for the image acquisition function is connected to the floating diffusion 38a, and the gate of the amplification transistor 42a used for the ID detection function is connected to the floating diffusion 38b. The source of the reset transistor 36 is connected only to the floating diffusion 38a.

  The source of the reset transistor 36 is connected only to the floating diffusion 38a of the two floating diffusions 38a and 38b.

  Further, in the pixel configuration of FIG. 14, it is assumed that the charge transfer from the floating diffusion 38 b to the floating diffusion 38 a is completely performed by selecting the transfer transistor 35.

<Specific Operation; Fourth Embodiment>
FIG. 13 is a timing chart illustrating pixel signal readout timings in the image acquisition function and the ID detection function in the solid-state imaging device 1 according to the fourth embodiment illustrated in FIG. Here, FIG. 13A shows the reading operation of the image acquisition function, and FIG. 13B shows the reading operation of the ID detection function.

  The scanning timing in the V sweep period and the H sweep period can be equivalent to that in the first embodiment shown in FIG. However, the configuration of the fourth embodiment enables more flexible timing setting.

  Specifically, image acquisition scanning is different from that of the first embodiment. For example, in the ID detection operation, charge transfer is performed from the charge generation unit 32 to the floating diffusion 38b, and the potential of the floating diffusion 38b is received by the amplifying transistor 42b to transmit a signal to the vertical signal line 19b.

  On the other hand, in the image acquisition scanning, the charge transferred to the floating diffusion 38b during the image frame accumulation period is transferred to the floating diffusion 38a by the transfer transistor 35. The potential of the floating diffusion 38a is received by the amplifying transistor 42a, and a signal is transmitted to the vertical signal line 19a.

  Thus, in the fourth embodiment, an image acquisition function and an ID detection function are provided by providing the transfer transistor 35 so as to divide the floating diffusion 38 into two portions in the floating diffusion 38 in the configuration of the first embodiment. The nodes that sense the potential are made different.

  In scanning within the unit pixel 3, scanning for ID detection is the same as in the first embodiment. On the other hand, in the image acquisition scan, as in the first embodiment, the vertical selection pulse SEL (1) is raised after the vertical selection pulse SEL (2) of the ID detection function falls. First, the node of the floating diffusion 38a is reset by the reset pulse RST, and the potential of the vertical signal line 19a at that time is clamped by the CDS processing unit 26a. Next, the charge of the floating diffusion 38b is transferred to the floating diffusion 38a by the transfer control pulse TX2. Then, a difference from the reset level is obtained by the CDS processing unit 26a based on the signal potential at that time, and is output as an image signal.

  Here, in the configuration of the fourth embodiment, once charge transfer is performed from the floating diffusion 38b to the floating diffusion 38a, ID detection scanning and image acquisition scanning can be performed independently.

  Therefore, the timing of image acquisition scanning can be freely set without being limited to ID detection scanning. In the first embodiment, the falling of the vertical selection pulse SEL (2) needs to be terminated within the ID detection 1H period (Th), but in the fourth embodiment, this is not necessary.

  As described above, the reset timing of the floating diffusion 38b can be set after the charge transfer as in the first embodiment, even before the charge transfer by the transfer control pulse TX2.

<Specific Operation; Modification of Fourth Embodiment>
FIG. 14 is a timing chart for explaining a modification of the image acquisition function and the ID detection function in the solid-state imaging device 1 of the fourth embodiment shown in FIG. The basic operation is the same as that shown in FIG. 13, but there is a slight difference in the drive timing of the transfer selection line TX1 for image acquisition.

  Specifically, in this modification, the transfer selection line TX1 for image acquisition is not selected every 1H period, but by controlling the transfer transistor 35, during the vertical blanking period of the low-speed frame rate scanning function. All lines are selected simultaneously, and signal charges are transferred from the floating diffusion 38b to the floating diffusion 38a all at once.

  In the scanning of each H period, the transfer control pulse TX1 is not selected, and the signal charge of the floating diffusion 38a is read out to the vertical signal line 19a when the vertical selection pulse SEL (1) is selected.

  In this way, for each pixel, the floating diffusion 38a as a charge storage unit is provided between the charge generation unit 32 and the amplification transistor 42 forming the pixel signal generation unit 5, and after all the pixels are exposed simultaneously, The signal charge generated by the generator 32 is transferred to the floating diffusion 38a at the same time.

  In each line, charge transfer from the floating diffusion 38b to the floating diffusion 38a is performed at the same time, so that a simultaneous shutter function, so-called global shutter operation, is possible. Here, the global shutter operation is a function that makes the exposure accumulation time of each pixel constant when the electronic shutter operation is performed (exposure at the same time).

  That is, at the timing of FIG. 13, charge transfer of each line is performed when each line is selected, and therefore the time of the charge accumulation period in the floating diffusion 38b differs depending on each line. Such a shutter operation is called a focal plane shutter. This means that the accumulation time differs in the vertical direction of the line, and the image is distorted when shooting moving objects. This is the same as a known problem specific to CMOS sensors. On the other hand, according to the modification of the fourth embodiment, such a phenomenon can be avoided by setting as shown in FIG.

  As described above, according to the solid-state imaging device 1 of the fourth embodiment (including modifications thereof), the configuration of the unit pixel 3 is modified to include two floating diffusion nodes, and therefore the timing of image acquisition scanning. Can be freely set without being limited to ID detection scanning, and is more convenient than the configuration of the first embodiment.

  In the first to fourth embodiments (including the respective modifications), in order to simplify the description, one charge generation unit 32 (such as a photodiode) is associated with one set of in-pixel transistors. However, as described in the summary of the first embodiment, a plurality of charge generation units 32 (such as photodiodes) may correspond to a combination of one set of in-pixel transistors. Such a configuration is an effective means for reducing the effective pixel size per pixel.

  The signal data output from the image sensor is based on the premise that the image acquisition function and the ID detection function are for each pixel, but this is based on the addition of nearby pixel signals within the pixel or signal line. It is also possible to perform signal processing and output.

  Further, the presence / absence of addition and the number of pixels to be added at that time may differ depending on the image acquisition function and the ID detection function. Such means effectively reduces the total number of pixels, and is effective when performing faster frame scanning or the like. Further, in the case of the ID detection function, means such as addition at the unit constituent pixels of the color filter matrix can be considered in order to compensate for the difference between the color filters arranged on each pixel.

  In the first to fourth embodiments (including each modification), it is assumed that the processing after the difference calculation of the ID detection function is performed in the sensor. Focusing on (that is, independent drive control for pixel signal readout processing), the ID detection signal read from the pixel is read out to the sensor at high speed, and the processing after the difference calculation is performed by a processor outside the sensor. An effective configuration is also possible.

  In addition, the scanning timing shown in the first to fourth embodiments (including the respective modifications) is an example, and only the characteristic part of the internal configuration of the pixel and the scanning method (that is, independent driving for pixel signal readout processing) Various timing setting variations are conceivable as long as the essence of the control is not lost.

  In the first to fourth embodiments (including each modification), the ID detection function detects a change timing of light that changes at high speed, and the same operation as this detects motion of a subject to be photographed. Can be used.

<Specific Configuration; Fifth Embodiment>
FIG. 15 is a diagram illustrating a fifth exemplary embodiment of a specific configuration example (internal configuration) of the column processing unit focusing on one vertical column in the solid-state imaging device 1. In the fifth embodiment, the two vertical scanning circuits (vertical scanners) 14a and 14b are scanned at different speeds in the same manner as in the first to fourth embodiments. Each of the signals is an image signal obtained by scanning at a high frame rate and a low frame rate, and an image having an extended dynamic range is obtained by the plurality of signals.

  That is, in the ID detection function of the first to fourth embodiments (including the respective modifications), the signal subjected to the first difference calculation process corresponds to information on a natural image obtained by high-speed frame scanning. . That is, when this signal is configured to be output to the outside of the sensor, a natural image signal by low-speed frame scanning and a natural image signal by high-speed frame scanning are simultaneously output from the sensor. In general, when signals with different accumulation times are obtained simultaneously, an image with a high dynamic range can be constructed from these data. Therefore, according to the configuration of this sensor, it is possible to acquire a high dynamic range image by calculating two types of signals having different charge accumulation times.

  For example, in the solid-state imaging device 1 of the fifth embodiment, similarly to the configuration of the second embodiment, the CDS / difference calculation processing unit 500 includes a column processing unit 260a that operates at the first frame rate and a second frame rate. The configuration is shared by two types of signal processing units having a sample and hold function, such as the column processing unit 260b operating in the above.

  For example, the CDS / difference calculation processing unit 500 corresponds to the coupling capacitor 502 corresponding to the coupling capacitor 402 of the second embodiment, the transistor 504 corresponding to the transistor 404, the amplifier 510 corresponding to the amplifier 410, and the transistor 416. A transistor 516 and a sample and hold capacitor 518 corresponding to the sample and hold capacitor 418 are included. These constitute the column processing unit 260a.

  Further, the CDS / difference calculation processing unit 500 includes a transistor 512 corresponding to the transistor 516 and a sample-and-hold capacitor 514 corresponding to the sample-and-hold capacitor 518 in parallel with the cascade circuit of the transistor 516 and the sample-and-hold capacitor 518. Have. The coupling capacitor 502, the transistor 504, and the amplifier 410 are also used as the column processing unit 260a, and the transistor 512 and the sample hold capacitor 514 constitute the column processing unit 260b.

  The transistor 512 also functions as a sample and hold switch, and has a drain connected to the output of the amplifier 510 and a source connected to one terminal of the sample and hold capacitor 514 at the node N2. The other terminal of the sample hold capacitor 514 is grounded. Further, an active H (high) sample hold pulse SH2 is supplied to the gate of the transistor 512 from a second vertical scanning circuit 14b (not shown).

  In the fifth embodiment, as in the second embodiment, a first horizontal scanning circuit (image acquisition horizontal scanner) 12a (not shown) and a horizontal signal line 18a (not shown) are connected to a column operating at the second frame rate. It is arranged on the same side as the processing unit 260b.

  In the CDS / difference calculation processing unit 500 having such a configuration, the normal image generation at the first frame rate is performed up to the node N1 where the difference calculation is performed and the amplifier 510 that amplifies the potential according to the second embodiment. This is shared by both the processing function and the normal image generation processing function at the second frame rate.

  Further, the amplifier 510 and later are branched by 512 and 516 having a sample hold switch function. For example, the one sample hold switch (transistor 516) and subsequent ones controlled by the sample hold pulse SH1 are for the image acquisition function, but here, the normal image generation processing function is processed at the first frame rate.

  For example, a signal obtained by subtracting the reference level at the node N1 is amplified by the amplifier 510 and held at the node N3. The signal charge acquired at the first frame rate is converted into the first horizontal signal by sequentially selecting the column switch S1 by a first vertical scanning circuit (image acquisition horizontal scanner) 14a (not shown) arranged in the vicinity. The image output 1 is sequentially transferred to the line 18a.

  On the other hand, the other sample hold switch (transistor 412) controlled by the sample hold pulse SH2 is also used for the image acquisition function, but here, the normal image generation processing function at the second frame rate is performed. For example, a signal obtained by subtracting the reference level at the node N1 is amplified by the amplifier 510 and held at the node N2. The signal charges acquired at the second frame rate are sequentially selected by a switch S2 by a second vertical scanning circuit (image acquisition horizontal scanner) 14b (not shown) disposed in the vicinity, whereby the second horizontal signal line. The images are sequentially transferred as image output 2 to 18b.

  In other words, in the fifth embodiment, the CDS / difference calculation processing unit 500 amplifies the signal clamped at the node N1 by the amplifier 410, and then branches by the transistors 512 and 516 having the sample hold switch function. Thereafter, the sample is held in the sample hold capacitors 514 and 518, respectively. Further, the signal charges held in the sample and hold capacitors 514 and 518 are respectively transferred to the horizontal signal lines 18a and 18b via column switches S1 and S2 which are sequentially selected by independent horizontal scanning circuits (horizontal scanners) 12a and 12b. The image is transferred and output to the outside as image output 1 and image output 2, respectively.

  For example, image output scanning by normal scanning is acquired as image output 1 by operating at the first frame rate, and acquired as image output 2 by image output scanning at a second frame rate faster than the first frame rate. To do.

  In the subsequent stage of the column processing unit 260a and the column processing unit 260b, a use signal that achieves a predetermined purpose based on the image output 1 and the image output 2 obtained by the column processing unit 260a and the column processing unit 260b, respectively. An application signal acquisition unit 100 for acquisition is provided.

  Here, as the use signal acquisition unit 100 of the fifth embodiment, as a functional element that performs a dynamic range expansion process for electronically expanding the dynamic range of an image by image data processing, a plurality of images having different charge accumulation times are used. A range expansion processing unit 120 for performing dynamic range expansion processing based on the above.

  The range expansion processing unit 120 includes a calculation image generation unit 122 that generates a calculation image by performing predetermined calculation processing based on the image output 1 from the column processing unit 260a and the image output 2 from the column processing unit 260b, and a column processing unit. One of the image output 1 from 260a, the image output 2 from the column processing unit 260b, and the calculation image output from the calculation image generation unit 122 (in this example, one image in which the D range is expanded). The output image selection unit 124 to be selected and output, and the pixel position (or image region) related to the dynamic range expansion processing in the imaging region of the solid-state imaging device 10 are designated, or the processing operation in the range expansion processing unit 120 is controlled. The process control part 126 which performs is provided.

<Specific Operation; Fifth Embodiment>
FIG. 16 is a timing chart illustrating pixel signal readout timing in the image acquisition function in the solid-state imaging device 1 according to the fifth embodiment illustrated in FIG. 15. Here, FIG. 16A shows the reading operation of the image acquisition function that operates at the first frame rate, and FIG. 16B shows the reading operation of the image acquisition function that operates at the second frame rate. .

  In the fifth embodiment, the column processing unit 260a in charge of the image output 1 outputs a normal image by processing at a normal (first) frame rate, as in the second embodiment. On the side of the column processing unit 260b in charge of the output 2, the difference calculation scan does not destroy the charge of the floating diffusion 38 during the frame scan of the normal image, and the second frame rate is higher than the first frame rate. Read the image signal.

  That is, as shown in FIG. 16A, scanning of the image output 1 is the same as that in the second embodiment shown in FIG. On the other hand, as an example, the image output 2 shows a case where the frame rate is scanned at a speed four times that of the image output 1 as shown in FIG. Therefore, the vertical sweep period of image output 2 is 1/120 sec, and the horizontal sweep period is 16 μsec.

  Here, in the vertical blanking period and horizontal blanking period of the image output 2, as in the case of the image output 1, timings of 24 counts and 100 counts are taken in terms of the number of output pixels, respectively. Pixel readout of the image output 2 is performed during a td period preceding the readout of the image output 1 during the horizontal blanking period.

  In the operation, as in the second embodiment, first, the corresponding vertical selection transistor 40 is selected by the vertical selection pulse SEL (2), and the signal potential due to the charges accumulated in the floating diffusion 38 until then is changed to the vertical signal. The signal potential is read to the line 19 and the signal potential is clamped to the node N1 by supplying the active H of the clamp pulse CLP2 to the transistor 504.

  Thereafter, the active H of the transfer control pulse TX (2) is supplied to the read selection transistor 34 to select the corresponding read selection transistor 34, and the charge newly accumulated in the charge generation unit 32 is supplied to the floating diffusion 38. The signal potential is transferred to the vertical signal line 19, and the potential corresponding to the difference from the previous potential is held at the node N1.

  The amplifier 510 amplifies this potential and simultaneously supplies the active H of the sample and hold pulse SH2 to the transistor 512 to turn on the transistor 512, thereby holding the signal potential in the sample and hold capacitor 514. Thereafter, the signal potential is read as the image output 2 in the horizontal transfer period by sequentially selecting the switch S2 in the second horizontal scanning circuit (horizontal scanner) 12b.

<Concept of dynamic range expansion processing function>
FIG. 17 is a diagram for explaining the processing mode of the dynamic range expansion processing. The range expansion processing unit 120 takes in a plurality of images having different exposure times (charge accumulation times) from the column processing units 260a and 260b, and performs a synthesis process in the arithmetic image generation unit 122.

  Here, the signal processing for wide dynamic range in the range expansion processing unit 120 can use a sum obtained by combining a short-time accumulation image and a long-time accumulation image into one sheet. In addition, both images can be properly used depending on the application. This selection is made by the output image selection unit 124. That is, if a charge accumulation time (corresponding to the shutter speed) is changed using an independent frame rate control function using the vertical scanning circuits 14a and 14b, and an object with a wide dynamic range including a bright part to a dark part is imaged. Good.

  With short-term storage (high-speed shutter), a dark subject is not captured, but a bright subject is clearly captured. On the other hand, when the image is accumulated for a long time (low speed shutter), a bright subject is saturated and flies, but a dark subject is clearly captured. In the dynamic range expansion process, only a good part may be cut out from these two images and used (for example, synthesized).

  In any case, the proportion of the image with a relatively low luminance level in the image is increased for a long-time accumulation (low-speed shutter) image, and the portion with a high luminance level is increased for a short-time accumulation (high-speed shutter) image. It is good. By such processing, it is possible to obtain an image having no whiteout due to saturation of a high luminance portion and having a good S / N of a low luminance portion.

  For example, the concept of dynamic range expansion is shown in FIGS. 17 (A) and 17 (B). The signal output of the solid-state imaging device 10 increases in proportion to the amount of incident light, but does not increase further when the output reaches a saturation level. For example, as indicated by a and b in FIG. 17A, the amount of light that saturates the solid-state imaging device 10 is L1.

  Here, when the charge accumulation time of the solid-state imaging device 10 is shortened using the frame rate independent control function using the vertical scanning circuit 14b, the relationship between the incident light quantity and the signal output changes as shown in c, and the solid-state imaging device. The amount of light at which 10 is saturated increases. When the maximum incident light amount increases, the charge accumulation time is adjusted to extract a signal along the curve of c. However, in the region of the light amount equal to or less than L1, the output a is larger when the charge accumulation time is longer. , S / N is good.

  Therefore, if the signal a for long-time charge accumulation is used more in the region below L1, and if the signal c for short-time charge accumulation is used more in the region above L1, the amount of saturated incident light is large and in a dark part. An image signal with a good S / N can be obtained. That is, the dynamic range can be expanded.

  Here, depending on the purpose of use of the imaging apparatus, the light quantity and the output can be used in a discontinuous relationship. In other words, when the low-light image is important, the long-time charge accumulation image is selectively output and used, while when the high-light image is important, the short-time charge accumulation image is selected and output. And use it. That is, one of the high-speed frame rate image and the low-speed frame rate image is selected and used. Without worrying about the responsiveness of the charge accumulation time control, it can be immediately switched to either.

  For example, the arithmetic image generation unit 122 adds a short-time accumulation image and a long-time accumulation image at a predetermined ratio, thereby combining them into a single image that is non-linearly or gently continuous. be able to.

  Here, it is assumed that the dynamic range of the image sensor is 60 dB, the long-time accumulation is set to an appropriate period in the vicinity of one frame period, for example, about 1/15 ms, and the short-time accumulation is an appropriate one less than one horizontal period. If the period is set to about 1/15 μs, for example, the sensor output corresponding to the light quantity during the long accumulation time corresponds to up to three digits with respect to the change in the light quantity. In addition, the sensor output corresponding to the light amount during the short accumulation time corresponds to up to three digits with respect to the change in the light amount, but is shifted by three digits from the light amount that can be detected during the long accumulation time.

  Therefore, a dynamic range of 6 digits, that is, 120 dB can be realized by the addition calculation image obtained by adding the images having different accumulation times. For example, an image that has a portion that saturates in the long accumulation time can be supplemented with an image that is detected in the short accumulation time, and even a saturation level that cannot be output with only one accumulation time is reproduced. Will be able to.

  For example, as described above, according to the fifth embodiment, the image output 1 system (column processing unit 260a) outputs an image as a normal image at a frame rate of 1/30 sec. On the other hand, in the system of image output 2 (column processing unit 260b), an image is output at a frame rate of 1/120 sec.

  Since these frame rates correspond to the charge accumulation time due to light irradiation in the charge generation unit 32 such as a photodiode, it means that image information with a four-fold different accumulation time is acquired simultaneously.

  These two image signals having different frame rates (that is, charge accumulation times) have an equivalent relationship to signals from two types of sensors having exactly four times the sensitivity with respect to the light signal intensity. Therefore, by adding these two image signals in the range expansion processing unit 120, for example, in the arithmetic image generation unit 122, the signal characteristics as shown in FIG. Compared to output (corresponding to image output 1), the detection signal range (dynamic range) can be expanded.

  As described above, as an application of time addition processing, if a plurality of pixel signals at the same position with different accumulation times are added, a signal output that is difficult to saturate for a wider incident light amount can be obtained as computation data, and dynamic Data that can expand the range can be acquired. It is possible to acquire an image with a wide dynamic range with respect to the amount of light with reduced whiteout and blackout.

  Further, since the dynamic range is expanded by combining two pixel signals having different accumulation times (the pixel signals having different accumulation times may be increased if necessary), a dedicated pixel such as an in-pixel memory is used. It does not require a structure, can be applied to a device having a normal pixel structure, and is not limited as a sensor device.

  However, in practice, in the simple addition process, the sensor output with respect to the light amount does not have an ideal knee characteristic that matches the visibility. That is, it does not match the human visual characteristic of identifying the brightness in proportion to the logarithm of the amount of light.

  In order to solve this problem, it is preferable to set the coefficient for the image signal to be processed in the addition operation by adjusting the amount of time change of the reference signal used for the comparison processing in consideration of the visibility. In particular, with respect to a processing target image acquired under a relatively short accumulation time in order to realize visibility correction without saturating a high level signal that would be saturated during a normal accumulation time It is preferable to adjust the amount of change.

  Specifically, it is preferable to change the slope over several stages without changing it linearly. In addition, it is not limited to changing in stages while having such linearity, and may be changed gradually and gradually according to a high-order function such as a quadratic function.

  The method of change at this time is to match the logarithmic characteristics of the sensitivity of the human eye, and to match the human eye's sensitivity to changes in brightness in the dark area, The gradation accuracy in the bright part is reduced so that the human eye is insensitive to the change in brightness in the bright part. Specifically, it is preferable that the coefficient is set to be large (high gain) by reducing the slope of the reference potential RAMP at the initial stage of AD conversion, and the slope of the reference potential RAMP is increased as AD conversion proceeds. A knee characteristic, which is a characteristic obtained by compressing a high luminance range in accordance with the human visual characteristic, is realized.

  In order to give such a change characteristic, the short-time accumulation sensitivity curve has a gain component and is subjected to gamma correction on the high luminance side, that is, a characteristic in which the high luminance range is compressed according to human visual characteristics. It is better to realize the knee characteristic.

  In this way, if the slope is gradually changed, it is possible not only to realize a wide dynamic range by combining different accumulation times, but also to perform gamma correction on the sensitivity characteristic to realize a more natural signal change characteristic. it can. Sensitivity differences between different accumulation times can be naturally connected, and more natural images can be synthesized.

  In the description of the fifth embodiment, as an example, the image output 2 is four times as high as the image output 1, but this speed can be arbitrarily changed. For example, the image output 2 can be changed by an external system. It is also possible to adjust and control the image output 2 so that the system of the image output 2 becomes an optimum image by detecting the overall brightness and appropriately changing the vertical scanning speed by the second vertical scanning circuit 14b.

  In the description of the fifth embodiment, the modification based on the second embodiment has been described. For example, the vertical signal line 19 includes a column processing unit 260b for the image output 2, the second horizontal signal line 18b, If a second horizontal scanning circuit (horizontal scanner) 12b is added, a modification based on another embodiment can be configured. In the fifth embodiment, the ID detection function is omitted. However, the column processing unit 260b for the image output 2 is provided while holding the ID detection calculation processing unit 27 provided in the second embodiment. It is also possible to add.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. Various changes or improvements can be added to the above-described embodiment without departing from the gist of the invention, and embodiments to which such changes or improvements are added are also included in the technical scope of the present invention.

  Further, the above embodiments do not limit the invention according to the claims (claims), and all combinations of features described in the embodiments are not necessarily essential to the solution means of the invention. Absent. The embodiments described above include inventions at various stages, and various inventions can be extracted by appropriately combining a plurality of disclosed constituent elements. Even if some constituent requirements are deleted from all the constituent requirements shown in the embodiment, as long as an effect is obtained, a configuration from which these some constituent requirements are deleted can be extracted as an invention.

  For example, a column processing unit 26 that processes pixel signals for normal images and an arithmetic processing unit 27 that performs arithmetic processing different from normal image processing are provided independently, and vertical scanning circuits 14a and 14b that perform vertical scanning with respect to each other. The first horizontal scanning circuits 12a and 12b that perform horizontal scanning (a drive control unit that collectively drives and controls the image sensor) need only be provided independently, and various applications can be considered as long as this is provided. Some examples are listed below.

  (Application 1) On the side of the arithmetic processing unit 27, if three-dimensional measurement is performed using a light cutting method system that applies a triangulation method and a detection method of slit light, based on distance information, Therefore, it is possible to perform control for extracting (extracting) an image within a specific distance range. Thereby, for example, only the image of the object in front is cut out from the sensor, and the background image portion is erased. With this function, it is possible to extract only a person who has a previous conversation or a person who is a conversation target in image communication in a mobile phone, a portable terminal (PDA), a TV phone, a TV conference, or the like. This can be used as a function for concealing private matters by deleting information such as the location of the person to be conversed. Further, if the image to be transmitted is limited to the cut out image, it can also be used as a function for reducing the amount of transmitted image information.

  It is also possible to perform a process of using another background instead of the deleted background, that is, replacing an image within a specific distance range with another image. That is, by using a landscape image acquired separately as the background, it is possible to replace the background with personal preference. At this time, if the distance information between the clipped object and the background is used, the images can be easily superimposed. The specific image extraction processing can be used as preprocessing for image recognition and object recognition processing. For example, in the face recognition processing, scanning that extracts the image of the face portion from the background is usually required as a step before the face portion recognition scanning. In general, since this is performed using only image information, it is not easy because it takes a long processing time. However, by using the above-described system, it is possible to easily perform face segmentation processing and the like.

  (Application Example 2) Two images can be synthesized according to distance information using two systems of the light cutting method. That is, control is performed such as displaying the image that is always in front of the two images. For example, as a simulation of a virtual space, it is possible to virtually arrange an ornament placed on a table in a room on another screen. Alternatively, it is possible to acquire in real time an image such as a person moving virtually in a room with a person and hiding in the shadow of an object. These can be used for interior layout simulation and interaction (mutual battle type) games.

  In addition, it is possible to construct a user interface by reflecting a manually created video with a keyboard and various buttons on the screen. In addition, depending on the application, it is possible to perform scanning such as displaying an image in the rear instead of displaying only the image in the foreground, and scanning such that an originally invisible image can be seen. In addition, not only two systems but also three or more images can be combined. In addition to real-time combining processing, control using a plurality of recorded images and recorded images and real-time are also possible. Variations such as synthesis with the acquired image are possible.

  Application Example 3 Using a light cutting method system, image information and distance information can be used for scanning feedback control of various devices and robots. For example, in remote control scanning such as telemedicine, it is possible to automatically control devices according to image distance information. Alternatively, when some device scan is performed on an object shown in an image, it is possible to take measures such as limiting the scanable space of the device so that the device does not touch the object. In addition, when installed in an autonomous robot or the like, the robot detects and stores the furniture arrangement information of the room, and the robot can create a map of the environmental information of the room. This can be used as basic data information when the robot moves and works in the room.

  (Application Example 4) By using a light cutting method system and analyzing distance information of an object on a time axis, it is possible to perform motion recognition and gesture recognition of the object. Since the three-dimensional distance information can be acquired in real time by using the optical cutting method system, by analyzing the position change of the object in the time axis direction and analyzing the characteristics of the movement, the human motion pattern (gesture ) Can be recognized. If this is used, for example, a user interface technique based on a gesture becomes possible. These can be used as user interfaces for personal computers, games, robots, various AV (Audio-Visual), IT (Information Technology) devices, and the like. In addition, this gesture recognition can be combined with information acquired from a normal image to further enhance the recognition target and the recognition effect.

  (Application example 5) Natural unevenness information can be used for object recognition, person recognition, or security purposes. For example, as a security application for performing person authentication, a person's face shape information is recognized and registered in advance, and when an unspecified person comes, it is pre-registered with unevenness information of the person's face. The unevenness information of the person is collated, and the person is specified depending on whether or not they match. At this time, in the sensor system using the optical cutting method system, normal images can be acquired at the same time. Therefore, as in Application Example 1, it can be used in combination with person authentication by image recognition. This is not limited to the face, and it is possible to perform authentication on various parts of the body.

  In addition, the sensor system using the optical cutting method system can perform motion detection and gesture recognition by analysis in the time axis direction as shown in Application Example 4. Therefore, this gesture can be used as personal authentication. Moreover, such authentication by unevenness can be used not only for a person but also for recognition of a normal object. Further, as described above, the unevenness information of each part of the subject is not accurately used as recognition and authentication data, but is analyzed as unevenness characteristics, for example, texture (texture) characteristics, and used as recognition and authentication data. It is also possible.

  (Application example 6) The system of the light cutting method can be used for vehicle rearward confirmation and vehicle exterior confirmation. For example, in the case of backward confirmation, driving scanning is performed while viewing a normal image from the sensor, and at the same time, the system measures the distance of the rear obstacle, and issues a warning when approaching the obstacle to a certain distance. Moreover, it is possible to apply to automatic control feedback of automobiles by setting signs with unevenness on general roads and reading them with a sensor system mounted on individual cars. For example, it is possible to construct a system in which an uneven sign is set on the side of a road, the car advances while reading this, and a warning is issued when the car is about to leave the road.

  (Application example 7) The light cutting system can be used in a car. For example, the presence / absence of a person sitting on the seat and the age of the person sitting are determined using a three-dimensional measurement function. This detection result can be fed back to seat belt warning, adjustment of the operation level of the airbag, and the like. Further, by using the gesture recognition function as in Application Example 2, it is possible to allow the driver to scan by gesture or hand gesture without touching a button or the like on the onboard device.

  (Application 8) The system of the light cutting method can be used for real-time three-dimensional modeling. Since a system using the light cutting method system can usually acquire an image and perform three-dimensional measurement almost simultaneously, it can perform three-dimensional modeling of an object and texture mapping by cutting and pasting an image. These are also possible in real time.

  (Application 9) The optical cutting system can be used for MPEG (Moving Picture Experts Group) 4 object extraction processing.

  (Application example 10) When a plurality of receivers connected by a network are installed in a place where the spatial position is known, the space of the transmitter imaged by two or more receivers by performing stereo matching The target position can be calculated. When performing stereo processing, it is necessary to synchronize in time with a plurality of receivers. By changing the blinking pattern (optical beacon) such as ID information transmitted from the transmitter incrementally with time, the time code in the transmission data can be included. It can be said that the moment when the same ID information is decoded is the same time. This makes it possible to easily synchronize among a plurality of receivers.

  (Application Example 11) In the above-described embodiment, the functional units that perform different signal processing based on the pixel signals output by the independent drive control are provided in the subsequent stage of the pixel unit 10, but each signal processing is performed. The functional unit may perform the same signal processing. In this case, the same signal processing result is output from the function unit that performs each signal processing. For example, when the drive capability of a functional unit that performs one signal processing is insufficient, the output of the functional unit that performs the other signal processing can be used to compensate for the lack of drive.

It is a schematic block diagram of the CMOS solid-state imaging device (CMOS image sensor) which is one Embodiment of the semiconductor device which concerns on this invention. It is a figure which shows an example of the relationship between the effective image area | region in a pixel part, and the reference | standard pixel area | region which gives optical black. It is a figure which shows 1st Embodiment of the specific structural example of the column processing part and arithmetic processing part which paid its attention to 1 vertical column in the solid-state imaging device shown in FIG. 4 is a timing chart for explaining pixel signal readout timing in an image acquisition function and an ID detection function in the solid-state imaging device of the first embodiment shown in FIG. 3. It is a timing chart explaining the coping method at the time of competition occurrence by the 1st vertical scanning circuit and the 2nd vertical scanning circuit performing independent vertical scanning. It is a timing chart explaining the modification 1 of the image acquisition function and ID detection function in the solid-state imaging device of 1st Embodiment shown in FIG. It is a timing chart explaining the other modification 2 of the image acquisition function and ID detection function in the solid-state imaging device of 1st Embodiment shown in FIG. It is a figure which shows 2nd Embodiment of the specific structural example of the column processing part and arithmetic processing part which paid its attention to 1 vertical column in the solid-state imaging device shown in FIG. It is a timing chart explaining the timing of pixel signal reading in the image acquisition function and the ID detection function in the solid-state imaging device of the second embodiment shown in FIG. It is a figure which shows 3rd Embodiment of the specific structural example of the column processing part and arithmetic processing part which paid its attention to 1 vertical column in the solid-state imaging device shown in FIG. It is a timing chart explaining the timing of pixel signal reading in the image acquisition function and the ID detection function in the solid-state imaging device of the third embodiment shown in FIG. It is a figure which shows 4th Embodiment of the specific structural example of the column process part and arithmetic processing part which paid its attention to 1 vertical column in the solid-state imaging device shown in FIG. 13 is a timing chart for explaining pixel signal readout timings in an image acquisition function and an ID detection function in the solid-state imaging device of the fourth embodiment shown in FIG. 12. 13 is a timing chart for explaining a modification of the image acquisition function and the ID detection function in the solid-state imaging device according to the fourth embodiment shown in FIG. It is a figure which shows 5th Embodiment of the specific structural example of the column process part which paid its attention to 1 vertical column in a solid-state imaging device. 16 is a timing chart illustrating pixel signal readout timing in an image acquisition function in the solid-state imaging device according to the fifth embodiment illustrated in FIG. 15. It is a figure explaining the processing mode of a dynamic range expansion process. It is the figure which showed the outline | summary of the mechanism as described in a nonpatent literature 1. FIG. It is the top view which showed the whole outline | summary of the solid-state imaging device of patent document 2. FIG.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Solid-state imaging device, 3 ... Unit pixel, 7 ... Drive control part, 10 ... Pixel part, 12 ... Horizontal scanning circuit, 12a ... 1st horizontal scanning circuit (horizontal scanner for image acquisition), 12b ... 2nd horizontal scanning circuit (ID detection horizontal scanner), 14 ... vertical scanning circuit, 14a ... first vertical scanning circuit (image acquisition vertical scanner), 14b ... second vertical scanning circuit (ID detection vertical scanner), 15 ... row control line, DESCRIPTION OF SYMBOLS 18 ... Horizontal signal line, 19 ... Vertical signal line, 20 ... Communication / timing control part, 26 ... Column processing part, 27 ... Operation processing part, 28 ... Output circuit, 100 ... Usage signal acquisition part, 120 ... Range expansion processing part 272: Difference calculation unit, 274 ... Analog memory array unit, 275 ... Bias circuit unit, 276 ... Comparator unit, 278 ... Data latch unit, 400, 500 ... CDS and difference calculation processing unit

Claims (47)

A unit signal generation unit that outputs a unit signal that detects a change in physical quantity is included in the unit component, and a semiconductor device for detecting a physical quantity distribution in which the unit component is arranged in a predetermined order is used. A physical information acquisition method for acquiring physical information for a predetermined purpose based on the unit signal acquired under a predetermined detection condition,
A physical information acquisition method characterized by performing independent drive control on each unit component, and thereby performing independent signal processing based on each unit signal output from the unit component. . 2. The physical information acquisition method according to claim 1, wherein the independent drive control has different sweep processing speeds related to reading of the unit signal. The physical information acquisition method according to claim 2, wherein the respective sweep processes are performed synchronously. As the independent signal processing, based on the unit signals output from the unit components acquired under conditions where the speed of the sweep processing is different, the physical information having different processing unit times is simultaneously applied. It acquires. The physical information acquisition method of Claim 2 characterized by the above-mentioned. The physical information acquisition method according to claim 1, wherein the unit signals output from the unit components are read out in a time division manner through a common signal line from the unit components. The physical information acquisition method according to claim 1, wherein each unit signal output from the unit component is read out via an independent signal line from the unit component. The physical information acquisition according to claim 1, wherein one of the independent signal processes includes a process of taking a difference between the unit signals output at different times from the same unit component. Method. The difference between the unit signals is stored in a predetermined storage medium, and thereafter, a difference is further calculated between the difference stored in the storage medium and the newly acquired difference between the unit signals. The physical information acquisition method according to claim 7, further comprising: performing processing. The physical information acquisition method according to claim 8, wherein the unit signals output at different times are acquired under conditions under different detection times by the independent drive controls. . The physical information acquisition according to claim 8, wherein the other of the independent signal processes includes a process of taking a difference between the unit signals output at different times from the same unit component. Method. The physical information acquisition method according to claim 10, wherein the process of obtaining the difference between one of the independent signal processes and the other of the independent signal processes is performed in a common processing system. The physical information acquisition method according to claim 1, wherein drive control is performed so as to avoid competition between both sweep processes for the same unit component. One of the independent drive controls is drive control so that a unit signal in which a change in physical quantity is detected is held in the unit component even after the unit signal is output from the unit component. The physical information acquisition method according to claim 1, wherein: One of the independent drive controls performs drive control on the unit component so as to acquire a natural image,
2. The physical information according to claim 1, wherein the other of the independent drive controls performs drive control on the unit component so as to detect timing of intensity change of a blinking signal of a light source. Acquisition method. The speed of the sweep processing related to the reading of the unit signal by one of the independent drive controls is related to the reading of the unit signal by the other drive control of the independent drive controls. The physical information acquisition method according to claim 14, wherein the physical information acquisition method is slower than a speed of the sweep process. The physical information acquisition method according to claim 1, further comprising performing predetermined signal processing based on each output signal obtained by the independent signal processing. The physical information acquisition method according to claim 16, wherein the predetermined signal processing involves synthesis processing between the respective output signals. 18. The physical information acquisition method according to claim 17, wherein each of the output signals is acquired under a condition in which detection periods by the independent drive control are different from each other. Based on each output signal obtained by the independent signal processing, an image signal is output, and a moving object detection process, a distance measurement process, an image filter process, an image smoothing process, and an image edge detection are performed. The physical information acquisition method according to claim 1, wherein at least one of the processes is performed. A unit signal generation unit that outputs a unit signal that detects a change in physical quantity is included in the unit component, and a semiconductor device for detecting a physical quantity distribution in which the unit component is arranged in a predetermined order is used. A physical information acquisition device that acquires physical information for a predetermined purpose based on the unit signal acquired under a predetermined detection condition,
A first drive control unit that performs first drive control on the unit component;
A first signal processing unit that performs signal processing based on the unit signal output from the unit component under the control of the first drive control unit;
A second drive control unit for performing second drive control on the unit component;
A second signal processing unit that performs signal processing based on the unit signal output from the unit component under the control of the second drive control unit;
A physical information acquisition apparatus comprising: The physical information acquisition apparatus according to claim 20, wherein the first signal processing unit and the second signal processing unit perform different signal processing. The physical information acquisition according to claim 20, wherein the first drive control unit and the second drive control unit have different sweep processing speeds related to reading of the unit signal. apparatus. The physical information acquisition apparatus according to claim 22, wherein the first drive control unit and the second drive control unit perform the respective sweep processes in synchronization. The first signal processing unit and the second signal processing unit are configured to perform processing based on the unit signals output from the unit components acquired under conditions where the speed of the sweep processing is different. The physical information acquisition apparatus according to claim 22, wherein the physical information having different unit times is acquired simultaneously. The first drive control unit and the second drive control unit read each unit signal output from the unit component in a time division manner via a common signal line from the unit component. 21. The physical information acquisition apparatus according to claim 20, wherein drive control is performed as described above. The first drive control unit and the second drive control unit are driven so as to read out the respective unit signals output from the unit component via independent signal lines from the unit component. 21. The physical information acquisition apparatus according to claim 20, wherein control is performed. The said 2nd signal processing part has a difference information acquisition part which performs the process which takes the difference between the said unit signals output at the different time from the said same unit component, respectively. Physical information acquisition device. The second signal processing unit includes a storage medium for storing a difference between the unit signals;
A second difference information acquisition unit that performs a process of obtaining a difference between the difference stored in the storage medium and a difference between the newly acquired unit signals. Item 27. The physical information acquisition apparatus according to Item 27. The first drive control unit and the second drive control unit perform drive control such that the unit signals output at the different times are acquired under different conditions of the detection times. 29. The physical information acquisition apparatus according to claim 28. The said 1st signal processing part has a 3rd difference information acquisition part which performs the process which takes the difference between the said unit signals each output at the different time from the said same unit component, It is characterized by the above-mentioned. 28. The physical information acquisition apparatus according to 28.   The physical information acquisition apparatus according to claim 30, wherein the first difference information acquisition unit and the third difference information acquisition unit have a common processing system. 21. The drive control unit according to claim 20, wherein the first drive control unit and the second drive control unit perform drive control so as to avoid competition of both sweep processes for the same unit component. The physical information acquisition device described. One of the first drive control unit and the second drive control unit is configured so that a unit signal in which a change in physical quantity is detected is detected in the unit component even after the unit signal is output from the unit component. The physical information acquisition apparatus according to claim 20, wherein drive control is performed so that The second signal processing unit includes a bias processing unit that provides an offset component to the unit signal output from the unit component or a processed signal corresponding to the unit signal. The physical information acquisition device described in 1. The first drive control unit performs drive control on the unit component so as to acquire a natural image,
The physical information acquisition apparatus according to claim 20, wherein the second drive control unit performs drive control on the unit component so as to detect timing of intensity change of a blinking signal of a light source. The first drive control unit and the second drive control unit are configured such that a speed of a sweep process related to reading of the unit signal by the first drive control unit is equal to the unit signal by the second drive control unit. 36. The physical information acquisition apparatus according to claim 35, wherein the physical information acquisition apparatus is set so as to be slower than a speed of a sweep process related to reading of. The first signal processing unit performs output processing of image information based on the unit signal output from the unit component,
The physical information acquisition apparatus according to claim 20, wherein the second signal processing unit performs arithmetic processing for executing a predetermined application using the unit signal output from the unit component. . 21. A third signal processing unit for further performing predetermined signal processing based on respective output signals obtained by the first signal processing unit and the second signal processing unit. The physical information acquisition device described in 1. The third signal processing unit includes a synthesis processing unit that performs synthesis processing between respective output signals obtained by the first signal processing unit and the second signal processing unit. Item 40. The physical information acquisition apparatus according to Item 38. In the first drive control unit and the second drive control unit, the output signals obtained by the first signal processing unit and the second signal processing unit have different detection periods. The physical information acquisition apparatus according to claim 38, wherein drive control is performed as described above. The second signal processing unit is a signal related to at least one of moving object detection processing, distance measurement processing, image filtering processing, image smoothing processing, and image edge detection processing as the predetermined application. 37. The physical information acquisition apparatus according to claim 36, wherein processing is performed.   21. The physical information acquisition apparatus according to claim 20, comprising the semiconductor device. A semiconductor device for detecting a physical quantity distribution including a unit signal generation unit that outputs a unit signal that detects a change in physical quantity in a unit constituent element, and the unit constituent elements are arranged in a predetermined order,
A semiconductor device comprising: individual signal lines capable of independently outputting the unit signals output from the same unit component. A semiconductor device for detecting a physical quantity distribution including a unit signal generation unit that outputs a unit signal that detects a change in physical quantity in a unit constituent element, and the unit constituent elements are arranged in a predetermined order,
A semiconductor device comprising a plurality of transfer units that transfer information indicating the detected change in the physical quantity to the unit signal generation unit. A semiconductor device for detecting a physical quantity distribution including a unit signal generation unit that outputs a unit signal that detects a change in physical quantity in a unit constituent element, and the unit constituent elements are arranged in a predetermined order,
A plurality of the unit signal generation units for detecting the same information indicating a change in the detected physical quantity;
A semiconductor device comprising: individual signal lines capable of independently outputting the unit signals output from each of the plurality of unit signal generation units. A first transfer unit that transfers the same information indicating the detected change in the physical quantity to one of the plurality of unit signal generation units;
A second transfer unit that transfers the same information indicating the change in the physical quantity transferred to one of the unit signal generation units to the other of the unit signal generation units. 46. The semiconductor device according to claim 45. The semiconductor device according to claim 46, wherein the first transfer unit is configured to be simultaneously controllable with respect to each of the unit components.

Images