JP4466612B2 - Solid-state imaging device and control method thereof - Google Patents

Description

  The present invention relates to a solid-state imaging device having a calculation function for executing various applications in addition to a normal image signal acquisition function and a control method thereof.

In recent years, there has been proposed a solid-state imaging device that realizes high-speed image processing or the like by providing an image sensor with various calculation functions of image information.
As one of such image sensors, a sensor having a normal real image acquisition, a three-dimensional distance calculation function and a moving object detection function has been proposed (for example, ISSCC / 2001 / SESSION6 / CMOS IMAGE SENSORS). WITH EMBEDDED PROCESSORS / 6.4 (2001 IEEE International Solid-State Circults Conference), and Japanese Patent Application No. 2000-107723, etc.
In addition to the image acquisition circuit similar to a normal image sensor, these have a function to detect temporal changes in light intensity. The thing which gave these arithmetic functions for every pixel has already been reported.
Various image processing can be realized by using the arithmetic function of such an image sensor. The typical principle of three-dimensional measurement will be described below.

FIG. 6 is an explanatory diagram illustrating the principle of trigonometry and the method of detecting slit light for performing three-dimensional measurement.
First, as shown in FIG. 6 (A), in the trigonometry, the sensor 2 and the light source (Light source) 3 are arranged apart from the object (Object) 1 for distance measurement. The light projecting unit 3 irradiates the object 1 with slit light, and is provided with a scanner (Scanning mirror) 4 that reflects the slit light.
In such an arrangement state, the scanner 4 repeats the operation of scanning the slit light of the light projecting unit 3 from right to left, for example.
Then, as shown in FIG. 6B, while the slit light of the light projecting unit 3 scans once from the right to the left, in the sensor 2, thousands to tens of thousands of frame sweeps are performed. When each of the imaging pixels (pixels) 2 detects the reflected light from the object 1 of the slit light, it outputs data indicating it.

Here, when attention is paid to one pixel, the distance from the sensor 2 of the object 1 when the reflected light is detected and the swing angle of the scanner 4 are uniquely determined in the line-of-sight direction of the pixel. That is, when the scanning of the scanner 4 and the counting of the number of frames of the sensor 2 are started at the same time, the swing angle of the scanner 4 is determined by knowing at which frame the slit light is detected. The distance from the object 1 to the object 1 is determined.
However, in an actual image sensor, the above-mentioned frame count number and distance are calibrated with a distance calibration object in advance, and the data is stored as a table on the system side, so that accurate absolute distance measurement is possible. Is possible.

In the method using triangulation as described above, a function required for the image sensor includes high-sensitivity detection of the projection slit light.
Normally, infrared light is used as the projection wavelength, but because there is a difference in the reflectance of infrared light depending on the non-measurement object, in order to enable measurement even for objects with poor reflectance texture It is necessary to improve the accuracy of detecting the passage of slit light.
In the prior art document 1, in order to perform this high sensitivity detection, an optical signal is calculated for each pixel. Hereinafter, this architecture will be described.

FIG. 7 is a block diagram showing the overall configuration of the image sensor in the prior art documents 1 and 2, and FIG. 8 is a circuit diagram showing the internal structure of one pixel of the image sensor shown in FIG.
In FIG. 7, in the image pickup unit 10 that picks up an image of a subject, a large number of pixels 11 constituting each photosensor are arranged in a matrix, and each pixel 11 is selected to take out an image pickup signal of each column. Vertical signal lines 12 are provided.
Further, outside the imaging unit 10, a V scanner unit 13 that scans a pixel 11 from which an imaging signal is extracted through a selection line in a vertical direction, a signal generation unit 14 that outputs a control signal to the V scanner unit 13, and a vertical signal The output circuit unit 15 receives the output signals of the columns (Columns) # 1 to # 192 from the line 12, performs necessary signal processing, and outputs the image signals of the columns.

In FIG. 8, in each pixel 11, a photodiode (PD) 21 that receives light, an amplifying transistor (QA) 22 that supplies current according to the intensity of light, and a current mirror circuit that amplifies the current. 23, a current copier circuit (Frame memories) 24 for storing the current signal, a two-stage chopper comparator 26 for comparing the current from the current copier circuit 24, and a bias circuit (offset genertor) for applying a bias to the current 27 etc.
Among these, the unit for reading the signal charge from the PD 21 includes a floating diffusion (FD) 31 part for extracting the signal charge from the PD 21, a transfer transistor 32 for transferring the signal charge of the PD 21 to the FD part 31, and A reset transistor 34 for resetting the FD unit 31, the above-described amplification transistor 22 that amplifies the signal charge of the FD unit 31 by converting it into a voltage signal, and the above-described current mirror circuit that amplifies the output current of the amplification transistor 22 23 and a switch (SA) 33 for controlling the output timing.
Further, the current copier circuit 24 has four circuits (M1 to M4) set in parallel, each of which functions as a frame memory, and can store a total of four frames of optical signals.

FIG. 9 is a timing chart showing the distance measuring operation in such an image sensor.
During one scan period when the laser scans once, operations of several thousand to several tens of thousands of frames are performed in the image sensor. Here, one scan period is usually 33 msec in accordance with the video frame rate when the distance information is displayed on the monitor.
In the following description, in order to distinguish from the video frame rate, scanning for one screen inside the image sensor is referred to as a sensor frame.

At the start of the sensor frame (1 Frame in FIG. 9), all the pixels in the FD unit 31, that is, amplification transistors, store the charges accumulated by the optical signal by the reset signal (RST) and the charge transfer signal (TX) of the FD unit 31. (QA) 22 is transferred to the gate.
Thereafter, pixels are selected line by line in the row direction of the image sensor, and a signal storing operation (φ1) and a reading operation (φ2, φ3) to the current copier circuit 24 are performed.
Here, in the storage operation φ1, the detection signal is stored in one frame memory, but the frame memory to be stored is changed every time the frame changes (Frame index: A, B, C, D).
Then, in the read operations φ2 and φ3, the memory of the previous two frames and the memory of the subsequent two frames are added and compared by the comparator 26,
f (k) + f (k-1)-(f (k-2) + f (k-3))-(Iz-Ic)
(Formula 1)
Enables the operation of
Here, the last Iz and Ic indicate bias currents by the bias circuit 27, which correspond to currents in the periods of φ2 and φ3, respectively, and are normally set to (Iz−Ic)> 0.

In the above equation, when the light projection slit is not detected, there is no time difference in the light detection light intensity of each pixel, so the calculation up to the fourth term of Equation 1 is 0, and only the bias portion remains and is negative. Value. If this is determined by the output of the comparator 26, it becomes an output of Low data.
On the other hand, when the projection slit light passes through the pixel, there is always a time region in which the added data of the first two frames is larger than the latter data (FIG. 6C), which is the set bias (Iz−). If Ic) is exceeded, the calculation result of Equation 1 becomes a positive value. That is, when attention is paid to a certain pixel, the data of the comparator 26 is normally low, but becomes high data when the slit light passes.
Therefore, for each pixel, if the frame count number at which the data of the comparator 26 becomes High is recorded on the system side, the distance to each point is determined by triangulation from the relationship between the count number and the angle of the projector. Can be measured uniquely.

On the other hand, in the image sensor described above, it is also possible to output a normal image by performing A / D conversion processing within the pixel.
At this time, the reference signal is stored in one of the frame memories M1 to M4, and thereafter, for each sensor frame scan, the optical signal charge is integrated and accumulated in the FD unit 31 in the pixel, The data is stored in the frame memories M1 to M4, and the comparator 26 compares the reference level with the size.
In the case of a pixel with high light intensity, the reference level is exceeded by charge accumulation by a small number of frame scans, but when the intensity is low, many frame operations are required. Therefore, as in the distance measurement, if the frame count number at which the data of the comparator 26 is inverted is stored on the system side, the count number corresponds to the actual image and can be displayed on the monitor.

However, in the image sensor as described above, since the arithmetic circuit is held for each pixel, the pixel size becomes large, and it is difficult to reduce the size of the sensor and increase the number of pixels.
In addition, since the circuit scale increases and the power consumption of the chip increases, for example, in the conventional document 1, the power consumption is 1 W or more.
Such an image sensor has sufficient space for placement and can be used in a relatively large system having a large power capability. However, low power consumption, low cost, and a large number of pixels of an actual image are required. It is not suitable for applications such as for consumers.
Further, for consumers, real images tend to require high-quality color images, as with normal imagers.

  The present invention has been made in view of such a situation, and an object of the present invention is to perform a normal real image acquisition function and a calculation function to execute various applications without complicating the configuration in each pixel. A solid-state imaging device and a control method that can realize both downsizing, low power consumption, low cost, multiple pixels (higher image quality) of an actual image, and the like. It is in.

In order to achieve the above object, the present invention provides a plurality of light receiving units that respectively constitute imaging pixels, a plurality of photoelectric conversion units that convert light received by the light receiving unit into electrical signals, and the plurality of photoelectric conversion units. A first signal processing unit comprising a signal line having a plurality of signal transmission paths for taking out the electric signal converted by the first signal processing unit for processing the electric signal transmitted in the first signal transmission direction by the signal line; A second signal processing unit that processes an electrical signal transmitted in the second signal transmission direction by the signal line, and performs different signal processing in the first signal processing unit and the second signal processing unit. A solid-state imaging device that includes: a light source for irradiating a subject; and a citation mirror that sweeps and irradiates the subject with light emitted from the light source. Irradiated The light reflected is detected by the light receiving portion Te, said by arithmetic processing by the second signal processing unit, the subject was determined by light section method, the output of the object image in the first signal processing unit It is characterized by performing processing .

According to the solid-state imaging device of the present invention, the image signal obtained by the imaging pixel is transmitted by the signal line in the first and second signal transmission directions, and the first and second signal processing units perform different signal processing. By doing so, for example, normal image signal output and other various arithmetic processes can be performed by separate circuits.
Therefore, circuit elements necessary for each signal processing are collectively arranged outside the pixel, the configuration inside each pixel can be minimized, and simplification can be achieved. Arithmetic functions that execute various applications can be strengthened by independent circuit configurations, realizing downsizing of the device, low power consumption, low cost, multiple pixels of real images (high image quality), etc. It becomes possible to do.

Next, embodiments of the present invention will be described with reference to the drawings.
The solid-state imaging device (hereinafter referred to as an image sensor or simply a sensor) according to the present embodiment shares an arithmetic circuit held for each pixel of the image sensor described in the conventional example for each column (Column = column). In addition, the image output processing and the arithmetic processing are completely separated by separate circuit blocks (first and second signal processing units), thereby achieving high image quality of the actual image and the arithmetic processing. In addition, an optimum design is possible.
Note that the image output process and the arithmetic process are selected by a switching signal from the outside of the sensor, and control is performed so that part or all of the operation of each circuit block is stopped when each process is not operating.
A sensor image output scanning method and an image processing operation method (here applied to distance measurement as an example here) will be described below according to a specific embodiment of the present invention.

FIG. 1 is a plan view showing the overall configuration of the image sensor according to the first embodiment of the present invention.
As shown in FIG. 1, this image sensor includes a pixel array unit 41, a pixel V scanner unit 42, a pixel H scanner unit 43, an IV conversion circuit unit 44, a CDS circuit unit 45, a current mirror circuit unit 46, an analog The memory array unit 47, the memory V scanner unit 48, the memory H scanner unit 49, the comparator unit 50, the bias circuit unit 51, the data latch unit 52, the output data bus unit 53, and the like.
The pixel array unit 41 includes a plurality of pixels 411 that detect light arranged in a two-dimensional matrix in the matrix direction, and a signal transmitted from each pixel 411 is a signal line (vertical signal line) that runs in the vertical direction. 54.

The pixel V scanner unit 42 and the pixel H scanner unit 43 scan the inside of the pixel array unit 41 in the vertical direction and the horizontal direction, and select one pixel 411.
The IV conversion circuit unit 44 converts a current output from each pixel 411 to the horizontal signal line 55 during normal image output scanning into a voltage signal. The CDS circuit unit 45 includes an IV conversion circuit unit. The output signal from 44 is subjected to a predetermined noise removal process and output as an image signal.
The current mirror circuit unit 46 amplifies the output current from each pixel 411 during arithmetic processing, and the analog memory array unit 47 temporarily stores the output from the current mirror circuit unit 46 in the current copier cell. is there.

The memory V scanner unit 48 and the memory H scanner unit 49 scan the current copier cell of the analog memory array unit 47 and take out data in the cell. The bias circuit unit 51 includes the analog memory array unit 47. A bias current is supplied to the current copier cell.
The comparator unit 50 performs a comparison operation on the data read from the analog memory array unit 47, and the data latch unit 52 latches the operation data of the comparator unit 50, and the latched data is output to the output data bus unit 53. More output.

  The vertical signal line 54 includes a switch SW1 for opening and closing the pixel array unit 41 and the horizontal signal line 55, a switch SW2 for opening and closing the pixel array unit 41 and the current mirror circuit unit 46, a current mirror circuit unit 46 and an analog memory. A switch SW3 for opening / closing the array unit 47 and a switch SW4 for opening / closing the analog memory array unit 47 and the bias circuit unit 51 are provided.

In the configuration as described above, during normal image output scanning, each pixel 411 of the pixel array unit 41 is sequentially scanned by the pixel H scanner unit and the pixel V scanner unit, and a specific one pixel 411 is selected. The sent current signal is transmitted in the upward direction (first signal transmission direction) in the figure, the switch SW1 is sequentially selected by the pixel H scanner unit 43, and is transferred to the horizontal signal line for each pixel signal. Thereafter, the signal is converted into a voltage signal by the IV conversion circuit unit 44, and further, FPN (Fixed pattern Noise) and reset noise (ktc noise) are removed by the CDS circuit unit 45 and output as an image signal output.
The scanning procedure for normal image output as described above is basically known in the art (see, for example, ISSCC / 2000 / SESSION6 / CMOS IMAGE SENSORS WITH EMBEDDED PROCESSORS / 6.1 (2000 IEEE International Solid-State Circults Conference)). Details are omitted here.
Further, during such image output scanning, the pixel array unit 41 and the current mirror circuit unit 46 are cut off by the switch SW2 of the vertical signal line 54 being turned off.

On the other hand, at the time of distance measurement, the switch SW1 is turned off, the switches SW2 and SW3 are turned on, and the current signal is arranged in the lower direction (second signal transmission direction) of the vertical signal line 54 in the figure. Is transmitted to.
At this time, in the pixel array unit 41, all the pixels in the same Row direction are simultaneously selected in the pixel V scanner unit 42, and signals from the respective columns (columns) are simultaneously output in parallel (that is, column parallel). Behave).
The signal transmitted to the current mirror circuit unit 46 is then held in the analog memory array unit 47, and thereafter, the data contents of the respective frames are compared by the comparator unit 50. After latching, the data is output from the output data bus unit 53.

FIG. 2 is a block diagram showing an internal configuration of each pixel 411 constituting the pixel array unit 41. FIG. 2 shows four pixels corresponding to four complementary colors Y, G, Cy, and Mg.
Each pixel 411 includes a photodiode (PD) 60 and five MOS transistors 61 to 65.
The light received by the PD 60 is converted into electric charges, and the electric charges are transferred to the floating diffusion (FD) unit 66 by the transfer transistor 61. The charges transferred to the FD unit 66 determine the gate potential of the amplification transistor 64, and accordingly, current is transmitted to the vertical signal line (SIG_n) 54 through the amplification transistor 64 and the selection transistor 65.
The reset transistor 63 is for resetting the FD section 66 to the power supply voltage, and the transfer selection transistor 62 is for selecting the gate of the transfer transistor 61.

Hereinafter, signal timings at the time of image output and distance measurement will be described.
FIG. 3 is a block diagram showing details of each block shown in FIG. 1, and FIG. 4 is a timing chart showing operations at the time of image output. FIG. 5 is a timing chart showing the operation during distance measurement.
First, the operation at the time of image output will be described based on FIG.
Here, all the switches SW2 below the vertical signal line 54 are turned off by the TSSW signal (Low; inactive), and the signal path is cut off.
First, when the specific row is selected by the pixel V scanner unit 42, the select line (SEL_n) for the 1H period becomes High (active), and the selection transistor 65 is turned on.

Next, a reset pulse is applied to the reset line (TDR_n−1), and the FD portion 66 is charged to the voltage power supply by the reset transistor 63. At this time, the signal in the reset state is transmitted to the signal line (SIG_n).
Further, the transfer gate selection line (TRG_n) connected to the gate of the transfer selection transistor 62 becomes High (active) in conjunction with the pixel H scanner unit 43, and at the same time, the transfer connected to the drain of the transfer selection transistor 62. The selection line (TDR_n) also becomes High (active), the transfer transistor 61 is selectively turned on only in a specific pixel, and the charge in the PD 60 is transferred to the FD unit 66.
Note that after the transfer of each pixel is completed, the transfer selection line (TDR_n) is set to Low (inactive) and then the transfer gate selection line (TRG_n) is set to Low (inactive).

By such an operation, a signal current corresponding to the light intensity is sent to the signal line (SIG_n). The current signal is converted into a voltage signal by the IV conversion circuit unit 44 via the horizontal signal line 55, and then noise is removed by the CDS circuit unit 45.
In the CDS circuit unit 45, the reset signal and the image signal are sequentially transferred, and the respective signals are clamped by the SHR pulse and the SHD pulse to remove noise.

Next, the operation during distance measurement will be described based on FIG.
As described above, the three-dimensional measurement method is similar to the conventional example in that triangulation is performed by detecting the passage of the projection slit for each pixel.
Further, at the time of scanning, unlike the image measurement (that is, image output), the switch SW1 above the vertical signal line 54 is turned off, and the connection between the signal line SIG and the horizontal signal line 55 is cut off.
Further, the video frame period (1 Scan period) and the sensor frame period (1 Frame) are the same as in the conventional example, and one line period (1 Row access period) is different from the conventional example.
FIG. 5 shows scanning in this one line period. Hereinafter, the scanning in the 1H period will be described.

In scanning at the time of distance measurement, four pixels constituting a color filter (for example, Y, G, Cy, Mg in the case of a complementary color mosaic filter) in the pixel array unit 41 are regarded as one light receiving pixel unit, and signals from the four pixels are received. Addition (fusion) is performed. That is, a signal for 4 pixels is handled as a signal for 1 pixel.
The purpose of this is to increase the sensitivity to compensate for the fact that the frame rate of the operation is about 100 times faster and the light reception time for each pixel is shorter when measuring the distance, as shown below. At the same time, when receiving the infrared slit reflected light during distance measurement, the difference in transmittance due to each filter is compensated.
Hereinafter, a unit in which these four pixels are regarded as one is expressed as ROU (Range operating unit) (FIG. 3).

In the distance measurement, since the columns are column-parallel operation that is read out in a lump, all the transfer gate selection lines TRG in the pixel array unit 41 always remain High (active).
In addition, in order to perform the addition (fusion) of the four pixels, the pixel V scanner unit 42 is swept so that two rows are selected simultaneously, unlike the time of image measurement.
That is, in FIG. 3, SEL_n and SEL_n + 1 are selected simultaneously. By selecting two specific rows, all the pixel selection transistors 65 in the two rows are turned on.
Thereafter, pulses are sequentially applied to TDR_n−1, TDR_n, and TDR_n + 1.

Here, the TDR line is connected to both the drain of the transfer selection transistor 62 in the pixel and the gate of the reset transistor 63 in the next line, and simultaneously with the charge transfer from the PD 60, the next row is reset. Is to do. That is, in this example, the transfer gate selection line and the reset line are shared by the TDR line.
Therefore, first, the pixel in the (n−1) th row is reset by applying the pulse of TDR_n−1, and the charge transfer of the pixel in the nth row is performed by applying the pulse of the TDR_n line, and at the same time, the pixel in the (n + 1) th row is reset. .
Then, charge transfer of pixels in the (n + 1) th row is performed by applying a pulse of TDRn + 1. At the same time, the next (n + 2) reset is performed at the same time, but this has no particular meaning (that is, no influence) in a series of scans.
As described above, in this example, when image reading is performed by simultaneous selection of a plurality of rows, first, the selection lines (SEL_n and SEL_n + 1) of the plurality of rows are activated and then transferred from the younger rows in the selection order of the selected rows. Active pulses are sequentially applied to the shared selection lines (TDR_n−1, TDR_n, TDR_n + 1) of the selection line and the reset line.

As a result of the above scanning, the charges due to light reception are simultaneously transferred to the FD portions 66 of the pixels connected in two rows, and the signal currents from the amplification transistors 64 of the two pixels simultaneously flow into the respective signal lines. Thereby, first, signal addition of 2 pixels is performed.
This current flows into the current mirror circuit 461 simultaneously with the turning on of the CMOS switch SW2 below the signal line. At this time, the odd and even columns are connected in front of the current mirror circuit 461, and the current from the two columns flows to one current mirror circuit 461.

In combination with the above-described simultaneous selection of two rows, signal addition of 4 pixels is completed. This current is amplified by the current mirror circuit 461, and a current corresponding to the signal is drawn from the signal line SIM_ (n + 1) / 2 in the memory array 47.
The analog memory array unit 47 configured by current copier cells has four current copier cells 471 corresponding to the one ROU. These correspond to memories for 4 sensor frames.
The memory array unit 47 is scanned by the memory V scanner unit 48 in synchronization with the scanning of the pixel array unit 41.
Since the current copier cells for four frames are arranged in the same manner as the pixel ROU, when two rows of the pixel array unit 41 are simultaneously selected, two rows of the memory V scanner unit 48 are also simultaneously selected. This corresponds to selecting a unit of four frames.
Hereinafter, the unit of the memory array unit 47 is referred to as RMU (Range memory unit) (FIG. 3).

At the same time that the switches SW2 and SW3 are turned on by the TSSW signal, one line selection line CCS_n1 is set to High, and the selection transistor 73 of one frame memory in the RMU is turned on.
A current flows from the PMOS transistor 71 constituting the memory cell to the current mirror circuit unit 46. At this time, the memory switch transistor 72 in the cell is ON and the drain and gate of the PMOS transistor 71 are biased to the same potential, so that the gate potential is at a potential determined according to the current flowing in the cell. Will be retained.
When the transistor 72 is turned off by the CCM_n1 signal after stabilizing in a steady state in this state, the gate and drain of the transistor 71 are separated, and the previous gate potential is dynamically stored and held in the gate of the transistor 71.
The above signal storage phase is assumed to be φ1.

At the subsequent timing, signal comparison between frames is performed in order to detect passage of the projection slit.
In the signal comparison between frames, as in the conventional example, the signal addition of two frames before and after is subtracted in time, and the above-described calculation (Equation 1) is executed.
In this read operation, the switches SW2 and SW3 are turned off, and the switch SW4 below the memory array is turned on by the BISW signal. At the same time, the selection line for the two frames from which the RMU is read becomes High.
As a result, the current signal read from the memory cell flows into the load transistor 74, and the potential of the memory signal line (SGM) becomes a potential determined by the current capability of the PMOS transistor 71 (for two frames) of the memory cell and the load transistor 74. Stabilized.

In order to execute the above (Equation 1), for example, the memory cell selection lines CCS_n3 and CCS_n4 are first selected at the same time, thereby simultaneously reading the previous two frames. As a result, this signal potential is input to the input gate of the chopper comparator connected to the vertical signal line 54. Then, the transistor initialization switches TC1 and TC2 of the two-stage chopper comparator are sequentially turned ON, and the comparator is initialized with the signal potential of the previous two frames.
Subsequently, in order to read out the signals of the latter two frames, the selection lines CCS_n1 and CCS_n2 are simultaneously selected, and the current is sent to the signal lines as in the previous two frames to determine the signal potential.

At this time, since the comparator is initialized in the previous two frames, if the potential is higher than the previous two frames, the comparator output becomes High, and if it is smaller, Low is output. Thereby, (Equation 1) is executed and light detection becomes possible.
The reading of the first two frames and the reading of the second two frames are indicated by φ2 and φ3 on the timing chart.
At this time, a bias current is supplied from the bias circuit unit 51 in each of φ2 and φ3 for signal weighting. This is the same as Iz and Ic in the conventional example, and the signal from the analog memory array unit 47 can be arbitrarily weighted. As an example of the bias circuit unit 51, one using a source follower circuit of a PMOS transistor biased by a column-common current mirror is used.
In addition, the bias circuit unit 51 biases the potential of the signal line in the previous stage of the comparator to the power supply voltage by turning off the switch SW4 provided between the analog memory array unit 47 except for the read operation.

In FIG. 3, the selection signal of each switch of SW2, SW3, SW4 and the selection signal of the CCM in the memory array are logically ANDed with the ADRn signal running vertically.
This is because when the operations of φ1 to φ3 are performed all over the column at the same time, an increase in power consumption may be recognized. Depending on the degree, the burden of wiring in the element increases, and proper operation is possible. There is also a risk that it will not be obtained.
Therefore, in order to cope with such a case, a circuit block below SW2 is divided (this unit is referred to as a divided area here), and this divided area is scanned serially, thereby suppressing a intensive increase in power consumption. It is what you do.
In addition, in the timing shown in FIG. 5, the case where the whole is divided | segmented into four as a divided area is assumed, a divided area is selected with an ADRn signal, and (phi) 1-phi3 is each with respect to each divided area 1-4. Operations are performed from time to time.

As described above, in this example, when signals from the pixel array 41 are read in parallel and calculation processing is performed for each column, a calculation area is formed by a plurality of columns, and the calculation processing is performed for each calculation area. By performing serially, it is possible to obtain an appropriate arithmetic operation in consideration of the load.
Each calculation area is selected by an address line, and the current mirror switch before and after the memory switch line, the comparator switch, the signal transistor load transistor selection line, the comparator initialization line, and the data latch enable line are The operation is performed only when the area is selected by the address.

It should be noted that the above two operations of image measurement (image output) and distance measurement can be executed independently and continuously in different time zones, and every video frame, etc. It is also possible to operate alternately every arbitrary video frame.
At this time, since image information and distance information are alternately output by the image sensor, real-time image processing combining image information and distance information becomes possible.
These can be selected at any time, for example, by a mode switching signal MSL by an external input.

In the image sensor of this example, various image processing operations can be performed in addition to the distance measurement by the above-described distance measurement circuit architecture.
First, there is motion detection as an example.
This is a function that extracts only a moving object in the imaging screen by basically the same operation as the distance measurement. In distance measurement, a difference calculation between frames that are temporally changed is always performed. Therefore, if there is a moving object in the image, the timing at which the subsequent frame signal intensity becomes larger than the previous signal intensity in the inter-frame difference of the comparator occurs as in the detection of the projection slit light.
Therefore, by detecting this, it is possible to detect a moving object.
In the case of this motion detection, it is not necessary to scan the sensor frame at high speed as in the distance measurement, and it is possible to slow down the frame rate up to the video frame in view of detection sensitivity.

Further, image information can be digitally output by the circuit configuration described above. This is basically the same as the conventional image output method, and uses the analog memory array unit 47 and the comparator unit 50.
In the above-described operation example, two lines are simultaneously read for adding four pixels at the time of distance measurement. However, in the digital image output using the analog memory array unit 47 and the comparator unit 50, scanning is performed for each line.
The switch SW2 is not turned on at the odd-numbered column and the even-numbered column at the same time, but is turned on separately according to the SEL_ODD signal and the SEL_EVEN signal. That is, the odd column reading and the even column reading are separately scanned in the 1H period.
In this case, two frame memories can be associated with one pixel in the read operation of each column. Therefore, if one frame memory is used to hold the reference signal and the other is used to hold the image signal, the reference signal and the image signal are sequentially compared by the integrated charge accumulation of the conventional example. As a result, an image signal can be acquired.

As another method for outputting an image, similarly to the above, the reference signal is read out during the 1H period to initialize the comparator unit 50, and then the image signal is read out, sent to the comparator unit 50, and biased. By ramping the current, it is possible to detect at which level the digital data is inverted and to extract image information.
Furthermore, the edge of an image can be detected by circuit modification.
This means that one comparator input is not limited to only one column as described above, but by selecting a switch, it is also possible to input from a neighboring column, thereby enabling comparison of the size of signals of neighboring pixels. To do.
As a result, it is possible to extract only a part having a large signal change, that is, to detect an edge in the image information.
It is also possible to perform various filter processes and smoothing processes using filters.

  As described above, in the image sensor of this example, the arithmetic circuit held for each pixel of the image sensor described in the conventional example is shared for each column, and the image output processing and arithmetic processing are separate circuits. By completely separating each block, the configuration within each pixel can be simplified, the entire device can be downsized and the quality of the actual image can be improved, and the design can be optimized for arithmetic processing. It becomes.

For example, it is possible to perform optimum processing by changing the order of sweeping each pixel in normal image output and arithmetic processing.
Further, as the order of pixel sweeping, serial processing in units of one pixel or a small number of pixel blocks and parallel processing by a plurality of signal lines can be used properly. For example, serial processing can be performed on a pixel-by-pixel basis during image information output processing, and parallel processing using a plurality of signal lines can be performed during other arithmetic processing, enabling optimization according to the characteristics of each signal processing. Become.
In addition, the number of pixels simultaneously transmitted to one signal line can be changed. A signal for each pixel is transmitted to the signal line at the time of image information output processing, and a signal of a plurality of pixels is simultaneously set to 1 at other arithmetic processing. By transmitting to the signal line of the book, optimization according to the characteristics of each signal processing becomes possible.
Further, by using the number of pixels corresponding to the combination of the color filters as the number of pixels simultaneously transmitted to one signal line at the time of calculation processing, high-precision calculation can be performed.

In the above-described example, the transfer selection line arranged in a certain row and the reset line wired in the next row can be shared by the TDR line, so that the wiring space can be reduced and the device can be miniaturized.
In the example described above, a switch is provided for each column or columns of the pixel array, a switch that is turned on at the time of signal readout is selected from these switches, and a column that is input to each current mirror circuit is selected. This is unique to arithmetic processing by combining and adding signals from multiple pixels through simultaneous selection of multiple rows in a pixel array or simultaneous selection of multiple columns, and handling multiple pixels as a single light-receiving pixel unit. This method can be adopted and can be optimized.

In the above example, when the analog memory array unit 47 compares the addition results of combinations of two or more frames among a plurality of frames corresponding to one light receiving pixel unit, the memory cell corresponding to each frame is a signal line. The cells are arranged in the form of a matrix, and the cells are always selected from the cells arranged on the counter electrode with the signal line in between. In addition, in the pixel array 41 and the analog memory array unit 47, a plurality of row selections at the time of selecting one light receiving pixel unit by the pixel array scanner and a plurality of row selections at the time of unit selection of a plurality of frames by the analog memory array scanner are synchronized. .
These enable highly accurate and efficient signal processing.

Next, a second embodiment of the present invention will be described.
First, the solid-state imaging device according to the first embodiment described above realizes a function of three-dimensional distance measurement based on normal color image output and a light cutting method. Can be realized by the conventional configuration described with reference to FIG.
That is, a light source that emits slit-shaped light and a sweeping mirror are arranged in the vicinity of the sensor (light receiving unit), and the subject is irradiated with slit light through the mirror while scanning the sweep mirror. Based on the relationship between the timing at which the pixel receives the reflected slit light from the subject and the scan angle of the mirror, it is possible to acquire distance information of each point of the subject by the principle of triangulation.

However, in this case, many parts such as an optical device for generating a light source and slit light outside the sensor, and a drive system for performing a scanning operation with a mirror are required, and miniaturization, power saving, etc. are required. Not easy. Further, an optical device for generating a slit-like light source is also a cylindrical lens, and it is necessary to satisfy the optical conditions, and it is not easy to reduce the size.
Therefore, in the second embodiment of the present invention, in the above-described three-dimensional distance measurement system based on the light cutting method, the mirror unit for performing the scanning operation is configured by a MEMS (Micro Electro Mechanical System) mirror, and a simple and compact system is realized. In addition to realizing this, it is intended to reduce power consumption.

FIG. 10 is a perspective view showing a configuration example of a MEMS mirror used in the three-dimensional distance measurement system according to the second embodiment of the present invention.
The MEMS mirror shown in FIG. 10 is an electromagnetically driven scanner-type mirror created on a silicon substrate (see, for example, “Hiroshi Miyajima, Journal of Microelectromechanical systems Vol. 10, No. 3 2001 p418-p424”).
In this MEMS mirror, a movable plate (mirror body) 120 having a mirror surface formed on the surface of a silicon substrate is attached to a metal base 121. A fixed piece 120B is formed on both sides of the movable plate 120 via a hinge part 120A, and the fixed piece 120B is joined to the metal base 121 to rotate using the flexibility and elasticity of the hinge part 120A. It is possible.

In the metal base 121, a sensor coil 122 and a drive coil 123 are disposed on the back side of the movable plate 120, and a magnet 124 and a yoke 125 are disposed with the movable plate 120 interposed therebetween.
Then, the drive coil 123 is energized via the flat cable 126 and the like, and the displacement of the movable plate 120 is detected by the detection signal of the sensor coil 122, and the current value to the drive coil 123 is controlled, so that the Lorentz force and the hinge The movable plate (mirror surface) 120 can be scanned using the balance of the torsional stress of the part.
It should be noted that the vibration frequency of the mirror, the swing angle of the mirror, and the start and stop of the operation can be controlled by external control.

In the example shown in FIG. 10, the galvanometric drive mode for statically controlling the swing angle of the mirror surface by adjusting the amount of current, and the resonance operation by synchronizing the vibration operation of the mirror surface and an external control signal. Thus, it is possible to select two drive modes of the resonant drive mode in which the mirror surface scan operation is performed.
In this MEMS mirror, the hinge portion 120A of the movable plate 120 is formed of a polyimide thin film, so that a low-frequency scanning operation that is difficult to realize with a normal silicon hinge is also possible.
Conventionally, the scanning mirror used in the light cutting method has a problem that the drive part such as a galvano motor and the mirror are individually formed, so that the size as a component is large and the power consumption is large. Therefore, in this example, as shown in FIG. 10, the entire system of the light cutting method can be downsized by using a MEMS mirror in which the mirror and the drive unit are integrally formed.

In this example, as an example of the MEMS mirror, an example of using an electromagnetically driven scanner has been shown. However, as another driving method, the thermal expansion of the hinge material due to current conduction to the mirror hinge portion is used. It is also possible to use a device that performs a scanning operation or a device that forms a hinge from a laminated film and performs a scanning operation using the difference in thermal expansion coefficient of each film.
Furthermore, if the mirror and the drive unit are integrally formed on the same substrate, the optical cutting method system can be similarly reduced in size.
Furthermore, since the MEMS mirror is generally formed on a semiconductor substrate, it is also possible to integrally form a light source such as a laser or an LED for irradiating the scan mirror on the same substrate as the MEMS mirror.

Next, a third embodiment of the present invention will be described.
FIG. 11 is an explanatory diagram showing two configuration examples of the three-dimensional distance measurement system according to the third embodiment of the present invention.
In the light cutting method system of this example, the laser hologram 100 is used as a light projecting unit for obtaining slit-like light.
The laser hologram has been commercialized and used as an AF light source for a digital still camera, for example. As shown in FIG. 11, the laser hologram is arranged in the light emission path from the laser light source 101 so that the laser light is slit-shaped light. And is supplied to the object 102.

Then, the reflected light from the object 102 is photographed by the sensor 110 described in the first embodiment through the lens 111, thereby performing three-dimensional distance measurement.
In this example, laser light is scanned by a mirror scanner (scanner type mirror) 103 similar to the example shown in FIG. 6A, but the MEMS mirror shown in FIG. 10 can also be used. is there.
By using such a laser hologram 100, it is possible to generate slit light only by adding a small optical system (hologram element) to the laser light source 101. Moreover, since the hologram element is formed of an inexpensive plastic substrate, it is advantageous in terms of cost.

Further, the laser hologram 100 may be arranged between the laser light source 101 and the mirror scanner 103 as shown in FIG. 11A, or as shown in FIG. Placed after the mirror scanner.
However, when the laser hologram 100 is arranged at the subsequent stage of the mirror scanner 103, the spot light emitted from the laser light source 101 is scanned by the mirror and then spread in a slit shape.

In addition, as a fourth embodiment of the present invention for simplifying the light source, a sensor similar to that in the first embodiment described above is used, and a light source such as an LED is used as the light source, and a mirror is used. By performing a difference calculation between the imaging frame when the LED is turned on and the imaging frame when the LED is turned off without performing scan scanning, an object in front of the background can be recognized. it can.
In this configuration, an approximate distance measurement can be performed by controlling the intensity of the LED light.

Next, an application example to the above-described embodiment will be described.
By using the imaging system by the solid-state imaging device according to each embodiment described above, it is possible to realize functions such as image processing and image recognition that are conventionally difficult to realize in various IT devices as described below. it can.
(Application 1)
In this example, using the light cutting method system according to the first to fourth embodiments described above, control is performed to extract an image within a specific distance range from an image based on distance information. .
Thereby, for example, only the image of the object in front is cut out from the sensor, and the background image portion is erased. With this function, it is possible to extract only a person who has a previous conversation or a person who is a conversation target in image communication in a mobile phone, a portable terminal (PDA), a TV phone, a TV conference, or the like. This can be used as a function for concealing private matters by deleting information such as the location of the person to be conversed. Further, if the image to be transmitted is limited to the cut out image, it can also be used as a function for reducing the amount of transmitted image information.

Further, in the above function, another background can be used instead of the deleted background. That is, by using a landscape image acquired separately as the background, it is possible to replace the background with personal preference. At this time, if the distance information between the clipped object and the background is used, the images can be easily superimposed.
The specific image extraction processing can be used as preprocessing for image recognition and object recognition processing. For example, in the face recognition process, an operation for extracting an image of the face part from the background is usually required as a process before the face part recognition operation. In general, since this is performed using only image information, it is not easy because it takes a long processing time. However, by using the above-described system, it is possible to easily perform face segmentation processing and the like.
In addition, although this example can also implement | achieve the same function using the LED irradiation system demonstrated in 4th Embodiment, since distance accuracy worsens with an LED irradiation system, it responded to distance information. Fine control is difficult.

(Application example 2)
In this example, two images are synthesized according to distance information by using two systems of the light cutting method shown in the first to fourth embodiments described above. That is, control is performed such as displaying the image that is always in front of the two images.
For example, as a simulation of a virtual space, it is possible to virtually arrange an ornament placed on a table in a room on another screen. Alternatively, it is possible to acquire in real time an image such as a person moving virtually in a room with a person and hiding in the shadow of an object. These can be used for interior layout simulation, interaction (mutual battle type) games, and the like.

In addition, it is possible to construct a user interface by reflecting a manually created video with a keyboard and various buttons on the screen.
Also, depending on the application, it is possible not only to display the image in the foreground, but also to display the image in the back, and to perform operations such as making an invisible one visible.
In addition, in this example, the system is not limited to two units, and it is possible to synthesize images with three or more units, not only real-time synthesis processing, but also control using a plurality of recorded images, Variations such as synthesis of recorded images and images acquired in real time are possible.

(Application 3)
In this example, image information and distance information are used for operation feedback control of various devices and robots using the optical cutting system shown in the first to fourth embodiments described above.
For example, in remote control operations such as telemedicine, it is possible to automatically control devices according to image distance information.
Alternatively, when any device operation is performed on the object shown in the video, it is possible to take measures such as limiting the operable space of the device so that the device does not touch the object.
In addition, when the above system is mounted on an autonomous robot or the like, and the robot detects and stores the arrangement information of the furniture in the room, the robot can create a map of the environmental information of the room. This can be used as basic data information when the robot moves and works in the room.

(Application 4)
In this example, the object motion recognition and gesture recognition are performed by analyzing the distance information of the object on the time axis using the optical cutting method system shown in the above-described first to fourth embodiments.
Since the systems shown in the first to fourth embodiments described above can acquire three-dimensional distance information in real time, the position change of the object is analyzed in the time axis direction, and the characteristics of the movement are analyzed. This makes it possible to recognize a human movement pattern (gesture).
If this is utilized, for example, a user interface technique based on gestures can be realized. These can be used as user interfaces for personal computers (PCs), games, robots, various AVs, IT devices, and the like.
In addition, this gesture recognition can be combined with information acquired from a normal image to further enhance the recognition target and the recognition effect.

(Application example 5)
In this example, natural unevenness information is used for object recognition, person recognition, or security.
For example, for security purposes to perform person authentication, human face shape information is recognized and registered in advance, and then when an unspecified person comes, information on the unevenness of that person's face and the person who has been registered in advance The unevenness information is collated, and a person is identified by whether or not they match. At this time, in the sensor systems of the first to fourth embodiments, normal images can be acquired at the same time, so that it can be used in combination with person authentication by image recognition as in Application Example 1. is there.
In addition, it is possible to perform authentication not only on the face but also on various parts of the body.

The sensor systems of the first to fourth embodiments can also perform motion detection and gesture recognition by analysis in the time axis direction as shown in the application example 4. Therefore, this gesture can be used as personal authentication.
Moreover, such authentication by unevenness can be used not only for a person but also for recognition of a normal object.
Further, as described above, the unevenness information of each part of the subject is not accurately used as recognition and authentication data, but is analyzed as unevenness characteristics, for example, texture (texture) characteristics, and used as recognition and authentication data. It is also possible.

(Application example 6)
In this example, the system of the light cutting method shown in the first to fourth embodiments described above is used for vehicle rearward confirmation and vehicle exterior confirmation.
For example, in the case of backward confirmation, a driving operation is performed while viewing a normal image from a sensor, and at the same time, the system measures the distance of the rear obstacle, and issues a warning when approaching the obstacle to a certain distance.
Moreover, it is possible to apply to automatic control feedback of a car by setting a sign with unevenness on a general road and reading it with the sensor system mounted on an individual car. For example, it is possible to construct a system in which an uneven sign is set on the side of a road, and the car proceeds while reading it, and a warning is issued when the car is likely to be off the road.

(Application example 7)
In this example, the optical cutting system shown in the first to fourth embodiments is used in a car.
For example, the presence / absence of a person sitting on the seat and the age of the person sitting are determined using a three-dimensional measurement function. This detection result can be fed back to seat belt warning, adjustment of the operation level of the airbag, and the like.
In addition, the gesture recognition function as in Application Example 2 is used so that the driver can operate the device mounted in the vehicle by gesture or hand gesture without touching a button or the like.

(Application 8)
In this example, the optical cutting system shown in the first to fourth embodiments is used for real-time three-dimensional modeling.
Since the system according to the first embodiment can acquire a normal image and perform three-dimensional measurement almost simultaneously, it can perform three-dimensional modeling of an object and texture mapping by cutting and pasting an image. These are also possible in real time.
(Application example 9)
As an application example 9, the optical cutting system shown in the first to fourth embodiments can be used for MPEG4 object extraction processing.

As described above, according to the solid-state imaging device of the present invention, the image signal obtained by the imaging pixel is transmitted to the first and second signal transmission paths through the signal line, and the first and second signal processing units. By performing signal processing different from each other, for example, normal image signal output and other various arithmetic processes can be performed by individual circuits.
According to the control method of the present invention, the image signal obtained by the imaging pixel is transmitted to the first and second signal transmission paths through the signal line, and the signal processing different from each other in the first and second signal processing steps. Thus, for example, normal image signal output and other various arithmetic processes can be performed by separate circuits.
Therefore, circuit elements necessary for each signal processing are collectively arranged outside the pixel, the configuration inside each pixel can be minimized, and simplification can be achieved. Arithmetic functions that execute various applications can be strengthened by independent circuit configurations, realizing downsizing of the device, low power consumption, low cost, multiple pixels of real images (high image quality), etc. It becomes possible to do.

1 is a plan view showing an overall configuration of an image sensor according to an embodiment of the present invention. FIG. 2 is a block diagram illustrating an internal configuration of each pixel of a pixel array unit in the image sensor illustrated in FIG. 1. It is a block diagram which shows the detail of each block of the image sensor shown in FIG. 2 is a timing chart illustrating an operation of the image sensor illustrated in FIG. 1 when outputting an image. It is a timing chart which shows the operation | movement at the time of the distance measurement of the image sensor shown in FIG. It is explanatory drawing which shows the principle of the trigonometry for performing three-dimensional distance measurement, and the detection method of slit light. It is a block diagram which shows the whole structure of the conventional image sensor. It is a circuit diagram which shows the internal structure of the pixel of the image sensor shown in FIG. It is a timing chart which shows the distance measurement operation | movement in the image sensor shown in FIG. It is a perspective view which shows the structural example of the MEMS mirror used for the three-dimensional distance measuring system by the 2nd Example of this invention. It is explanatory drawing which shows the structural example of the three-dimensional distance measurement system by the 3rd Example of this invention.

Explanation of symbols

41... Pixel array unit 42. Pixel V scanner unit 43 43 Pixel H scanner unit 44. IV conversion circuit unit 45. CDS circuit unit 46. ... Analog memory array part, 48 ... Memory V scanner part, 49 ... Memory H scanner part, 50 ... Comparator part, 51 ... Bias circuit part, 52 ... Data latch part, 53 ... Output data bus part, 54 …… Vertical signal line, 55 …… Horizontal signal line, 60 …… Photodiode, 61 …… Transfer transistor, 62 …… Transfer selection transistor, 63 …… Reset transistor, 64 …… Amplification transistor, 65 …… Select transistor , 66 …… Floating diffusion part.

Claims (34)

A plurality of light-receiving units that respectively constitute imaging pixels, a plurality of photoelectric conversion units that convert light received by the light-receiving unit into electric signals, and a plurality of signal transmissions that extract electric signals converted by the plurality of photoelectric conversion units A signal line having a path;
A first signal processor for processing an electric signal transmitted in the first signal transmission direction by the signal line; and a second signal for processing an electric signal transmitted in the second signal transmission direction by the signal line. A solid-state imaging device that performs different signal processing in the first signal processing unit and the second signal processing unit,
A light source for irradiating the subject, and a citation mirror for sweeping and irradiating the light emitted from the light source to the subject, and the light reflected by the illuminating object by the light source and the citation mirror The object is detected by a light-cutting method by being detected by a light receiving unit and processed by the second signal processing unit, and the first signal processing unit performs output processing of the image of the subject.
Solid-state image sensor. The light source that shines out of the slit-like light
The solid-state imaging device 請 Motomeko 1 wherein. The sweep mirror is Der Ru obtained by forming a mirror driver on a movable plate which is formed of a semiconductor or other materials comprising a mirror body
The solid-state imaging device 請 Motomeko 1 wherein. The sweep mirror has a base for holding the movable plate to be capable of rocking displacement, the movable plate is attached to the base through the hinge portion, you swings by the deformation of the hinge portion
The solid-state imaging device 請 Motomeko 1 wherein. The sweeping mirror includes a magnetic circuit that applies a magnetic field to the movable plate held by the base, and the mirror driving unit drives the movable plate by Lorentz force when energized in the magnetic field. a driving coil including
請 Motomeko 4 solid-state imaging device according. That the mirror drive unit having a detection coil for detecting the displacement of the movable plate in said magnetic field
The solid-state imaging device 請 Motomeko 5 wherein. The mirror drive unit, by energizing the hinge portion of the movable plate, drive the movable plate by using a thermal expansion of the hinge material
請 Motomeko 4 solid-state imaging device according. The mirror drive unit, by energizing the hinge portion of the movable plate, drive the movable plate by utilizing a difference in thermal expansion of the laminated film constituting the hinge portion
請 Motomeko 4 solid-state imaging device according. The mirror drive unit, that have a galvano drive mode and a resonant drive mode as the drive mode
請 Motomeko 4 solid-state imaging device according. Polyimide material was used for the hinge part
請 Motomeko 4 solid-state imaging device according. The light source and the sweeping mirror are provided on the same substrate .
The solid-state imaging device 請 Motomeko 1 wherein. Vibration frequency, deflection angle of the mirror of the mirror in the sweep mirror, the start of the operation, that having a means for controlling the external signal to stop
The solid-state imaging device 請 Motomeko 1 wherein. The light source is a laser light source, that having a laser hologram converts the laser beam into a slit-shaped light
The solid-state imaging device 請 Motomeko 1 wherein. Provided with the laser hologram between the light source and the sweep mirror
The solid-state imaging device 請 Motomeko 1 wherein. Provided with the laser hologram on the object side of the sweep mirror
The solid-state imaging device 請 Motomeko 1 wherein. Based on the distance information obtained by the arithmetic processing of the second signal processing unit, the second signal processing unit further performs processing for extracting an image within a specific distance range from the image of the subject. intends row
The solid-state imaging device 請 Motomeko 1 wherein. Based on the distance information obtained by the arithmetic processing of the second signal processing unit, the second signal processing unit further converts an image within a specific distance range from the subject image to another image. It intends line the process of replacing
The solid-state imaging device 請 Motomeko 1 wherein. The other images, Ru image der prepared by the operator by hand
The solid-state imaging device 請 Motomeko 17 wherein. The other images, Ru image der input from other media
The solid-state imaging device 請 Motomeko 17 wherein. The other images, Ru image der picked up by the solid-state imaging device
The solid-state imaging device 請 Motomeko 17 wherein. Said second signal processing section extracts a specific image, it recognizes an object by image recognition
The solid-state imaging device 請 Motomeko 16 wherein. Said second signal processing section extracts a specific image, we recognize an object by the feature extraction by temporal analysis of the positional information of the subject
The solid-state imaging device 請 Motomeko 16 wherein. It said second signal processing section extracts a specific image, you recognize an object by the feature extraction by the image recognition, and temporal analysis of positional information of the subject
The solid-state imaging device 請 Motomeko 16 wherein. The second signal processing unit extracts a specific image, recognizes the object by image recognition and feature extraction by temporal analysis of the position information of the object, and obtains the recognition result by other processes. by comparing the recognition result of the object, it recognizes an object
The solid-state imaging device 請 Motomeko 16 wherein. A plurality of light-receiving units that respectively constitute imaging pixels, a plurality of photoelectric conversion units that convert light received by the light-receiving unit into electric signals, and a plurality of signal transmissions that extract electric signals converted by the plurality of photoelectric conversion units A signal line having a path, and a first signal processing unit for processing an electric signal transmitted in the first signal transmission direction by the signal line; and a signal line transmitted in the second signal transmission direction by the signal line. In a control method of a solid-state imaging device having a second signal processing unit for processing a processed electric signal ,
A first signal processing step for processing an electric signal transmitted in the first signal transmission direction by the signal line; and a second for processing the electric signal transmitted in the second signal transmission direction by the signal line. Signal processing steps, and performing different signal processing in the first signal processing step and the second signal processing step ,
In the previous SL second signal processing step, by the light emitted from the light source sweeps irradiated to an object, the reflected light from the object is detected by the light receiving unit, for processing the detection signal, the Measure the subject by light cutting ,
In the first signal processing step, output processing of the image of the subject is performed.
The method of the solid-state imaging device. Based on the distance information obtained by the calculation process of the second signal processing step, the second signal processing unit performs a process of extracting an image within a specific distance range from the subject image. U
Control method of a solid-state imaging device 請 Motomeko 25 wherein. Based on the distance information obtained by the calculation processing of the second signal processing step, the second signal processing unit replaces an image within a specific distance range with another image from the subject image. It intends line processing
Control method of a solid-state imaging device 請 Motomeko 25 wherein. The other images, Ru image der prepared by the operator by hand
Control method of a solid-state imaging device 請 Motomeko 27 wherein. The other images, Ru image der input from other media
Control method of a solid-state imaging device 請 Motomeko 27 wherein. The other images, Ru image der picked up by the solid-state imaging device
Control method of a solid-state imaging device 請 Motomeko 27 wherein. The second signal processing unit extracts a specific image and recognizes the subject by image recognition.
Control method of a solid-state imaging device 請 Motomeko 26 wherein. In the second signal processing unit extracts a specific image, we recognize an object by the feature extraction by temporal analysis of the positional information of the subject
Control method of a solid-state imaging device 請 Motomeko 26 wherein. In the second signal processing unit extracts a specific image, we recognize an object by the feature extraction by the image recognition, and temporal analysis of positional information of the subject
Control method of a solid-state imaging device 請 Motomeko 26 wherein. The second signal processing unit extracts a specific image, recognizes the object by image recognition and feature extraction by temporal analysis of the position information of the object, and obtains the recognition result by other processing. Recognize the subject by comparing the recognition results of the subject
Control method of a solid-state imaging device 請 Motomeko 26 wherein.

Images