WO2020054184A1

WO2020054184A1 - Solid-state imaging element, imaging device, and solid-state imaging element control method

Info

Publication number: WO2020054184A1
Application number: PCT/JP2019/026210
Authority: WO
Inventors: 拓永瀬; 亮太郎高田
Original assignee: ソニーセミコンダクタソリューションズ株式会社
Priority date: 2018-09-12
Filing date: 2019-07-02
Publication date: 2020-03-19
Also published as: JP2020043504A

Abstract

The purpose of the present invention is to reduce the number of wirings in a CDS processing circuit of a solid-state imaging element performing CDS processing. A data output unit alternately and repeatedly outputs predetermined reset data and signal data corresponding to an exposure amount. One pair of reset data holding units alternately hold the repeatedly output reset data. A pixel data generation unit repeatedly performs processing for generating pixel data corresponding to a difference between the reset data held in one of the one pair of reset data holding units and signal data output subsequent to the reset data. One pair of pixel data holding units alternately hold the repeatedly generated pixel data. An output-side selection unit alternately selects one of the one pair of pixel data holding units and outputs the pixel data held in the selected pixel data holding unit.

Description

Solid-state imaging device, imaging device, and method of controlling solid-state imaging device

技術 The present technology relates to a solid-state imaging device, an imaging device, and a method for controlling a solid-state imaging device. More specifically, the present invention relates to a solid-state imaging device that generates a pixel signal having a difference between a reset level and a signal level, an imaging device, and a method for controlling the solid-state imaging device.

Conventionally, a correlated double sampling (CDS) process has been performed in a solid-state imaging device for the purpose of reducing fixed pattern noise and the like. A data processing unit that performs the CDS processing calculates a difference between a reset level at the time of initialization of a pixel and a signal level at the end of exposure of the pixel, and outputs the difference as a pixel signal with reduced fixed pattern noise. I do. For example, there has been proposed a solid-state imaging device in which an ADC (Analog to Digital Converter) is arranged for each pixel, and a reset level and a signal level from the ADC are transferred to a data processing unit to perform a CDS process (for example, Patent Reference 1).

International Publication No. WO 2016/136448

In the above-described solid-state imaging device, since the ADC is arranged for each pixel, AD (Analog to Digital) conversion can be performed simultaneously for all pixels. For this reason, in the above-described solid-state imaging device, an image signal can be read at a higher speed than in a case where an ADC is arranged for each column and AD conversion is performed for each column. However, in the data processing unit that performs the CDS processing, the number of signal lines for transmitting pixel signals may increase as the number of pixels increases.

The present technology has been developed in view of such a situation, and has an object to reduce the number of wirings in a circuit that performs CDS processing in a solid-state imaging device that performs CDS processing.

The present technology has been made to solve the above-described problems, and a first aspect of the present technology includes a data output unit that repeatedly and alternately outputs predetermined reset data and signal data corresponding to an exposure amount, A pair of reset data holding units for alternately holding the repeatedly output reset data; and a pair of the reset data held in one of the pair of reset data holding units and the signal data output after the reset data. A pixel data generating unit that repeats a process of generating a difference as pixel data, a pair of pixel data holding units that alternately holds the repeatedly generated pixel data, and a pair of the pixel data holding units that are alternately selected. A solid-state imaging device including an output-side selection unit that outputs the pixel data held in the selected pixel data holding unit, and control thereof It is the law. This brings about an effect that pixel data is repeatedly generated from the reset data and the signal data alternately held in the pair of reset data holding units.

In the first aspect, the data output unit may start outputting the reset data while outputting the signal data. This brings about an effect that the reading speed of the image data is increased.

Further, in the first aspect, an input-side selection unit for alternately selecting the pair of reset data holding units and supplying the reset data held in the selected reset data holding unit to the pixel data generation unit is provided. It may be further provided. This brings about an effect that pixel data is generated from reset data and signal data read from a selected one of the pair of reset data holding units.

Further, in the first aspect, the pixel data generation unit includes a pair of pixel data generation circuits, and one of the pair of pixel data generation circuits is held in one of the pair of reset data holding units. The difference between the reset data and the signal data output after the reset data is output as the pixel data to one of the pair of pixel data holding units. A difference between the reset data held in the other data holding unit and the signal data output after the reset data may be output to the other of the pair of pixel data holding units as the pixel data. This brings about an effect that the wiring distance between the reset data holding unit and the pixel data generation circuit is shortened.

Further, in the first aspect, a pixel for generating a predetermined reset level and a signal level corresponding to the exposure amount, a process of converting the reset level into a digital signal and holding the digital signal as the reset data, To a digital signal and holding the signal data. This brings about an effect that the CDS processing is performed on the AD-converted data for each pixel.

In the first aspect, the pixel is disposed on a predetermined light receiving substrate, and the analog-to-digital conversion unit, the data output unit, the pair of reset data holding units, the pixel data generation unit, and the pair of The pixel data holding unit and the output side selection unit may be arranged on a predetermined circuit board. This brings about an effect that the circuit scale per board is reduced as compared with the case where the circuit board is provided on a single board.

In the first aspect, the pixel is disposed on a predetermined light receiving substrate, the analog-to-digital converter is disposed on a first circuit substrate, and the data output unit and the pair of reset data holding units are provided. Part of the pixel data generation unit, the pair of pixel data holding units, and the output-side selection unit may be disposed on the first circuit board, and the rest may be disposed on a second circuit board. This brings about an effect that the circuit scale per board is reduced as compared with the case where the circuit board is provided on two boards.

According to a second aspect of the present technology, there is provided a data output unit that alternately and repeatedly outputs predetermined reset data and signal data corresponding to an exposure amount, and a pair of resets that alternately hold the repeatedly output reset data. A data holding unit, and a pixel data generation unit that repeats a process of generating a difference between the reset data held in one of the pair of reset data holding units and the signal data output after the reset data as pixel data as pixel data. A pair of pixel data holding units for alternately holding the repeatedly generated pixel data, and alternately selecting the pair of pixel data holding units and outputting the pixel data held in the selected pixel data holding unit An image pickup apparatus comprising: an output-side selector that performs the above-described pixel data processing; and a signal processor that processes the pixel data. This brings about an effect that pixel data is repeatedly generated and processed from reset data and signal data alternately held in the pair of reset data holding units.

In the first aspect, the data output unit, the pair of reset data holding units, the pixel data generation unit, the pair of pixel data holding units, and the output side selection unit are arranged in a solid-state imaging device. You may. This brings about an effect that the CDS processing is performed in the solid-state imaging device.

In the first aspect, the data output unit, the pair of reset data holding units, and the pixel data generation unit are arranged in a solid-state imaging device, and the pair of pixel data holding units and the output side selection unit are arranged. May be disposed outside the solid-state imaging device. This brings about the effect that the circuit scale in the solid-state imaging device is reduced.

FIG. 1 is a block diagram illustrating a configuration example of an imaging device according to a first embodiment of the present technology. FIG. 2 is a diagram illustrating an example of a stacked structure of the solid-state imaging device according to the first embodiment of the present technology. FIG. 2 is a plan view illustrating a configuration example of a light receiving substrate according to the first embodiment of the present technology. FIG. 2 is a block diagram illustrating a configuration example of a circuit board according to the first embodiment of the present technology. FIG. 2 is a block diagram illustrating a configuration example of a cluster according to the first embodiment of the present technology. FIG. 3 is a perspective view illustrating an example of a connection relationship between a pixel and a circuit in a cluster according to the first embodiment of the present technology. FIG. 2 is a circuit diagram illustrating a configuration example of a P-phase transfer unit according to the first embodiment of the present technology. FIG. 2 is a block diagram illustrating a configuration example of a data processing unit according to the first embodiment of the present technology. 5 is a timing chart illustrating an example of an operation of the solid-state imaging device according to the first embodiment of the present technology. FIG. 7 is a diagram for describing a circuit control method until a first CDS process is performed according to the first embodiment of the present technology. FIG. 7 is a diagram for describing a circuit control method when performing the second and third CDS processes according to the first embodiment of the present technology. FIG. 11 is a diagram for describing a control method in a comparative example having two frame memories according to the present technology. FIG. 11 is a diagram for describing a control method in a comparative example having three frame memories of the present technology. 5 is a flowchart illustrating an example of an operation of the solid-state imaging device according to the first embodiment of the present technology. FIG. 13 is a block diagram illustrating a configuration example of a data processing unit according to a second embodiment of the present technology. FIG. 11 is a diagram illustrating an example of a stacked structure of a solid-state imaging device according to a third embodiment of the present technology. FIG. 13 is a block diagram illustrating a configuration example of a data processing unit according to a third embodiment of the present technology. FIG. 21 is a block diagram illustrating a configuration example of a data processing unit according to a fourth embodiment of the present technology. FIG. 21 is a block diagram illustrating a configuration example of a DSP (Digital Signal Processing) circuit according to a fourth embodiment of the present technology. FIG. 2 is a block diagram illustrating a schematic configuration example of a vehicle control system. FIG. 4 is an explanatory diagram illustrating an example of an installation position of an imaging unit.

Hereinafter, a mode for implementing the present technology (hereinafter, referred to as an embodiment) will be described. The description will be made in the following order.
1. First Embodiment (an example in which data is alternately stored in a pair of frame memories)
2. 2. Second embodiment (an example in which data is alternately held in a pair of frame memories and a pair of CDS processing units are arranged)
3. 3. Third embodiment (an example in which data is alternately held in a pair of frame memories and a circuit for performing CDS processing is distributed and arranged on two substrates)
4. 4. Fourth embodiment (an example in which data is alternately held in a pair of frame memories and a part of the circuit is arranged outside the solid-state imaging device)
5. Example of application to moving objects

<1. First Embodiment>
[Configuration Example of Imaging Device]
FIG. 1 is a block diagram illustrating a configuration example of an imaging device 100 according to the first embodiment of the present technology. The imaging apparatus 100 is an apparatus for imaging image data, and includes an optical unit 110, a solid-state imaging device 200, and a DSP circuit 120. Further, the imaging device 100 includes a display unit 130, an operation unit 140, a bus 150, a frame memory 160, a storage unit 170, and a power supply unit 180. As the imaging apparatus 100, for example, in addition to a digital camera such as a digital still camera, a smartphone, a personal computer, an in-vehicle camera, or the like having an imaging function is assumed.

The optical unit 110 collects light from a subject and guides the light to the solid-state imaging device 200. The solid-state imaging device 200 generates image data by photoelectric conversion in synchronization with a vertical synchronization signal VSYNC. Here, the vertical synchronization signal VSYNC is a periodic signal of a predetermined frequency indicating the timing of imaging. The solid-state imaging device 200 supplies the generated image data to the DSP circuit 120 via the signal line 209.

The DSP circuit 120 performs predetermined signal processing on image data from the solid-state imaging device 200. The DSP circuit 120 outputs the processed image data to the frame memory 160 via the bus 150. Note that the DSP circuit 120 is an example of a signal processing unit described in the claims.

The display unit 130 displays image data. As the display unit 130, for example, a liquid crystal panel or an organic EL (Electro Luminescence) panel is assumed. The operation unit 140 generates an operation signal according to a user operation.

The bus 150 is a common path through which the optical unit 110, the solid-state imaging device 200, the DSP circuit 120, the display unit 130, the operation unit 140, the frame memory 160, the storage unit 170, and the power supply unit 180 exchange data with each other.

The frame memory 160 stores image data. The storage unit 170 stores various data such as image data. The power supply unit 180 supplies power to the solid-state imaging device 200, the DSP circuit 120, the display unit 130, and the like.

[Configuration example of solid-state imaging device]
FIG. 2 is a diagram illustrating an example of a stacked structure of the solid-state imaging device 200 according to the first embodiment of the present technology. The solid-state imaging device 200 includes a circuit board 202 and a light receiving substrate 201 laminated on the circuit board 202. These substrates are electrically connected via connection parts such as vias. Note that, in addition to vias, connection can also be made by inductive coupling communication technology such as Cu—Cu bonding, bumps, and TCI (ThruChip Interface).

FIG. 3 is a plan view illustrating a configuration example of the light receiving substrate 201 according to the first embodiment of the present technology. The pixel array unit 210 is provided on the light receiving substrate 201. In the pixel array section 210, a plurality of pixel blocks 211 are arranged in a two-dimensional lattice. A plurality of pixels 212 are arranged in each of the pixel blocks 211. For example, eight pixels 212 of 2 rows × 4 columns are arranged for each pixel block 211. Note that the number of pixels in the pixel block 211 is not limited to eight.

The pixel 212 generates an analog signal by photoelectric conversion. Here, the level of the analog signal generated when the pixel 212 is initialized is hereinafter referred to as a “reset level”, and the level of the analog signal according to the exposure amount generated at the end of the exposure of the pixel 212 is hereinafter referred to as “reset level”. Signal level.

FIG. 4 is a block diagram illustrating a configuration example of the circuit board 202 according to the first embodiment of the present technology. The circuit board 202 includes a DAC (Digital to Analog Converter) 220, a vertical drive circuit 230, a timing control circuit 240, a time code generator 250, an AD converter 260, and a data processor 270.

The DAC 220 generates an analog ramp signal RMP that changes in a slope shape by DA (Digital to Analog) conversion. The DAC 220 supplies the generated ramp signal RMP to the AD converter 260.

The AD converter 260 converts, for each pixel, an analog signal from the pixel into a digital signal. In the AD converter 260, the clusters 300 are arranged in a two-dimensional lattice. The cluster 300 is provided for each pixel block 211. If the number of pixel blocks 211 is N (N is an integer), N clusters 300 are also provided. The pixel block 211 and the cluster 300 are connected one-to-one.

Each of the clusters 300 converts an analog signal from the corresponding pixel block 211 into a digital signal for each pixel and supplies the digital signal to the data processing unit 270. A digital signal corresponding to the reset level of the analog signal is hereinafter referred to as “P-phase data”, and a digital signal corresponding to the signal level is hereinafter referred to as “D-phase data”. The P-phase data is an example of reset data described in the claims, and the D-phase data is an example of signal data described in the claims.

The vertical drive circuit 230 drives each of the clusters 300 under the control of the timing control circuit 240.

(4) The timing control circuit 240 controls the DAC 220, the vertical drive circuit 230, and the data processing unit 270 in synchronization with the vertical synchronization signal VSYNC.

(4) The time code generation unit 250 generates a time code. This time code indicates a time within a period during which the ramp signal changes in a slope shape. The time code generation unit 250 counts a count value, for example, in synchronization with a clock signal having a constant frequency, and generates data indicating the count value as a time code. The time code generation unit 250 supplies the generated time code to the AD conversion unit 260.

The data processing unit 270 executes a predetermined process including a CDS process on the P-phase data and the D-phase data from the AD conversion unit 260. The data processing unit 270 supplies the DSP circuit 120 with image data including the processed data.

[Example of cluster configuration]
FIG. 5 is a block diagram illustrating a configuration example of the cluster 300 according to the first embodiment of the present technology. The cluster 300 includes comparison circuits 311 to 318, data storage units 321 to 328, and data storage units 331 to 338. In the AD converter 260, a data transfer unit 261 is arranged for each column of the cluster 300. When the number of columns of the cluster 300 is M (M is an integer), M data transfer units 261 are also arranged. When the number of clusters 300 in a certain column is L (L is an integer), those L clusters 300 share one data transfer unit 261 corresponding to the column.

The data transfer unit 261 transfers data from the time code generation unit 250 to the cluster 300, and transfers data from the cluster 300 to the data processing unit 270. The data transfer unit 261 includes a P-phase transfer unit 262, a write data transfer unit 264, and a D-phase transfer unit 265.

$ P-phase transfer unit 262 transfers P-phase data from cluster 300 to data processing unit 270. The write data transfer unit 264 transfers the time code from the time code generation unit 250 to the cluster 300. The D-phase transfer unit 265 transfers the D-phase data from the cluster 300 to the data processing unit 270.

The comparison circuits 311 to 318 are connected one-to-one with the eight pixels 212 in the pixel block 211 corresponding to the cluster 300. For example, when the pixels 212 are arranged in a Bayer array, R (Red), Gb (Green), B (Blue), and Gr are placed on the left 2 rows x 2 columns of the 2 rows x 4 columns in the pixel block 211. (Green) pixels are arranged. Similarly, R, Gb, B, and Gr pixels are also arranged in the right 2 rows × 2 columns. In this arrangement, the left Gb pixel is connected to the comparison circuit 311, and the left B pixel is connected to the comparison circuit 312. The left R pixel is connected to the comparison circuit 313, and the left Gr pixel is connected to the comparison circuit 314. On the other hand, the right Gb pixel is connected to the comparison circuit 315, and the right B pixel is connected to the comparison circuit 316. The right R pixel is connected to the comparison circuit 317, and the right Gr pixel is connected to the comparison circuit 318.

{Circle around (4)} The ramp signals from the DAC 220 are input to the comparison circuits 311 to 318. Then, the comparison circuits 311 to 318 compare the analog signal from the connected pixel 212 with the ramp signal. Comparison circuit 311 outputs the comparison result to

data storage units

321 and 322, and comparison circuit 312 outputs the result to

data storage units

323 and 324. Comparison circuit 313 outputs the comparison result to

data storage units

325 and 326, and comparison circuit 314 outputs the result to

data storage units

327 and 328. On the other hand, comparison circuit 315 outputs the comparison result to

data storage units

331 and 332, and comparison circuit 316 outputs the result to data storage units 333 and 334. Comparison circuit 317 outputs the comparison result to

data storage units

335 and 336, and comparison circuit 318 outputs the result to

data storage units

337 and 338.

The data storage unit 321 holds P-phase data. The data storage unit 321 holds the time code from the write data transfer unit 264 as P-phase data at the timing when the comparison result is inverted after the output of the reset level. Then, the data storage unit 321 outputs the held P-phase data to the data processing unit 270 via the P-phase transfer unit 262. The configuration of the

data storage units

323, 325, 327, 331, 333, 335, and 337 is the same as that of the data storage unit 321.

The data storage unit 322 holds D-phase data. The data storage unit 322 holds the time code from the write data transfer unit 264 as D-phase data at the timing when the comparison result is inverted after the output of the signal level. Then, the data storage unit 322 outputs the held D-phase data to the data processing unit 270 via the D-phase transfer unit 265. The configuration of the

data storage units

324, 326, 328, 332, 334, 336, and 338 is the same as that of the data storage unit 322.

With the above configuration, the cluster 300 alternately converts the reset level and the signal level of each pixel from the pixel block 211 into digital signals. This AD conversion is repeatedly executed in all pixels in synchronization with the vertical synchronization signal VSYNC. Then, the data transfer unit 261 repeatedly and alternately outputs the P-phase data and the D-phase data to the data processing unit 270 in synchronization with the vertical synchronization signal VSYNC. The data transfer unit 261 is an example of a data output unit described in the claims.

The numbers of the comparison circuits and the data storage units in the cluster 300 are not limited to eight and sixteen, respectively. For example, when the number of pixels in the pixel block 211 is K (K is an integer), the number of comparison circuits in the cluster 300 is K, and the number of data storage units is K × 2.

FIG. 6 is a perspective view illustrating an example of a connection relationship between the pixel 212 and a circuit in the cluster 300 according to the first embodiment of the present technology.

座標 Let the coordinates of the pixel 212 in the i-th row and j-th column in the pixel block 211 be (i, j). Gb, B, R, and Gr pixels are arranged at coordinates (0, 0), (0, 1), (1, 0), and (1, 1) on the left side. Further, Gb pixels, B pixels, R pixels, and Gr pixels are also arranged at the coordinates (0, 2), (0, 3), (1, 2), and (1, 3) on the right side. Such an array is called a Bayer array. Note that the pixels can be arranged in an arrangement other than the Bayer arrangement. For example, R, G, B and W (White) pixels can be arranged.

Ｇ The left Gb pixel is connected to the comparison circuit 311, and the left B pixel is connected to the comparison circuit 312. The left R pixel is connected to the comparison circuit 313, and the left Gr pixel is connected to the comparison circuit 314. On the other hand, the right Gb pixel is connected to the comparison circuit 315, and the right B pixel is connected to the comparison circuit 316. The right R pixel is connected to the comparison circuit 317, and the right Gr pixel is connected to the comparison circuit 318.

データ Also, data storage units 321 to 328 are arranged in the left memory 320 in FIG. In the right memory 330, data storage units 331 to 338 are arranged.

[Configuration Example of P-Phase Transfer Unit]
FIG. 7 is a circuit diagram illustrating a configuration example of the P-phase transfer unit 262 according to the first embodiment of the present technology. The P-phase transfer unit 262 is provided with a shift register including a predetermined number of flip-flops 263. These flip-flops 263 are connected in series. Each of the data storage units holding the P-phase data, such as the left

data storage units

321, 323, 325, and 327, is connected to input terminals of flip-flops 263 different from each other by wired-OR connection. The shift register holds the P-phase data bit by bit in synchronization with the clock signal CLK and outputs the data to the data processing unit 270.

The configurations of the write data transfer unit 264 and the D-phase transfer unit 265 are the same as those of the P-phase transfer unit 262.

[Configuration example of data processing unit]
FIG. 8 is a block diagram illustrating a configuration example of the data processing unit 270 according to the first embodiment of the present technology. The data processing unit 270 includes

frame memories

271, 272, 275, and 276,

selectors

273 and 277, and a CDS processing unit 274. The AD converter 260 includes a clock supply unit 266 in addition to the cluster 300, the P-phase transfer unit 262, the write data transfer unit 264, and the D-phase transfer unit 265. Note that, in the figure, the cluster 300 is omitted.

The clock supply unit 266 supplies the clock signal CLK to each of the P-phase transfer unit 262, the write data transfer unit 264, and the D-phase transfer unit 265.

The

frame memories

271 and 272 alternately hold P-phase data repeatedly output from the P-phase transfer unit 262. The P-phase data from each of the M P-phase transfer units 262 is input to the frame memory 271 via the signal line 291. P-phase data from each of the M P-phase transfer units 262 is input to the frame memory 272 via a signal line 292.

(4) A control signal from the timing control circuit 240 is input to the

frame memories

271 and 272. This control signal is a signal for instructing writing or reading of data. When writing is instructed,

frame memories

271 and 272 fetch and hold P-phase data of all pixels. Each of these frame memories has such a capacity that it can hold at least one frame of P-phase data.

On the other hand, when reading is instructed,

frame memories

271 and 272 output the held P-phase data to selector 273 via

signal lines

293 and 294. The

frame memories

271 and 272 are an example of a pair of reset data holding units described in the claims.

The selector 273 alternately selects the

frame memories

271 and 272 in accordance with the timing control circuit 240 selection signal, and outputs the P-phase data from the selected memory to the CDS processing unit 274 via the signal line 296. Note that the selector 273 is an example of an input-side selection unit described in the claims.

The CDS processing unit 274 repeats the CDS process of generating the difference between the P-phase data from the selector 273 and the D-phase data from the M D-phase transfer units 265 as pixel data in synchronization with the vertical synchronization signal VSYNC. Is what you do. The CDS processing unit 274 supplies the generated pixel data to the

frame memories

275 and 276 via the

signal lines

297 and 298. The CDS processing unit 274 is an example of a pixel data generation unit described in the claims.

The

frame memories

275 and 276 alternately hold pixel data repeatedly generated by the CDS processing unit 274. The capacity of each of these frame memories is large enough to hold at least one frame of pixel data. Control signals from the timing control circuit 240 are also input to the

frame memories

275 and 276. When writing is instructed by the control signal, the

frame memories

275 and 276 fetch and hold the pixel data of all the pixels.

On the other hand, when reading is instructed,

frame memories

275 and 276 output the held pixel data to selector 277 via signal lines 299 and 299-1. The

frame memories

275 and 276 are examples of a pair of pixel data holding units described in the claims.

Here, data is output from the P-phase transfer unit 262 and the D-phase transfer unit 265 in cluster 300 units. For this reason, the

frame memories

275 and 276 hold pixel data in the order of output in units of clusters 300. However, in general image data, pixel data is arranged in raster (row) order. Therefore, the timing control circuit 240 transmits a control signal to the

frame memories

275 and 276 so that the reading order of the pixels is in the raster order. Thereby, the data processing unit 270 can rearrange and output the pixel data in the raster order.

The selector 277 alternately selects the

frame memories

275 and 275 in accordance with a selection signal from the timing control circuit 240, and outputs pixel data from the selected memory to the DSP circuit 120. Note that the selector 277 is an example of an output-side selection unit described in the claims.

Note that, in the figure, each of the signal lines 291 to 299-1 is represented by one line for convenience of description, but physically, not one, but a plurality of lines are wired. For example, a total of M signal lines are wired as signal lines 291, one from each of the M P-phase transfer units 262.

FIG. 9 is a timing chart illustrating an example of an operation of the solid-state imaging device 200 according to the first embodiment of the present technology. At timing T0, the vertical synchronization signal VSYNC falls, and after timing T0, the DAC 220 decreases the ramp signal RMP in a slope shape. On the other hand, the left memory 320 holds P-phase data based on the result of comparison between the ramp signal RMP and the reset level.

(4) The P-phase transfer unit 262 transfers the P-phase data to the frame memory 271 for a certain period after the timing T1, and the frame memory 271 holds the P-phase data. On the other hand, the DAC 220 starts supplying the ramp signal RMP immediately after the timing T1, and the left memory 320 retains the D-phase data based on the comparison result between the ramp signal RMP and the signal level.

Then, at timing T2, the vertical synchronizing signal VSYNC falls, and after timing T2, the DAC 220 lowers the ramp signal RMP like a slope. The left memory 320 holds the second P-phase data. On the other hand, immediately after the timing T2, the D-phase transfer unit 265 starts the first transfer of the D-phase data. The CDS processing unit 274 reads the first P-phase data from the frame memory 271 and obtains a CDS result (that is, pixel data) of a difference from the first D-phase data. The frame memory 275 holds the CDS result.

At the timing T3 during the transfer of the D-phase data, the P-phase transfer unit 262 starts the second transfer of the P-phase data, and the frame memory 272 holds the P-phase data. On the other hand, immediately after the timing T3, the DAC 220 starts supplying the ramp-shaped ramp signal RMP, and the left memory 320 holds the second D-phase data.

(4) Subsequently, at timing T4, the vertical synchronization signal VSYNC falls, and after timing T4, the DAC 220 decreases the ramp signal RMP in a slope shape. The left memory 320 holds the third P-phase data. On the other hand, immediately after the timing T4, the D-phase transfer unit 265 starts the second transfer of the D-phase data. Further, the CDS processing unit 274 reads the second P-phase data from the frame memory 275 and obtains a CDS result (that is, pixel data) of a difference from the second D-phase data. The frame memory 276 holds the CDS result. The first CDS result is read from the frame memory 275 in raster order, and output from the selector 277.

At the timing T5 during the second transfer of the D-phase data, the P-phase transfer unit 262 starts the third transfer of the P-phase data, and the frame memory 271 holds the P-phase data. On the other hand, immediately after the timing T5, the DAC 220 starts supplying the ramp-shaped ramp signal RMP, and the left memory 320 holds the third D-phase data. Hereinafter, the same read control is repeatedly executed.

As described above, the cluster 300 including the left memory 320 alternately and repeatedly holds the time code as P-phase data and D-phase data in synchronization with the vertical synchronization signal VSYNC. Also, the P-phase transfer unit 262 repeatedly transfers P-phase data in synchronization with the vertical synchronization signal VSYNC, while the D-phase transfer unit 265 repeats the D-phase data in synchronization with the vertical synchronization signal VSYNC. Forward.

Here, it takes a certain time to transfer the D-phase data for all pixels. On the other hand, since the time until the result of comparison with the reset level is inverted is relatively short, the time from the falling of the vertical synchronization signal VSYNC to the holding of the P-phase data is generally longer than the transfer period of the D-phase data. Is also shorter. Therefore, the P-phase data is held during the transfer of the D-phase data. For this reason, if the transfer of the P-phase data is started immediately after the P-phase data is held, a period occurs in which the P-phase data and the D-phase data are simultaneously transferred.

(4) The

frame memories

271 and 272 alternately hold the P-phase data output repeatedly. When one of the

frame memories

271 and 272 holds the second and subsequent P-phase data, the P-phase data is read from the other. The CDS processing unit 274 generates a difference between the read P-phase data and the subsequently transferred D-phase data as pixel data.

{Then, the

frame memories

275 and 276 alternately hold the repeatedly generated pixel data. When one of the

frame memories

275 and 276 holds the second and subsequent pixel data, the selector 277 reads and outputs the pixel data from the other in raster order.

FIG. 10 is a diagram for describing a circuit control method until the first CDS process is performed in the first embodiment of the present technology. In the figure, a shows the state of the data processing unit 270 when holding the first P-phase data, and b in the figure shows the state of the data processing unit 270 when performing the first CDS processing.

When the first P-phase data is transferred, the timing control circuit 240 writes the data in the frame memory 271 by a control signal as illustrated in a in FIG. Next, the first D-phase data is transferred as illustrated in FIG. Here, it is assumed that the second transfer of P-phase data is started during the first transfer of D-phase data.

At this time, the timing control circuit 240 writes the second P-phase data in the frame memory 272 by the control signal. The selector 273 selects the frame memory 271, reads out the first P-phase data from the memory, and supplies the data to the CDS processing unit 274. The CDS processing unit 274 generates the difference between the first P-phase data and the D-phase data as net pixel data. The timing control circuit 240 writes the first pixel data into the frame memory 275 according to the control signal.

FIG. 11 is a diagram for describing a circuit control method when performing the second and third CDS processes according to the first embodiment of the present technology. In the figure, a shows the state of the data processing unit 270 when performing the second CDS processing, and b in the figure shows the state of the data processing unit 270 when performing the third CDS processing.

(4) As illustrated in FIG. 3A, the second D-phase data is transferred. It is assumed that the third transfer of P-phase data is started during the second transfer of D-phase data. The timing control circuit 240 writes the third P-phase data in the frame memory 271 by the control signal. Further, the selector 273 selects the frame memory 272, reads the second P-phase data from the memory, and supplies it to the CDS processing unit 274. The CDS processing unit 274 generates a difference between the second P-phase data and the D-phase data as pixel data.

(4) The timing control circuit 240 writes the second pixel data to the frame memory 276 according to the control signal. On the other hand, the selector 277 selects the frame memory 275, reads out the first pixel data from the memory in raster order, and outputs the pixel data to the DSP circuit 120.

(4) Next, as exemplified by b in the figure, the third D-phase data is transferred. It is assumed that the fourth transfer of P-phase data is started during the third transfer of D-phase data. The timing control circuit 240 writes the fourth P-phase data in the frame memory 272 according to the control signal. Further, the selector 273 selects the frame memory 271, reads out the third P-phase data from the memory, and supplies it to the CDS processing unit 274. The CDS processing unit 274 generates a difference between the third P-phase data and the D-phase data as pixel data.

(4) The timing control circuit 240 writes the third pixel data in the frame memory 275 by the control signal. On the other hand, the selector 277 selects the frame memory 276, reads out the second pixel data from the memory in raster order, and outputs it to the DSP circuit 120.

As illustrated in FIGS. 10 and 11, while reading the P-phase data from one of the

frame memories

271 and 272 and performing the CDS processing, the timing control circuit 240 stores the next Can be written. Therefore, even when the transfer of the next P-phase data is started during the transfer of the D-phase data, the data processing unit 270 can generate the pixel data by the CDS processing. In this configuration, the transfer of the next P-phase data can be started during the transfer of the D-phase data, so that the reading speed (frame rate) of the image data (frame) can be increased.

{Circle around (2)}

Frame memories

271 and 272 are used only for writing P-phase data, and are used only for writing

frame memories

275 and 276. With this configuration, an increase in the number of wirings in the data processing unit 270 can be suppressed.

(4) In order to verify the effects of increasing the reading speed and suppressing the number of wires, a comparative example having two frame memories and a comparative example having three frame memories are assumed.

FIG. 12 is a diagram for explaining a control method in a comparative example having two frame memories according to the present technology. In the figure, a shows the state of the comparative example when the first P-phase data is held, and b in the figure shows the state of the comparative example when the first CDS processing is executed.

The selectors A, B, C, and D, the frame memories A and B, and the CDS processor are arranged in the data processor of this comparative example. The selector A selects one of the P-phase data and the pixel data and outputs it to the frame memory A, and the selector B selects one of the P-phase data and the pixel data and outputs it to the frame memory B. The selector C selects one of the frame memories A and B, reads out pixel data from the memory, and outputs the pixel data to a subsequent circuit. The selector D selects one of the frame memories A and B, reads pixel data from the memory, and outputs the pixel data to the CDS processing unit.

(4) When the first P-phase data is transferred in this comparative example, the selector A selects the P-phase data and supplies it to the frame memory A, as illustrated in FIG. P-phase data is written in the frame memory A.

Next, when the first D-phase data is transferred, the selector D reads the first P-phase data from the frame memory A as illustrated in FIG. The difference between the P-phase data and the D-phase data is output as pixel data. The selector B selects the pixel data and outputs it to the frame memory B. Pixel data is written to the frame memory B.

比較 In this comparative example, as illustrated in FIG. 13B, both the frame memories A and B are used during the first D-phase data transfer. For this reason, even if the second P-phase data is transferred during the first transfer of the D-phase data, there is no memory to write to, and the second and subsequent CDS processes cannot be executed. For this reason, in this comparative example, the period of the vertical synchronization signal VSYNC needs to be sufficiently long. Therefore, the reading speed (frame rate) of the image data is reduced as compared with the configuration of FIG.

FIG. 13 is a diagram for describing a control method in a comparative example having three frame memories of the present technology. In the figure, a shows the state of the comparative example when the first P-phase data is held, and b in the figure shows the state of the comparative example when the first CDS processing is executed.

The selectors A, B, C, D, and E, the frame memories A, B, and C, and the CDS processor are arranged in the data processor of this comparative example. The selector A selects one of the P-phase data and the pixel data and outputs it to the frame memory A, and the selector B selects one of the P-phase data and the pixel data and outputs it to the frame memory B. The selector C selects one of the P-phase data and the pixel data and outputs it to the frame memory C. The selector D selects one of the frame memories A, B, and C, reads pixel data from the memory, and outputs the pixel data to a subsequent circuit. The selector E selects one of the frame memories A, B, and C, reads pixel data from the memory, and outputs the pixel data to the CDS processing unit.

Next, when the first D-phase data is transferred, the selector E reads the first P-phase data from the frame memory A as illustrated in FIG. The difference between the P-phase data and the D-phase data is output as pixel data. The selector C selects the pixel data and outputs it to the frame memory C. Pixel data is written in the frame memory C. Here, when the transfer of the second P-phase data is started during the transfer of the first D-phase data, the selector B selects the P-phase data and outputs it to the frame memory B. The P-phase data is written into the frame memory B.

In this comparative example, as illustrated by b in the same figure, even if the transfer of the second P-phase data is started during the transfer of the first D-phase data, the frame memory B is empty. Can be written with P-phase data. Therefore, the reading speed of the image data is equivalent to the configuration of FIG.

However, in this comparative example, the number of wires in the data processing unit is increased as compared with the configuration of FIG. Since the number of wirings is proportional to the number of arrows indicating the data output destination, comparing the number of arrows, the number of the arrows is 11 in FIG. 11, but increases to 18 in the comparative example of FIG. ing.

As described above, in the comparative examples of FIGS. 12 and 13, the frame memory is not divided into a memory for writing only P-phase data and a memory for writing only pixel data. With this configuration, if the number of frame memories is two, it becomes difficult to improve the reading speed of image data. In addition, when three frame memories are used, the number of wirings increases.

In contrast, as illustrated in FIG. 11, the data processing unit 270 is provided with

frame memories

271 and 272 for writing only P-phase data and frame

memories

275 and 276 for writing only pixel data. According to this configuration, even if the transfer of the second P-phase data is started during the transfer of the first D-phase data, the data can be written to the empty one of the

frame memories

271 and 272. For this reason, the read speed of the image data can be improved as compared with the comparative example having two frame memories. In addition, since it is not necessary to arrange a selector at the preceding stage for each frame memory, the number of wirings can be reduced as compared with the comparative example having three frame memories. Further, the number of selectors can be reduced as compared with the comparative example. By reducing the number of wirings and the like, an increase in manufacturing cost can be suppressed.

[Operation example of solid-state imaging device]
FIG. 14 is a flowchart illustrating an example of an operation of the solid-state imaging device 200 according to the first embodiment of the present technology. This operation is started, for example, when a predetermined application for capturing image data is executed.

(4) The cluster 300 in the solid-state imaging device 200 generates P-phase data, and the data transfer unit 261 transfers the data (Step S901). The timing control circuit 240 writes P-phase data in one of the frame memories 271 and 272 (step S902). Next, the cluster 300 generates the D-phase data, and the data transfer unit 261 transfers the data (Step S903). The CDS processing unit 274 generates pixel data by a CDS operation (Step S904). The timing control circuit 240 writes the pixel data into one of the frame memories 275 and 276 (step S905), and the selector 277 reads out the pixel data from the other in raster order and outputs it (step S906). After step S906, the solid-state imaging device 200 repeatedly executes step S901 and subsequent steps in synchronization with the vertical synchronization signal VSYNC.

As described above, in the first embodiment of the present technology, the

frame memories

271 and 272 alternately hold the P-phase data, and the

frame memories

275 and 276 alternately hold the pixel data after the CDS processing. This eliminates the need to arrange a selector for selecting either the P-phase data or the pixel data for each frame memory, and reduces the number of wires as compared with a comparative example in which the selector is arranged for each frame memory. it can.

<2. Second Embodiment>
In the above-described first embodiment, the selector 273 is arranged between the

frame memories

271 and 272 and the CDS processing unit 274. However, in this configuration, the wiring distance between the

frame memories

271 and 272 and the CDS processing unit 274 is increased by the amount of the selector 273, and the propagation delay may be increased. The data processing unit 270 according to the second embodiment differs from the first embodiment in that the selector 273 is omitted.

FIG. 15 is a block diagram illustrating a configuration example of the data processing unit 270 according to the second embodiment of the present technology. The data processing unit 270 of the second embodiment differs from the first embodiment in that

CDS processing units

278 and 279 are provided instead of the selector 273 and the CDS processing unit 274.

The D-phase data from the D-phase transfer unit 265 is input to both the

CDS processing units

278 and 279. The CDS processing unit 278 reads out the P-phase data stored in the frame memory 271 and generates a difference from the subsequently transferred D-phase data as pixel data. On the other hand, the CDS processing unit 279 reads out the P-phase data held in the frame memory 272 and generates a difference from the subsequently transferred D-phase data as pixel data. Then, the CDS processing unit 278 outputs the pixel data to the frame memory 275, and the CDS processing unit 279 outputs the pixel data to the frame memory 276.

The

CDS processing units

278 and 279 are an example of a pair of pixel data generation circuits described in the claims.

As described above, in the second embodiment of the present technology, the CDS processing unit 278 reads the P-phase data from the frame memory 271 and the CDS processing unit 279 reads the pixel data from the frame memory 272. There is no need for a selector between the units. Thus, the wiring distance between the frame memory and the CDS processing unit can be reduced, and the propagation delay can be reduced.

<3. Third Embodiment>
In the above-described first embodiment, the circuits in the solid-state imaging device 200 are distributed and arranged on the light receiving substrate 201 and the circuit substrate 202. However, in this configuration, as the number of pixels increases, the scale of a circuit disposed on each of the substrates may increase. The solid-state imaging device 200 according to the third embodiment is different from the first embodiment in that circuits in the solid-state imaging device 200 are dispersed and arranged on three substrates.

FIG. 16 is a diagram illustrating an example of a stacked structure of the solid-state imaging device 200 according to the third embodiment of the present technology. The solid-state imaging device 200 according to the third embodiment includes an upper circuit board 203 and a lower circuit board 204 instead of the circuit board 202. The upper circuit board 203 is disposed between the light receiving board 201 and the lower circuit board 204. The upper circuit board 203 is an example of a first circuit board described in the claims, and the lower circuit board 204 is an example of a second circuit board described in the claims.

FIG. 17 is a block diagram illustrating a configuration example of the data processing unit 270 according to the third embodiment of the present technology. In the solid-state imaging device 200 according to the third embodiment, a part of the circuit in the data processing unit 270 is disposed on the upper circuit board 203 as the upper data processing unit 281, and the rest is formed as the lower data processing unit 282. It is arranged on the side circuit board 204.

For example, a circuit including the

frame memories

271 and 272, the selector 273, and the CDS processing unit 274 is provided on the upper circuit board 203 as the upper data processing unit 281. On the other hand, a circuit including the

frame memories

275 and 276 and the selector 277 is provided on the lower circuit board 204 as the lower data processing unit 282.

{Circle around (2)} In the second embodiment, the DAC 220, the vertical drive circuit 230, the AD converter 260, and the time code generator 250 are arranged on the upper circuit board 203 as in the first embodiment.

The

frame memories

271 and 272, the selector 273, and the CDS processing unit 274 in the data processing unit 270 are arranged on the upper circuit board 203, and the rest are arranged on the lower circuit board 204. The circuit to perform is not limited to this configuration.

As described above, in the solid-state imaging device 200 according to the third embodiment of the present technology, the circuits in the solid-state imaging device 200 are dispersed and arranged on three substrates. Thus, the circuit scale for each substrate can be reduced.

<4. Fourth Embodiment>
In the above-described first embodiment, the circuit for CDS processing and the circuit for converting the reading order of the pixels into the raster order are arranged in the solid-state imaging device 200. The circuit scale of the solid-state imaging device 200 may increase as the number increases. The imaging apparatus 100 according to the fourth embodiment is different from the first embodiment in that a circuit for converting the reading order of pixels into a raster order is arranged outside the solid-state imaging device 200.

FIG. 18 is a block diagram illustrating a configuration example of the data processing unit 270 according to the fourth embodiment of the present technology. The data processing unit 270 according to the fourth embodiment differs from the first embodiment in that the frame memories 275 and 256 and the selector 277 are not provided. The CDS processing unit 274 according to the fourth embodiment outputs pixel data to the DSP circuit 120.

FIG. 19 is a block diagram illustrating a configuration example of a DSP circuit 120 according to the fourth embodiment of the present technology. The DSP circuit 120 includes a timing control circuit 121, frame

memories

122 and 123, a selector 124, and a post-processing unit 125.

The configurations of the timing control circuit 121, the

frame memories

122 and 123, and the selector 124 are the same as those of the timing control circuit 240, the

frame memories

275 and 276, and the selector 277 of the first embodiment.

The selector 124 supplies the selected pixel data to the post-processing unit 125. The post-processing unit 125 executes various image processing such as demosaic processing and white balance correction processing. Note that the post-processing unit 125 is an example of a signal processing unit described in the claims.

As described above, according to the fourth embodiment of the present technology, a circuit for converting the reading order of the pixels into the raster order is arranged in the DSP circuit 120 outside the solid-state imaging device 200. The circuit scale of the solid-state imaging device 200 can be reduced as compared with the case where the device is arranged in the device 200.

<5. Example of application to moving objects>
The technology (the present technology) according to the present disclosure can be applied to various products. For example, the technology according to the present disclosure is realized as a device mounted on any type of moving object such as an automobile, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, a personal mobility, an airplane, a drone, a ship, and a robot. You may.

FIG. 20 is a block diagram illustrating a schematic configuration example of a vehicle control system that is an example of a moving object control system to which the technology according to the present disclosure can be applied.

Vehicle control system 12000 includes a plurality of electronic control units connected via communication network 12001. In the example shown in FIG. 20, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an outside information detection unit 12030, an inside information detection unit 12040, and an integrated control unit 12050. As a functional configuration of the integrated control unit 12050, a microcomputer 12051, an audio / video output unit 12052, and a vehicle-mounted network I / F (interface) 12053 are illustrated.

The drive system control unit 12010 controls the operation of the device related to the drive system of the vehicle according to various programs. For example, the drive system control unit 12010 includes a drive force generation device for generating a drive force of the vehicle such as an internal combustion engine or a drive motor, a drive force transmission mechanism for transmitting the drive force to the wheels, and a steering angle of the vehicle. It functions as a control mechanism such as a steering mechanism for adjusting and a braking device for generating a braking force of the vehicle.

The body control unit 12020 controls the operation of various devices mounted on the vehicle body according to various programs. For example, the body-related control unit 12020 functions as a keyless entry system, a smart key system, a power window device, or a control device for various lamps such as a head lamp, a back lamp, a brake lamp, a blinker, and a fog lamp. In this case, a radio wave or a signal of various switches transmitted from a portable device replacing the key can be input to the body control unit 12020. The body control unit 12020 receives the input of these radio waves or signals and controls a door lock device, a power window device, a lamp, and the like of the vehicle.

外 Out-of-vehicle information detection unit 12030 detects information external to the vehicle on which vehicle control system 12000 is mounted. For example, an imaging unit 12031 is connected to the outside-of-vehicle information detection unit 12030. The out-of-vehicle information detection unit 12030 causes the imaging unit 12031 to capture an image outside the vehicle, and receives the captured image. The out-of-vehicle information detection unit 12030 may perform an object detection process or a distance detection process of a person, a vehicle, an obstacle, a sign, a character on a road surface, or the like based on the received image.

The imaging unit 12031 is an optical sensor that receives light and outputs an electric signal according to the amount of received light. The imaging unit 12031 can output an electric signal as an image or can output the information as distance measurement information. The light received by the imaging unit 12031 may be visible light or non-visible light such as infrared light.

The in-vehicle information detection unit 12040 detects information in the vehicle. The in-vehicle information detection unit 12040 is connected to, for example, a driver status detection unit 12041 that detects the status of the driver. The driver state detection unit 12041 includes, for example, a camera that captures an image of the driver, and the in-vehicle information detection unit 12040 determines the degree of driver fatigue or concentration based on the detection information input from the driver state detection unit 12041. The calculation may be performed, or it may be determined whether the driver has fallen asleep.

The microcomputer 12051 calculates a control target value of the driving force generation device, the steering mechanism or the braking device based on the information on the inside and outside of the vehicle acquired by the outside information detection unit 12030 or the inside information detection unit 12040, and the drive system control unit A control command can be output to 12010. For example, the microcomputer 12051 implements functions of ADAS (Advanced Driver Assistance System) including a collision avoidance or a shock mitigation of a vehicle, a following operation based on a distance between vehicles, a vehicle speed maintaining operation, a vehicle collision warning, or a vehicle lane departure warning. Cooperative control for the purpose.

Further, the microcomputer 12051 controls the driving force generation device, the steering mechanism, the braking device, and the like based on the information about the surroundings of the vehicle obtained by the outside information detection unit 12030 or the inside information detection unit 12040, so that the driver 120 It is possible to perform cooperative control for automatic driving or the like in which the vehicle travels autonomously without depending on the operation.

マイクロ Also, the microcomputer 12051 can output a control command to the body system control unit 12020 based on information on the outside of the vehicle acquired by the outside information detection unit 12030. For example, the microcomputer 12051 controls the headlamp according to the position of the preceding vehicle or the oncoming vehicle detected by the outside information detection unit 12030, and performs cooperative control for the purpose of preventing glare such as switching a high beam to a low beam. It can be carried out.

The sound image output unit 12052 transmits at least one of a sound signal and an image signal to an output device capable of visually or audibly notifying a passenger of the vehicle or the outside of the vehicle of information. In the example of FIG. 20, an audio speaker 12061, a display unit 12062, and an instrument panel 12063 are illustrated as output devices. The display unit 12062 may include, for example, at least one of an on-board display and a head-up display.

FIG. 21 is a diagram illustrating an example of an installation position of the imaging unit 12031.

では In FIG. 21, the image pickup unit 12031 includes

image pickup units

12101, 12102, 12103, 12104, and 12105.

The

imaging units

12101, 12102, 12103, 12104, and 12105 are provided, for example, at positions such as a front nose, a side mirror, a rear bumper, a back door of the vehicle 12100, and an upper portion of a windshield in the vehicle interior. The imaging unit 12101 provided on the front nose and the imaging unit 12105 provided above the windshield in the passenger compartment mainly acquire an image in front of the vehicle 12100. The

imaging units

12102 and 12103 provided in the side mirror mainly acquire images of the side of the vehicle 12100. The imaging unit 12104 provided in the rear bumper or the back door mainly acquires an image behind the vehicle 12100. The imaging unit 12105 provided above the windshield in the passenger compartment is mainly used for detecting a preceding vehicle, a pedestrian, an obstacle, a traffic light, a traffic sign, a lane, and the like.

FIG. 21 shows an example of the imaging range of the imaging units 12101 to 12104. The imaging range 12111 indicates the imaging range of the imaging unit 12101 provided on the front nose, the imaging ranges 12112 and 12113 indicate the imaging ranges of the

imaging units

12102 and 12103 provided on the side mirrors, respectively, and the imaging range 12114 indicates 13 shows an imaging range of an imaging unit 12104 provided in a rear bumper or a back door. For example, a bird's-eye view image of the vehicle 12100 viewed from above is obtained by superimposing image data captured by the imaging units 12101 to 12104.

At least one of the imaging units 12101 to 12104 may have a function of acquiring distance information. For example, at least one of the imaging units 12101 to 12104 may be a stereo camera including a plurality of imaging elements or an imaging element having pixels for detecting a phase difference.

For example, based on the distance information obtained from the imaging units 12101 to 12104, the microcomputer 12051 calculates a distance to each three-dimensional object in the imaging ranges 12111 to 12114 and a temporal change of the distance (relative speed with respect to the vehicle 12100). , It is possible to extract, as a preceding vehicle, a three-dimensional object that travels at a predetermined speed (for example, 0 km / h or more) in a direction substantially the same as that of the vehicle 12100, which is the closest three-dimensional object on the traveling path of the vehicle 12100 it can. Further, the microcomputer 12051 can set an inter-vehicle distance to be secured before the preceding vehicle and perform automatic brake control (including follow-up stop control), automatic acceleration control (including follow-up start control), and the like. In this way, it is possible to perform cooperative control for automatic driving or the like in which the vehicle travels autonomously without depending on the operation of the driver.

For example, the microcomputer 12051 converts the three-dimensional object data relating to the three-dimensional object into other three-dimensional objects such as a motorcycle, a normal vehicle, a large vehicle, a pedestrian, a telephone pole, and the like based on the distance information obtained from the imaging units 12101 to 12104. It can be classified and extracted and used for automatic avoidance of obstacles. For example, the microcomputer 12051 distinguishes obstacles around the vehicle 12100 into obstacles that are visible to the driver of the vehicle 12100 and obstacles that are difficult to see. Then, the microcomputer 12051 determines a collision risk indicating a risk of collision with each obstacle, and when the collision risk is equal to or more than the set value and there is a possibility of collision, via the audio speaker 12061 or the display unit 12062. By outputting an alarm to the driver through forced driving and avoidance steering via the drive system control unit 12010, driving assistance for collision avoidance can be performed.

At least one of the imaging units 12101 to 12104 may be an infrared camera that detects infrared light. For example, the microcomputer 12051 can recognize a pedestrian by determining whether or not a pedestrian exists in the captured images of the imaging units 12101 to 12104. The recognition of such a pedestrian is performed by, for example, extracting a feature point in an image captured by the imaging units 12101 to 12104 as an infrared camera, and performing a pattern matching process on a series of feature points indicating the outline of the object to determine whether the object is a pedestrian. Is performed by a procedure for determining When the microcomputer 12051 determines that a pedestrian is present in the images captured by the imaging units 12101 to 12104 and recognizes the pedestrian, the audio image output unit 12052 outputs a rectangular outline to the recognized pedestrian for emphasis. The display unit 12062 is controlled so that is superimposed. Further, the sound image output unit 12052 may control the display unit 12062 so as to display an icon or the like indicating a pedestrian at a desired position.

As described above, an example of the vehicle control system to which the technology according to the present disclosure can be applied has been described. The technology according to the present disclosure can be applied to, for example, the imaging unit 12031 in the configuration described above. Specifically, the imaging device 100 in FIG. 1 can be applied to the imaging unit 12031. By applying the technology according to the present disclosure to the imaging unit 12031, it is possible to reduce the number of wirings and suppress an increase in manufacturing cost.

The above-described embodiment is an example for embodying the present technology, and matters in the embodiment and matters specifying the invention in the claims have a corresponding relationship. Similarly, the matters specifying the invention in the claims and the matters in the embodiments of the present technology with the same names have a correspondence relationship. However, the present technology is not limited to the embodiments, and can be embodied by variously modifying the embodiments without departing from the gist thereof.

効果 Note that the effects described in this specification are merely examples, are not limited, and may have other effects.

Note that the present technology may have the following configurations.
(1) a data output unit for alternately and repeatedly outputting predetermined reset data and signal data corresponding to an exposure amount;
A pair of reset data holding units that alternately hold the repeatedly output reset data,
A pixel data generation unit that repeats a process of generating a difference between the reset data held in one of the pair of reset data holding units and the signal data output after the reset data as pixel data,
A pair of pixel data holding units that alternately hold the repeatedly generated pixel data,
A solid-state imaging device comprising: an output-side selection unit that alternately selects the pair of pixel data holding units and outputs the pixel data held in the selected pixel data holding unit.
(2) The solid-state imaging device according to (1), wherein the data output unit starts outputting the reset data while outputting the signal data.
(3) The above-mentioned (1) further includes an input-side selection unit that alternately selects the pair of reset data holding units and supplies the reset data held in the selected reset data holding unit to the pixel data generation unit. ) Or (2).
(4) The pixel data generation unit includes a pair of pixel data generation circuits,
One of the pair of pixel data generation circuits sets a difference between the reset data held in one of the pair of reset data holding units and the signal data output after the reset data as the pixel data. Output to one of the pixel data holding units,
The other of the pair of pixel data generation circuits sets a difference between the reset data held in the other of the pair of reset data holding units and the signal data output after the reset data as the pixel data. The solid-state imaging device according to the above (1) or (2), which outputs the image data to the other of the pixel data holding units.
(5) a pixel that generates a predetermined reset level and a signal level according to the exposure amount;
The (1) further includes an analog-to-digital conversion unit that performs a process of converting the reset level to a digital signal and holding the reset data as the data, and a process of converting the signal level to a digital signal and holding the signal data as the data. The solid-state imaging device according to any one of (1) to (4).
(6) The pixel is disposed on a predetermined light receiving substrate,
The analog-to-digital conversion unit, the data output unit, the pair of reset data holding units, the pixel data generation unit, the pair of pixel data holding units, and the output side selection unit are disposed on a predetermined circuit board. The solid-state imaging device according to (5).
(7) The pixel is disposed on a predetermined light receiving substrate,
The analog-to-digital converter is disposed on a first circuit board,
A part of the data output unit, the pair of reset data holding units, the pixel data generation unit, the pair of pixel data holding units, and the output side selection unit are disposed on the first circuit board, and the rest is The solid-state imaging device according to (5), which is disposed on a second circuit board.
(8) a data output unit for alternately and repeatedly outputting predetermined reset data and signal data corresponding to the exposure amount;
A pair of reset data holding units that alternately hold the repeatedly output reset data,
A pixel data generation unit that repeats a process of generating a difference between the reset data held in one of the pair of reset data holding units and the signal data output after the reset data as pixel data,
A pair of pixel data holding units that alternately hold the repeatedly generated pixel data,
An output-side selection unit that alternately selects the pair of pixel data holding units and outputs the pixel data held in the selected pixel data holding unit;
An image pickup apparatus comprising: a signal processing unit that processes the pixel data.
(9) The device according to (8), wherein the data output unit, the pair of reset data holding units, the pixel data generation unit, the pair of pixel data holding units, and the output side selection unit are arranged in a solid-state imaging device. Imaging device.
(10) The data output unit, the pair of reset data holding units, and the pixel data generation unit are arranged in a solid-state imaging device,
The imaging device according to (8), wherein the pair of pixel data holding units and the output side selection unit are arranged outside the solid-state imaging device.
(11) a data output procedure for alternately and repeatedly outputting predetermined reset data and signal data corresponding to the exposure amount;
A process of generating, as pixel data, a difference between the reset data held in one of a pair of reset data holding units that alternately holds the repeatedly output reset data and the signal data output after the reset data. Pixel data generation procedure to be repeated,
An output-side selecting step of alternately selecting a pair of pixel data holding units for alternately holding the repeatedly generated pixel data and outputting the pixel data held in the selected pixel data holding unit. A method for controlling an image sensor.

REFERENCE SIGNS LIST 100 imaging device 110 optical unit 120 DSP circuit 121

timing control circuit

122, 123, 160, 271, 272, 275, 276

frame memory

124, 273, 277 selector 125 post-processing unit 130 display unit 140 operation unit 150 bus 170 storage unit 180 Power supply unit 200 Solid-state imaging device 201 Light receiving substrate 202 Circuit substrate 203 Upper circuit substrate 204 Lower circuit substrate 210 Pixel array unit 211 Pixel block 212 Pixel 220 DAC
230 Vertical drive circuit 240 Timing control circuit 250 Time code generation unit 260 AD conversion unit 261 Data transfer unit 262 P-phase transfer unit 263 Flip-flop 264 Write data transfer unit 265 D-phase transfer unit 266 Clock supply unit 270

Data processing unit

274, 278 279 CDS (Correlated Double Sampling) processing unit 281 Upper data processing unit 282 Lower data processing unit 300 Cluster 311 to 318 Comparison circuit 320 Left memory 321 to 328, 331 to 338 Data storage unit 330 Right memory 12031 Imaging unit

Claims

A data output unit for alternately and repeatedly outputting predetermined reset data and signal data corresponding to the exposure amount,
A pair of reset data holding units that alternately hold the repeatedly output reset data,
A pixel data generation unit that repeats a process of generating a difference between the reset data held in one of the pair of reset data holding units and the signal data output after the reset data as pixel data,
A pair of pixel data holding units that alternately hold the repeatedly generated pixel data,
A solid-state imaging device comprising: an output-side selection unit that alternately selects the pair of pixel data holding units and outputs the pixel data held in the selected pixel data holding unit.
The solid-state imaging device according to claim 1, wherein the data output unit starts outputting the reset data while outputting the signal data.
2. The solid according to claim 1, further comprising: an input side selection unit that alternately selects the pair of reset data holding units and supplies the reset data held in the selected reset data holding unit to the pixel data generation unit. Imaging device.
The pixel data generation unit includes a pair of pixel data generation circuits,
One of the pair of pixel data generation circuits sets a difference between the reset data held in one of the pair of reset data holding units and the signal data output after the reset data as the pixel data. Output to one of the pixel data holding units,
The other of the pair of pixel data generation circuits sets a difference between the reset data held in the other of the pair of reset data holding units and the signal data output after the reset data as the pixel data. 2. The solid-state imaging device according to claim 1, wherein the image data is output to the other of the pixel data holding units.
A pixel that generates a predetermined reset level and a signal level according to the exposure amount,
2. An analog-to-digital converter for converting the reset level to a digital signal and holding the reset data as the reset data, and converting the signal level to a digital signal and holding the signal data as the analog data. The solid-state imaging device according to any one of the preceding claims.
The pixel is disposed on a predetermined light receiving substrate,
The analog-to-digital conversion unit, the data output unit, the pair of reset data holding units, the pixel data generation unit, the pair of pixel data holding units, and the output side selection unit are arranged on a predetermined circuit board. Item 6. A solid-state imaging device according to Item 5.
The pixel is disposed on a predetermined light receiving substrate,
The analog-to-digital converter is disposed on a first circuit board,
A part of the data output unit, the pair of reset data holding units, the pixel data generation unit, the pair of pixel data holding units, and the output side selection unit are disposed on the first circuit board, and the rest is The solid-state imaging device according to claim 5, which is arranged on a second circuit board.
A data output unit for alternately and repeatedly outputting predetermined reset data and signal data corresponding to the exposure amount,
A pair of reset data holding units that alternately hold the repeatedly output reset data,
A pixel data generation unit that repeats a process of generating a difference between the reset data held in one of the pair of reset data holding units and the signal data output after the reset data as pixel data,
A pair of pixel data holding units that alternately hold the repeatedly generated pixel data,
An output-side selection unit that alternately selects the pair of pixel data holding units and outputs the pixel data held in the selected pixel data holding unit;
An image pickup apparatus comprising: a signal processing unit that processes the pixel data.
9. The imaging device according to claim 8, wherein the data output unit, the pair of reset data holding units, the pixel data generation unit, the pair of pixel data holding units, and the output side selection unit are arranged in a solid-state imaging device.
The data output unit, the pair of reset data holding units and the pixel data generation unit are arranged in a solid-state imaging device,
The imaging device according to claim 8, wherein the pair of pixel data holding units and the output-side selection unit are arranged outside the solid-state imaging device.
A data output procedure for alternately and repeatedly outputting predetermined reset data and signal data corresponding to the exposure amount,
A process of generating, as pixel data, a difference between the reset data held in one of a pair of reset data holding units that alternately holds the repeatedly output reset data and the signal data output after the reset data. A pixel data generation procedure to be repeated,
An output-side selecting step of alternately selecting a pair of pixel data holding units for alternately holding the repeatedly generated pixel data and outputting the pixel data held in the selected pixel data holding unit. A method for controlling an image sensor.