WO2021106370A1

WO2021106370A1 - Solid-state image sensor, image-capturing system, and control method for solid-state image sensor

Info

Publication number: WO2021106370A1
Application number: PCT/JP2020/037515
Authority: WO
Inventors: 智裕山崎
Original assignee: ソニーセミコンダクタソリューションズ株式会社
Priority date: 2019-11-28
Filing date: 2020-10-02
Publication date: 2021-06-03
Also published as: JP2021087108A

Abstract

In this solid-state image sensor that performs TDI processing, miniaturization of pixels is facilitated.　The solid-state image sensor is provided with a pixel circuit, an analog-to-digital converter, and an arithmetic circuit. The pixel circuit generates a plurality of mixed signals each containing a visible light component and an invisible light component, and a plurality of visible light signals obtained by photoelectrically converting visible light. The analog-to-digital converter converts each of the plurality of mixed signals and the plurality of visible light signals into digital signals. The arithmetic circuit generates, from the digital signals, integrated data indicating the integrated value of the invisible light component included in each of the plurality of mixed signals.

Description

Solid-state image sensor, image pickup system, and control method for solid-state image sensor

This technology relates to a solid-state image sensor. More specifically, the present invention relates to a solid-state image sensor for capturing an infrared image, an image pickup system, and a control method for the solid-state image sensor.

Conventionally, time delay integration (TDI) sensors have been used in the fields of FA (Factory Automation), aerial photography, and medical treatment. This TDI sensor is a sensor that performs TDI processing that integrates the amount of electric charge while shifting the time according to the moving speed of the subject. For example, a solid-state image sensor that performs TDI processing to capture an infrared image while blinking an infrared light source has been proposed (see, for example, Non-Patent Document 1). This solid-state image sensor outputs a floating diffusion layer that holds a mixed signal in which infrared rays and visible light are photoelectrically converted, a floating diffusion layer that holds a visible light signal in which only visible light is photoelectrically converted, and a difference between these signals. A circuit is provided for each pixel.

In the above-mentioned conventional technique, the visible light component is removed by obtaining the difference between the mixed signal and the visible light signal. An infrared image can be obtained by removing the visible light component. However, in the above-mentioned solid-state image sensor, it is necessary to provide two floating diffusion layers for each pixel in order to remove the visible light component, and the pixels are miniaturized as compared with the case where one floating diffusion layer is provided for each pixel. There is a problem that it becomes difficult.

This technology was created in view of such a situation, and aims to facilitate the miniaturization of pixels in a solid-state image sensor that performs TDI processing.

The present technology has been made to solve the above-mentioned problems, and the first aspect thereof is a plurality of mixed signals each containing a visible light component and an invisible light component, and a plurality of photoelectrically converted visible lights. A pixel circuit that generates a visible light signal, an analog digital converter that converts each of the plurality of mixed signals and the plurality of visible light signals into a digital signal, and the invisible one included in each of the plurality of mixed signals. It is a solid-state image sensor including an arithmetic circuit that generates integrated data indicating an integrated value of an optical component from the digital signal, and a control method thereof. This has the effect of generating integrated data from the digital signal.

Further, in the first aspect, the plurality of mixed signals include first and second mixed signals, and the plurality of visible light signals include first and second visible light signals, and the digital signal. Includes first and second mixed data obtained by converting the first and second mixed signals and first and second visible light data obtained by converting the first and second visible light signals. The arithmetic circuit includes the first subtraction process for obtaining the difference between the first mixed data and the first visible light data as the difference data, and the addition data by adding the difference data and the second mixed data. The addition process for acquiring the above and the second subtraction process for obtaining the difference between the addition data and the second visible light data may be performed in order. This has the effect of generating integrated data by operations in the order of addition, subtraction, and addition.

Further, in the first aspect, the plurality of mixed signals include first and second mixed signals, and the plurality of visible light signals include first and second visible light signals, and the digital signal. Includes first and second mixed data obtained by converting the first and second mixed signals and first and second visible light data obtained by converting the first and second visible light signals. The arithmetic circuit acquires the addition process of adding the first mixed data and the second mixed data to acquire the addition data, and the difference between the addition data and the first visible light data as difference data. The first subtraction process and the second subtraction process for obtaining the difference between the difference data and the second visible light data may be performed in order. This has the effect of generating integrated data by operations in the order of addition, subtraction, and subtraction.

Further, in the first aspect thereof, a frame memory is further provided, and the arithmetic circuit adds the digital signal from the analog-digital converter and the read data read from the frame memory to obtain additional data. An adder to output, a subtractor to output the difference between the read data and the digital signal as difference data, and a selector to select either the adder data and the difference data and output to the frame memory. You may prepare. This has the effect of generating integrated data by the operations of the adder and subtractor.

Further, in this first aspect, it is also possible to further include a synchronization control unit that operates an invisible light source that irradiates invisible light in synchronization with the pixel circuit. This has the effect of integrating invisible light components from the invisible light source.

Further, in the first aspect, the pixel circuit and a part of the analog-digital converter are arranged on a predetermined light receiving chip, and the rest of the analog digital converter and the arithmetic circuit are laminated on the light receiving chip. It may be arranged on the circuit chip. This brings about the effect that integrated data is generated in the solid-state image sensor having a laminated structure.

The second aspect of the present technology is to generate an invisible light source that irradiates invisible light, a plurality of mixed signals each containing a visible light component and an invisible light component, and a plurality of visible light signals obtained by photoelectrically converting visible light. The pixel circuit, the analog digital converter that converts each of the plurality of mixed signals and the plurality of visible light signals into digital signals, and the integrated value of the invisible light component contained in each of the plurality of mixed signals. It is an imaging system including an arithmetic circuit that generates the integrated data shown from the digital signal. This has the effect of generating integrated data of the invisible light component irradiated by the invisible light source from the digital signal.

It is a block diagram which shows one configuration example of the image pickup system in the 1st Embodiment of this technique. It is a figure for demonstrating the use example of the image pickup system in 1st Embodiment of this technique. It is a figure which shows an example of the laminated structure of the solid-state image pickup device in the 1st Embodiment of this technique. It is a block diagram which shows one structural example of the light receiving chip in the 1st Embodiment of this technique. It is a block diagram which shows one structural example of the circuit chip in 1st Embodiment of this technique. It is a figure which shows one configuration example of the pixel AD (Analog to Digital) conversion part in the 1st Embodiment of this technique. It is a block diagram which shows one configuration example of ADC (Analog to Digital Converter) in the 1st Embodiment of this technique. It is a circuit diagram which shows one configuration example of the pixel circuit, the differential input circuit and the positive feedback circuit in the 1st Embodiment of this technique. It is a block diagram which shows one structural example of the signal processing circuit in 1st Embodiment of this technique. It is a circuit diagram which shows one structural example of the arithmetic circuit in 1st Embodiment of this technique. It is a figure which shows an example of the state of the arithmetic circuit at the time of holding the 1st frame in 1st Embodiment of this technique. It is a figure which shows an example of the state of the arithmetic circuit at the time of calculating the difference between the 2nd frame and the 1st frame in the 1st Embodiment of this technique. It is a figure which shows an example of the state of the arithmetic circuit at the time of adding a 3rd frame and read data in 1st Embodiment of this technique. It is a figure which shows an example of the state of the arithmetic circuit at the time of calculating the difference between the 4th frame and read data in the 1st Embodiment of this technique. It is a figure which shows an example of the state of the solid-state image sensor at the time of holding the 1st frame in 1st Embodiment of this technique. It is a figure which shows an example of the state of the solid-state image sensor at the time of calculating the difference between the 2nd frame and the 1st frame in the 1st Embodiment of this technique. It is a figure which shows an example of the state of the solid-state image sensor at the time of adding a 3rd frame and read data in 1st Embodiment of this technique. It is a figure which shows an example of the state of the solid-state image sensor at the time of calculating the difference between the 4th frame and the read data in the 1st Embodiment of this technique. It is a figure which shows an example of TDI processing in 1st Embodiment of this technique. It is a figure for demonstrating control of the solid-state image sensor in 1st Embodiment of this technique. This is an example of a flowchart showing an example of the operation of the imaging system according to the first embodiment of the present technology. It is a figure which shows an example of the state of the arithmetic circuit at the time of holding the 1st frame in 2nd Embodiment of this technique. It is a figure which shows an example of the state of the arithmetic circuit at the time of adding the 2nd frame and the 1st frame in the 2nd Embodiment of this technique. It is a figure which shows an example of the state of the arithmetic circuit at the time of calculating the difference between the 3rd frame and read data in the 2nd Embodiment of this technique. It is a figure which shows an example of the state of the arithmetic circuit at the time of calculating the difference between the 4th frame and read data in the 2nd Embodiment of this technique. It is a figure which shows an example of the state of the solid-state image sensor at the time of holding the 1st frame in 2nd Embodiment of this technique. It is a figure which shows an example of the state of the solid-state image sensor at the time of adding the 1st frame and the 2nd frame in the 2nd Embodiment of this technique. It is a figure which shows an example of the state of the solid-state image sensor at the time of calculating the difference between the 3rd frame and the read data in the 2nd Embodiment of this technique. It is a figure which shows an example of the state of the solid-state image sensor at the time of calculating the difference between the 4th frame and the read data in the 2nd Embodiment of this technique.

Hereinafter, embodiments for carrying out the present technology (hereinafter referred to as embodiments) will be described. The explanation will be given in the following order.
1. 1. First Embodiment (Example of generating integrated data from a digital signal)
2. Second embodiment (an example in which integrated data is generated from a digital signal and the frequency of a light emission control signal is reduced)

<1. First Embodiment>
[Configuration example of imaging system]
FIG. 1 is a block diagram showing a configuration example of an imaging system according to the first embodiment of the present technology. This imaging system is a system for capturing a near-infrared image, and includes an imaging device 100 and a near-infrared light source 500.

The image pickup device 100 includes an optical unit 110, a solid-state image sensor 200, a storage unit 120, a synchronization control unit 130, and a communication unit 140.

The optical unit 110 collects the incident light and guides it to the solid-state image sensor 200. The solid-state image sensor 200 captures image data. The solid-state image sensor 200 supplies image data (in other words, a frame) to the storage unit 120 via a signal line 209.

The storage unit 120 stores frames. The communication unit 140 reads the frame from the storage unit 120 and transmits it to the outside.

The synchronization control unit 130 operates the solid-state image sensor 200 and the near-infrared light source 500 in synchronization with each other. The synchronization control unit 130 supplies a vertical scanning signal having a predetermined frequency via the signal line 208. Here, the vertical scanning signal is a periodic signal indicating the imaging timing of the frame. Further, the synchronization control unit 130 supplies a light emission control signal synchronized with the vertical scanning signal to the near infrared light source 500 via the signal line 209. As the light emission control signal, for example, a periodic signal having a frequency half that of the vertical synchronization signal is used.

Although the synchronization control unit 130 is provided inside the image pickup apparatus 100, it can also be arranged outside the image pickup apparatus 100.

The near-infrared light source 500 irradiates near-infrared light under the control of the synchronization control unit 130. Although the near-infrared light source 500 irradiates the near-infrared light, a light source other than the near-infrared light source 500, such as an ultraviolet light source, may be provided as long as it irradiates invisible light. Further, the near-infrared light source 500 is an example of the invisible light source described in the claims.

FIG. 2 is a diagram for explaining a usage example of the imaging system in the first embodiment of the present technology. As illustrated in the figure, the image pickup apparatus 100 and the near-infrared light source 500 are used in a factory or the like where a belt conveyor 510 is provided.

The belt conveyor 510 moves the subject 511 in a predetermined direction at a constant speed. The image pickup apparatus 100 is fixed in the vicinity of the belt conveyor 510 together with the near-infrared light source 500, and images the subject 511 while blinking the near-infrared light source 500 to generate image data of an infrared image. This image data is used, for example, for inspection of the presence or absence of defects. As a result, FA is realized especially in a dark place.

The imaging system captures a subject 511 moving at a constant speed, but the imaging system is not limited to this configuration. The image pickup system may move at a constant speed to take an image of the subject, such as aerial photography. An imaging system can also be applied to an optical heart rate monitor to observe blood flow.

[Structure example of solid-state image sensor]
FIG. 3 is a diagram showing an example of a laminated structure of the solid-state image sensor 200 according to the first embodiment of the present technology. The solid-state image sensor 200 includes a circuit chip 202 and a light receiving chip 201 laminated on the circuit chip 202. These chips are electrically connected via a connection such as a via. In addition to vias, it can also be connected by Cu-Cu bonding or bumps.

FIG. 4 is a block diagram showing a configuration example of the light receiving chip 201 according to the first embodiment of the present technology. The light receiving chip 201 is provided with a pixel array unit 210 and a peripheral circuit 212.

A plurality of pixel circuits 220 are arranged in a two-dimensional grid pattern in the pixel array unit 210. Further, the pixel array unit 210 is divided into a plurality of pixel blocks 211. In each of these pixel blocks 211, for example, a pixel circuit 220 having 4 rows × 2 columns is arranged.

For example, a circuit that supplies a DC (Direct Current) voltage is arranged in the peripheral circuit 212.

FIG. 5 is a block diagram showing a configuration example of the circuit chip 202 according to the first embodiment of the present technology. A DAC (Digital to Analog Converter) 251, a pixel drive circuit 252, a time code generation unit 253, a pixel AD conversion unit 254, and a vertical scanning circuit 255 are arranged on the circuit chip 202. Further, a control circuit 256, a signal processing circuit 400, an image processing circuit 260, and an output circuit 257 are arranged on the circuit chip 202.

The DAC 251 generates a reference signal by DA (Digital to Analog) conversion within a predetermined AD conversion period. For example, a saw blade-shaped lamp signal is used as a reference signal. The DAC 251 supplies the reference signal to the pixel AD conversion unit 254.

The time code generation unit 253 generates a time code indicating the time within the AD conversion period. The time code generation unit 253 is realized by, for example, a counter. As the counter, for example, a Gray code counter is used. The time code generation unit 253 supplies the time code to the pixel AD conversion unit 254.

The pixel drive circuit 252 drives each of the pixel circuits 220 to generate an analog pixel signal.

The pixel AD conversion unit 254 performs AD conversion that converts each analog signal (that is, a pixel signal) of the pixel circuit 220 into a digital signal. The pixel AD conversion unit 254 is divided by a plurality of clusters 300. The cluster 300 is provided for each pixel block 211 and converts the analog signal in the corresponding pixel block 211 into a digital signal.

The pixel AD conversion unit 254 generates image data in which digital signals are arranged by AD conversion as a frame and supplies the image data to the signal processing circuit 400. In this frame, a set of digital signals arranged in the horizontal direction is hereinafter referred to as a "line". Each line is assigned a row address, which is an address indicating the position of the line in the vertical direction.

The vertical scanning circuit 255 drives the pixel AD conversion unit 254 to execute AD conversion.

The signal processing circuit 400 performs predetermined signal processing on the frame. As signal processing, various processes including TDI processing are executed. The signal processing circuit 400 supplies the processed frame to the image processing circuit 260.

The image processing circuit 260 executes predetermined image processing on the frame from the signal processing circuit 400. As image processing, image recognition processing, black level correction processing, image correction processing, demosaic processing, and the like are executed. The image processing circuit 260 supplies the processed frame to the output circuit 257.

The output circuit 257 outputs the frame after image processing to the outside.

The control circuit 256 controls the operation timings of the DAC 251, the pixel drive circuit 252, the vertical scanning circuit 255, the signal processing circuit 400, the image processing circuit 260, and the output circuit 257 in synchronization with the vertical synchronization signal VSYNC.

[Configuration example of pixel AD conversion unit]
FIG. 6 is a diagram showing a configuration example of the pixel AD conversion unit 254 according to the first embodiment of the present technology. A plurality of ADCs 310 are arranged in a two-dimensional grid pattern in the pixel AD conversion unit 254. The ADC 310 is arranged for each pixel circuit 220. When the number of rows and columns of the pixel circuit 220 is N rows (N is an integer) and M columns (M is an integer), N × M ADC 310s are arranged.

In each of the clusters 300, the same number of ADC 310s as the number of pixel circuits 220 in the pixel block 211 are arranged. When the pixel circuit 220 having 4 rows × 2 columns is arranged in the pixel block 211, the ADC 310 having 4 rows × 2 columns is also arranged in the cluster 300.

The ADC 310 performs AD conversion on an analog pixel signal generated by the corresponding pixel circuit 220. The ADC 310 compares the pixel signal and the reference signal in the AD conversion, and holds the time code when the comparison result is inverted. Then, the ADC 310 outputs the held time code as a digital signal after AD conversion.

In addition, a repeater unit 360 is arranged for each row of the cluster 300. When the number of columns of the cluster 300 is M / 2, M / 2 repeater units 360 are arranged. The repeater unit 360 transfers the time code. The repeater unit 360 transfers the time code from the time code generation unit 253 to the ADC 310. Further, the repeater unit 360 transfers a digital signal from the ADC 310 to the signal processing circuit 400. This transfer of digital signals is also referred to as "reading" the digital signals.

Further, in the figure, the numbers in parentheses indicate an example of the reading order of the digital signals of the ADC 310. For example, the odd-numbered column digital signal in the first row is read first, and the even-numbered column digital signal in the first row is read second. The odd-numbered column digital signal in the second row is read out third, and the even-numbered column digital signal in the second row is read out third. Hereinafter, similarly, the odd-numbered columns and even-numbered columns of the digital signals in each row are read out in order.

Although the ADC 310 is arranged for each pixel circuit 220, the configuration is not limited to this. A plurality of pixel circuits 220 may be configured to share one ADC 310.

[ADC configuration example]
FIG. 7 is a block diagram showing a configuration example of the ADC 310 according to the first embodiment of the present technology. The ADC 310 includes a differential input circuit 320, a positive feedback circuit 330, a latch control circuit 340, and a plurality of latch circuits 350.

Further, the pixel circuit 220 and a part of the differential input circuit 320 are arranged on the light receiving chip 201, and the rest of the differential input circuit 320 and the circuit in the subsequent stage are arranged on the circuit chip 202.

The differential input circuit 320 compares the pixel signal from the pixel circuit 220 with the reference signal from the DAC 251. The differential input circuit 320 supplies a comparison result signal indicating the comparison result to the positive feedback circuit 330.

The positive feedback circuit 330 adds a part of the output to the input (comparison result signal) and supplies it to the latch control circuit 340 as an output signal VCO.

The latch control circuit 340 causes a plurality of latch circuits 350 to hold the time code when the output signal VCO is inverted according to the control signal xWORD from the vertical scanning circuit 255.

The latch circuit 350 holds the time code from the repeater unit 360 according to the control of the latch control circuit 340. The latch circuit 350 is provided for the number of bits of the time code. For example, when the time code is 15 bits, 15 latch circuits 350 are arranged in the ADC 310. Further, the held time code is read out by the repeater unit 360 as a digital signal after AD conversion.

According to the configuration illustrated in the figure, the ADC 310 converts the pixel signal from the pixel circuit 220 into a digital signal.

[Configuration example of pixel circuit, differential input circuit and positive feedback circuit]
FIG. 8 is a circuit diagram showing a configuration example of a pixel circuit 220, a differential input circuit 320, and a positive feedback circuit 330 according to the first embodiment of the present technology.

The pixel circuit 220 includes a reset transistor 221, a floating diffusion layer 222, a transfer transistor 223, a photodiode 224, and an emission transistor 225. As the reset transistor 221 and the transfer transistor 223 and the emission transistor 225, for example, an nMOS (n-channel Metal Oxide Semiconductor) transistor is used.

The photodiode 224 generates an electric charge by photoelectric conversion. The discharge transistor 225 discharges the electric charge accumulated in the photodiode 224 according to the drive signal OFG from the pixel drive circuit 252.

The transfer transistor 223 transfers an electric charge from the photodiode 224 to the floating diffusion layer 222 according to the transfer signal TX from the pixel drive circuit 252.

The floating diffusion layer 222 accumulates the transferred electric charge and generates a voltage according to the amount of electric charge.

The reset transistor 221 initializes the floating diffusion layer 222 according to the reset signal RST from the pixel drive circuit 252.

The differential input circuit 320 includes pMOS (p-channel Metal Oxide Semiconductor)

transistors

321 and 324 and 326, and

nMOS transistors

322, 323, 325 and 327. Of these, the

nMOS transistors

322, 323 and 325 are arranged on the light receiving chip 201, and the rest are arranged on the circuit chip 202.

The

nMOS transistors

322 and 325 form a differential pair, and the source of these transistors is commonly connected to the drain of the nMOS transistor 323. Further, the drain of the nMOS transistor 322 is connected to the drain of the pMOS transistor 321 and the gate of the

pMOS transistors

321 and 324. The drain of the nMOS transistor 325 is connected to the drain of the pMOS transistor 324, the gate of the pMOS transistor 326, and the drain of the reset transistor 221. Further, a reference signal REF from the DAC 251 is input to the gate of the nMOS transistor 322.

A predetermined bias voltage Vb is applied to the gate of the nMOS transistor 323, and a predetermined ground voltage is applied to the source of the nMOS transistor 323.

The

pMOS transistors

321, 324 and 326 form a current mirror circuit. A power supply voltage VDDH is applied to the sources of the

pMOS transistors

321, 324 and 326. This power supply voltage VDDH is higher than the power supply voltage VDDL described later.

A power supply voltage VDDL is applied to the gate of the nMOS transistor 327. Further, the drain of the nMOS transistor 327 is connected to the drain of the pMOS transistor 326, and the source is connected to the positive feedback circuit 330.

The positive feedback circuit 330 includes

pMOS transistors

331, 332, 334 and 335, and

nMOS transistors

333, 336 and 337. The

pMOS transistors

331 and 332 and the nMOS transistor 333 are connected in series with the power supply voltage VDDL. Further, a drive signal INI2 from the vertical scanning circuit 255 is input to the gate of the pMOS transistor 331. The connection points of the pMOS transistor 332 and the nMOS transistor 333 are connected to the source of the nMOS transistor 327.

A ground voltage is applied to the source of the nMOS transistor 333, and a drive signal INI1 from the vertical scanning circuit 255 is input to the gate.

The

pMOS transistors

334 and 335 are connected in series with the power supply voltage VDDL. Further, the drain of the pMOS transistor 335 is connected to the gate of the pMOS transistor 332 and the drain of the

nMOS transistors

336 and 337. The control signal TESTVCO from the vertical scanning circuit 255 is input to the gates of the pMOS transistor 335 and the nMOS transistor 337. Further, the gates of the pMOS transistor 334 and the nMOS transistor 336 are connected to the connection points of the pMOS transistor 332 and the nMOS transistor 333.

The output signal VCO is output from the connection point of the pMOS transistor 335 and the nMOS transistor 337. A ground voltage is applied to the sources of the

nMOS transistors

336 and 337.

Note that each of the pixel circuit 220, the differential input circuit 320, and the positive feedback circuit 330 is not limited to the circuit illustrated in FIG. 8 as long as it has the functions described in FIG. 7.

[Example of signal processing circuit configuration]
FIG. 9 is a block diagram showing a configuration example of the signal processing circuit 400 according to the first embodiment of the present technology. The signal processing circuit 400 includes a plurality of selectors 405, a plurality of arithmetic circuits 410, and a frame memory 420.

The selector 405 is arranged for each column of the cluster 300, in other words, for each repeater unit 360. When two rows of ADC 310s are arranged in the cluster 300, a selector 405 is arranged in every two rows. Further, the arithmetic circuit 410 is arranged for each row of the ADC 310. When the ADC 310 has M columns, M / 2 selectors 405 and M arithmetic circuits 410 are arranged.

As described above, the repeater unit 360 outputs an odd-numbered row of digital signals and an even-numbered row of digital signals in order.

The selector 405 selects the output destination of the digital signal according to the control of the control circuit 256. When an odd-numbered sequence is output by the repeater unit 360, the selector 405 outputs a digital signal to the arithmetic circuit 410 corresponding to the odd-numbered sequence. On the other hand, when an even-numbered sequence is output, the selector 405 outputs a digital signal to the arithmetic circuit 410 corresponding to the even-numbered sequence.

The arithmetic circuit 410 performs TDI processing for integrating infrared light components while shifting the time based on the digital signal from the selector 405.

Here, as described above, the synchronization control unit 130 blinks the near-infrared light source 500 in synchronization with the vertical synchronization signal. For example, the synchronization control unit 130 turns on the near-infrared light source 500 within the exposure period of the odd-numbered frames and turns off the near-infrared light source 500 during the exposure period of the even-numbered frames. As a result, mixed light including infrared light and visible light is incident on the solid-state image sensor 200 within the odd-th exposure period, and a pixel signal containing the infrared light component and the visible light component is generated within the exposure period. Will be done. On the other hand, within the even-numbered exposure period, visible light is incident, and a pixel signal containing visible light components and not including infrared light components from the near-infrared light source 500 is generated.

Hereinafter, an analog pixel signal containing an infrared light component and a visible light component will be referred to as a "mixed signal". On the other hand, an analog pixel signal containing a visible light component and not containing an infrared light component from the near-infrared light source 500 is referred to as a "visible light signal". The mixed signal may include an infrared light component and a visible light component of the near-infrared light source 500, as well as an infrared light component and an ultraviolet light component from a natural light source other than the near-infrared light source 500. .. Similarly, the visible light signal may contain an infrared light component or an ultraviolet light component from a natural light source in addition to the visible light component.

Further, the digital signal obtained by AD-converting the mixed signal is referred to as "mixed data", and the digital signal obtained by AD-converting the visible light signal is referred to as "visible light data".

The arithmetic circuit 410 generates line data for one line from K (K is an integer) mixed data and K visible light data in TDI processing. First, the arithmetic circuit 410 causes the frame memory 420 to hold the first mixed data of the K pieces. Then, the arithmetic circuit 410 reads the read data for one line from the frame memory 420, obtains the difference between the first visible light data and the read data among the K pieces, and updates the frame memory 420 with the difference value. Subsequently, the arithmetic circuit 410 reads the updated read data from the frame memory 420, adds the second mixed data and the read data, and updates the frame memory 420 with the added value. Further, the arithmetic circuit 410 reads the updated read data from the frame memory 420, obtains the difference between the second visible light data and the read data, and updates the frame memory 420 with the difference value.

The arithmetic circuit 410 repeats the above processing for the mixed data and visible light data of K set to generate line data for one line. The arithmetic circuit 410 repeats such line data generation processing, and causes the frame memory 420 to hold the line data for one frame (N line, etc.).

The frame memory 420 holds a frame. The held frame is output to the image processing circuit 260 as a TDI frame.

[Example of arithmetic circuit configuration]
FIG. 10 is a circuit diagram showing a configuration example of the arithmetic circuit 410 according to the first embodiment of the present technology. The arithmetic circuit 410 includes a buffer 411,

selectors

412, 413 and 416, an adder 414, and a subtractor 415.

The buffer 411 delays and outputs the digital signal from the selector 405.

The selector 412 outputs the read data from the frame memory 420 to either the selector 413 or the subtractor 415 according to the control of the control circuit 256.

The selector 413 selects either the read data from the selector 412 or the data having a decimal value of "0" according to the control of the control circuit 256, and outputs the data to the adder 414.

The adder 414 adds the digital signal from the buffer 411 and the data from the selector 413 and outputs the data as the addition data to the selector 416.

The subtractor 415 obtains the difference between the read data from the selector 412 and the digital signal from the buffer 411, and outputs the difference data to the selector 416 as the difference data.

The selector 416 outputs either the addition data from the adder 414 or the difference data from the subtractor 415 to the frame memory 420 according to the control of the control circuit 256.

FIG. 11 is a diagram showing an example of the state of the arithmetic circuit 410 when holding the first frame in the first embodiment of the present technology.

The pixel AD conversion unit 254 generates mixed data of the first frame while the near-infrared light source 500 is lit. This mixed data includes an infrared light component and a visible light component as described above. The infrared light component in the first frame is IR1, and the visible light component in the first frame is V1.

Mixed data (IR1 + V1) in the corresponding column of the first frame is input to the buffer 411. The buffer 411 delays the mixed data and outputs it to the adder 414.

The selector 413 selects the data of "0" and outputs it to the adder 414. The adder 414 adds "0" to the mixed data and outputs it to the selector 416. The selector 416 outputs the data from the adder 414 to the frame memory 420.

By the above control, the mixed data (IR1 + V1) for one line is held in the frame memory 420.

FIG. 12 is a diagram showing an example of the state of the arithmetic circuit 410 when calculating the difference between the second frame and the first frame in the first embodiment of the present technology.

The pixel AD conversion unit 254 generates visible light data in the second frame while the near-infrared light source 500 is turned off. The visible light component in the visible light data in the second frame has substantially the same value as the visible light component V1 in the first frame. Here, "substantially the same" means that the two values to be compared are completely the same, or the difference between them is within a predetermined allowable value.

Visible light data (V1) in the corresponding column of the second frame is input to the buffer 411. The buffer 411 delays the visible light data and outputs it to the subtractor 415.

The selector 412 reads the mixed data (IR1 + V1) from the frame memory 420 and supplies it to the subtractor 415 as read data.

The subtractor 415 obtains the difference data by subtracting the visible light data (V1) from the read data (IR1 + V1). By this subtraction, the visible light component V1 is removed.

The selector 416 outputs the difference data (IR1) from the subtractor 415 to the frame memory 420.

By the above control, the mixed data (IR1 + V1) for one line held in the frame memory 420 is updated by the difference data (IR1).

FIG. 13 is a diagram showing an example of the state of the arithmetic circuit 410 when the third frame and the read data are added according to the first embodiment of the present technology.

The pixel AD conversion unit 254 generates mixed data of the third frame while the near-infrared light source 500 is lit. This mixed data includes an infrared light component and a visible light component as described above. The infrared light component in the third frame is IR2, and the visible light component in the third frame is V2.

Mixed data (IR2 + V2) in the corresponding column of the third frame is input to the buffer 411. The buffer 411 delays the mixed data and outputs it to the adder 414.

The selector 412 reads the difference data (IR1) from the frame memory 420 and supplies it as read data to the adder 414 via the selector 413.

The adder 414 adds the mixed data (IR2 + V2) and the read data (IR1) and outputs the added data to the selector 416. This added data includes an infrared light component IR2 and a visible light component V2 in the third frame, and an infrared light component IR1 in the first frame.

The selector 416 outputs the addition data (IR1 + IR2 + V2) from the adder 414 to the frame memory 420.

By the above-mentioned control, the difference data (IR1) for one line held in the frame memory 420 is updated by the addition data (IR1 + IR2 + V2).

FIG. 14 is a diagram showing an example of the state of the arithmetic circuit 410 when calculating the difference between the fourth frame and the third frame in the first embodiment of the present technology.

The pixel AD conversion unit 254 generates visible light data in the fourth frame while the near-infrared light source 500 is turned off. The visible light component in the visible light data in the 4th frame has substantially the same value as the visible light component V2 in the 3rd frame.

Visible light data (V2) in the corresponding column of the 4th frame is input to the buffer 411. The buffer 411 delays the visible light data and outputs it to the subtractor 415.

The selector 412 reads the addition data (IR1 + IR2 + V2) from the frame memory 420 and supplies it to the subtractor 415 as read data.

The subtractor 415 obtains the difference data by subtracting the visible light data (V2) from the read data (IR1 + IR2 + V2). By this subtraction, the visible light component V2 is removed.

The selector 416 outputs the difference data (IR1 + IR2) from the subtractor 415 to the frame memory 420.

By the above-mentioned control, the addition data (IR1 + IR2 + V2) for one line held in the frame memory 420 is updated by the difference data (IR1 + IR2).

By the operations illustrated in FIGS. 11 to 14, the integrated data of the differences between the two sets of mixed data and visible light data can be obtained. This integrated data shows the integrated value of the infrared light component IR1 in the first frame and the infrared light component IR2 in the third frame.

The arithmetic circuit 410 performs the arithmetics exemplified in FIGS. 11 to 14 for the mixed data and the visible light data of the K set, and generates the integrated data for one line. For example, if K is set to "4", the operation is executed for 8 frames. As a result, the integrated data for one line obtained by integrating the infrared light components of the first frame, the third frame, the fifth frame, and the seventh frame is generated as the line data of the TDI frame. The arithmetic circuit 410 repeats the generation of line data to generate a TDI frame in which a plurality of line data are arranged.

FIG. 15 is a diagram showing an example of a state of the solid-state image sensor 200 when holding the first frame in the first embodiment of the present technology.

The plurality of pixel circuits 220 generate a pixel signal (that is, a mixed signal) of the first frame while the near-infrared light source 500 is lit. This mixed signal contains an infrared light component R1 and a visible light component V1. Each of the ADC 310s (not shown) AD-converts the pixel signals (mixed signals) of the corresponding pixel circuits 220 to generate mixed data. Each of the ADC 310s in any one of the plurality of rows (for example, the first row) supplies mixed data to the arithmetic circuit 410.

The arithmetic circuit 410 of each column outputs mixed data to the frame memory 420. Here, in the frame memory 420, a plurality of memories 421 are arranged in a two-dimensional grid pattern. The memory 421 is provided for each pixel circuit 220. It is assumed that the mixed data (IR1 + V1) of the first frame from the arithmetic circuit 410 is held in the memory 421 of the first line.

FIG. 16 is a diagram showing an example of the state of the solid-state image sensor 200 when calculating the difference between the second frame and the first frame in the first embodiment of the present technology.

The plurality of pixel circuits 220 generate a pixel signal (visible light signal) of the second frame while the near-infrared light source 500 is turned off. The visible light component in the visible light signal in the second frame has substantially the same value as the visible light component V1 in the first frame. Each of the ADC 310s (not shown) AD-converts the pixel signal (visible light signal) of the corresponding pixel circuit 220 to generate visible light data. Of the plurality of rows, each of the ADC 310s in the row adjacent to the row read in the first frame (for example, the second row) supplies visible light data to the arithmetic circuit 410.

The arithmetic circuit 410 of each column reads the mixed data (IR1 + V1) as read data from the frame memory 420, and obtains the difference data between the read data and the visible light data (V1) by subtraction. By this subtraction, the visible light component V1 is removed.

The memory 421 of the first line in the frame memory 420 is updated by the difference data (IR1).

FIG. 17 is a diagram showing an example of the state of the solid-state image sensor 200 when the third frame and the read data in the first embodiment of the present technology are added.

The plurality of pixel circuits 220 generate a pixel signal (mixed signal) of the third frame while the near-infrared light source 500 is lit. This mixed signal contains an infrared light component R2 and a visible light component V2. Each of the ADC 310s (not shown) AD-converts the pixel signals (mixed signals) of the corresponding pixel circuits 220 to generate mixed data. Of the plurality of rows, each of the ADC 310s in the row adjacent to the row read in the second frame (for example, the third row) supplies the mixed data to the arithmetic circuit 410.

The arithmetic circuit 410 of each column reads the difference data (IR1) from the frame memory 420 as read data and adds it to the mixed data (IR2 + V2).

The memory 421 of the first line in the frame memory 420 is updated by the addition data (IR1 + IR2 + V2).

FIG. 18 is a diagram showing an example of the state of the solid-state image sensor 200 when calculating the difference between the fourth frame and the read data in the first embodiment of the present technology.

The plurality of pixel circuits 220 generate a pixel signal (visible light signal) in the fourth frame while the near-infrared light source 500 is turned off. The visible light component in the visible light signal in the 4th frame has substantially the same value as the visible light component V2 in the 3rd frame. Each of the ADC 310s (not shown) AD-converts the pixel signal (visible light signal) of the corresponding pixel circuit 220 to generate visible light data. Of the plurality of rows, each of the ADC 310s in the row adjacent to the row read in the first frame (for example, the fourth row) supplies visible light data to the arithmetic circuit 410.

The arithmetic circuit 410 of each column reads the addition data (IR1 + IR2 + V2) as read data from the frame memory 420, and obtains the difference data between the read data and the visible light data (V2) by subtraction. By this subtraction, the visible light component V2 is removed.

The memory 421 of the first line in the frame memory 420 is updated by the difference data (IR1 + IR2).

FIG. 19 is a diagram showing an example of TDI processing in the first embodiment of the present technology. For example, it is assumed that the frame memory 420 is initialized, the frame F1 is first imaged, and then the frames F2, F3, and F4 are imaged in order. The arrows in the figure indicate the moving direction of the subject. As illustrated in the figure, it is assumed that the subject moves one line at a time in the direction in which the row address increases along the vertical direction.

The synchronization control unit 130 turns on the near-infrared light source 500 and causes the solid-state image sensor 200 to generate the frame F1. The signal processing circuit 400 in the solid-state image sensor 200 holds the mixed data of each column of the line L1 of the frame F1 in the frame memory 420. Since the near-infrared light source 500 is lit, the mixed data includes the infrared light component R1 and the visible light component V1.

Next, the synchronization control unit 130 turns off the near-infrared light source 500 and causes the solid-state image sensor 200 to generate the frame F2. The signal processing circuit 400 obtains the difference data (IR1) between the visible light data of each column of the line L2 of the frame F2 and the mixed data (IR1 + V1).

Subsequently, the synchronization control unit 130 turns on the near-infrared light source 500 and causes the solid-state image sensor 200 to generate the frame F3. Since the near-infrared light source 500 is lit, the mixed data in the frame F3 includes the infrared light component R2 and the visible light component V2. The signal processing circuit 400 adds the mixed data of each column of the line L3 of the frame F3 and the difference data (IR1).

The synchronization control unit 130 turns off the near-infrared light source 500 and causes the solid-state image sensor 200 to generate the frame F4. The signal processing circuit 400 obtains the difference data (IR1 + IR2) between the visible light data (V2) of each column of the line L4 of the frame F4 and the addition data (IR1 + IR2 + V2).

The signal processing circuit 400 repeats the above processing for the mixed data and visible light data of the K set to generate integrated data for one line. When K is "4", the integrated data for one line of the TDI frame is generated by the difference calculation between the visible light data of the frame F8 and the addition data. This integrated data shows the integrated values of the infrared light components IR1, IR2, IR3 and IR4 in the first frame, the third frame, the fifth frame and the seventh frame. Line data for the second and subsequent lines of the TDI frame are also generated by the same control.

When the moving speed of the subject is fast, it is necessary to shorten the exposure time in order to prevent blurring. If the exposure time is shortened, the image may become dark, but by performing the TDI process, it is possible to integrate a plurality of lines of the same pattern to improve the brightness. Further, as the number of integrated lines increases, noise is reduced due to the smoothing effect. By improving the brightness and reducing the noise, the image quality of the frame (that is, the image data) can be improved as compared with the case where the TDI processing is not performed.

Further, the solid-state image sensor 200 can generate a TDI frame of an infrared image by integrating infrared light components. As a result, FA in a dark place can be realized.

Here, consider a comparative example in which the mixed signal and the visible light signal are held in separate floating diffusion layers and the visible light component is removed by an analog pixel circuit that outputs the difference between them. In this comparative example, since it is necessary to provide two floating diffusion layers for each pixel, the circuit scale of the pixel circuit may increase and it may be difficult to miniaturize the pixels. On the other hand, in the solid-state image sensor 200, the mixed signal and the visible light signal are AD-converted, and the visible light component is removed by the arithmetic circuit 410 (in other words, the digital circuit). Therefore, as illustrated in FIG. 8, only one floating diffusion layer is required in the pixel circuit 220, the circuit scale of the pixel circuit 220 can be reduced as compared with the comparative example, and the pixels can be easily miniaturized. ..

The signal processing circuit 400 integrates the infrared light components of four lines, but the number of integrated lines is not limited to four as long as it is two or more. Further, the signal processing circuit 400 integrates four lines from the first line for the first four frames, but the present invention is not limited to this configuration. For example, when the moving direction of the subject is opposite, the signal processing circuit 400 may integrate four lines from the last line for the first four frames.

FIG. 20 is a diagram for explaining the control of the solid-state image sensor according to the first embodiment of the present technology. The synchronization control unit 130 operates the near-infrared light source 500 in synchronization with the solid-state image sensor 200 to irradiate the near-infrared light.

The pixel drive circuit 252 in the solid-state image sensor 200 drives the pixel circuit 220 under the control of the synchronization control unit 130. While the near-infrared light source 500 is lit, the pixel circuit 220 photoelectrically converts mixed light including visible light and infrared light, and outputs a plurality of pixel signals (mixed signals) each containing visible light component and infrared light component. Generate. On the other hand, while the near-infrared light source 500 is turned off, the pixel circuit 220 photoelectrically converts visible light to generate a plurality of pixel signals (visible light signals) each containing a visible light component.

For example, it is assumed that the near-infrared light source 500 is turned on during the exposure of the odd-numbered frames F1 and F3, and the near-infrared light source 500 is turned off during the exposure of the even-numbered frames F2 and F4. In this case, each of the frames F1 and F3 includes a mixed signal for each pixel, and each of the frames F2 and F4 contains a visible light signal for each pixel.

Each of the ADC 310s in the pixel AD conversion unit 254 converts a plurality of mixed signals and a plurality of visible light signals into digital signals in order.

The arithmetic circuit 410 generates integrated data indicating the integrated value of the infrared light component included in each of the plurality of mixed signals from the digital signal by TDI processing. For example, in the TDI process, the arithmetic circuit 410 holds the mixed data of the frame F1 in the frame memory 420. Further, the arithmetic circuit 410 obtains the difference between the mixed data and the visible light data of the frame F2 as the difference data, and updates the frame memory 420 with the difference data. Then, the arithmetic circuit 410 adds the difference data and the mixed data of the frame F3 to acquire the addition data, and updates the frame memory 420 with the addition data. Subsequently, the arithmetic circuit 410 obtains the difference between the added data and the visible light data as the difference data, and updates the frame memory 420 with the difference data.

[Operation example of imaging system]
FIG. 21 is an example of a flowchart showing an example of the operation of the imaging system according to the first embodiment of the present technology. This operation is started, for example, when a predetermined application for capturing an infrared image is executed.

The near-infrared light source 500 lights up according to the control of the solid-state image sensor 200 (step S901). The ADC 310 in the solid-state image sensor 200 AD-converts a pixel signal containing an infrared light component and a visible light component (step S902). The arithmetic circuit 410 holds the mixed data of the first frame in the frame memory 420 (step S903).

The near-infrared light source 500 is turned off according to the control of the solid-state image sensor 200 (step S904). The ADC 310 in the solid-state image sensor 200 AD-converts a pixel signal containing a visible light component (step S905). The arithmetic circuit 410 subtracts the visible light data from the data held in the frame memory 420 to obtain the difference data (step S906).

Then, the solid-state image sensor 200 determines whether or not the integrated number of infrared light components is K times (step S907). When the number of integrations is K (step S907: Yes), the solid-state image sensor 200 determines whether or not the number of lines held in the frame memory 420 is N (step S911). When the number of lines is N (step S9111: Yes), the solid-state image sensor 200 ends the process for generating the TDI frame. When the number of lines is less than N (step S9111: No), the solid-state image sensor 200 changes the row address in the access destination frame memory 420 and repeatedly executes step S901 and subsequent steps.

When the number of integrations is less than K (step S907: No), the near-infrared light source 500 lights up according to the control of the solid-state image sensor 200 (step S908). The ADC 310 in the solid-state image sensor 200 AD-converts a pixel signal containing an infrared light component and a visible light component (step S909). The arithmetic circuit 410 adds the difference data and the visible light data, and updates the frame memory 420 with the added data (step S910). After step S910, the imaging system repeatedly executes step S904 and subsequent steps.

As described above, according to the first embodiment of the present technology, since the arithmetic circuit 410 generates integrated data in which infrared light components are integrated from digital signals (mixed data and visible light data), analog pixels. The circuit does not need to integrate the infrared light component. As a result, the circuit scale of the pixel circuit can be reduced and the pixels can be easily miniaturized.

<2. Second Embodiment>
In the first embodiment described above, the arithmetic circuit 410 generates a TDI frame by blinking the near-infrared light source 500 for each line and alternately performing addition and subtraction. However, in this configuration, as the number of lines in one frame increases, it is necessary to shorten the blinking interval (in other words, increase the frequency of the light emission control signal). The arithmetic circuit 410 of the second embodiment is different from the first embodiment in that the frequency of the light emission control signal is reduced to 1/2 by performing addition and subtraction twice in succession. ..

FIG. 22 is a diagram showing an example of the state of the arithmetic circuit 410 when holding the first frame in the second embodiment of the present technology. In the second embodiment, a light emission control signal having a frequency 1/2 that of the first embodiment is supplied. As a result, the near-infrared light source 500 repeats the operation of continuously turning on the lights for two frames and then turning off the lights continuously for the next two frames.

The pixel AD conversion unit 254 generates mixed data of the first frame while the near-infrared light source 500 is lit. This mixed data includes an infrared light component IR1 and a visible light component V1.

FIG. 23 is a diagram showing an example of the state of the arithmetic circuit 410 when adding the second frame and the first frame in the second embodiment of the present technology.

The pixel AD conversion unit 254 generates mixed data of the second frame while the near-infrared light source 500 is lit. This mixed data includes infrared and visible light components. The infrared light component in the second frame is IR2, and the visible light component in the second frame is V2.

Mixed data (IR2 + V2) in the corresponding column of the second frame is input to the buffer 411. The buffer 411 delays the mixed data and outputs it to the adder 414.

The selector 412 reads the mixed data (IR1 + V1) from the frame memory 420 and supplies it as read data to the adder 414 via the selector 413.

The adder 414 adds the mixed data (IR2 + V2) and the read data (IR1 + V1) and outputs the added data to the selector 416. The added data includes the infrared light component IR1 and the visible light component V1 in the first frame, and the infrared light component IR2 and the visible light component V2 in the second frame.

The selector 416 outputs the addition data (IR1 + IR2 + V1 + V2) from the adder 414 to the frame memory 420.

By the above control, the mixed data (IR1 + V1) for one line held in the frame memory 420 is updated by the addition data (IR1 + IR2 + V1 + V2).

FIG. 24 is a diagram showing an example of the state of the calculation circuit when calculating the difference between the third frame and the read data in the second embodiment of the present technology.

The pixel AD conversion unit 254 generates visible light data in the third frame while the near-infrared light source 500 is turned off. The visible light component in the visible light data in the third frame has a value V1'close to the visible light component V1 in the first frame.

Visible light data (V1') in the corresponding column of the third frame is input to the buffer 411. The buffer 411 delays the visible light data and outputs it to the subtractor 415.

The selector 412 reads the addition data (IR1 + IR2 + V1 + V2) from the frame memory 420 and supplies it to the subtractor 415 as read data.

The subtractor 415 obtains the difference data by subtracting the visible light data (V1') from the read data (IR1 + IR2 + V1 + V2). By this subtraction, the visible light component V1 is removed.

The selector 416 outputs the difference data (IR1 + IR2 + V2) from the subtractor 415 to the frame memory 420.

By the above-mentioned control, the addition data (IR1 + IR2 + V1 + V2) for one line held in the frame memory 420 is updated by the difference data (IR1 + IR2 + V2).

FIG. 25 is a diagram showing an example of the state of the calculation circuit when calculating the difference between the fourth frame and the read data in the second embodiment of the present technology.

The pixel AD conversion unit 254 generates visible light data in the fourth frame while the near-infrared light source 500 is turned off. The visible light component in the visible light data in the 4th frame has a value V2'close to the visible light component V1 in the 2nd frame.

Visible light data (V2') in the corresponding column of the 4th frame is input to the buffer 411. The buffer 411 delays the visible light data and outputs it to the subtractor 415.

The selector 412 reads the difference data (IR1 + IR2 + V2) from the frame memory 420 and supplies it to the subtractor 415 as read data.

The subtractor 415 obtains the difference data by subtracting the visible light data (V2') from the read data (IR1 + IR2 + V2). By this subtraction, the visible light component V2 is removed.

When generating one line from the 8th frame, the arithmetic circuit 410 subsequently adds the difference data and the mixed data of the 5th frame, and adds the added data and the mixed data of the 6th frame. Then, the arithmetic circuit 410 calculates the difference between the added data and the visible light data in the 7th frame, and calculates the difference between the difference data and the visible light data in the 8th frame. As a result, one line of TDI frames is generated from 8 frames. The second and subsequent lines of the TDI frame are also generated by the same calculation. In this way, each of addition and subtraction is executed twice.

FIG. 26 is a diagram showing an example of the state of the solid-state image sensor 200 when holding the first frame in the second embodiment of the present technology.

The plurality of pixel circuits 220 generate a pixel signal (mixed signal) of the first frame while the near-infrared light source 500 is lit. This mixed signal contains an infrared light component R1 and a visible light component V1. Each of the ADC 310s (not shown) AD-converts the pixel signals (mixed signals) of the corresponding pixel circuits 220 to generate mixed data. Each of the ADC 310s in any one of the plurality of rows (for example, the first row) supplies mixed data to the arithmetic circuit 410.

The arithmetic circuit 410 of each column outputs mixed data to the frame memory 420. It is assumed that the mixed data (IR1 + V1) of the first frame from the arithmetic circuit 410 is held in the memory 421 of the first line.

FIG. 27 is a diagram showing an example of the state of the solid-state image sensor 200 when adding the first frame and the second frame in the second embodiment of the present technology.

The plurality of pixel circuits 220 generate a pixel signal (mixed signal) of the second frame while the near-infrared light source 500 is lit. This mixed signal contains an infrared light component R2 and a visible light component V2. Each of the ADC 310s (not shown) AD-converts the pixel signals (mixed signals) of the corresponding pixel circuits 220 to generate mixed data. Of the plurality of rows, each of the ADC 310s in the row adjacent to the row read in the first frame (for example, the second row) supplies the mixed data to the arithmetic circuit 410.

The arithmetic circuit 410 in each column reads the mixed data (IR1 + V1) from the frame memory 420 as read data and adds it to the mixed data (IR2 + V2) in the second frame.

The memory 421 of the first line in the frame memory 420 is updated by the addition data (IR1 + IR2 + V1 + V2).

FIG. 28 is a diagram showing an example of the state of the solid-state image sensor 200 when calculating the difference between the third frame and the read data in the second embodiment of the present technology.

The plurality of pixel circuits 220 generate a pixel signal (visible light signal) of the third frame while the near-infrared light source 500 is turned off. The visible light component in the visible light signal in the third frame has a value V1'close to the visible light component V1 in the third frame. Each of the ADC 310s (not shown) AD-converts the pixel signal (visible light signal) of the corresponding pixel circuit 220 to generate visible light data. Of the plurality of rows, each of the ADC 310s adjacent to the row read in the second frame (for example, the third row) supplies visible light data to the arithmetic circuit 410.

The arithmetic circuit 410 of each column reads the added data (IR1 + IR2 + V1 + V2) as read data from the frame memory 420, and obtains the difference data between the read data and the visible light data (V1') by subtraction. By this subtraction, the visible light component V1 is removed.

The memory 421 of the first line in the frame memory 420 is updated by the difference data (IR1 + IR2 + V2).

FIG. 29 is a diagram showing an example of the state of the solid-state image sensor 200 when calculating the difference between the fourth frame and the read data in the second embodiment of the present technology.

The plurality of pixel circuits 220 generate a pixel signal (visible light signal) in the fourth frame while the near-infrared light source 500 is turned off. The visible light component in the visible light signal in the 4th frame has a value V2'close to the visible light component V2 in the 2nd frame. Each of the ADC 310s (not shown) AD-converts the pixel signal (visible light signal) of the corresponding pixel circuit 220 to generate visible light data. Of the plurality of rows, each of the ADC 310s in the row adjacent to the row read in the third frame (for example, the fourth row) supplies visible light data to the arithmetic circuit 410.

The arithmetic circuit 410 of each column reads the difference data (IR1 + IR2 + V2) from the frame memory 420 as read data, and obtains the difference data between the read data and the visible light data (V2') by subtraction. By this subtraction, the visible light component V2 is removed.

As illustrated in FIGS. 22 to 29, the arithmetic circuit 410 adds the mixed data of the first frame and the mixed data of the second frame. Then, the arithmetic circuit 410 acquires the difference between the addition data and the visible light data in the third frame as the difference data, and obtains the difference between the difference data and the visible light data in the fourth frame. In this way, the arithmetic circuit 410 performs addition and subtraction twice in succession. Therefore, the frequency of the light emission control signal can be halved as compared with the first embodiment.

As described above, in the second embodiment of the present technology, since the arithmetic circuit 410 performs addition and subtraction twice in succession, it is different from the first embodiment in which addition and subtraction are performed once. By comparison, the frequency of the light emission control signal can be halved.

Note that the above-described embodiment shows an example for embodying the present technology, and the matters in the embodiment and the matters specifying the invention in the claims have a corresponding relationship with each other. Similarly, the matters specifying the invention within the scope of claims and the matters in the embodiment of the present technology having the same name have a corresponding relationship with each other. However, the present technology is not limited to the embodiment, and can be embodied by applying various modifications to the embodiment without departing from the gist thereof.

It should be noted that the effects described in the present specification are merely examples and are not limited, and other effects may be obtained.

The present technology can have the following configurations.
(1) A pixel circuit that generates a plurality of mixed signals each containing a visible light component and an invisible light component and a plurality of visible light signals obtained by photoelectrically converting visible light.
An analog-to-digital converter that converts each of the plurality of mixed signals and the plurality of visible light signals into a digital signal.
A solid-state imaging device including an arithmetic circuit that generates integrated data indicating an integrated value of the invisible light component included in each of the plurality of mixed signals from the digital signal.
(2) The plurality of mixed signals include the first and second mixed signals.
The plurality of visible light signals include first and second visible light signals.
The digital signal includes first and second mixed data obtained by converting the first and second mixed signals and first and second visible light data obtained by converting the first and second visible light signals. Including
The arithmetic circuit adds the first subtraction process for obtaining the difference between the first mixed data and the first visible light data as the difference data, and the addition of the difference data and the second mixed data. The solid-state imaging device according to (1) above, wherein an addition process for acquiring data and a second subtraction process for obtaining a difference between the added data and the second visible light data are sequentially performed.
(3) The plurality of mixed signals include the first and second mixed signals.
The plurality of visible light signals include first and second visible light signals.
The digital signal includes first and second mixed data obtained by converting the first and second mixed signals and first and second visible light data obtained by converting the first and second visible light signals. Including
The arithmetic circuit uses the difference between the addition process of adding the first mixed data and the second mixed data to acquire the addition data and the difference between the addition data and the first visible light data as difference data. The solid-state imaging device according to (1) above, wherein the first subtraction process to be acquired and the second subtraction process for obtaining the difference between the difference data and the second visible light data are sequentially performed.
(4) Further equipped with a frame memory
The arithmetic circuit
An adder that adds the digital signal from the analog-to-digital converter and the read data read from the frame memory and outputs the adder data.
A subtractor that outputs the difference between the read data and the digital signal as difference data, and
The solid-state image sensor according to any one of (1) to (3), further comprising a selector that selects either the addition data or the difference data and outputs the data to the frame memory.
(5) The solid-state image sensor according to any one of (1) to (4) above, further comprising a synchronization control unit that operates an invisible light source that irradiates invisible light in synchronization with the pixel circuit.
(6) The pixel circuit and a part of the analog-to-digital converter are arranged on a predetermined light receiving chip.
The solid-state image sensor according to any one of (1) to (5), wherein the rest of the analog-to-digital converter and the arithmetic circuit are arranged on a circuit chip laminated on the light receiving chip.
(7) An invisible light source that irradiates invisible light,
A pixel circuit that generates a plurality of mixed signals each containing a visible light component and an invisible light component and a plurality of visible light signals obtained by photoelectrically converting visible light.
An analog-to-digital converter that converts each of the plurality of mixed signals and the plurality of visible light signals into a digital signal.
An imaging system including an arithmetic circuit that generates integrated data indicating an integrated value of the invisible light component included in each of the plurality of mixed signals from the digital signal.
(8) A signal generation procedure for generating a plurality of mixed signals each containing a visible light component and an invisible light component and a plurality of visible light signals obtained by photoelectric conversion of visible light.
An analog-to-digital conversion procedure for converting each of the plurality of mixed signals and the plurality of visible light signals into a digital signal, and
A method for controlling a solid-state image sensor, which comprises a calculation procedure for generating integrated data indicating an integrated value of the invisible light component included in each of the plurality of mixed signals from the digital signal.

100 Image sensor 110 Optical unit 120 Storage unit 130 Synchronous control unit 140 Communication unit 200 Solid-state image sensor 201 Light receiving chip 202 Circuit chip 210 Pixel array unit 211 Pixel block 212 Peripheral circuit 220 Pixel circuit 221 Reset transistor 222 Floating diffusion layer 223 Transfer transistor 224 Photodiode 225 Ejection transistor 251 DAC
252 Pixel drive circuit 253 Time code generator 254 Pixel AD converter 255 Vertical scanning circuit 256 Control circuit 257 Output circuit 260 Image processing circuit 300 Cluster 310 ADC
320

Differential input circuit

321, 324, 326, 331, 332, 334, 335

pMOS transistor

322, 323, 325, 327, 333, 336, 337 nMOS transistor 330 Positive feedback circuit 340 Latch control circuit 350 Latch circuit 360 Repeater section 400

Signal processing circuit

405, 412, 413, 416 Selector 410 Arithmetic circuit 411 Buffer 414 Adder 415 Adder 420 Frame memory 421 Memory 500 Near infrared light source 510 Belt conveyor 511 Subject

Claims

A pixel circuit that generates a plurality of mixed signals each containing a visible light component and an invisible light component and a plurality of visible light signals obtained by photoelectrically converting visible light.
An analog-to-digital converter that converts each of the plurality of mixed signals and the plurality of visible light signals into a digital signal.
A solid-state imaging device including an arithmetic circuit that generates integrated data indicating an integrated value of the invisible light component included in each of the plurality of mixed signals from the digital signal.
The plurality of mixed signals include the first and second mixed signals.
The plurality of visible light signals include first and second visible light signals.
The digital signal includes first and second mixed data obtained by converting the first and second mixed signals and first and second visible light data obtained by converting the first and second visible light signals. Including
The arithmetic circuit adds the first subtraction process for obtaining the difference between the first mixed data and the first visible light data as the difference data, and the addition of the difference data and the second mixed data. The solid-state imaging device according to claim 1, wherein an addition process for acquiring data and a second subtraction process for obtaining a difference between the added data and the second visible light data are sequentially performed.
The plurality of mixed signals include the first and second mixed signals.
The plurality of visible light signals include first and second visible light signals.
The digital signal includes first and second mixed data obtained by converting the first and second mixed signals and first and second visible light data obtained by converting the first and second visible light signals. Including
The arithmetic circuit uses the difference between the addition process of adding the first mixed data and the second mixed data to acquire the addition data and the difference between the addition data and the first visible light data as difference data. The solid-state imaging device according to claim 1, wherein the first subtraction process to be acquired and the second subtraction process for obtaining the difference between the difference data and the second visible light data are performed in order.
With more frame memory,
The arithmetic circuit
An adder that adds the digital signal from the analog-to-digital converter and the read data read from the frame memory and outputs the adder data.
A subtractor that outputs the difference between the read data and the digital signal as difference data, and
The solid-state image sensor according to claim 1, further comprising a selector that selects either the addition data or the difference data and outputs the data to the frame memory.
The solid-state image sensor according to claim 1, further comprising a synchronization control unit that operates an invisible light source that irradiates invisible light in synchronization with the pixel circuit.
The pixel circuit and a part of the analog-to-digital converter are arranged on a predetermined light receiving chip.
The solid-state image sensor according to claim 1, wherein the rest of the analog-to-digital converter and the arithmetic circuit are arranged on a circuit chip laminated on the light receiving chip.
An invisible light source that irradiates invisible light,
A pixel circuit that generates a plurality of mixed signals each containing a visible light component and an invisible light component and a plurality of visible light signals obtained by photoelectrically converting visible light.
An analog-to-digital converter that converts each of the plurality of mixed signals and the plurality of visible light signals into a digital signal.
An imaging system including an arithmetic circuit that generates integrated data indicating an integrated value of the invisible light component included in each of the plurality of mixed signals from the digital signal.
A signal generation procedure for generating a plurality of mixed signals each containing a visible light component and an invisible light component and a plurality of visible light signals obtained by photoelectric conversion of visible light.
An analog-to-digital conversion procedure for converting each of the plurality of mixed signals and the plurality of visible light signals into a digital signal, and
A method for controlling a solid-state image sensor, which comprises a calculation procedure for generating integrated data indicating an integrated value of the invisible light component included in each of the plurality of mixed signals from the digital signal.