JP2012244357A - Imaging method and imaging apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To eliminate influence from external light and noise with a simple circuit structure to improve measuring accuracy, when picking up images of an object by reflection light from the light emitted for measuring depth.SOLUTION: One frame period includes: two exposure periods A, B; one light emission period corresponding to one exposure period B; and three output periods C, A, B. In the output period C, random noise without including image signals is output, in the output period A, image signals and random noise in the exposure period A are output, and in the output period B, the image signals in the exposure period A and image signals and random noise in the exposure period B are output. Signals being output in the three periods are calculated to eliminate the image signals and the random noise by the external light in order to obtain image signals by only the emitted light.

Description

  The present invention relates to an imaging method and an imaging apparatus, and more particularly to an imaging method and an imaging apparatus suitable for irradiating emitted light onto a subject and imaging the subject from the reflected light.

  In recent years, a device that emits a planar pattern to irradiate a subject, captures the reflected light, and measures the depth of the subject has been commercialized and used in home game machines and the like. When such a depth measuring device is used outdoors as a vehicle-mounted object recognition device or the like, interference caused by outside light increases. In an object farther away, the irradiation light from the emitted flat pattern and the reflected light from the subject are weakened, so the influence of noise such as random noise cannot be ignored. As a general method for improving the S / N ratio with respect to external light, there is a method in which light emission is pulsed to increase a peak light emission amount and an image is picked up only during a light emission period by a shutter function of the image pickup apparatus. In general, as the light emission period is shortened, the amount of light emission during that period can be increased. Therefore, if light is received only during that period, the SN ratio with respect to external light can be improved.

  In this imaging method using pulsed light emission, an imaging device having a rolling shutter function in which the exposure timing is shifted for each row cannot be used, and a global shutter function for exposing all pixels at the same timing is necessary. In recent years, a CMOS type image pickup device having such a global shutter function has been developed. For example, Patent Document 1 discloses that sufficient exposure can be achieved with a quick operation by reducing the restriction of exposure time when the global shutter function is realized. Techniques for securing time are disclosed.

JP 2004-140149 A

First, a conventionally known global shutter function will be described.
FIG. 11 is a circuit configuration diagram showing a first conventional example of a CMOS type imaging device having a global shutter function, which is drawn around the pixel portion 100. The signal charge photoelectrically converted by the photodiode (PD) 101 is completely transferred to the floating diffusion layer (FD) 102 via the transfer gate (TG) 103. Then, a signal voltage proportional to the signal charge is read to the vertical signal line 200 connected to the current source 201 by the source follower amplifier 105 and the row selection gate (LS) 106. Thereafter, the signal is converted into a digital signal by an AD converter (ADC) 300 and output to a horizontal output circuit. In the example of FIG. 11, a blooming sweep gate (BG) 107 is provided, and the signal charge of the photodiode 101 can be swept out by increasing the gate voltage (BG voltage). Thereafter, a pulse is applied to the voltage (TG voltage) of the transfer gate 103 to transfer the signal charge to the floating diffusion layer 102. The period until that time is the exposure period. By controlling the BG voltage and the TG voltage with the same pulse for all the pixels, the exposure timing of all the pixels can be made equal, that is, a global shutter function is realized. The reset gate (RG) 104 is for resetting the electric charge accumulated in the floating diffusion layer 102.

  FIG. 12 is a timing chart illustrating an example of the operation of the imaging apparatus in FIG. A period from time T1 to time T6 is one frame period (for example, 17 msec), and is a period for capturing one image in the moving image. Among these, the accumulation of signal charges in the photodiode 101 starts from the time T3 when the BG pulse is lowered, and the signal charges accumulated when the TG pulse is raised starts to be transferred to the floating diffusion layer 102. The transfer is completed at the falling time T4. At this time, the exposure period of the signal charges transferred and held in the floating diffusion layer 102 is a period from T3 to T4. The signal charge information held by the floating diffusion layer 102 is then sequentially transmitted to the vertical signal line 200 in the form of voltage for each row. That is, when the pulse is input to the row selection gate LS (n) in the pixel portion 100 of the nth row, and when the pulse is input to the row selection gate LS (n + 1) of the pixel portion 100 of the (n + 1) th row, the vertical selection is performed. Each signal information is transmitted to the signal line 200, converted into a digital signal by the AD converter 300, and then output through a horizontal output circuit. Although the output is performed in order for each row, the exposure period is the same for all the pixels, and the global shutter function is realized.

  In this manner, the SN ratio with respect to external light can be relatively improved by shortening the exposure time using an imaging apparatus having a global shutter function. However, when the outside light is strong, such as in the daytime in fine weather, the influence of the outside light cannot be ignored only by the shutter operation, and it may be necessary to devise a high-performance outside light removal. Further, since the signal amount generally decreases with the high-speed shutter, it is necessary to further reduce noise such as random noise. When light emission is pulsed, the peak light quantity can be increased by shortening the pulse width, but the area is usually reduced and the total signal quantity is reduced. Note that the fundamental noise among the random noise generated in the circuit of FIG. 11 is so-called “kTC noise” generated when the floating diffusion layer 102 is reset by the RG pulse. Here, k is the Boltzmann constant, T is the absolute temperature, C is the capacitance of the floating diffusion layer 102, and the square of the dispersion of the random noise charge is expressed as kTC, so it is generally called “kTC noise”.

  FIG. 13 is a circuit configuration diagram showing a second conventional example of a CMOS type imaging device having a global shutter function, which is configured to suppress the kTC noise. In the configuration of FIG. 11, a second photodiode (PD2) 108 and a second transfer gate (TG2) 109 are arranged between the photodiode (PD) 101 and the floating diffusion layer (FD) 102. Further, between the vertical signal line 200 and the AD converter 300, first and second sample hold circuits (SH1, SH2) 202, 203 and a differential amplifier (AMP) 204 are arranged. In this configuration, the number of transistors increases and the aperture ratio decreases, but kTC noise can be suppressed by the following operation. The added second photodiode (PD2) 108 is shielded from the upper part and does not perform photoelectric conversion, but has a structure similar to that of the photodiode 101 in that it can be depleted and charge can be completely transferred to the next stage. ing.

  FIG. 14 is a timing chart showing an example of the operation of the imaging apparatus of FIG. The signal charge is accumulated in the photodiode (PD) 101 from the time T3 when the BG pulse is lowered. When the TG pulse is applied, the accumulated signal charge is transferred to the second photodiode (PD2) 108, and the TG pulse is raised. At the lowered time T4, the transfer of the signal charge is finished and held by the second photodiode 108. The period from time T3 to T4 is common to all pixels in the exposure period. Thereafter, signals are sequentially read out to the floating diffusion layer (FD) 102 and the vertical signal line 200 for each subsequent row. In the pixel unit 100 in the n-th row, first, the reset gate RG (n) 104 becomes a low potential by the time T7, and kTC noise is sequentially held in the floating diffusion layer 102. At this time, by setting the row selection gate LS (n) 106 to a high potential, voltage information of this kTC noise is transmitted to the vertical signal line 200 via the source follower amplifier 105. At the timing of this time T7, the voltage information of kTC noise is held by sample-holding by the first sample-hold circuit (SH1) 202. Next, by applying a pulse to the second transfer gate TG2 (n) 109, the signal charge of the second photodiode (PD2) 108 is transferred to the floating diffusion layer 102, and the held kTC noise charge and Is added. Since the signal + noise voltage information is transmitted to the vertical signal line 200 at the timing of the next time T8, it is sampled and held by the second sample hold circuit (SH2) 203. By taking the difference between the two by the differential amplifier (AMP) 204, kTC noise can be removed and only signal information can be output.

  According to the first conventional example described above (FIGS. 11 and 12), the time width of the light emission pulse is narrowed to reduce the influence of external light, but it is necessary to reduce the influence of noise such as kTC noise. . Further, according to the second conventional example (FIGS. 13 and 14), kTC noise can be suppressed, but the circuit configuration is complicated and complicated timing control is required.

  An object of the present invention is to provide an imaging method and an imaging apparatus that improve the measurement accuracy by eliminating the influence of external light and noise with a simple circuit configuration when imaging a subject from reflected light of light emitted for depth measurement. There is.

  The present invention provides an imaging method of irradiating a subject with emitted light and imaging the subject from the reflected light, providing a plurality of exposure periods within one frame period, exposing each period, and the plurality of exposures. Providing a light emission period that coincides with one exposure period in the period, emitting each of the video signals obtained in the plurality of exposure periods, calculating the plurality of output video signals, The video signal due to the external light is removed and the video signal based only on the light emission is acquired.

  The imaging method of the present invention is to provide an exposure period and a light emission period within one frame period, and provide a plurality of output periods within one frame period to output a signal obtained in each period, In the first output period of the plurality of output periods, random noise not including a video signal is output, and in the second and subsequent output periods, the video signal and random noise obtained in the exposure period are output, and the output A video signal from which random noise is removed is obtained by calculating the signals of the plurality of output periods.

  The present invention relates to an photodiode arranged in an array that generates a signal charge by photoelectrically converting reflected light from the subject in an imaging apparatus that irradiates the subject with emitted light and images the subject from the reflected light. A group, an output circuit that sequentially reads video signals from signal charges generated by the photodiode group, an arithmetic circuit that calculates an output signal from the output circuit, an exposure operation of the photodiode group, and a readout of the output circuit And a pulse generation circuit for controlling the light emission operation of the light emitting device, wherein the pulse generation circuit provides a plurality of exposure periods within one frame period for the photodiode group. Exposing in each period, causing the light-emitting device to emit light by providing a light-emitting period that matches one exposure period of the plurality of exposure periods, The output circuit outputs the respective video signals obtained in the plurality of exposure periods, and the arithmetic circuit calculates the plurality of video signals output from the output circuit and removes the video signal due to the external light. The video signal is obtained only by light emission.

  According to the present invention, when the subject is irradiated with the emitted light and the subject is imaged from the reflected light, it is possible to remove the influence of external light and obtain a video signal based only on the light emission. Furthermore, the influence of random noise can be removed with a simple circuit configuration, and a video signal with a high S / N ratio can be acquired. As a result, the depth of an outdoor subject or a distant subject can be accurately measured.

3 is a timing chart showing an exemplary embodiment of an imaging method according to the present invention. The circuit block diagram which shows the typical Example of the imaging device by this invention. 6 is a timing chart (first embodiment) illustrating a first embodiment of an imaging method. 9 is a timing chart (second embodiment) illustrating a second embodiment of the imaging method. 9 is a timing chart (third embodiment) illustrating a third embodiment of the imaging method. FIG. 10 is a potential diagram illustrating a signal transmission operation according to the third embodiment. 9 is a timing chart illustrating a fourth example of the imaging method (Example 4). FIG. 10 is a configuration diagram illustrating a fifth embodiment of the imaging apparatus (embodiment 5). The timing chart which shows one Example of the imaging method in the imaging device of FIG. The figure which showed the timing chart of FIG. 9 in detail. The circuit block diagram which shows the 1st prior art example of an imaging device. FIG. 12 is a timing chart illustrating an example of the operation of the imaging apparatus in FIG. 11. FIG. The circuit block diagram which shows the 2nd prior art example of an imaging device. FIG. 14 is a timing chart illustrating an example of the operation of the imaging apparatus in FIG. 13.

  In the present invention, when the video signal is read out by the image pickup apparatus, the video signal exposed during the period of light emission in one frame and the video signal exposed during the period of no light emission are obtained, and the influence of external light is obtained by their calculation. Rather, it was determined to obtain an image based only on the reflection of the emitted light.

  FIG. 1 is a timing chart showing an exemplary embodiment of an imaging method according to the present invention. In one frame period (from time T1 to T6), one light emission period (from time T4 to T5) and two exposure periods A (from time T3 to T4) and B (from time T4 to T5) are provided. Further, there are three output periods, namely, an output period A (from time T4 to T5), an output period B (from time T5 to T6), and an output period C (from time T2 to T4).

  In the example of FIG. 1, the exposure period A is a non-light emission period, and the signal A obtained in the exposure period A is a video signal based only on external light. Since the exposure period B coincides with the light emission period, the signal B obtained in the exposure period B is a video signal by external light and light emission. Regarding output, after resetting the floating diffusion layer 102 at time T2, only random noise (kTC noise) is output in the output period C, and after transferring the signal A to the floating diffusion layer 102 at time T4, the output period A A signal obtained by adding the signal A to the kTC noise is output. After the signal B is transferred to the floating diffusion layer 102 at time T5, a signal obtained by adding the signal B to the kTC noise and the signal A is output in the output period B.

  The following calculation is performed from these output signals. By subtracting the output signal of the output period C from the output signal of the output period A, a signal A (signal of the exposure period A) based only on outside light excluding kTC noise can be obtained. Further, by subtracting the output signal of the output period A from the output signal of the output period B, a signal B (signal of the exposure period B) by external light and light emission excluding the kTC noise and the signal A is obtained. A video signal based only on light emission can be obtained by the calculation of the signals A and B. In the calculation, the signals in each output period can be subtracted by multiplying by an appropriate coefficient. For example, the exposure period A and the exposure period B can be subtracted. If the lengths are equal, each coefficient is 1. Even when the light emission period is made to coincide with the exposure period A, desired information can be obtained by the same calculation.

  FIG. 2 is a circuit configuration diagram showing a typical embodiment of the imaging apparatus according to the present invention. The internal configuration of the pixel portion 100 is the same as that shown in FIG. 11 and includes one photodiode (PD) 101 per pixel. In the imaging device, such pixel portions (photodiodes) are arranged in an array. As a control function for reading a video signal from each pixel, a pulse generation circuit 600 is provided to supply a pulse group 601 into the pixel. The pulse group 601 includes gate control pulses such as BG, TG, RG, and LS (n). The pulse generation circuit 600 sends a control signal to the light emitting device 700 so as to emit light for a predetermined light emission period. The light emitting device 700 may be built in the imaging device or externally attached.

The horizontal output circuit 400 sequentially outputs signals AD-converted in each column. The arithmetic circuit 500 calculates a plurality of video signals output from each pixel, removes the influence of external light and random noise (kTC noise), and obtains a video signal based only on light emission necessary for depth measurement. In some cases, a video signal using only external light can be obtained and used for subject recognition or the like.
Hereinafter, a plurality of embodiments according to the present invention will be described in detail.

  FIG. 3 is a timing chart showing the first embodiment of the imaging method of the present invention. The imaging apparatus has the configuration shown in FIG. 2 and supplies a pulse group 601 for gate control from a pulse generation circuit 600. The gate control pulse includes a BG (blooming sweep gate) pulse, a TG (transfer gate) pulse, an RG (reset gate) pulse, an LS (row selection gate) pulse, and the like. The functions of these gate pulses are the same as those in FIGS. 11 and 12, and repeated description is omitted. In this embodiment, two exposure periods A and B are provided in one frame and are read independently.

  First, the first exposure period A and the output period A will be described. At time T3, the BG pulse falls to start accumulating signal charge in the photodiode 101, and at time T4, the TG pulse falls to complete signal charge transfer from the photodiode 101 to the floating diffusion layer 102. This period from time T3 to T4 is the first exposure period (exposure period A). The signal charges transferred to the floating diffusion layer 102 are sequentially set to high levels in the row selection gates LS (n), LS (n + 1),... Corresponding to each pixel in the period from time T4 to T5 (output period A). As a result, the voltage is sequentially transmitted to the vertical signal line 200 in the form of voltage, and is output via the AD converter 300 and the horizontal output circuit 400. The charge remaining in the floating diffusion layer 102 is reset by applying an RG pulse at an appropriate time T9 after the signal transmission to the series of vertical signal lines 200 is completed.

  Next, the second exposure period B and its output period B will be described. At time T4, new signal charges start to be accumulated in the photodiode 101. At time T5, the TG pulse is lowered again, and the signal charge transfer from the photodiode 101 to the floating diffusion layer 102 is completed. This period from time T4 to T5 is the second exposure period (exposure period B). The signal charges transferred to the floating diffusion layer 102 sequentially increase LS (n), LS (n + 1),... In the period from time T5 to T6 (output period B), as in the first time. As a result, the voltage is sequentially transmitted to the vertical signal line 200 in the form of voltage, and output through the AD converter 300 and the horizontal output circuit. The electric charge remaining in the floating diffusion layer 102 is reset by applying an RG pulse at an appropriate time T2 after the signal transmission to the series of vertical signal lines 200 is completed.

  As described above, in the embodiment of FIG. 3, two exposure periods A and B are set in one frame period, and the respective signal information is output in two output periods A and B. For example, when the light emission period coincides with the exposure period B, a video signal based only on the external light of the exposure period A is obtained from the output signal of the output period A, and the external light of the exposure period B is obtained from the output signal of the output period B. A video signal by light emission is obtained. By calculating (subtracting) these two output signals by the calculation circuit 500, the video signal due to external light can be removed, and the video signal based only on light emission can be obtained. In other words, when measuring the depth of the subject by imaging the reflected light during the light emission period, the depth information can be accurately acquired. Further, as an accompanying effect, an image using only external light that is not affected by light emission can be obtained from the signal of the exposure period A, which is suitable for recognition of a subject.

  FIG. 4 is a timing chart showing a second embodiment of the imaging method of the present invention. The change from the first embodiment (FIG. 3) is that the reset gate RG is kept at a low level between the output period A and the output period B, and the floating diffusion layer 102 is not reset. As a result, the signal charge in the exposure period A is not reset after being output in the output period A but is held in the floating diffusion layer 102 and is read again in the output period B. That is, the output signal in the output period A is a video signal by the external light in the exposure period A, whereas the output signal in the output period B is the video signal by the light emission in the exposure period B and the video signal by the external light in the exposure period B. To which the video signal of the external light during the exposure period A is added. As in the case of the first embodiment (FIG. 3), the arithmetic circuit 500 calculates (subtracts) these two output signals by multiplying them by a predetermined coefficient, thereby removing the video signal due to the external light, and the video only based on the light emission. A signal can be obtained. In addition, there is an accompanying effect that a video signal using only external light can be obtained. In the configuration of this embodiment, there is an advantage that the margin of timing setting is increased by reducing one RG pulse.

  FIG. 5 is a timing chart showing a third embodiment of the imaging method of the present invention. The difference from the second embodiment (FIG. 4) is that an output period C is further added between time T2 and T4 to output kTC noise. Specifically, after the RG pulse is applied at time T2 to reset the floating diffusion layer 102, pulses are sequentially applied to LS (n), LS (n + 1),. The voltage information of the kTC noise to be transmitted is transmitted to the vertical signal line 200. That is, in this output period C, there is no video signal and only kTC noise is output. Thereafter, as in the second embodiment (FIG. 4), the image signal generated by the external light in the exposure period A is output in the output period A, and the light output in the exposure period B and the video signal generated by the external light are output in the output period B. The video signal added by is output.

  Considering kTC noise in the output periods A and B, when the signal charge of the photodiode 101 is transferred to the floating diffusion layer 102 by applying a TG pulse at time T4, the signal charge in the exposure period A is transferred to the floating diffusion layer 102. kTC noise is retained. That is, in the output period A, kTC noise is output in addition to the video signal (video signal generated by the external light during the exposure period A). Similarly, in the output period B, kTC noise is output in addition to the video signal (the video signal due to light emission and external light during the exposure period B and the video signal due to external light during the exposure period A). By calculating these three output signals of the output periods A, B, and C, it is possible to obtain a video signal based only on light emission and a video signal based only on outside light with the kTC noise removed.

  FIG. 6 is a potential diagram showing the signal transmission operation of this embodiment, and shows the potential distribution inside the image sensor at each time in the timing chart of FIG.

At time T2, the RG pulse is lowered to disconnect the FD (floating diffusion layer 102) from the power supply Vdd. At this time, kTC noise 404 remains in the FD. In this state, the LS pulse is applied and the signal is read from the FD in the output period C. Further, a signal charge 401 is generated and accumulated in the PD (photodiode 101) by photoelectric conversion.
At time T3, a BG pulse is applied to deplete the PD and sweep the signal charge 401 to the power source. When the BG pulse is lowered, signal charge 402 is newly accumulated in the PD (exposure period A).

At time T4, a TG pulse is applied to deplete the PD, and the signal charge 402 generated by the PD is transferred to the FD after T3 (exposure period A). As a result, the kTC noise 404 and the signal charge 402 are held in the FD. In this state, the output period A is when the LS pulse is applied and the signal is read from the FD. When the transfer of the signal charge 402 to the FD is completed, accumulation of the signal charge 403 newly starts in the PD (exposure period B).
At time T5, the TG pulse is applied again to deplete the PD, and the signal charge 403 generated in the PD is transferred to the FD after T4 (exposure period B). As a result, the kTC noise 404, the signal charge 402, and the signal charge 403 are held in the FD. In this state, the output period B is when the LS pulse is applied and the signal is read from the FD.

In this way, the kTC noise 404 can be output in the output period C, the kTC noise 404 and the signal charge 402 can be output in the output period A, and the kTC noise 404, the signal charge 402, and the signal charge 403 can be output in the output period B, respectively. Information of each signal charge can be separated and acquired by a subsequent calculation.
The imaging method of this embodiment is as described as a typical example in FIG.

The present invention can also be applied to an imaging apparatus having two photodiodes per pixel described in FIG.
FIG. 7 is a timing chart showing the fourth embodiment of the imaging method of the present invention, which is applied to the imaging apparatus of FIG. The imaging device includes a second photodiode (PD2) 108 and a second transfer gate (TG2) 109, and first and second sample and hold circuits (SH1, SH2) 202, 203 and a differential amplifier (AMP). 204), kTC noise can be removed and only signal information can be output.

  This embodiment differs from the timing chart of FIG. 14 in that two exposure periods A and B and corresponding output periods A and B are provided in one frame period. In FIG. 7, the display of the RG pulse, TG2 pulse, and LS pulse shown in FIG. 14 is omitted. For example, when the light emission period coincides with the exposure period B, a video signal based only on external light is obtained from the signal in the output period A, and a video signal based on light emission and a video signal based on external light are obtained from the signal in the output period B. In these video signals, kTC noise is removed by the differential amplifier 204. By calculating these two output signals, it is possible to remove an image due to external light and obtain an image based only on light emission, thereby obtaining more accurate depth information. Further, as an accompanying effect, an image using only external light that is not affected by light emission can be obtained from the signal of the exposure period A, which is suitable for recognition of a subject.

  FIG. 8 is a block diagram showing a fifth embodiment of the imaging apparatus according to the present invention. Although this embodiment also has two photodiodes, the second transfer gate (TG2) 109 and the reset gate (RG) 104 are driven by a pulse common to all pixels, as compared with the imaging device of FIG. I have to. Further, as in the third embodiment (FIG. 5), the kTC noise is read out first through the same path and is subtracted in the subsequent processing, so that the sample hold circuit (SH1, SH2) 202, before the AD converter 300, 203 and the differential amplifier 204 become unnecessary. The second photodiode (PD2) 108 is shielded from light as in FIG. 13 and can hold signal charges.

  In this embodiment, two exposure periods are set. During this period, after the signal charge of the photodiode (PD) 101 is swept from the blooming sweep gate (BG) 107, the second photodiode ( PD2) is a period until transfer to 108, and can be freely selected regardless of other operations. Further, the second photodiode 108 transfers the signal charge to the floating diffusion layer 102, and then transfers another signal charge from the photodiode 101 and holds it, so that the two exposure periods do not overlap. It can be set freely within the range. By operating at the timing described below, the influence of external light can be suppressed with extremely high accuracy.

  FIG. 9 is a timing chart showing an embodiment of an imaging method in the imaging apparatus of FIG. As in FIG. 1, two exposure periods A and B and three output periods C, A, and B are provided in one frame. In each output period C, A, B, kTC noise, signal A (exposure period A signal), and signal B (exposure period B signal) are added and output. Also in this example, the light emission period is made to coincide with the exposure period B.

  The difference from FIG. 1 is that the two exposure periods A and B can be set freely within a range that does not overlap. For example, the time difference between the exposure period A and the exposure period B (time difference between T3 and T4) is set to about 30 μsec. This corresponds to the time required for a car traveling at 100 km / h to move 1 mm, and the images obtained by external light in the exposure period A and the exposure period B are substantially the same image (the intensity is proportional to the exposure time). . Therefore, the influence of external light can be almost completely removed by calculation in a later stage even for a subject with a fast movement that is seen everyday. Strictly speaking, the setting of the light emission period needs to consider the speed of light. However, since it moves 300 m in 1 μsec, it may coincide with the exposure period B. When the maximum recognition distance specification is as long as 150 m, for example, the light emission intensity can be slightly increased by setting the end time of the light emission period 1 μsec earlier than the end time of the exposure period B in consideration of the speed of light.

  FIG. 10 is a diagram showing in more detail a gate pulse added to the timing chart of FIG.

The output period C is entered after the floating diffusion layer 102 is separated by falling the RG pulse at time T2. Here, pulses are sequentially applied to LS (n), LS (n + 1),... To transmit voltage information of kTC noise to the vertical signal line 200 and output it.
At time T3, the BG pulse falls and the exposure period A starts. This time T3 can be arbitrarily selected, but is set to the latter half of the output period C when the exposure time is shortened. Thereafter, when the TG pulse is applied at time T11 to transfer the signal charges to the second photodiode 108, the exposure time A ends and the exposure period B starts instead. Immediately after the TG pulse falls at time T11, the TG2 pulse is applied, and the signal charge in the exposure period A is transferred from the second photodiode 108 to the floating diffusion layer 102.

At time T4, the TG2 pulse falls and the second photodiode 108 is emptied. Thereafter, the TG pulse is applied again at an arbitrary time T12, and the signal charge is applied from the photodiode 101 to the second photodiode 108. Forward. At this point, the exposure period B ends.
At time T4, the signal charge in the exposure period A is transferred to the floating diffusion layer 102 and is combined with the kTC noise charge. After this, the output period A starts. Here, pulses are sequentially applied to LS (n), LS (n + 1),..., And noise and signals are transmitted to the vertical signal line 200 and output.

  After the output period A is completed, the TG2 pulse is applied again at time T5, and the signal charge in the exposure period B is transferred from the second photodiode 108 to the floating diffusion layer 102. At this time, the floating diffusion layer 102 holds all of the kTC noise charge and the signal charges of the two exposure periods A and B. After this, the output period B starts. Here, pulses are sequentially applied to LS (n), LS (n + 1),..., And noise and signals are transmitted to the vertical signal line 200 and output.

  As described above, in this embodiment, the two consecutive exposure periods A and B can be set to be extremely short. Therefore, the timing difference between the two exposure periods can be extremely small, and the two subjects can be seen in a state where the subject seen in daily life hardly moves. You can get a picture. The calculation at the latter stage is the same as in FIG. 1, but since the image by outside light is almost the same during the two exposure periods, the influence of outside light can be almost completely eliminated, and the depth can be accurately obtained both outdoors and at a distant subject. It can be measured.

  In the above embodiment, only the case where there are two exposure periods has been described, but it is also possible to provide a larger number of exposure periods in order to expand the dynamic range. Even in that case, the accuracy of depth measurement can be similarly improved by emitting light only during a part of the plurality of exposure periods.

100 ... pixel portion,
101 ... Photodiode (PD),
102 ... floating diffusion layer (FD),
103: Transfer gate (TG),
104 ... Reset gate (RG),
105 ... Source follower amplifier,
106 ... row selection gate (LS),
107 ... Blooming sweep gate (BG),
108 ... second photodiode (PD2),
109 ... second transfer gate (TG2),
200 ... vertical signal line,
201 ... current source,
202... First sample and hold circuit (SH1),
203 ... second sample and hold circuit (SH2),
204... Differential amplifier (AMP),
300 ... AD converter (ADC),
400 ... horizontal output circuit,
401, 402, 403 ... signal charges,
404 ... noise charge,
500 ... arithmetic circuit,
600 ... pulse generation circuit,
700: Light-emitting device.

Claims (8)

In an imaging method of illuminating a subject with emitted light and imaging the subject from the reflected light,
A plurality of exposure periods are provided within one frame period, and exposure is performed in each period.
Providing a light emission period corresponding to one of the plurality of exposure periods to emit light;
Output each video signal obtained in the plurality of exposure periods,
An imaging method comprising: calculating the plurality of output video signals, removing a video signal due to external light, and acquiring a video signal based only on light emission. In an imaging method of illuminating a subject with emitted light and imaging the subject from the reflected light,
An exposure period and a light emission period are provided within one frame period,
Providing a plurality of output periods within one frame period and outputting signals obtained in each period;
In a first output period among the plurality of output periods, random noise not including a video signal is output,
In the second and subsequent output periods, the video signal and random noise obtained during the exposure period are output,
An imaging method comprising: calculating the output signals of a plurality of output periods to obtain a video signal from which random noise has been removed. In an imaging method of illuminating a subject with emitted light and imaging the subject from the reflected light,
Within one frame period, there are provided first and second exposure periods, one light emission period corresponding to one of the first and second exposure periods, and first, second and third output periods,
In the first output period, random noise not including a video signal is output,
In the second output period, the video signal and random noise of the first exposure period are output,
In the third output period, the video signal of the first exposure period, the video signal of the second exposure period, and random noise are output,
An imaging method, wherein the output signals of a plurality of output periods are calculated, and a video signal by external light and random noise are removed to obtain a video signal by only light emission. The imaging method according to claim 1 or 3,
An imaging method characterized by further obtaining a video signal based only on outside light by the calculation. In an imaging device that irradiates a subject with emitted light and images the subject from the reflected light,
A group of photodiodes arranged in an array for photoelectrically converting reflected light from the subject to generate signal charges;
An output circuit for sequentially reading video signals from signal charges generated by the photodiode group;
An arithmetic circuit for calculating an output signal from the output circuit;
A pulse generation circuit for controlling the light emission operation of the light emitting device, and generating a pulse group for controlling the exposure operation of the photodiode group and the read operation of the output circuit,
The pulse generation circuit provides the photodiode group with a plurality of exposure periods within one frame period and exposes each of the periods, and causes the light emitting device to perform exposure within one of the plurality of exposure periods. Providing a light emission period that coincides, and causing the output circuit to output the respective video signals obtained in the plurality of exposure periods;
The imaging device is characterized in that the arithmetic circuit calculates a plurality of video signals output from the output circuit, removes the video signal due to external light, and acquires a video signal based only on light emission. In an imaging device that irradiates a subject with emitted light and images the subject from the reflected light,
A group of photodiodes arranged in an array for photoelectrically converting reflected light from the subject to generate signal charges;
An output circuit for sequentially reading video signals from signal charges generated by the photodiode group;
An arithmetic circuit for calculating an output signal from the output circuit;
A pulse generation circuit for controlling the light emission operation of the light emitting device, and generating a pulse group for controlling the exposure operation of the photodiode group and the read operation of the output circuit,
In one frame period, the pulse generation circuit provides an exposure period for the photodiode group for exposure, the light emitting device for light emission with a light emission period, and a plurality of output periods for the output circuit. Output the signal obtained in each period,
The output circuit outputs random noise that does not include a video signal in a first output period of the plurality of output periods, and a video signal and random noise obtained in the exposure period in a second and subsequent output periods. Output
The arithmetic circuit calculates an output signal of a plurality of output periods to obtain a video signal from which random noise has been removed. In an imaging device that irradiates a subject with emitted light and images the subject from the reflected light,
A group of photodiodes arranged in an array for photoelectrically converting reflected light from the subject to generate signal charges;
An output circuit for sequentially reading video signals from signal charges generated by the photodiode group;
An arithmetic circuit for calculating an output signal from the output circuit;
A pulse generation circuit for controlling the light emission operation of the light emitting device, and generating a pulse group for controlling the exposure operation of the photodiode group and the read operation of the output circuit,
The pulse generation circuit exposes the photodiode group by providing first and second exposure periods within one frame period, and causes the light-emitting device to coincide with one of the first and second exposure periods. Providing one light emission period to emit light, providing the output circuit with first, second, and third output periods to output signals obtained in the respective periods;
The output circuit outputs random noise not including a video signal in the first output period, and outputs a video signal and random noise in the first exposure period in the second output period, In the third output period, the video signal of the first exposure period, the video signal of the second exposure period, and random noise are output,
The arithmetic circuit calculates an output signal of a plurality of output periods, and obtains a video signal based only on light emission by removing a video signal due to external light and random noise. In the imaging device according to claim 5 or 7,
The imaging device is characterized in that the arithmetic circuit calculates the plurality of outputted signals and further acquires a video signal based only on external light.

Images