JP7182244B2 - Imaging device - Google Patents

Description

The present invention relates to an imaging device.

Patent Literature 1 discloses an apparatus for detecting a biological condition. This biological condition detecting device detects a pulse and body motion as biological conditions. Further, the biological state detection device includes a light emitting element and a light receiving element, and the light emitting element irradiates the living body with light, and the light receiving element captures the light including the component reflected from the living body. A biological condition is detected based on the light captured by the light receiving element.

JP 2008-264302 A

In recent years, sensor technology capable of non-contact detection of biological information such as pulse has been studied. As such a technique, for example, a technique for optically obtaining biological information is known, as in Patent Document 1. When the sensor is not in contact with the subject, there is a space between the sensor and the subject. Therefore, in addition to the light component containing the biological information of the subject, the light component not containing the biological information, so-called background light, is likely to enter the sensor. This background light can become noise.

An object of the present invention is to provide an imaging apparatus capable of improving the accuracy of biological information.

One aspect of the present invention is an imaging device for obtaining information about the pulsation of a subject, which includes irradiating the subject with measurement light having a wavelength in the visible light region or the near-infrared light region and irradiating the subject with the measurement light. a first signal associated with a first image containing the subject when illuminated with the measuring light, the first signal having a light source and a photodiode that generate a charge in response to the light and that are mutually switchable between; A signal acquisition unit that obtains a second signal related to a second image including the subject when irradiation is stopped, a control unit that controls the operation of the light source and the signal acquisition unit, and the first image and the second image based on the first signal. and a computing unit that obtains information about pulsation by using the difference of the second image based on the signal, and the signal acquiring unit is arranged in N rows and M columns (M and N are integers of 2 or more), and the received light and a pixel array unit that controls the pixel array unit and generates a first signal based on the charges generated in response to the light received by the pixels when irradiated with the measurement light. and forming a second signal based on the charge generated in response to the light received by the pixel when the irradiation of the measurement light is stopped.

According to the imaging device described above, information about pulsation is obtained by using the difference between the first image and the second image in the calculation unit. According to this difference, the influence of background light as a noise component contained in the first image is reduced. Therefore, it is possible to improve the accuracy of information on pulsation, which is biological information.

In one embodiment, a pixel includes a photodiode, a first charge storage portion and a second charge storage portion that store charges, and a charge distribution portion that selectively transfers charges to the first charge storage portion and the second charge storage portion. and the control unit controls the charge distributing unit to transfer the charge from the photodiode to the first charge storage unit so as to be synchronized with the irradiation of the measurement light, and stop and synchronize with the irradiation of the measurement light. The charge may be transferred from the photodiode to the second charge storage unit as shown in FIG. According to this configuration, the first and second signals can be preferably obtained.

In one form, the pixel further includes a drain for receiving electric charge, and the control section controls the electric charge distributing section to transfer the electric charge from the photodiode to the first electric charge accumulation section and the second electric charge accumulation section. The period during which the charge is transferred from the photodiode to the drain and the measurement light is applied may be shorter than the period during which the charge is transferred from the photodiode to the drain. According to this configuration, new charge may be transferred from the photodiode to the first and second charge storage units during a period other than the period during which charges are transferred from the photodiode to the first and second charge storage units. do not have. Therefore, it is possible to suppress an increase in the noise components of the first and second signals. Furthermore, since the period during which the measurement light is irradiated is shorter than the period during which charges are transferred to the drain, it is possible to suppress a decrease in the S/N ratio of the signal obtained during the period during which the measurement light is irradiated.

In one form, the control unit changes the row to be selected each time transfer to the first charge storage unit and transfer to the second charge storage unit are performed once in each pixel, Charges may be output from the pixels row by row. According to this operation, the first and second signals can be preferably obtained.

In one embodiment, the control unit changes the row to be selected after multiple times of transfer to the first charge storage unit and multiple times of transfer to the second charge storage unit in each pixel. Charges may be output from the pixels row by row. Also by this operation, the first and second signals can be preferably obtained.

In one embodiment, the light source irradiates the subject with a first measurement light, which is the measurement light, and irradiates a second measurement light having a wavelength in the visible light region or the near-infrared light region different from the first measurement light. , and stop irradiation of the subject with the first measurement light and the second measurement light. is transferred from the photodiode to the first charge storage unit in synchronization with the irradiation of the first measurement light, and the third charge is transferred from the photodiode in synchronization with the irradiation of the second measurement light. Charges may be transferred from the photodiode to the second charge accumulation section in synchronization with the stoppage of irradiation of the first measurement light and the second measurement light. According to this configuration, it is possible to obtain two types of images by irradiating two different measuring beams. Therefore, it is possible to increase the types of information that can be acquired regarding pulsation.

In one embodiment, the light source irradiates the subject with a first measurement light, which is the measurement light, and irradiates a second measurement light having a wavelength in the visible light region or the near-infrared light region different from the first measurement light. , and stop irradiation of the subject with the first measurement light and the second measurement light. transferring charge from the photodiode to the first charge storage synchronously with the irradiation, and transferring charge from the photodiode to the second charge storage synchronously with the cessation of the irradiation of the first and second measuring light; is transferred, and by controlling the charge distributing section in the second pixel adjacent to the first pixel, the charge is transferred from the photodiode to the first charge storage section in synchronization with the irradiation of the second measurement light. , the charge may be transferred from the photodiode to the second charge accumulator in synchronization with the termination of the irradiation of the first measurement light and the second measurement light. According to this configuration, two types of images can be obtained by irradiating two different measurement lights with a simple pixel configuration. Therefore, it is possible to increase the types of information that can be acquired regarding pulsation.

In one embodiment, when transferring charges from the photodiode to the first charge accumulation unit, the control unit may stop transfer to the first charge accumulation unit after stopping irradiation of the measurement light. According to this operation, it is possible to reliably capture the light originating from the subject irradiated with the measurement light.

In one form, the calculation unit may obtain the number of heartbeats per unit time as the information about the pulsation.

Advantageous Effects of Invention According to the present invention, an imaging device capable of improving the accuracy of biometric information is provided.

FIG. 1 is a diagram showing the configuration of an imaging device according to the first embodiment. FIG. 2 is a diagram showing a configuration of a signal acquisition unit included in the imaging apparatus shown in FIG. 1. As shown in FIG. 3 is a diagram showing a connection configuration of pixels included in the signal acquisition unit shown in FIG. 2. FIG. FIG. 4 is a diagram showing a configuration of a pixel included in the signal acquisition unit shown in FIG. 2; FIG. 5 is a timing chart showing the operation of the imaging device according to the first embodiment. FIG. 6 is a timing chart showing the operation of the imaging device according to the second embodiment. FIG. 7 is a diagram showing the configuration of an imaging device according to the third embodiment. FIG. 8 is a diagram showing a configuration of a pixel included in the signal acquisition unit shown in FIG. 7; FIG. 9 is a timing chart showing the operation of the imaging device according to the third embodiment. FIG. 10 is a timing chart showing the exposure operation and transfer operation of the imaging device according to the fourth embodiment. Part (a) of FIG. 11 is a timing chart showing the operation of the imaging device according to the first modification, and part (b) of FIG. 11 is a timing chart showing the operation of the imaging device according to the second modification.

[First Embodiment]
BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same elements are denoted by the same reference numerals, and overlapping descriptions are omitted.

[Imaging device]
As shown in FIG. 1, the imaging device 1 obtains information about the pulsation of the subject 100. FIG. Information on pulsation includes, for example, pulse rate, pulse variability, blood pressure, and the like. The imaging device 1 is installed at a distance from the subject 100 . For example, the imaging device 1 is not attached to the body of the subject 100 but is installed via a visible physical space between the subject 100 and the subject 100 .

The imaging device 1 has a light source 2 , a signal acquisition section 3 , a control section 4 and a calculation section 6 . The imaging device 1 emits measurement light L having a wavelength in the visible light region or the near-infrared light region from the light source 2 toward the subject 100 . For example, the wavelength of the measurement light L is 870 nanometers. Therefore, since the measurement light L has a wavelength band longer than that of visible light, the subject 100 does not recognize the irradiation of the measurement light L. The measurement light L irradiated to the subject 100 is absorbed and reflected according to the pulsation state of the subject 100 . The signal acquisition unit 3 is operated by the control unit 4 to acquire an image of the subject 100 receiving the measurement light L. FIG. Then, the calculation unit 6 uses the image to obtain information about the pulsation. For example, the image contains signal strength that varies periodically. The period of this periodic change corresponds to the pulse rate.

〔light source〕
The light source 2 emits the measurement light L having a wavelength in the visible light region or the near-infrared light region, as described above. Note that the wavelength of the measurement light L is not limited to this numerical value, and may be appropriately selected according to the characteristics of the object to be measured, the characteristics of the measurement parameters, and the like. The light source 2 is controlled by the control unit 4 so as to switch between emission of the measurement light L and stop thereof. In other words, the light source 2 does not emit the measuring light L all the time while the imaging device 1 is operating, but switches between emitting and stopping. This operation of repeating irradiation and stopping is called a "blinking operation". The blinking operation of the light source 2 is controlled by control signals provided from the controller 4 . This blinking operation is synchronized with the operation of the signal acquisition unit 3, which will be described later.

[Signal Acquisition Unit]
The signal acquisition unit 3 is a camera as a so-called image acquisition unit that acquires an image of the subject 100 . The signal acquisition unit 3 is connected to the control unit 4 and operates according to control signals provided from the control unit 4 . The operation here includes, for example, an exposure operation and a transfer operation. The signal acquisition unit 3 is also connected to the calculation unit 6 to provide the calculation unit 6 with the first signal and the second signal. The formats of the first signal and the second signal are not particularly limited.

[Control part]
As described above, the operation of the light source 2 and the operation of the signal acquisition section 3 are synchronized. Therefore, the control unit 4 generates control signals for these synchronous operations and provides them to the light source 2 and the signal acquisition unit 3 . According to the synchronous operation, the signal acquisition section 3 obtains the first signal when the measurement light L is applied, and the signal acquisition section 3 obtains the second signal when the measurement light L is not applied. In other words, synchronization includes at least an operation of matching the timings of the irradiation of the measurement light L and the exposure of the signal acquisition unit 3, and an operation of matching the timings of stopping the irradiation of the measurement light L and the exposure of the signal acquisition unit 3. include. A specific operation of the control unit 4 is shown by timing charts such as FIG. 5, and a more detailed description of the control unit 4 will be given later.

[Calculation part]
The calculation unit 6 uses the signal provided from the signal acquisition unit 3 to obtain information about pulsation.
For example, the calculation unit 6 obtains a plurality of images G1 based on the first signal and obtains a plurality of images G2 based on the second signal. Next, after obtaining a plurality of difference images GS, the calculation unit 6 uses the difference images GS to obtain information on pulsation (graph BS).

Here, the first signal includes a noise signal component in addition to the true signal component. A noise component may include, for example, a component due to background light. Then, the second signal obtained when the measurement light L is not irradiated does not contain the true signal component but contains the noise component. Therefore, the noise component can be subtracted by obtaining the difference between the first signal and the second signal. Therefore, the calculation unit 6 can obtain a signal with reduced noise components and a good S/N ratio. As a result, the accuracy of information on pulsation can also be improved.

[Signal Acquisition Unit]
As shown in FIG. 2 , the signal acquisition section 3 has a pixel array section 7 and a peripheral circuit section 8 . The pixel array section 7 and the peripheral circuit section 8 are integrated on the same semiconductor chip. The pixel array section 7 has a plurality of pixels X(i, j) arranged in a two-dimensional matrix. Here, i=1 to m; j=1 to n: m and n are integers. The pixel array section 7 constitutes a rectangular imaging area. A horizontal scanning circuit 9A is provided on the upper side of the pixel array section 7 . A horizontal scanning circuit 9B is provided on the lower side of the pixel array section 7 . A vertical scanning circuit 11 is provided on the left side of the pixel array section 7 . The horizontal scanning circuits 9A and 9B and the vertical scanning circuit 11 are connected to the control section 4 and drive the pixels X using control signals received from the control section 4 .

That is, the pixels X in the pixel array section 7 are sequentially scanned by the horizontal scanning circuits 9A and 9B and the vertical scanning circuit 11 . As a result, readout of pixel signals and electronic shutter operation are executed. The imaging device 1 scans each pixel X row by row in the vertical direction. As a result, the signals output from the pixels X are read out via the vertical output signal lines B1 and B2 provided for each column of pixels as pixel signals for each row of each pixel. Signal reading from each pixel X is generally the same as in a normal CMOS image sensor.

As shown in FIG. 3, pixels X are connected to each other via a plurality of signal lines. The multiple signal lines include reset signal lines, transfer signal lines, drain signal lines, and distribution signal lines. Also, the pixels X are connected to each other via selection signal lines. The selection signal line is connected to the pixel X readout buffer amplifiers A1 and A2.

As shown in FIG. 2, a signal processing section 12A is provided on one output side (upper side) of the pixel array section 7 . A signal processing section 12B is provided on the other output side (lower side) of the pixel array section 7 . The signal processing units 12A and 12B perform, for example, analog-to-digital conversion processing. That is, the signal processing units 12A and 12B each have an AD conversion circuit provided for each column. Specifically, the signal processing units 12A and 12B each have a folding integration type AD conversion circuit 14 and a cyclic AD conversion circuit 16 . The signal processing units 12A and 12B are connected to the computing unit 6 and provide the computing unit 6 with digital signals (first signal and second signal).

FIG. 4 is a diagram showing the configuration of the pixel X. As shown in FIG. The pixel X has a photodetector D, a readout buffer amplifier A1, and a readout buffer amplifier A2. The photodetector D receives light and generates an electric charge. The charges are output as corresponding voltages to the signal processing units 12A and 12B via the floating diffusion portion 28 and the readout buffer amplifiers A1 and A2, respectively.

The photodetection section D has a photodiode PD and a charge transfer section 17 . The photodetector D has a structure based on the principle of a lateral electric field controlled charge modulator (LEFM) developed by the inventors of the present invention. The lateral electric field control charge modulation device performs electric field control of the charge transport path by a lateral electric field by a plurality of gates provided on its side surface, thereby performing high-speed electron transport control.

The photodiode PD generates charges according to the light received through the aperture AP. The charges are provided to the charge transfer section 17 . The photodiode PD generates charge in response to light having a wavelength of, for example, 870 nanometers. It should be noted that the photodiode PD only needs to generate charges corresponding to the light of the wavelength to be detected.

The charge transfer unit 17 receives charges provided from the photodiode PD. The charge transfer unit 17 temporarily accumulates the charges and provides them to the readout buffer amplifier A1 or the readout buffer amplifier A2. The charge transfer section 17 has a charge collection region 18, a drain 19, a charge distribution section 21, and charge processing sections 22A and 22B.

A charge collection region 18 collects the charge generated in the photodiode PD. Then, the charge distributing section 21 transfers the collected charges to one of the charge processing sections 22A and 22B and the drain 19 . Then, the charge processing units 22A and 22B provide outputs corresponding to the transferred charges to the readout buffer amplifiers A1 and A2.

A power supply VDD is connected to the drain 19 . Electric charge continues to be generated while the photodiode PD receives light. On the other hand, the transfer of charges to the charge processing units 22A and 22B is prohibited while predetermined processing is being performed on the charges in the charge processing units 22A and 22B. Therefore, the drain 19 receives charges generated during the period in which transfer of charges to the charge processing units 22A and 22B is prohibited. In other words, no charge is accumulated in the charge processing portions 22A and 22B during the period when the drain 19 receives charges.

The charge distribution unit 21 distributes the charge according to the blinking operation of the light source 2 . The charge distribution section 21 has distribution electrodes 24A and 24B and a drain gate electrode 25 . Distribution signal lines are connected to the distribution electrodes 24A and 24B, respectively. The distribution electrodes 24A and 24B control charge transfer from the charge collection region 18 to the charge processing section 22A or the charge processing section 22B according to a control signal provided from the distribution signal line. A drain signal line is connected to the drain gate electrode 25 . The drain gate electrode 25 controls charge transfer from the charge collection region 18 to the drain 19 in response to a control signal provided from the drain signal line.

The charge processing units 22A and 22B differ from each other only in arrangement and connection configuration. Accordingly, the charge processing section 22A will be described in detail, and the charge processing section 22B will be described as necessary.

The charge processing section 22</b>A has a charge storage section 27 (first charge storage section), a floating diffusion section 28 and a reset drain 29 . Note that the charge storage unit 27 of the charge processing unit 22B is a second charge storage unit. The charge accumulation portion 27 is adjacent to the charge collection region 18 . Floating diffusion 28 is adjacent to charge storage 27 . Also, the floating diffusion portion 28 is connected to the read buffer amplifier A1 through the output line 31. As shown in FIG. A reset drain 29 is adjacent to the floating diffusion 28 . A power supply VDD is connected to the reset drain 29 .

These areas are areas that temporarily store charge. Charge transfer between these regions is controlled by voltages applied from several electrodes. Therefore, the charge processing section 22A has a transfer gate electrode 32 and a reset gate electrode 33 . The transfer gate electrode 32 controls charge transfer from the charge accumulation portion 27 to the floating diffusion portion 28 according to a control signal provided from the transfer signal line. The reset gate electrode 33 controls charge transfer from the floating diffusion 28 to the reset drain 29 according to a control signal provided from the reset signal line.

The read buffer amplifier A1 and the read buffer amplifier A2 are different only in arrangement and connection configuration. Therefore, the read buffer amplifier A1 will be described in detail, and the read buffer amplifier A2 will be described as necessary.

The read buffer amplifier A1 has a signal read transistor TA and a switching transistor TS. In the signal readout transistor TA, the drain is connected to the power supply VDD, the gate is connected to the floating diffusion portion 28 of the charge processing portion 22A, and the source is connected to the switching transistor TS. In the switching transistor TS, the drain is connected to the source of the signal readout transistor TA, the gate is connected to the selection signal line, and the source is connected to the vertical output signal line B1. The read buffer amplifier A1 outputs a voltage corresponding to the charges accumulated in the floating diffusion 28 to the vertical output signal line B1 according to a control signal provided to the gate of the switching transistor TS.

[Control part]
The signal acquisition unit 3 described above operates according to a control signal provided from the control unit 4 . This operation is explained by the timing chart shown in FIG. The operation of the imaging device 1 will be described below with reference to the timing chart shown in FIG.

FIG. 5 shows temporal changes in the control signal group S1 for the light source 2, the control signal group S2 for the charge distribution unit 21, and the control signal group S3 for the transfer operation, in order from the top. ing. Furthermore, the control signal group S1 includes a lighting signal NP for the light source 2 . The control signal group S2 includes a distribution signal TG1, a distribution signal TG2, and a drain signal TD. The control signal group S3 includes a reset signal RT, a transfer signal TX, and a selection signal SEL.

Further, FIG. 5 shows the temporal change of the control signal group S4, which includes the signal VP read from the read buffer amplifiers A1 and A2 and the signal for AD-converting the signal VP, following the control signal group S3. each shown. The control signal group S4 includes a clock signal F for the folding integration AD converter circuit 14 and a clock signal C for the cyclic AD converter circuit 16. FIG. The signal VP is an example of the voltage at each point P in FIG. That is, the signal VP is a voltage provided to the signal processing units 12A and 12B. Although FIG. 5 shows a case where voltages having the same waveform are input to the signal processing units 12A and 12B, the waveforms of the voltages input to the signal processing units 12A and 12B are not limited to this mode. They can be different from each other.

Also, in the following description, for convenience, a low level (0) is indicated as "L" and a high level (1) is indicated as "H". For example, "Set the reset signal RT to 'L'. ' means setting the reset signal RT to a low level (0). Conversely, 'set the reset signal RT to "H". ' means setting the reset signal RT to a high level (1).

First, the control unit 4 sets the drain signal TD to "H". On the other hand, the control unit 4 sets other signals (lighting signal NP, distribution signals TG1 and TG2, reset signal RT, transfer signal TX, selection signal SEL) to "L".

[Reset operation]
Next, the control unit 4 switches the i-th reset signal RT from "L" to "H". In synchronization with this switching, the control unit 4 also switches the selection signal SEL from "L" to "H". Then, the control unit 4 maintains the state in which the reset signal RT and the selection signal SEL are "H" for a predetermined period (for example, 0.78 microseconds). During this predetermined period, the control unit 4 does not output the clock signals F and C for driving the folding integration type AD conversion circuit 14 and the cyclic AD conversion circuit 16 . After a predetermined period of time, the controller 4 switches the reset signal RT from "H" to "L". On the other hand, the control unit 4 maintains the state where the selection signal SEL is "H".

[First AD conversion operation]
Next, the control unit 4 performs the first AD conversion operation. The first AD conversion operation is for digital correlated double sampling (CDS). Specifically, the control unit 4 provides the clock signal C to the cyclic AD converter circuit 16 after providing the clock signal F to the folding integration AD converter circuit 14 . The control unit 4 maintains the state in which the selection signal SEL is "H" while the clock signals F and C are being output.

[First exposure operation]
Next, the controller 4 switches the lighting signal NP from "L" to "H". In synchronization with this switching, the control unit 4 also switches the distribution signal TG1 from "L" to "H". Further, in synchronization with this switching, the control section 4 switches the drain signal TD from "H" to "L". Then, the state in which the lighting signal NP is "H", the distribution signal TG1 is "H", and the drain signal TD is "L" is maintained for a predetermined period (for example, 0.5 microseconds). During this predetermined period, the control unit 4 does not output the clock signal for driving the folding integration type AD conversion circuit 14 and the cyclic AD conversion circuit 16 . After the predetermined period has elapsed, the control unit 4 switches the lighting signal NP and the distribution signal TG1 from "H" to "L". Further, the control unit 4 switches the drain signal TD from "L" to "H".

Next, the control unit 4 maintains a state in which the lighting signal NP, the distribution signals TG1 and TG2, the reset signal RT, and the transfer signal TX are "L" and the drain signal TD and the selection signal SEL are "H" for a predetermined period. do.

[Second exposure operation]
Next, the control unit 4 switches the distribution signal TG2 from "L" to "H". Further, in synchronization with this switching, the control section 4 switches the drain signal TD from "H" to "L". Then, the state in which the distribution signal TG2 is "H" and the drain signal TD is "L" is maintained for a predetermined period (for example, 0.5 microseconds). The control unit 4 does not output the clock signal for driving the folding integration type AD conversion circuit 14 and the cyclic AD conversion circuit 16 even during this predetermined period.

[Transfer operation]
After the predetermined period has elapsed, the control unit 4 switches the distribution signal TG2 from "H" to "L". In synchronization with this switching, the control unit 4 switches the drain signal TD from "L" to "H" and switches the transfer signal TX from "L" to "H". Then, the state in which the transfer signal TX is "H" is maintained for a predetermined period (for example, 0.78 microseconds). The control unit 4 does not output the clock signal for driving the folding integration type AD conversion circuit 14 and the cyclic AD conversion circuit 16 even during this predetermined period. After the predetermined period has elapsed, the control unit 4 switches the transfer signal TX from "H" to "L".

[Second AD conversion operation]
The control unit 4 performs a second AD conversion operation. The second AD conversion operation is for AD converting the voltage output by the transfer operation. Specifically, the control unit 4 provides the clock signal F to the folding integration type AD conversion circuit 14 for a predetermined period. After completing the folding AD conversion operation, the control unit 4 subsequently outputs the clock signal C to the cyclic AD conversion circuit 16 for a predetermined period.

Next, while changing the target row (while increasing the variable i by 1), the control unit 4 performs the reset operation, the first AD conversion operation, the first and second exposure operations, the transfer operation, and the second AD conversion operation described above. Perform conversion operation. That is, the reading method of the imaging device 1 is a so-called rolling shutter.

The reset operation, the first AD conversion operation, the first and second exposure operations, the transfer operation, and the second AD conversion operation constitute storage and readout operations. That is, the operation of the first embodiment regards storage and readout as a single operation. Performing this store and read operation takes, for example, 80.6 microseconds. In FIG. 5, the period of storage and readout operations is shown as period _T1H . That is, it can be said that the period _T1H is a readout period for one row. The period _T1H may be defined as a period during which the selection signal SEL is "H", or the reset signal for the (i+1) row from the timing when the reset signal RT for the i row is switched from "L" to "H". It may be a period until the timing when the signal RT is switched from "L" to "H". Then, in order to obtain one image, the accumulation and readout operations are repeated, for example, 412 times. That is, one frame is 33 milliseconds.

Note that, in the several operations described above, the exposure operation, the transfer operation, and the second AD conversion operation may be performed in this order. For example, the exposure operation and the first AD conversion operation may be performed in parallel. Also, after performing the exposure operation, the first AD conversion operation may be performed, and then the transfer operation may be performed. That is, the exposure operation may be performed at desired timing within the period before transfer.

[Information Acquisition Operation]
Next, the calculation unit 6 performs processing for obtaining information on pulsation. Specifically, as shown in FIG. 1, the calculation unit 6 forms RAW format images G1 and G2 using the digital signals output from the signal processing units 12A and 12B. Specifically, the image G1 (first image) is formed using the digital signal of the signal processing unit 12A. Also, an image G2 (second image) is formed using the digital signal of the signal processing unit 12B. Next, the calculation unit 6 obtains a differential image GS using the acquired images G1 and G2. The calculation unit 6 sets a processing area in the difference image GS and obtains an average luminance within the processing area. The processing from formation of these images G1 and G2 to acquisition of luminance averages is repeated a predetermined number of times.

Next, the calculation unit 6 graphs (graph BS) the relationship between the intensity of the average brightness and time. Through this process, a graph showing changes in hemoglobin concentration over time is obtained. Since the hemoglobin concentration is responsive to pulsation, periodic time changes in the hemoglobin concentration correspond to the pulsation. This graph BS is used to obtain information on various pulsations.

For example, the number of heartbeats (so-called pulse rate) per unit time (one minute) may be obtained. Also, a graph (graph BS in FIG. 1) showing changes in hemoglobin concentration over time may be obtained.

By the way, as described above, the pulsation corresponds to the change in hemoglobin concentration, and the hemoglobin concentration corresponds to the intensity (voltage value) of the pixel X output. In other words, the variation in the output of the pixel X corresponds to the variation (pulsation) in the hemoglobin concentration. On the other hand, the output of the pixel X includes a component of environmental light (background light) incident on the pixel X in addition to the component caused by the change (pulsation) in the hemoglobin concentration. The fluctuation width of the light intensity caused by the background light component may be larger than the fluctuation width caused by the fluctuation of the hemoglobin concentration (pulsation). In other words, signal components resulting from fluctuations in hemoglobin concentration can be buried in background light components.

The imaging device 1 of this embodiment acquires the image G1 at the timing when the measurement light L is irradiated. Therefore, the image G1 certainly includes signal components resulting from irradiation of the measurement light L. FIG. Further, the imaging device 1 acquires the image G2 at the timing when the measurement light L is turned off. The image G2 does not contain a signal component caused by the irradiation of the measurement light L, and contains a component (noise component) not caused by the irradiation of the measurement light L. As shown in FIG. Such an operation may be called a lock-in operation (synchronous operation). An image G1 containing a signal component and a noise component and an image G2 not containing a signal component are obtained, and the image G2 is subtracted from the image G1 (for example, the difference in luminance value is obtained) to obtain the true signal. Ingredients can be extracted. According to such a lock-in operation, it is possible to obtain a true signal component variation that is smaller than the variation width of the noise component.

[Second embodiment]
An imaging device 1 according to the second embodiment will be described. The imaging device 1 of the second embodiment differs from the imaging device 1 of the first embodiment in the readout operation from the pixel array section 7 . Specifically, the readout operation of the imaging device 1 of the first embodiment was a so-called rolling shutter method. On the other hand, the readout operation of the imaging device 1 of the second embodiment is a so-called global shutter method.

In other words, the imaging device 1 of the second embodiment differs from the imaging device 1 of the first embodiment only in the operation method. 1 in common. Details of the control performed by the control unit 4 will be described below.

[Control part]
6 shows temporal changes in the control signal group S1 for the light source 2, the control signal group S2 for the charge distribution unit 21, and the control signal group S3 for the transfer operation, as in FIG. each shown. Note that the timing chart for the AD conversion operation is omitted in FIG.

First, the control unit 4 sets the drain signal TD to "H". On the other hand, the control unit 4 sets other signals (lighting signal NP, distribution signals TG1 and TG2, reset signal RT, transfer signal TX, selection signal SEL) to "L".

Next, the controller 4 continuously performs a plurality of exposure operations. In the first embodiment, the controller 4 alternately performs one exposure operation and one transfer operation. On the other hand, the control unit 4 of the second embodiment performs multiple exposure operations based on the global shutter method.

Specifically, in the period T2a, the control unit 4 sets the lighting signal NP and the distribution signal TG1 to "H" and the distribution signal TG2 and the drain signal TD to "L" (first exposure). NP, distribution signal TG1 and distribution signal TG2 are set to "L" and drain signal TD is set to "H"; ” (second exposure) are repeated a predetermined number of times (for example, 412 times).

Further, the control unit 4 keeps the states of the reset signal RT, the transfer signal TX, and the selection signal SEL at "L" during the period T2a.

Next, the control unit 4 performs multiple read operations in the period T2b.

[Reset operation]
First, the i-th reset signal RT is switched from "L" to "H". In synchronization with this switching, the control unit 4 also switches the i-th row selection signal SEL from "L" to "H". Then, the state in which the reset signal RT and the selection signal SEL are "H" is maintained for a predetermined period (for example, 0.78 microseconds). After the predetermined period has elapsed, the control unit 4 switches the reset signal RT from "H" to "L". On the other hand, the control unit 4 maintains the selection signal SEL at "H".

[Transfer operation]
Next, the control unit 4 switches the transfer signal TX from "L" to "H". After the predetermined period has elapsed, the control unit 4 switches the transfer signal TX from "H" to "L". On the other hand, the control unit 4 maintains the selection signal SEL at "H".

[AD conversion operation]
During the period T2c from the reset operation to the transfer operation, an AD conversion operation for correlated double sampling is performed. Specifically, the control section 4 outputs the clock signals F and C during this period T2c.

Next, the control unit 4 performs a readout operation for the pixels X in the (i+1)th row.

Here, from the timing when the transfer signal TX of the i-th row is switched from "H" to "L", the timing of starting the readout operation of the pixels X of the i+1-th row (timing of switching the reset signal RT of the i+1-th row). A period T2d up to the second AD conversion operation of the folding integration type AD conversion circuit 14 and the cyclic AD conversion circuit 16 is performed.

The above readout operation is repeated from the pixel X on the first row to the pixel X on the nth row. In addition, in the read operation, the control unit 4 keeps the drain signal TD at "H".

In the above operation, the read operation is performed multiple times after the exposure operation is performed multiple times. For example, 412 exposure operations for 824 microseconds are followed by 412 read operations for 78.6 microseconds. That is, one frame is 33 milliseconds.

[Information Acquisition Operation]
Next, the calculation unit 6 performs processing for obtaining information on pulsation. This processing is the same as the processing in the first embodiment.

The imaging apparatus 1 of the second embodiment can also provide the same effects as those of the first embodiment.

[Third embodiment]
An imaging device 1A of the third embodiment will be described. As shown in FIG. 7, the imaging device 1A of the third embodiment includes an image G1A obtained by irradiating the subject 100 with the measurement light L1, an image G1B obtained by irradiating the subject 100 with the measurement light L2, and a measurement An image G2 obtained without irradiating the subject 100 with the light L1 and L2 is acquired, and information is obtained using these images G1A, G1B, and G2. Therefore, the imaging device 1A of the third embodiment differs from the imaging device 1 of the first embodiment in the configuration of the light source 2A and the pixels XA. In short, the imaging apparatus 1A of the third embodiment performs a 3-tap 2-wavelength type operation. The configuration of the light source 2A and pixels XA, and the operation of the imaging device 1A will be described below.

〔light source〕
The light source 2 of the first embodiment could irradiate only the measurement light L1. On the other hand, the light source 2A of the third embodiment can emit not only the measurement light L1 (first measurement light) but also the measurement light L2 (second measurement light). The light source 2A has light generators 2a and 2b. The light generator 2a emits measurement light L1 having a first wavelength in the visible light region or the near-infrared light region. The light generator 2b emits measurement light L2 having a second wavelength in the visible light region or the near-infrared light region. Here, the first wavelength is different from the second wavelength. For example, the first wavelength is 870 nm and the second wavelength is 780 nm.

Also, the light generators 2a and 2b can be turned on and/or off independently of each other. Therefore, the light source 2 emits the measurement light L1 from the light generator 2a and turns off the light generator 2b, emits the measurement light L2 from the light generator 2b and turns off the light generator 2a, and an operation of extinguishing both of the measurement lights L1 and L2 is selectively performed. These operations are switched based on control signals provided from the control unit 4 .

[Pixel]
As shown in FIG. 8, the pixel XA of the third embodiment has an additional charge processing section 22C in addition to the charge processing sections 22A and 22B. The charge processing section 22C has the same configuration as the charge processing section 22A. Therefore, the charge storage section 27 of the charge processing section 22C is the third charge storage section. Furthermore, the pixel XA has an added readout buffer amplifier A3 in addition to the readout buffer amplifiers A1 and A2. Furthermore, the charge distribution unit 21A further includes a distribution electrode 24C. The read buffer amplifier A3 also has the same circuit configuration and connection configuration as the read buffer amplifier A1.

[Control part]
The pixel XA described above operates according to a control signal provided from the control section 4 . This operation is explained by the timing chart shown in FIG. The operation of the imaging device 1A will be described below with reference to the timing chart shown in FIG.

[Reset operation]
First, the controller 4 performs a reset operation. Specific control is the same as the reset operation in the first embodiment. That is, the control unit 4 switches the reset signal RT from "L" to "H" and switches the selection signal SEL from "L" to "H". After the predetermined period has elapsed, the control unit 4 switches the reset signal RT from "H" to "L".

[First exposure operation]
Next, the control section 4 performs the first exposure operation. Specific control is the same as the first exposure operation of the first embodiment.

[Second exposure operation]
Next, the controller 4 performs a second exposure operation. Specific control is the same as the second exposure operation of the first embodiment.

[Third exposure operation]
Next, the controller 4 performs a third exposure operation. Specifically, the controller 4 switches the lighting signal NP2 from "L" to "H". In synchronization with this switching, the control unit 4 also switches the distribution signal TG3 from "L" to "H". Further, in synchronization with this switching, the control section 4 switches the drain signal TD from "H" to "L". Then, the state in which the lighting signal NP2 is "H", the distribution signal TG3 is "H", and the drain signal TD is "L" is maintained for a predetermined period (for example, 0.5 microseconds). After the predetermined period has elapsed, the control unit 4 switches the lighting signal NP2 and the distribution signal TG1 from "H" to "L". Further, the control unit 4 switches the drain signal TD from "L" to "H".

[First AD conversion operation]
Here, in the first embodiment, the exposure operation is performed after performing the first AD conversion operation. However, the relationship between the first AD conversion operation and the exposure operation is not limited to this order. In the third embodiment, the first AD conversion operation and the exposure operation are performed in parallel. Therefore, the control section 4 provides control signals for the first AD conversion operation to the signal processing sections 12A and 12B during the period in which the first to third exposure operations are performed. Specific control is the same as the first AD conversion operation of the first embodiment.

[Transfer operation]
Next, the control unit 4 performs a transfer operation. Specific control is the same as the transfer operation of the first embodiment.

[Second AD conversion operation]
The control unit 4 performs a second AD conversion operation. Specific control is the same as the second AD conversion operation of the first embodiment.

[Information Acquisition Operation]
Next, the calculation unit 6 performs processing for obtaining information on pulsation. For example, by the same processing as in the first embodiment, information on pulsation is obtained using the image G1A based on the first wavelength and the image G2 based on the background light, and the image G1B based on the second wavelength and the image based on the background light are obtained. and G2 may be used to obtain information about pulsation.

Further, by using the relationship between the image G1A based on the first wavelength and the image G1B based on the second wavelength, information about the oxygen saturation may be obtained from the ratio of the oxygenated hemoglobin concentration to the deoxygenated hemoglobin concentration.

[Fourth embodiment]
An imaging device according to the fourth embodiment will be described. The light source and the signal acquisition unit of the imaging device of the fourth embodiment are different in configuration from the light source 2 provided in the imaging device 1 of the first embodiment, and are also different in operation from the light source 2 and the signal acquisition unit 3 . That is, the light source of the imaging device of the fourth embodiment has the same configuration as the light source 2A included in the imaging device 1A of the third embodiment. Specifically, when the light source 2A irradiates the measurement light L1, an image G1A is obtained by the pixels X1. Next, when the light source 2A irradiates the measurement light L2, an image G1B is obtained by another pixel X2 adjacent to the pixel X1. In other words, the imaging device of the fourth embodiment has the same configuration (2-tap type) as the pixel X of the first embodiment, resulting in the same result as the imaging device 1A of the third embodiment (acquisition of images G1A, G1B, and G2). is obtained. In short, the imaging apparatus of the fourth embodiment performs a 2-tap 2-wavelength type operation.

In addition, in the imaging apparatus of the fourth embodiment, the signal acquisition unit 3 has the same configuration as in the first embodiment, and the light source 2A has the same configuration as in the third embodiment. Therefore, the description of the physical configuration of the imaging apparatus of the fourth embodiment will be omitted, and the operation of the imaging apparatus will be described.

FIG. 10 shows temporal changes in the control signal group S1A for the light source 2, the control signal groups S2A and S2B for the charge distribution unit 21, and the control signal group S3 for the transfer operation, in order from the top. each shown. The control signal group S2A is for the pixels X in the odd columns (Odd Col.) targeted for the measurement light L1. The control signal group S2B is for the pixels X of even columns (Even Col.) targeted for the measurement light L2. "Even" and "odd" may correspond to the number of columns in the pixel array section 7, for example. In other words, the odd-numbered column pixels X are located in the odd-numbered columns (j=1, 3, 5, . . . ) in the pixel array section 7 . Pixels X in even columns are located in even columns (j=2, 4, 6 . . . ) in the pixel array section 7 . That is, pixels X in odd columns are adjacent to pixels X in even columns.

The selection of the pixel X targeted for the measurement light L1 and the pixel X targeted for the measurement light L2 is an example. The pixels X targeted for the measurement light L1 and the pixels X targeted for the measurement light L2 only need to be close to each other, and are not limited to adjacent columns. For example, it may be sorted by row. That is, the pixels X targeted for the measurement light L1 may be pixels X located in even rows, and the pixels X targeted for the measurement light L2 may be pixels X located in odd rows. Further, the pixels X targeted for the measurement light L1 and the pixels X targeted for the measurement light L2 may be arranged in a grid pattern.

In this way, it can be said that the configuration in which the target measurement light is different for each pixel is spatially resolved.

Further, it is necessary to independently operate the pixels X in the odd-numbered columns and the pixels X in the even-numbered columns in the exposure operation. On the other hand, the transfer operations need not be independent from each other like the exposure operations. Therefore, it is indicated only by the control signal group S3 for the transfer operation.

[Reset operation]
The control unit 4 performs a reset operation. Specific control is the same as the reset operation in the first embodiment. That is, the control unit 4 switches the reset signal RT from "L" to "H" and switches the selection signal SEL from "L" to "H". After the predetermined period has elapsed, the control unit 4 switches the reset signal RT from "H" to "L".

[First Exposure Operation of Pixel X in Odd Column]
Next, the control unit 4 performs the first exposure operation for the pixels X in the odd-numbered columns. The control unit 4 switches the lighting signal NP1 from "L" to "H", switches the distribution signal TG1 of the pixels X in the odd columns from "L" to "H", and further switches the drain signals TD of the pixels X in the odd columns. from "H" to "L". Then, the controller 4 maintains this state for a predetermined period (for example, 0.5 microseconds). After a predetermined period has elapsed, the control unit 4 switches the lighting signal NP1 from "H" to "L", switches the distribution signal TG1 from "H" to "L", and furthermore, switches the drain signal TD of the pixels X in the odd columns. from "L" to "H".

[Second exposure operation of pixels X in odd columns]
Next, the control unit 4 performs the second exposure operation for the pixels X in the odd columns. The control unit 4 switches the distribution signal TG2 of the pixels X in the odd columns from "L" to "H", and switches the drain signal TD of the pixels X in the odd columns from "H" to "L". After the predetermined period has elapsed, the control unit 4 switches the distribution signal TG2 of the odd-numbered pixel X from "H" to "L" and switches the drain signal TD of the odd-numbered pixel X from "L" to "H." .

[First Exposure Operation of Pixel X in Even Column]
Next, the control unit 4 performs the first exposure operation for the pixels X in the even columns. The control unit 4 switches the lighting signal NP2 from "L" to "H", switches the distribution signal TG1 of the pixels X in the even columns from "L" to "H", and further switches the drain signals TD of the pixels X in the even columns. from "H" to "L". Then, the control unit 4 maintains this state for a predetermined period. After the predetermined period has elapsed, the control unit 4 switches the lighting signal NP2 from "H" to "L" and switches the distribution signal TG1 from "H" to "L". Further, the control unit 4 switches the drain signal TD of the pixels X in the even columns from "L" to "H".

[Second exposure operation of pixels X in even columns]
Next, the control unit 4 performs the second exposure operation for the pixels X in the even columns. The control unit 4 switches the distribution signal TG2 of the pixels X in the even columns from "L" to "H", and switches the drain signal TD of the pixels X in the even columns from "H" to "L". After the predetermined period has elapsed, the control unit 4 switches the distribution signal TG2 of the even-numbered pixel X from "H" to "L", and switches the drain signal TD of the even-numbered pixel X from "L" to "H". .

[Transfer operation]
Next, the control unit 4 performs a transfer operation. Specific control is the same as the transfer operation of the first embodiment.

[AD conversion operation]
In the series of operations described above, the first AD conversion operation for correlated double sampling is performed before the transfer operation is performed. This first AD conversion operation may be performed in parallel with the exposure operation as in the third embodiment. Also, after the transfer operation, the second AD conversion operation is performed. Specific control of the first and second AD conversion operations is the same as the first and second AD conversion operations of the first embodiment.

Through the above steps, the exposure operation, readout operation, and AD conversion operation for the i-th row are completed. Next, the control unit 4 performs the same operation as the control described above for the pixels X and XE in the i+1-th row and odd-numbered columns.

[Information Acquisition Operation]
Next, the calculation unit 6 performs processing for obtaining information on pulsation. Images G1A, G1B, and G2 are also obtained by the operation of the fourth embodiment. Therefore, processing similar to that of the third embodiment may be performed.

Although embodiments of the present invention have been described, the present invention is not limited to the above embodiments.

[Modification 1]
In the first embodiment, the charge is transferred from the photodiode PD to the charge storage unit 27 (distribution signal TG1: "H") in synchronization with irradiation of the measurement light L (lighting signal NP: "H"). did. As shown in FIG. 5, in the first embodiment, switching of the lighting signal NP from "L" to "H" and switching of the sorting signal TG1 from "L" to "H" occur at the same timing. gone. Also, the switching of the lighting signal NP from "H" to "L" and the switching of the distribution signal TG1 from "H" to "L" were performed at the same timing. That is, the time during which the lighting signal NP maintains "H" is equal to the time during which the sorting signal TG1 maintains "H". However, to "synchronize" the irradiation of the measurement light L and the transfer to the charge storage unit 27 is not limited to the matching of these switching timings and the matching of the periods.

For example, as shown in part (a) of FIG. 11, the switching of the lighting signal NP from "L" to "H" and the switching of the distribution signal TG1 from "L" to "H" are out of timing. may Similarly, the switching of the lighting signal NP from "H" to "L" and the switching of the distribution signal TG1 from "H" to "L" may be out of timing. That is, the time during which the lighting signal NP maintains "H" may be different from the time during which the sorting signal TG1 maintains "H". In other words, "synchronization" in the embodiment means that the period during which the lighting signal NP is "H" overlaps with the period during which the sorting signal TG1 is "H". Therefore, the start timing and the end timing of these periods may be completely the same, or if there is an overlapping part, the start timing and the end timing may be different.

For example, first, the control unit 4 switches the distribution signal TG1 from "L" to "H". Then, after the predetermined period has elapsed, the control unit 4 switches the lighting signal NP from "L" to "H".
The deviation between the sorting signal TG1 and the lighting signal NP may be defined, for example, by a ratio (eg, 5%) to the period during which the lighting signal NP is set to "H". Next, after a predetermined period has elapsed, the control section 4 switches the lighting signal NP from "H" to "L". After the switching is performed, the control unit 4 switches the distribution signal TG1 from "H" to "L". According to this operation, the period during which the lighting signal NP is "H" completely overlaps the period during which the sorting signal TG1 is "H". That is, the time during which the sorting signal TG1 is at "H" is longer than the time during which the lighting signal NP is at "H". According to this operation, the light caused by the subject 100 irradiated with the measurement light L can be reliably captured.

Also, the lighting signal NP and the distribution signal TG1 are switched from "L" to "H" at the same time, and after a predetermined period of time has elapsed, the lighting signal NP is first switched from "H" to "L", and then the distribution signal TG1 is switched. You may switch from "H" to "L."

[Modification 2]
In the above-described embodiment, a predetermined standby period is provided between the period during which the distribution signal TG1 is "H" and the period during which the distribution signal TG2 is "H". That is, the period during which the drain signal TD is "H" is set between the period during which the distribution signal TG1 is "H" and the period during which the distribution signal TG2 is "H". As shown in part (b) of FIG. 11, for example, this period may be omitted. That is, at the same time as switching the distribution signal TG1 from "H" to "L", the distribution signal TG2 is switched from "L" to "H". Before and after this switching timing, the drain signal TD continues to be "L".

DESCRIPTION OF SYMBOLS 1, 1A... Imaging device 2, 2A... Light source 3... Signal acquisition part 4... Control part 6... Calculation part 7... Pixel array part 8... Peripheral circuit part 9A, 9B... Horizontal scanning circuit 11 Vertical scanning circuit 12A Signal processing unit 12B Signal processing unit 14 Folding integration type AD conversion circuit 16 Cyclic AD conversion circuit 17 Charge transfer unit 18 Charge collection region 19 Drain 21 charge distributing section 22A, 22B, 22C charge processing section 24A, 24B distributing electrode 25 drain gate electrode 27 charge storage section (first charge storage section, second charge storage section, third charge storage section) Accumulation part), 28... floating diffusion part, 29... reset drain, 32... transfer gate electrode, 33... reset gate electrode, 100... subject, A1, A2, A3... readout buffer amplifier, AP... aperture, D... light detection part, TA... signal reading transistor, TS... switching transistor, PD... photodiode, VDD... power supply, X... pixel.

Claims (1)

An imaging device for obtaining information about pulsation of a subject,
a light source capable of switching between irradiating the subject with measurement light having a wavelength in the visible light region or the near-infrared light region and stopping irradiation of the subject with the measurement light;
having a photodiode that generates a charge in response to light, a first signal relating to a first image including the subject when the measurement light is applied, and including the subject when the measurement light is no longer applied a signal acquisition unit for obtaining a second signal related to the second image;
a control unit that controls operations of the light source and the signal acquisition unit;
a calculation unit that obtains information about the pulsation by using a difference between the first image based on the first signal and the second image based on the second signal;
The signal acquisition unit is
a pixel array section including a plurality of pixels arranged in N rows and M columns (M and N are integers of 2 or more) and generating charges corresponding to received light;
controlling the pixel array unit, forming the first signal based on the charge generated in accordance with the light received by the pixel when the measurement light is applied, and stopping the irradiation of the measurement light; a peripheral circuit section that forms the second signal based on the charge generated in response to the light received by the pixel when the
The pixels are
the photodiode;
a first charge storage unit and a second charge storage unit that store the charge;
a charge distribution unit that selectively transfers the charge to the first charge storage unit and the second charge storage unit;
The light source irradiates the subject with a first measurement light, which is the measurement light, and irradiates a second measurement light having a wavelength in a visible light region or a near-infrared light region different from the first measurement light. , and stop irradiation of the subject with the first measurement light and the second measurement light, and
The control unit
By controlling the charge distributing section in the first pixel, the charge is transferred from the photodiode to the first charge storage section so as to be synchronized with irradiation of the first measurement light, and the first measurement is performed. transferring the charge from the photodiode to the second charge storage unit in synchronization with stopping irradiation of the light and the second measurement light;
By controlling the charge distribution section in the second pixel adjacent to the first pixel, the charge is transferred from the photodiode to the first charge storage section in synchronization with irradiation of the second measurement light. and transferring the charge from the photodiode to the second charge storage unit in synchronization with stopping the irradiation of the first measurement light and the second measurement light.

Images