JP2010192659A - Imaging device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging device that outputs a high quality image by avoiding a halation and the like in response to conditions of an object. <P>SOLUTION: The imaging device 101 includes a controller 6 for setting the initial potential barrier height of a discharge gate 212a which corresponds to a start potential of discharge by controlling a control potential through a control potential generating portion 212e in response to an output signal of a pixel circuit 211. By this controller 6, a dynamic range of the imaging device 101 is optimized in response to the object conditions. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an imaging apparatus capable of imaging with changing output characteristics in accordance with the state of a subject.

  2. Description of the Related Art Conventionally, an image pickup apparatus including a semiconductor image pickup device (hereinafter also simply referred to as an image pickup device) such as a CCD or a CMOS image sensor has been used in various devices, and examples thereof include a TV door phone and a network camera. . This image pickup apparatus includes a plurality of light receiving elements arranged in a grid pattern in the image pickup element, and converts the received light into an electric signal by each light receiving element, thereby outputting an image based on the electric signal. Specifically, the image sensor accumulates an amount of charge corresponding to the amount of light received by each light receiving element during a preset charge accumulation period, and the amount of the accumulated charge (hereinafter also referred to as accumulated charge). The electric signal according to is output. The imaging device forms a frame as a plane image based on the electrical signal output from the imaging element, and outputs a moving image by continuing the frame in time series. Here, the image sensor generally has a characteristic of accumulating an amount of electric charge proportional to the amount of received light (hereinafter also referred to as the amount of received light). Therefore, an electrical signal output from the image sensor is Although it shows a so-called linear characteristic that increases linearly with the increase in the amount of received light, if charge is accumulated until it reaches a capacity (hereinafter also referred to as charge storage capacity) that can store the charge of the image sensor. An electrical signal output from the image sensor (hereinafter also referred to as a saturated state) becomes constant at a predetermined saturation potential. Since the dynamic range of the image sensor is defined in this way, when a relatively dark subject is imaged so as not to reach saturation, the gradation of brightness according to the magnitude of the electrical signal output from each image sensor Appears in the image, but when a relatively bright subject that is in a saturated state is imaged, the image is completely white, that is, whites are blown out. In view of this, there is known an imaging apparatus that avoids whiteout of an image even in a relatively narrow dynamic range by changing the charge accumulation period according to the brightness of the subject. That is, in this imaging apparatus, when a bright subject is imaged, the charge accumulation period is shortened so that the accumulation of charges does not reach a saturation state to avoid overexposure.

  However, when imaging a subject that includes a part with a large difference in brightness in one field of view, such as when imaging a person with the sun behind, if the charge accumulation period is shortened according to the bright part, the dark part is completely black. In other words, the black portion is crushed, but if the charge accumulation period is lengthened in accordance with the dark portion, the bright portion will be blown out.

  Therefore, attempts have been made to expand the dynamic range of the image sensor (hereinafter also referred to as a wide dynamic range) by obtaining output characteristics different from the linear characteristics by various methods.

  For example, in the image sensor disclosed in Patent Document 1, when the amount of charge generated and accumulated by the light receiving element is large (more specifically, the charge generated and accumulated by the photodiode is set in advance). In the case of overflowing to the outside beyond the potential barrier (potential barrier), a so-called charge discharging operation is performed in which all of the accumulated charge accumulated so far is temporarily discharged at a predetermined intermediate timing of the charge accumulation period. This avoids a saturation state at the output timing of the electric signal, thereby realizing a wide dynamic range.

Further, for example, Patent Document 2 includes a MOS transistor that inputs a photocurrent generated according to the amount of received light, and a bias unit that biases the MOS transistor to a state that is equal to or lower than a threshold voltage and allows a subthreshold current to flow. A technique for realizing a wide dynamic range by logarithmically compressing a photocurrent with the MOS transistor is disclosed.
Japanese Patent No. 3996618 Japanese Patent Laid-Open No. 03-192864

  By the way, in the imaging device disclosed in Patent Document 1, when the amount of received light exceeds a predetermined amount of light, the accumulated charge is increased at a predetermined intermediate timing of the charge accumulation period (the larger the amount of received light). (Accelerated), because the output potential is determined based on the charge that has been discharged once and then re-accumulated during the remaining period, the magnitude of the amount of received light, the beginning and end of the charge discharge timing, and the length of the charge discharge period For example, there is a concern that gradation inversion occurs in the final output signal. For example, it is assumed that one output frame is composed of gradation gradations having a large contrast. At this time, the charge is not discharged in the dark part, and the output gradually increases linearly (linear characteristics) as the amount of received light increases. Is discharged at an accelerated rate, so the output that was monotonically increasing up to the middle of the gradation gradation turns to zero. As a result, even if an attempt is made to correct the amount of re-accumulated charge based on the ratio of the re-accumulation period set after discharging the charge and the entire charge accumulation period, gradation gradation (as long as there is a point where the output becomes zero) In this case, tone reversal can occur in any part of the dark to bright tone change.

On the other hand, the output of the image sensor disclosed in Patent Document 2 exhibits logarithmic characteristics that asymptotically approach a saturation potential. As a result, this imaging device can theoretically realize a very wide dynamic range. However, in order to realize an output signal with a wide dynamic range (for example, a light quantity ratio of 1:10 ⁶ (about 120 [dB])), the output is compressed as the received light quantity increases in order to surely avoid a saturated state. It will be. For this reason, the contrast of the output signal is lowered and the number of gradations is substantially lowered.

  In response to the problems such as the reduction in contrast and the reduction in the number of gradations, it is conceivable to amplify the output by an analog amplifier. However, in this case, the noise component is also amplified at the same time.

  As described above, an imaging apparatus having a conventional wide dynamic range imaging device has an advantage of a wide dynamic range. On the other hand, depending on the condition of the subject, the output image has low contrast and gradation. It has a drawback that it may be lowered.

  The present invention has been made in view of the above points, and an object of the present invention is to provide an imaging device capable of high-quality imaging while avoiding overexposure depending on the state of the subject. It is in.

  The inventors of the present application, for example, by creating an image pickup device of Patent Document 1 by appropriately setting a manufacturing process, even if a charge discharging operation is executed, all of the accumulated charges accumulated so far are discharged. Without charge generation and charge discharge based on the amount of received light, and an output characteristic similar to a logarithmic characteristic can be obtained due to the equilibrium state, and a potential barrier related to discharge of charge generated and accumulated by a photodiode It has been found that the dynamic range of the imaging device can be easily changed by appropriately controlling the initial height of the imaging device. Therefore, we focused on performing imaging while optimizing the dynamic range in accordance with the condition of the subject.

  The imaging device of the present invention generates charges by receiving light, and is provided corresponding to each of the plurality of light receiving elements arranged in a lattice so as to form a plurality of rasters, A plurality of pixel circuits for accumulating the charges generated in the light receiving element and outputting an output signal corresponding to the accumulated amount of the charges, and a frame configured based on the output signal in time series An image pickup device that continuously outputs, and the image pickup device outputs an overflow drain that accumulates charges discharged from each pixel circuit, and a control potential that outputs a control potential corresponding to the amount of charges accumulated in the overflow drain. A potential barrier is formed between the potential generator, each of the pixel circuits, and the overflow drain, and the potential failure is determined according to the control potential. A charge discharge gate that changes the discharge mode of the charge from the pixel circuit to the overflow drain by changing the height of the pixel circuit, and a part of the charge accumulated in each pixel circuit, It further has an accumulated charge discharging circuit capable of executing an accumulated charge discharging operation for discharging to the overflow drain before outputting an output signal of each pixel circuit.

  The imaging apparatus controls the control potential through the control potential generation unit according to the output signal of the pixel circuit, whereby the initial height of the potential barrier of the charge discharge gate corresponding to the charge discharge start potential Is further provided.

  According to this configuration, the accumulated charge discharging circuit has a control potential generating unit that outputs a control potential, and the control potential generating unit outputs a control potential corresponding to the amount of charge accumulated in the overflow drain. The charge discharge gate forms a potential barrier between each pixel circuit and the overflow drain, and changes the discharge mode of charge from each pixel circuit to the overflow drain according to the height of the potential barrier. That is, by accumulating charges according to the amount of light received by the light receiving element of each pixel circuit, discharging of the charges to the overflow drain is started beyond the potential barrier constituted by the charge discharging gate. As the charge is discharged, the charge accumulated in the overflow drain increases, so that the control potential, which is the output of the control potential generation unit, changes, thereby changing the height of the potential barrier of the charge discharge gate. More specifically, the height of the potential barrier is lowered. As a result, the amount of charge discharged to the overflow drain beyond the potential barrier further increases, and the amount of charge stored in the overflow drain also increases accordingly, so that the control potential further changes, and the potential barrier of the charge discharge gate changes. Is further changed (lower). Thus, after discharge of charge to the overflow drain is started beyond the potential barrier, discharge of charge is promoted according to the discharge amount of the charge (in other words, the amount of light received by the light receiving element). Become. Therefore, before the start of charge discharge, the amount of accumulated charge increases linearly with the increase in the amount of light received by the light receiving element. Is non-linear, specifically increasing in a logarithmic function and charge accumulation is suppressed, so that the accumulated charge is prevented from being saturated at the output timing of the output signal of the pixel circuit. Thus, the output characteristic of the light receiving element with respect to the amount of received light is a characteristic that combines the range of the linear characteristic and the range of the logarithmic characteristic, and the dynamic range can be expanded.

  Here, in the above configuration, the control unit further changes the charge discharge start timing by setting the initial height of the potential barrier of the charge discharge gate corresponding to the charge discharge start potential. As a result, the timing at which the charge accumulation characteristic with respect to the amount of light received by the light receiving element changes from the linear characteristic to the logarithmic characteristic is changed. This means that the output characteristics of the pixel circuit with respect to the amount of light received by the light receiving element are changed. Specifically, by setting the initial height of the potential barrier relatively low so that the charge discharge start timing is advanced, the range of the output characteristics of the pixel circuit becomes logarithmic, and dynamic While the range expands relatively, the initial height of the potential barrier is set relatively high so that the charge discharge start timing is delayed, so that the output characteristics of the pixel circuit are in a linear characteristic range. The dynamic range is enlarged and the dynamic range is relatively reduced.

  The control unit further sets the initial height of the potential barrier according to the output signal of the pixel circuit, thereby changing the dynamic range according to the output signal of the pixel circuit, that is, the state of the subject. The dynamic range is optimized according to the situation. As a result, it is possible not only to avoid overexposure but also to avoid unnecessary reduction in the contrast of the output image and reduction in the number of gradations.

  According to a second invention, in the first invention, further comprising an output signal processing unit that outputs the frame configured to include imaging data obtained by converting an output signal of each pixel circuit into a digital signal, and the control unit includes: The initial height of the potential barrier is controlled so that the maximum value of the imaging data included in the frame becomes a preset maximum output value of the imaging device. The “maximum output value” here includes, in addition to the maximum output value of the image sensor itself, a value that is lower than the maximum output value by a predetermined amount and can be regarded as a substantially maximum output value. .

  By making the maximum value of the imaging data included in the frame the maximum output value of the image sensor, the output increases up to the maximum output value of the image sensor while avoiding overexposure of the image, which is unnecessary. The output is not compressed, and a decrease in contrast and a decrease in the number of gradations are avoided.

  According to a third aspect, in the second aspect, the control unit is configured to output a potential designation signal corresponding to the output signal, and the control potential generation unit inputs an accumulated charge amount of the overflow drain. An inverting amplifier that outputs a higher control potential as the amount of accumulated charge increases, and an adder that adds a charge discharge designated potential corresponding to a potential designation signal from the control unit to the control potential. The charge discharge gate is configured to lower the potential barrier height as the control potential is higher, and the control unit further includes a maximum value of the imaging data included in the frame, When the maximum output value of the image sensor is exceeded, the potential specifying signal is output so that the charge discharge specified potential is increased, while the maximum value of the imaging data is the maximum value. When below the output value is configured to output the potential designation signal so that the charge discharging specified potential decreases.

  In this way, the height of the potential barrier of the charge discharge gate can be changed according to the accumulated charge amount of the overflow drain by the control potential generator having a simple circuit configuration including the inverting amplifier and the adder. The height can also be changed.

  Thus, when the maximum value of the imaging data included in the frame exceeds the maximum output value of the imaging element, in other words, when the control unit generates whitening due to saturation, the control potential generating unit The initial height of the potential barrier of the charge discharging gate is reduced through the above, and as described above, when the charge discharge start timing is advanced and the dynamic range is expanded, the maximum value is lower than the maximum output value, in other words, When saturation does not occur, by raising the charge discharge designated potential, the initial height of the potential barrier of the charge discharge gate increases, and as described above, the charge discharge start timing is delayed and the dynamic range is reduced. Thus, it can be realized that the maximum value becomes the maximum output value by controlling the initial height of the potential barrier.

  According to a fourth invention, in any one of the first to third inventions, the first imaging that does not execute the accumulated charge discharging operation by the accumulated charge discharging circuit in the imaging mode of each pixel circuit with respect to the image sensor. An imaging mode switching unit that switches between a mode and a second imaging mode for executing the stored charge discharging operation, and the imaging device outputs the pixel circuit each time a predetermined number of frames are output. Outputs an imaging mode determination frame configured to include at least a part of a first imaging mode raster configured based on the output signal captured in the first imaging mode, and the control unit further includes: Based on the output signal constituting the first imaging mode raster in the imaging mode determination frame, the imaging mode of each pixel circuit is set through the imaging mode switching unit. Serial to specify the first or second imaging mode.

  In the first imaging mode, since the accumulated charge discharging operation by the accumulated charge discharging circuit is not executed, the output signal increases linearly as the amount of light received by the light receiving element increases. For this reason, when imaging in the first imaging mode, the imaging apparatus can output an image with high contrast and high gradation, but the image may be overexposed due to saturation. On the other hand, in the second imaging mode, as described above, since the accumulated charge discharging operation is executed, the dynamic range is expanded.

  Since the control unit designates the imaging mode of each pixel circuit through the imaging mode switching unit based on the output signal imaged in the first imaging mode included in the imaging mode determination frame, the dynamic range is relatively narrow. If there is no problem with the output image even if the image is captured in the one imaging mode, the imaging is performed in the first imaging mode. On the other hand, when a problem such as overexposure can occur when the image is captured in the first image capturing mode, the image capturing is performed by switching to the second image capturing mode having a relatively wide dynamic range. By switching between the first imaging mode and the second imaging mode, an image with higher contrast and higher gradation can be output while avoiding overexposure.

  Here, at the time of imaging in the second imaging mode, as described above, in order to set the initial height of the potential barrier, the determination on the imaging data included in the frame is performed, whereas the first imaging mode Such a determination is not performed during imaging. Therefore, the configuration for switching between the first imaging mode and the second imaging mode has an advantage that the determination process regarding the imaging data of the frame can be omitted at least during imaging in the second imaging mode.

  In a fifth aspect based on the fourth aspect, the imaging device further includes a clear switch that clears the accumulated charge in the overflow drain, and the imaging mode switching unit turns on the clear switch to turn on the clear switch. The accumulated charge discharging circuit is set to the first imaging mode in which the accumulated charge discharging operation is not performed, and the accumulated charge discharging circuit is set to the second imaging mode in which the accumulated charge discharging operation is performed by turning off the clear switch. It is configured as follows.

  When the stored charge in the overflow drain is cleared by turning on the clear switch, even if the charge is discharged beyond the potential barrier of the charge discharging gate, the charge is cleared, so that the overflow drain is substantially charged. Does not accumulate. As a result, the height of the potential barrier is not changed as the control potential changes as described above. That is, the discharge of charge is not promoted, and therefore the accumulated charge discharging operation is not performed. On the other hand, when the clear switch is turned off, as described above, when the charge is discharged beyond the potential barrier of the charge discharging gate, the charge is accumulated in the overflow drain, which causes a change in the control potential. Along with this, the height of the potential barrier changes (becomes lower), facilitating the discharge of charges. That is, the accumulated charge discharging operation is performed.

  As described above, the imaging apparatus of the present invention can output a high-quality image while avoiding overexposure by optimizing the dynamic range according to the condition of the subject.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. The following description of the preferred embodiment is merely exemplary in nature.

  As shown in FIG. 1, the imaging apparatus 1 of the present invention temporarily stores the imaging element 2 that converts incident light into an electrical signal and outputs it as imaging data, and imaging data output from the imaging element 2. The buffer memory 3, the imaging data transferred from the buffer memory 3, and the image processing unit 4 for constructing the image data based on the imaging data, and the imaging mode of the imaging device 2 will be described in detail later. An imaging mode switching unit 5 that switches between a narrow dynamic range (NDR) mode and a wide dynamic range (WDR) mode with a relatively wide dynamic range, and imaging based on imaging data stored in the buffer memory 3 The imaging mode is designated to the mode switching unit 5 and the overall operation of the imaging device 2 is controlled. A control unit 6 which comprises a.

  As shown in FIG. 1, the image pickup device 2 is arranged in a matrix by being arranged side by side in the main scanning direction (vertical direction in FIG. 1) and the sub-scanning direction (horizontal direction in FIG. 1) orthogonal thereto. A sensor area 21 having a plurality of pixel circuits 211 arranged in the pixel circuit 211 and an accumulated charge discharging circuit 212 connected to the pixel circuit 211, and the sensor area 21 arranged on the right side in FIG. The output signal processing unit 22 and the peripheral circuit 23 disposed below the sensor area 21 in FIG. 1 are provided.

  As shown in FIGS. 1 and 2 (where FIG. 2 shows a state in which the pixel circuit 211 is rotated 90 ° clockwise with respect to FIG. 1), the pixel circuits 211 are equally spaced (for example, at an interval of 7.8 μm). For example, 640 pixels are arranged in the main scanning direction and 480 pixels are arranged in the sub-scanning direction, so that a total of 640 × 480 pixel circuits 211 are provided in one image sensor 2. . As a result, the imaging device 2 constitutes a so-called 1/3 inch imaging device, and the imaging device 1 can output an image having a resolution equivalent to VGA.

  As shown in FIG. 3, each pixel circuit 211 has a photodiode 211a as a light receiving element and a transfer for transferring the charge generated by the photodiode 211a and accumulated in the first node N1 to the second node N2. A gate 211b, a reset switch 211c that can clear charges accumulated in the second node N2, an amplifier 211d that amplifies and outputs a signal corresponding to the potential of the second node N2, an amplifier 211d and an output node NO And a pixel selection switch 211e that outputs a signal amplified by the amplifier 211d from the pixel circuit 211 at a predetermined timing.

  The photodiode 211a is composed of a semiconductor pn junction, and receives incident light by a so-called photovoltaic effect and generates an amount of electric charge corresponding to the amount of received light. The cathode side of the photodiode 211a is connected to the first node N1, while the anode side is connected to the ground node N4 that supplies the ground potential VSS and is biased to the ground potential VSS. Accordingly, the charge generated in the photodiode 211a becomes a negative charge, and this negative charge is stored in the first node N1 as a capacitor capable of storing the charge of the photodiode 211a.

  The transfer gate 211b is a switch element that is interposed between the first node N1 and the second node N2 and is configured by, for example, a transistor, and its on / off state is controlled by the transfer control signal TG. As a result, the transfer gate 211b has a function of controlling the transfer timing of the charge accumulated in the first node N1 from the first node N1 to the second node N2.

  The reset switch 211c is a switch element that is interposed between the second node N2 and the power supply potential node N3 that supplies the power supply potential VDD, and is configured by, for example, a transistor. The reset switch 211c is turned on and off by a reset control signal RF. Be controlled. Accordingly, the reset switch 211c has a function of removing the charge at the second node N2 at a predetermined timing (that is, before transferring the charge from the first node N1 to the second node N2). The power supply potential VDD is set to a positive potential because the charge at the second node N2 is a negative charge.

  As described above, the amplifier 211d amplifies and outputs a signal corresponding to the electric charge of the second node N2, the input node is connected to the second node N2, and the output node is connected to the pixel selection. It is connected to the output node NO via the switch 211e, and is constituted by, for example, a source follower circuit.

  The pixel selection switch 211e is a switch element that is interposed between the amplifier 211d and the output node NO and is configured by, for example, a transistor, and the ON / OFF of the pixel selection switch 211e is controlled by the pixel selection control signal PS. Thus, the pixel selection switch 211e has a function of controlling the output timing of the signal amplified by the amplifier 211d from the amplifier 211d to the output node NO, and hence to the output signal processing unit 22.

  As shown in FIGS. 1 and 2, one accumulated charge discharging circuit 212 is provided for one pixel circuit 211 in this embodiment.

  The accumulated charge discharging circuit 212 is a circuit for discharging the charges accumulated in the first node N1 as necessary, and as shown in FIG. 3, the ground potential is supplied by the ground node N4 that supplies the ground potential VSS. The overflow drain 212b biased to VSS, the charge drain gate 212a interposed between the fifth node N5 as the overflow drain capacitance and the first node N1 of the pixel circuit 211, the fifth node N5 and the power supply potential A clear switch 212c interposed between the power supply potential node N6 for supplying VDD2 and an inverting amplifier 212d whose input node is connected to the fifth node N5 and whose output node is connected to the charge discharge gate 212a. And.

  Here, the power supply potential VDD2 is a switchable power supply potential, and is switched by the imaging mode switching unit 5 to either the power supply potential VDD or a power supply potential (however, a positive potential) as an intermediate potential lower than the power supply potential VDD.

  The charge discharge gate 212a has a function of controlling transfer (that is, discharge) of charges accumulated in the first node N1 from the first node N1 to the fifth node N5. In the present embodiment, the charge discharging gate 212a is configured by a transistor equivalently functioning as a variable resistor. As will be described in detail later, the charge discharging gate 212a is controlled based on the output potential Vg of the inverting amplifier 212d. The height of the potential barrier constituted by 212a changes and the charge discharging operation is changed and controlled.

  The overflow drain 212b accumulates the charge transferred from the first node N1 to the fifth node N5 through the charge discharge gate 212a.

  The clear switch 212c is a switch element constituted by a transistor, and its on / off state is controlled by a clear control signal RO. When the clear switch 212c is on, the power supply potential VDD2 supplied from the power supply potential node N6 causes the charge of the fifth node N5 to disappear, whereas when the clear switch 212c is off, the charge of the fifth node N5 does not disappear. As electric charges (electrons) are discharged from the first node N1 and accumulated in the overflow drain 212b, the potential of the fifth node N5 decreases. Although described in detail later, the imaging mode of the imaging device 2 (each pixel circuit 211) is changed between the NDR mode and the WDR mode by the switching control of the power supply potential of the power supply potential VDD2 and the control of the clear switch 212c. Can be switched.

  The clear control signal RO output from the imaging mode switching unit 5 is commonly input to the clear switch 212c of each accumulated charge discharging circuit 212 connected in the main scanning direction (the vertical direction in FIG. 3) in the imaging element 2. The accumulated charge discharging circuits 212 connected in the main scanning direction simultaneously execute the charge discharging operation by controlling the clear control signal RO to the Hi level. The image sensor 2 has a plurality of such structural units in the sub-scanning direction (left-right direction in the drawing in FIG. 3) (not shown), and the charge discharging operation in each row in the sub-scanning direction is performed in the imaging mode. It is independently controlled by RO_1, RO_2,.

  That is, in the present embodiment, the image pickup mode can be switched between the NDR mode and the WDR mode with each accumulated charge discharging circuit 212 connected in the main scanning direction as one unit, and in the sub scanning direction in one frame described later. In addition, it is possible to mix a row imaged in the NDR mode and a row imaged in the WDR mode (see FIG. 11 and the like).

  The inverting amplifier 212d is configured such that the output potential Vg is low when the potential of the fifth node N5 as the input potential is high, and conversely, the output potential Vg is high when the potential of the fifth node N5 is low. This output potential Vg is applied to the gate of the transistor constituting the charge discharge gate 212a. As a result, the resistance value (potential barrier) of the charge discharging gate 212a changes according to the potential of the fifth node N5.

  Here, the operation of the sensor area 21 will be described. When light enters the sensor area 21, a charge is generated in the photodiode 211a, and the charge is accumulated in the first node N1. The charges at the first node N1 are continuously accumulated while light is incident on the photodiode 211a in a predetermined charge accumulation period set during one frame period. Thus, when the accumulated amount of charge at the first node N1 exceeds the potential barrier as the potential of the charge discharge gate 212a, the charge overflows from the first node N1 to the fifth node N5.

  Here, the power supply potential VDD2 is a power supply potential configured to be switchable. In the present embodiment, two power sources (the potentials are, for example, VDD and VDD × 3/4) are prepared in advance. Based on a signal (not shown) output by the imaging mode switching unit 5, the potential of VDD2 is set to one by switching these systems. Note that this system switching is possible in units of rasters.

  Regarding the switching of the power supply potential VDD2, the system may be set to VDD and VDD × α, and the latter may be generated by a DA converter provided in the imaging mode switching unit 5.

  When the potential of the power supply potential VDD2 is switched to the same potential as the power supply potential VDD and the clear switch 212c is on, the potential of the fifth node N5 is substantially the same as the power supply potential VDD. The electric charge overflowing to the fifth node N5 is attracted to the fifth node N5 and disappears (that is, VDD becomes a clear potential). Accordingly, since the potential of the fifth node N5 remains unchanged at the power supply potential VDD, the output potential Vg of the inverting amplifier 212d remains low, and the potential barrier of the charge discharge gate 212a remains high and does not change. Therefore, when the clear switch 212c is on, the amount of accumulated charge at the first node N1 increases linearly as the amount of light received by the photodiode 211a increases, and the amount of charge accumulated at the photodiode 211a at a predetermined amount of received light is obtained. It reaches capacity and saturates. Therefore, the output signal output from the pixel circuit 211 also increases linearly with the increase in the amount of light received by the photodiode 211a and becomes saturated at a predetermined amount of received light (in this way, reaches saturation). The potential of the output signal is also referred to as “saturation potential”). Although this output characteristic has a drawback that the dynamic range is relatively narrow, there is an advantage that a linear output characteristic can be obtained below a predetermined received light quantity that does not lead to saturation. Thus, in a state where the clear switch 212c is turned on, the imaging mode of the imaging device 2 is the NDR mode as the first imaging mode.

  On the other hand, when the clear switch 212c is turned off, the electric charge discharged from the first node N1 to the fifth node N5 is accumulated in the overflow drain 212b, and according to the discharge amount, the fifth node N5 is stored. The potential of becomes gradually lower. For this reason, the output potential Vg of the inverting amplifier 212d gradually increases, and accordingly, the potential barrier of the charge discharge gate 212a gradually decreases. This facilitates discharge of charges from the first node N1 to the fifth node N5. As the charge discharge is promoted, the potential of the fifth node N5 is further lowered, and the potential barrier height of the charge discharge gate 212a is further lowered. Thus, the discharge of charges from the first node N1 to the fifth node N5 is accelerated at an accelerated rate.

  In this embodiment, in a predetermined period before the clear switch 212c is turned off, the power supply potential VDD2 is switched to a potential lower than the power supply potential VDD (for example, the clear potential is VDD × 3/4), and the clear switch 212c is turned on. As a result, the discharge charges accumulated in the overflow drain 212b are once cleared. Thus, the potential of the fifth node N5 is set to a potential lower than the power supply potential VDD, and the output potential Vg output through the inverting amplifier 212d in the predetermined period becomes at least higher than that in the above-described NDR mode. . Therefore, the height of the potential barrier formed by the charge discharging gate 212a is lower than that in the NDR mode. As a result, the charge overflows from the first node N1 to the fifth node N5 before reaching the saturation potential. That is, in terms of function, the charge discharge gate 212a is controlled to start discharging charges when the amount of received light is smaller than in the NDR mode.

  Furthermore, in the present embodiment, the gate width and gate length of the charge discharging gate 212a are adjusted so as to limit the charge transfer (that is, the gate width is narrowed or the gate length is lengthened). Even when the charge discharge gate 212a is opened (a state in which the output Vg of the inverting amplifier 212d is increased and the charge can be discharged completely, that is, a state where the height of the potential barrier is minimum), the charge discharge gate 212a is predetermined. The charge discharge amount (or charge discharge speed) is suppressed to such an extent that the charge accumulated in the first node N1 is not discharged to the overflow drain 212b during the charge discharge period. That is, in the imaging device 1 of the present embodiment, even if the charge discharging operation is performed, not all of the charges accumulated in the first node N1 so far are discharged, and only a part thereof is discharged.

  As a method for limiting the charge transfer, in addition to adjusting the physical shape of the gate portion described above, the impurity concentration of the transistor constituting the charge discharge gate 212a may be adjusted low.

As described above, by introducing the control that once clears the overflow drain 212b at the intermediate potential and the configuration that suppresses the discharge of the electric charge, the imaging device of the present embodiment
i) When the amount of received light is small and the charge generated by the photodiode 211a does not overflow beyond the potential barrier of the charge discharge gate 212a (as described above, the barrier height is lower than that of the NDR mode), the output characteristics Indicates a linear characteristic,
ii) When the amount of received light increases and the charge generated by the photodiode 211a overflows beyond the potential barrier of the charge discharge gate 212a, the charge discharge amount depends on the amount of charge newly generated by the photodiode 211a. Is dynamically adjusted (and autonomously)
This brings about a new action.

  Accordingly, by turning off the clear switch 212c during at least a part of the charge accumulation period, the amount of charge accumulated at the first node N1 increases linearly with an increase in the amount of light received by the photodiode 211a. Without saturation, saturation is suppressed even if the amount of received light increases. At this time, the output signal output from the pixel circuit 211 exhibits a characteristic similar to the logarithmic characteristic that gradually approaches the upper limit value corresponding to the saturation as the amount of light received by the photodiode 211a increases. . As described above, in the image pickup device of the present embodiment, the output when the amount of received light is small exhibits a linear characteristic, but when the amount of received light increases, the output exhibits a logarithmic characteristic. It has a hybrid output characteristic in which the generated charges are connected at a light amount that starts to overflow beyond the potential barrier of the charge discharging gate 212a. As a result, a wide dynamic range can be secured in the same way as an image pickup device having a normal logarithmic characteristic, and the degree of compression in a high-luminance region can be reduced by the presence of a linear characteristic portion as compared to an image pickup device having a logarithmic characteristic. As a result, the gradation becomes smaller and the gradation in the high luminance region is ensured.

  Further, as described above, if the potential for clearing the overflow drain 212b can be adjusted (VDD × α), the height of the potential barrier of the charge discharging gate 212a can be arbitrarily set, so the linear characteristic is switched to the logarithmic characteristic. The degree of freedom of point setting increases, and it becomes possible to cope with all imaging conditions.

  Thus, by providing a period during which the clear switch 212c is turned off, the imaging mode of the imaging device 2 becomes the WDR mode as the second imaging mode.

  However, although this output characteristic has the advantage of a relatively wide dynamic range, it does not change in that the output is compressed below a predetermined received light quantity that does not lead to saturation. The problem remains that the number decreases.

  After a predetermined charge accumulation period elapses, the transfer control signal TG turns on the transfer gate 211b to transfer the accumulated charge at the first node N1 to the second node N2. The signal corresponding to the charge transferred to the second node N2 is amplified by the amplifier 211d at a predetermined timing and output as an output signal from the output node NO.

  Next, components other than the sensor area 21 of the image sensor 2 will be described.

  The output signal processing unit 22 includes a CDS (not shown) and an AD converter.

  640 CDSs and AD converters are provided in the main scanning direction corresponding to each pixel circuit 211.

  The CDS removes noise included in the output signal output from the output node NO by so-called correlated double samples.

  The AD converter converts the output signal output from the output node NO from an analog signal to a digital signal.

  The peripheral circuit 23 outputs a timing signal for synchronization when the pixel circuit 211 captures an image. This timing signal also includes the transfer control signal TG, the reset control signal RF, and the pixel selection control signal PS.

  Here, the operation of the image sensor 2 will be described. First, a timing signal is output from the peripheral circuit 23 for each pixel row (hereinafter also referred to as a raster) extending in the main scanning direction (the vertical direction in FIG. 3). The timing signal output from the peripheral circuit 23 is output to each pixel circuit 211 and each accumulated charge discharging circuit 212 through a control signal line (not shown). Next, imaging is performed in each pixel circuit 211 based on the timing signal. At this time, an output signal is output from each pixel circuit 211 as described above. The image is scanned in 480 columns while being shifted in the sub-scanning direction in units of rasters composed of a plurality of pixel circuits 211 and the like that are continuous in the main scanning direction, whereby an output signal of 640 × 480 pixels for one frame can be obtained. . This output signal is output to the output signal processing unit 22 through the readout signal line, and the output signal processing unit 22 performs noise removal and digitization. Then, this digital output signal (hereinafter also referred to as imaging data) is transferred to the buffer memory 3 through the imaging data path. The output signal processing unit 22 is shared by all the rasters (480), and 640 output signal processing units 22 are mounted on the image sensor 2 of the present embodiment.

  In this embodiment, each pixel circuit 211 is provided with one accumulated charge discharging circuit 212. However, as shown in FIG. 14, one pixel is formed by four pixel circuits 211 adjacent in the main scanning direction and the sub-scanning direction. A group may be configured, and one accumulated charge discharging circuit 212 may be shared by the pixel group. Here, the charge accumulation period for accumulating charges in the first node N1 is determined by the timing at which the so-called electronic shutter is released. If a so-called rolling shutter operation is used as a method for releasing this electronic shutter, the shutter is released in units of rasters. Therefore, the timing of discharging charges is superimposed on the same accumulated charge discharging circuit 212 between the raster pixel circuits 211 adjacent in the sub-scanning direction, which may adversely affect the charge discharging operation from the pixel circuit 211. . In order to avoid this, it is preferable to perform a rolling shutter operation in which the shutter is released in units of two rasters that the accumulated charge discharging circuit 212 straddles. Further, since the charges discharged from the four pixel circuits 211 are accumulated in one overflow drain 212b, there is a possibility that the capacity that can accumulate the charges in the overflow drain 212b may be exceeded. In this case, the charge overflows from the overflow drain 212b to the surrounding photodiode 211a, and the image quality of the image output from the imaging device 1 deteriorates. In this regard, as shown in FIG. 2, the configuration in which one accumulated charge discharging circuit 212 is provided for each pixel circuit 211 can secure a sufficient overflow drain capacity for one pixel circuit 211. Is advantageous. In addition, as shown in FIG. 2, providing the one accumulated charge discharging circuit 212 for each pixel circuit 211 without sharing the accumulated charge discharging circuit 212 with the plurality of pixel circuits 211 causes the accumulated charge discharging circuit 212 to discharge. Since the generated charge does not affect the surrounding pixel circuit 211, it is advantageous in that a rolling shutter operation can be performed in units of one raster.

  Further, as shown in FIG. 15, the accumulated charge discharging circuits 212 may be arranged in a staggered manner with respect to the pixel circuit 211, and substantially two accumulated charge discharging circuits 212 may be assigned by the four pixel circuits 211. By doing so, it is possible to increase the overflow drain capacity for one pixel circuit 211 compared to the configuration of FIG. However, when the rolling shutter operation is used, since the shutter is released in units of rasters, the effect of shifting the timing at which charges are discharged from the pixel circuit 211 of the adjacent raster to the same accumulated charge discharging circuit 212 is affected in the sub-scanning direction. There is a risk of spreading not only to adjacent pixel circuits 211 but also to all pixel circuits 211. For this reason, it is preferable to use a so-called global shutter operation in which the electronic shutter operation is performed at the same timing for all the pixel circuits 211 mounted on the image sensor 2.

  Further, as shown in FIG. 16, substantially four accumulated charge discharging circuits 212 may be assigned by four pixel circuits 211. In this way, the overflow drain capacity for one pixel circuit 211 can be further increased. However, the global shutter operation is preferably used because the influence of the timing at which charges are discharged from the pixel circuit 211 of the adjacent raster to the same accumulated charge discharging circuit 212 is spread to all the pixel circuits 211 as described above.

  14 to 16, when one accumulated charge discharging circuit 212 is shared by a plurality of pixel circuits 211, each pixel circuit 211 and the accumulated charge discharging circuit 212 are connected as shown in FIG. A charge discharging gate 212a may be provided between them.

  14 to 16, it is preferable that the charge discharging capability of the charge discharging gate 212a is suppressed as described above.

  Next, components other than the imaging device 2 of the imaging device 1 will be described. As described above, the buffer memory 3 is a storage device that temporarily stores the imaging data output from the imaging device 2, and is a dual port memory having a storage capacity of two frames in this embodiment.

  The image processing unit 4 has a function of converting imaging data transferred from the buffer memory 3 into image data by performing image processing. In the image processing performed by the image processing unit 4, for example, as illustrated in FIG. 5, before the light enters the image sensor 2, green filters are arranged in a staggered manner so as to correspond to the pixel circuit 211. The image data having red, green, and blue color information obtained by passing through a so-called Bayer color filter in which red and blue filters are alternately arranged between the green filters is obtained. There is a so-called interpolation process in which the actual color of target imaging data is estimated from the color of surrounding imaging data. The image processing unit 4 has a storage device for storing imaging data (so-called RAW data) in order to perform image processing as described above. Note that this storage device may be used as the buffer memory 3.

  The imaging mode switching unit 5 sends a clear control signal RO for turning on and off the clear switch 212c to the imaging device 2 based on a control signal from the control unit 6, thereby changing the imaging mode of each imaging device 2 to NDR. It has a function to set the mode or WDR mode. However, as described above, switching of the imaging mode is performed in units of columns with each charge discharging circuit 212 connected in the main scanning direction as one unit.

  Although not shown, the control unit 6 includes a CPU and a calculation unit configured by so-called wired logic. The control unit 6 controls the output signal processing unit 22 and the peripheral circuit 23 to control the imaging device 2. Control the whole operation. The control unit 6 also has a function of determining an appropriate imaging mode for each pixel circuit 211 and instructing the imaging mode switching unit 5 to enter the determined imaging mode. At this time, the control unit 6 sends a mode designation signal indicating the determined imaging mode to the imaging mode switching unit 5, and based on this mode designation signal, the imaging mode switching unit 5 sends the clear control signal RO to the imaging element 2. send. In the illustrated example, the function block of the imaging mode switching unit 5 is provided separately from the control unit 6, but the function of the imaging mode switching unit 5 may be included in the control unit 6.

  Next, imaging mode determination processing executed by the control unit 6 will be described with reference to the flowchart shown in FIG.

  As described above, the imaging data output from the imaging device 2 constitutes a frame of 640 (pixels) × 480 (raster). Thus, the imaging device 1 provides a moving image by outputting 30 frames per second. The control unit 6 determines an appropriate imaging mode based on the imaging mode determination frame. Therefore, as shown in FIG. 6, the control unit 6 instructs to insert an imaging mode determination frame every time a predetermined number of frames are output. In the present embodiment, while the imaging apparatus 1 is set to 30 frames / second, the control unit 6 outputs 14 frames in order to determine an appropriate imaging mode every 0.5 seconds. It is instructed to insert an imaging mode determination frame every time. The imaging mode determination frame is configured by imaging data when each pixel circuit 211 performs imaging in the first imaging mode, and the control unit 6 basically includes imaging data in a saturated state within the imaging mode determination frame. An appropriate imaging mode is determined on the basis of whether or not is included.

  First, when each variable in the flowchart shown in FIG. 7 is defined, FC is a frame counter for counting the number of captured frames, INSERT is an insertion flag indicating whether or not an imaging mode determination frame is inserted, and INSERT_R is an imaging mode determination frame. This is a control constant for controlling the timing, and is set to 15 here. PM is a parameter indicating the imaging mode, and TEMP is a save variable for saving the parameter PM.

  In the flowchart, first, in step S1, variables used in the following steps are initialized. Specifically, 0 is input to the frame counter FC, 0 is input to the insertion flag INSERT, and the NDR mode is input to the parameter PM.

  In the subsequent step S2, it is determined whether or not an instruction for imaging has been given by the user. If the user has instructed imaging, the process proceeds to step S3. On the other hand, if the user has not instructed imaging, the process ends. When the imaging apparatus 1 is equipped with an operating system such as a runtime monitor, a loop that returns to step S1 or step S2 after a predetermined break period may be configured.

  In step S3, it is determined whether or not an imaging mode determination frame is to be inserted. Specifically, when the insertion flag INSERT is 0, the process proceeds to step S4 to insert an imaging mode determination frame, whereas when the insertion flag INSERT is not 0, the process proceeds to step S12 and the imaging mode determination frame is inserted. Do not insert.

  First, when the description is continued assuming that an imaging mode determination frame is inserted (that is, when the insertion flag INSERT is 0), in step S4, the parameter PM is saved in the save variable TEMP.

  In subsequent step S5, the imaging mode switching unit 5 is instructed to set the imaging mode to the NDR mode, and imaging is performed. For this reason, when the image is captured in the NDR mode until now, the image is captured as it is in the NDR mode, whereas when the image is captured in the WDR mode so far, the mode is switched to the NDR mode. . An NDR mode frame obtained as a result is an imaging mode determination frame.

  In subsequent step S6, the imaging mode determination frame obtained in step S5 is stored in the buffer memory 3.

  In subsequent step S7, it is determined whether or not the imaging mode determination frame stored in the buffer memory 3 includes imaging data in a saturated state (hereinafter also referred to as saturation determination). At this time, the fact that the imaging data is in a saturated state may be, for example, the upper limit value of 255 if the imaging data is output in 8 bits. In this embodiment, the imaging data is output in 8 bits, but the output bits may be changed as appropriate, for example, 10 bits or 12 bits. Further, when a predetermined value (for example, 250) is exceeded, it may be determined that the imaging data is in a saturated state. By doing so, it is possible to avoid the influence of an error that may occur in the output signal in the amplifier 211d and the AD converter of the output signal processing unit 22. Note that the imaging data is in a saturated state, in other words, when the imaging data is output as an image, whiteout occurs.

  The saturation determination may be performed on all the imaging data included in the imaging mode determination frame, but may be performed on only a part of the imaging data. For example, determination may be made only for imaging data having green color information among imaging data having red, green, and blue color information obtained by the Bayer array color filter described above. Green color information has the highest correlation with luminance in human visual characteristics, and is arranged in a staggered manner within one frame, so the number of image data to be determined is halved, and the processing time and processing resources required for determination are reduced. This is advantageous in terms. Further, when calculating the strictness, the luminance component (Y component) may be calculated (YC separation) from the red, green, and blue color information of the imaging data, and the saturation determination may be performed based on the luminance component. Further, the saturation determination may be performed by extracting imaging data having the largest value from the imaging mode determination frame and determining whether or not the imaging data exceeds a predetermined value. Furthermore, when there is a predetermined number or more of saturated imaging data, it may be determined that the imaging mode determination frame includes saturated imaging data.

  If it is determined that the imaging mode determination frame includes saturated imaging data, the process proceeds to step S8, while the imaging mode determination frame does not include saturation imaging data. If so, the process proceeds to step S9.

  In Step S8, since the imaging mode determination frame includes the imaging data in the saturated state, more specifically, the imaging data that has a large amount of light received by the imaging device 2 and is saturated when imaging in the NDR mode is output. Therefore, the WDR mode is input to the parameter PM. Then, the process proceeds to step S10.

  In step S9, since the imaging mode determination frame does not include saturated imaging data, more specifically, the amount of light received by the imaging device 2 is small, and saturated imaging data is output even when imaging is performed in the NDR mode. Therefore, the NDR mode is input to the parameter PM. Then, the process proceeds to step S14.

  In step S10, it is determined whether or not the parameter PM saved in the save variable TEMP in step S4 is in the WDR mode. Thus, when the save variable TEMP is in the WDR mode, the process proceeds to step S11. On the other hand, when the save variable TEMP is not in the WDR mode, the process proceeds to step S14.

  In step S 11, the frame stored immediately before the imaging mode determination frame stored in step S 6 is extracted from the buffer memory 3 and output to the image processing unit 4. Then, the process proceeds to step S15. On the other hand, in step S14, the most recently stored frame in the buffer memory 3 is output to the image processing unit 4. That is, the imaging mode determination frame stored in the buffer memory 3 in step S6 is output to the image processing unit 4. Then, the process proceeds to step S15.

  Here, the frame stored in the buffer memory 3 immediately before the imaging mode determination frame in step S11 is a frame captured in the WDR mode indicated by the save variable TEMP. That is, the frame output to the image processing unit 4 before the imaging mode determination frame is inserted is output twice to the image processing unit 4. That is, as shown in FIG. 8A, the imaging mode of the frame output to the image processing unit 4 before inserting the imaging mode determination frame is the WDR mode, and the image processing unit after inserting the imaging mode determination frame When the imaging mode of the frame output to 4 is also in the WDR mode, if the frame is output to the image processing unit 4 in this state, the NDR mode including the imaging data in the saturated state between the frames of the WDR mode that are continuously output Are output to the image processing unit 4. If a state in which the imaging mode is switched for a moment is output as a moving image, a person watching the moving image will see a white-out image for a moment and feel uncomfortable. In order to prevent this, instead of outputting the NDR mode imaging mode determination frame to the image processing unit 4, the WDR mode frame output to the image processing unit 4 before inserting the imaging mode determination frame is continued twice. Output.

  On the other hand, the imaging mode of the frame output to the image processing unit 4 before inserting the imaging mode determination frame shown in FIG. 8B is the NDR mode, and the image processing unit after inserting the imaging mode determination frame When the imaging mode of the frame to be output to 4 is also the NDR mode, or the imaging mode of the frame output to the image processing unit 4 before inserting the imaging mode determination frame shown in FIG. When the imaging mode of the frame output to the image processing unit 4 after the imaging mode determination frame is inserted is the WDR mode, or before the imaging mode determination frame is inserted as shown in FIG. The imaging mode of the selected frame is the WDR mode, and the imaging mode of the frame output to the image processing unit 4 after inserting the imaging mode determination frame is the NDR mode. The, since not occur the state in which imaging mode described above is switched a moment, in step S14, and outputs the image pickup mode determining frame as it is to the image processing unit 4.

  In the present embodiment, as described above, a frame imaged in the NDR mode is used as the imaging mode determination frame. However, a frame imaged in the WDR mode can be used instead. This is because it is possible to specify the value of imaging data in the WDR mode when saturation occurs in the NDR mode by imaging in the WDR mode when the output is just saturated when imaging in the NDR mode. .

  Now, when a frame imaged in the WDR mode is used as the imaging mode determination frame, the problem is that the imaging mode determination frame in the WDR mode is inserted in the state where the image is captured in the NDR mode, and the subsequent imaging is performed here. This is a case where it is determined that an image is captured in the NDR mode as a mode. In this case, it is only necessary to perform control so that the frame imaged in the NDR mode is output twice in step S11 described above. To generalize this, if frames that are continuous before and after the imaging mode determination frame are captured in an imaging mode that is different from the imaging mode used in the imaging mode determination frame, the imaging mode determination frame is discarded and the imaging mode determination is performed. That is, control is performed so that the frame immediately before the frame is output again.

  Furthermore, as described above, since the frame captured in the WDR mode as well as the NDR mode can be adopted as the imaging mode determination frame, the current (that is, the frame captured immediately before the imaging mode determination frame is inserted). When the imaging mode is the WDR mode, the imaging mode determination frame is set to the WDR mode. When the current imaging mode is the NDR mode, the imaging mode determination frame is inserted as the NDR mode. It is possible to eliminate the problem that the imaging mode is switched between the determination frame and before and after the determination frame. When this control is employed, it is possible to avoid outputting the same frame twice as described above.

  Note that the imaging mode determination frame is used only for determination of the imaging mode, and is not output to the image processing unit 4. The number of frames per unit time output from the pixel circuit 2 is determined per unit time output from the imaging device 1. It may be more than the number of frames. For example, by configuring the pixel circuit 2 to output 32 frames per second, the imaging mode determination frame (two imaging mode determination frames are output per second) is output to the image processing unit 4. Even without this, the imaging apparatus 1 can output 30 frames per second. In this case, output of the same frame twice as in step S11 in the flowchart described above is avoided.

  In step S15 which has shifted from step S11 and step S14, 1 is added to the frame counter FC.

  In subsequent step S16, the frame counter FC is divided by the control constant INSERT_R, and the remainder is input to the insertion flag INSERT. Then, the process returns to step S2. In this embodiment, INSERT_R is set to 15, and an imaging mode determination frame can be inserted every time 14 frames are output. Note that the control constant INSERT_R may be changed as appropriate. That is, a mode setting by user designation is separately provided in the image pickup apparatus 1. For example, for a subject moving at high speed, the control constant INSERT_R is decreased, and control for increasing the switching opportunity between the NDR mode and the WDR mode is performed. It is preferable to do.

  Next, when the insertion flag INSERT is not 0 and the imaging mode determination frame is not inserted, the process proceeds from step S3 to step S12.

  In step S12, the imaging mode switching unit 5 is instructed to set the imaging mode to the value of the parameter PM, and imaging is performed. Therefore, the image is captured in the NDR mode as it is while the image is captured in the NDR mode, and the image is captured in the WDR mode as it is while the image is captured in the WDR mode.

  In the subsequent step S13, the frame obtained in step S12 is stored in the buffer memory 3. Then, the process proceeds to step S14.

  In step S14, as described above, the most recently stored frame in the buffer memory 3 is output to the image processing unit 4. That is, the frame stored in the buffer memory 3 in step S12 is output to the image processing unit 4. Then, it returns to step S2 through step S15 and step S16.

  In this manner, during imaging by the imaging apparatus 1, an appropriate imaging mode is the NDR mode or the WDR mode based on the imaging mode determination frame configured by imaging data captured in the NDR mode. Is determined, and the imaging mode of the imaging device 2 is set to an appropriate imaging mode based on the determination result. In other words, when imaging in the NDR mode with a relatively narrow dynamic range is possible, imaging is performed in the NDR mode. Imaging is performed. Thus, the imaging apparatus 1 can take high-quality images while avoiding overexposure and the like by making use of the advantages of each of the NDR mode and the WDR mode and compensating for each defect.

  In the present embodiment, the imaging mode of a frame to be output after that is determined based only on whether or not the imaging mode determination frame includes saturated imaging data. As shown in FIG. 9, when it is determined in step S7 that the imaging mode determination frame includes saturated imaging data, the average value of all the imaging data included in the imaging mode determination frame is a predetermined value. You may make it transfer to step S7A which determines whether it is larger than this.

  In step S7A, when the average value of all the imaging data included in the imaging mode determination frame is larger than a predetermined value, the process proceeds to step S8, and the subsequent imaging mode is set to the WDR mode, while the imaging mode determination frame is set. When the average value of all the imaging data included in is less than or equal to a predetermined value, the process proceeds to step S9, and the subsequent imaging mode is set to the NDR mode. As a result, the imaging mode is set to the WDR mode only when the imaging mode determination frame includes saturated imaging data and the average value of all the imaging data included in the imaging mode determination frame is greater than a predetermined value. In other cases, even when saturated imaging data is included in the imaging mode determination frame, the imaging mode is set to the NDR mode. As a result, when the imaging data in a saturated state (that is, high brightness) is spread over a relatively wide range (when the average value is larger than the predetermined value) in the imaging mode determination frame, the overexposed area is wide. Therefore, when the imaging mode is set to the WDR mode to avoid over-exposure of the image, and when the imaging data in the saturated state is only local (when the average value is equal to or less than a predetermined value), the over-exposed region is localized. Therefore, the imaging mode is set to the NDR mode. By doing so, an image closer to human vision can be obtained.

  Note that since humans generally tend to gaze at the center of the image, in step S7A, not the average value of all the imaging data included in the imaging mode determination frame, but the imaging data included in the center of the imaging mode determination frame. It may be determined whether the average value of is greater than a predetermined value. In this way, it is possible to determine the imaging mode that matches the human sense.

  Moreover, as long as it is the control of the same content as the control shown in FIG. 9, you may change suitably, for example, you may replace the order of step S7 and step S7A.

  Furthermore, as shown in FIG. 10, when it is determined in step S7 that the imaging mode determination frame includes saturated imaging data, the frequency of occurrence of imaging data below a predetermined value is greater than a predetermined reference value. Alternatively, the process may proceed to step S7B for determining whether the value is smaller.

  In step S7B, when the occurrence frequency of imaging data below a predetermined value in the imaging mode determination frame is smaller than a predetermined reference value, the process proceeds to step S8, and the subsequent imaging mode is set to the WDR mode. If the occurrence frequency of the imaging data is greater than or equal to a predetermined reference value, the process proceeds to step S9, and the subsequent imaging mode is set to the NDR mode. Thus, the imaging mode is changed only when the imaging mode determination frame includes saturated imaging data and the frequency of occurrence of imaging data below a predetermined value in the imaging mode determination frame is smaller than a predetermined reference value. In the WDR mode, otherwise, the imaging mode is set to the NDR mode even when the imaging mode determination frame includes saturated imaging data. That is, as described above, when the imaging data in the saturation state in the imaging mode determination frame is spread over a relatively wide range (when the occurrence frequency is smaller than the reference value), the whiteout region becomes wide. When the imaging mode is set to the WDR mode to avoid overexposure of the image, and when the imaging data in the imaging mode determination frame is only in a saturated state (when the occurrence frequency is equal to or higher than the reference value), the overexposed region is Since it is determined to be local and the entire image is dark, the imaging mode is set to the NDR mode.

  In step S7B as well, only the imaging data included in the central portion of the imaging mode determination frame may be subject to frequency measurement in the same manner as described above.

  Further, the control may be appropriately changed as long as the control has the same content as the control shown in FIG. Then, when the occurrence frequency of imaging data greater than or equal to a predetermined value is greater than the predetermined value, the imaging mode may be changed to the WDR mode, or the order of step S7 and step S7B may be switched.

  Further, in the present embodiment, the imaging mode of a frame subsequent to the imaging mode determination frame is determined based on the saturation determination result, and the imaging mode is changed in real time. Frames that are output in the imaging mode determined by the saturation determination only when the result determined in is held for a predetermined time (for example, 0.5 seconds), and then the saturation determination is performed again and both results match. May be imaged. In this way, the amount of incident light normally changes gradually, but the result of the saturation determination is that the result of the saturation determination before and after the saturation of the imaging data due to the noise included in the imaging data, for example. Even if they are changed every time, it is possible to prevent the actual imaging mode from being changed frequently.

  Furthermore, in the present embodiment, the entire pixels of the imaging mode determination frame are configured by imaging data in the NDR mode. However, the imaging device 2 includes a pixel circuit 211 and an accumulated charge discharging circuit 212 as shown in FIG. Is compatible with 1: 1, and an NDR mode and a WDR mode can be individually set for each pixel circuit 211, for example, as shown in FIG. The mode determination frame may be configured such that rasters configured with imaging data in the NDR mode and rasters configured in the WDR mode are alternated. In this way, when performing saturation determination, saturation determination can be performed using half of the imaging data in the imaging mode determination frame as a determination target. In addition, it is also possible to further extract image data having green color information, and perform saturation determination using the extracted image data as a determination target. As a result, processing time can be shortened and processing resources can be reduced. This is advantageous in reducing the cost of the imaging apparatus. Note that the sampling for reducing the determination target of saturation determination is effective even in an image sensor equivalent to a VGA of 640 × 480 pixels as in the present embodiment, and for example, 1280 × 1024 pixels (so-called SXGA) or 1600 × 1200 pixels. This is particularly effective when a high-resolution image sensor such as (so-called UXGA) is used.

  As shown in FIG. 14, when one accumulated charge discharging circuit 212 is shared between two pixel circuits 211 adjacent in the sub-scanning direction, there is a problem due to the shutter being released in units of one raster as described above. Since it is assumed, as shown in FIG. 11B, the imaging mode determination frame may be configured so that the imaging data in the NDR mode and the imaging data in the WDR mode are alternated in units of two rasters. At this time, one pixel group shown in FIG. 14 needs to be completely included in the raster imaged in the WDR mode or the NDR mode, and one pixel group extends over both the WDR mode and the NDR mode. Should not be set.

  In the above-described embodiment, the imaging mode determination frame is periodically inserted and the optimal imaging mode is periodically determined. However, as a modification, each frame output in time series from the imaging device 2 is determined. The NDR mode imaging data (raster) and the WDR mode imaging data (raster) are configured, and the raster is composed of two temporally continuous frames, thereby equivalently configuring the imaging mode determination frame. Depending on the determination result based on the imaging mode determination frame, rasters may be appropriately combined between two temporally continuous frames and output from the imaging apparatus 1.

  That is, in this modification, based on the imaging mode switched by the imaging mode switching unit 5, the imaging device 2 has a raster (first imaging mode raster) based on an output signal imaged in the NDR mode constituting the frame, A raster (second imaging mode raster) based on an output signal imaged in the WDR mode is output. Then, the control unit 6 should be selected based on an output signal (imaging data stored in the buffer memory 3) included in one of the first imaging mode raster and the second imaging mode raster. The imaging mode is determined, and based on the determination result, the first imaging mode raster or the second imaging mode raster is selected from the frame and the frame captured immediately before this frame, and is combined. is there.

  Specifically, when the frame counter FC is an even number, as shown in FIG. 12A, the odd-numbered row (number of rows is counted from 1) in the sub-scanning direction (vertical direction in the drawing). ) Raster (hereinafter also referred to as odd-numbered raster) is constituted by imaging data in NDR mode, while a raster in even-numbered rows (hereinafter also referred to as even-numbered raster) is constituted by imaging data in WDR mode. That is, in one frame, rasters captured in the NDR mode and rasters captured in the WDR mode are alternately arranged so as to be adjacent to each other.

  On the other hand, when the frame counter FC is an odd number, as shown in FIG. 12 (b), contrary to FIG. 12 (a), the odd raster is composed of imaging data in the WDR mode, while the even raster is in the NDR mode. It is comprised by the imaging data of. That is, the positions of rasters captured in the NDR mode and rasters captured in the WDR mode are exchanged between adjacent frames.

  The raster imaged in the NDR mode and the raster imaged in the WDR mode may be reversed. Then, as shown in FIG. 13, when the frame counter FC is 1, the control unit 6 stores the frame stored in the buffer memory 3 when the frame counter FC is 0 (hereinafter also referred to as the 0th frame). The odd-numbered raster imaging data (that is, the NDR mode imaging data) is extracted from the frame, and the even-numbered raster imaging data from the frame (hereinafter also referred to as the first frame) stored in the buffer memory 3 when the frame counter FC is 1. (Ie, NDR mode imaging data) is extracted. In this way, an imaging mode determination frame constituted by only the imaging data of the NDR mode is equivalently created by the 0th and first frames. Then, saturation determination is performed by extracting imaging data having further green color information from the equivalent imaging mode determination frame. For the saturation determination, the above-described various determination methods can be employed.

  Here, when the imaging data determination frame includes saturated imaging data, the control unit 6 sets the even-numbered raster imaging data (that is, WDR mode imaging data) of the 0th frame and the first frame. The odd-numbered raster image data (that is, WDR mode image data) is combined and output to the image processing unit 4 (DMA transfer).

  On the other hand, when the imaging data determination frame does not include saturated imaging data, the control unit 6 uses the odd-numbered raster imaging data (that is, NDR mode imaging data) of the 0th frame and the first frame. Are combined with the even raster image data (that is, the NDR mode image data) and output to the image processing unit 4 (DMA transfer).

  In parallel with the synthesis of the 0th frame and the 1st frame and output to the image processing unit 4, the imaging device 2 uses the frame memory FC 2 for the frame counter FC (hereinafter also referred to as the second frame) to the buffer memory 3. Output to.

  Then, the saturation determination is performed in the same manner as described above based on the even-numbered raster image data of the first frame and the odd-numbered raster image data of the second frame. At this time, since the control unit 6 has already performed the saturation determination for the first frame, it is only necessary to perform the saturation determination for the odd-numbered raster imaging data (that is, the NDR mode imaging data) in the second frame. At this time, if the result of the saturation determination of the first frame already performed is different from the result of the saturation determination of the second frame newly performed, the determination result based on the second frame having a later chronological order is obtained. You may make it give priority.

  In this embodiment, a raster imaged in the NDR mode is extracted from two consecutive frames, and these are combined to form an equivalent imaging mode determination frame. However, the raster imaged in the NDR mode is used instead. A raster imaged in the WDR mode may be used.

  In addition, since the phase between images is generally very high between adjacent rasters, an image of 640 × 480 pixels is not constructed as an imaging mode determination frame, but an image is captured in the NDR mode obtained by one imaging. Saturation determination or the like may be performed based on only the raster (or the raster imaged in the WDR mode).

Thereafter, by repeating the same control as described above, the synthesized frames are continuously output to the image processing unit 4. Even in this way, since the frame based on the imaging data in the NDR mode and the frame based on the imaging data in the WDR mode are switched and outputted, the contrast is unnecessarily lowered while avoiding overexposure, etc. It is possible to obtain a moving image that avoids a decrease in gradation. Further, by performing this control, the same effect as that of so-called progressive scan can be obtained, so that an incidental effect that a moving image with less image blur can be obtained even for a subject moving at high speed can be obtained.
(Embodiment 2)
Next, the imaging apparatus 101 according to the second embodiment of the present invention will be described with reference to FIG. The imaging apparatus 101 according to the second embodiment is different from the imaging apparatus 1 according to the first embodiment in operation in the WDR mode. Specifically, in the imaging apparatus 101 according to the second embodiment, in the WDR mode in which the charge discharging operation is performed, the dynamic range is changed by changing the potential (accumulated charge amount) at which the charge discharging starts. .

  The imaging apparatus 101 according to the second embodiment is different from the first embodiment in the power supply potential node, the inverting amplifier, the function of the control unit, and the function of the imaging mode switching unit. Therefore, the same configurations as those of the first embodiment are denoted by the same reference numerals, and the configurations different from those of the first embodiment will be mainly described.

  In the imaging device 101, the input node of the control potential generator 212e is connected to the fifth node N5 and the imaging mode switching unit 5, and the power supply potential supplied to the fifth node N5 by turning on the clear switch 212c. However, the power supply potential VDD supplied from the power supply potential node N3 is set.

  Similar to the first embodiment, the control unit 6 controls the overall operation of the image sensor 2 and transmits a mode designation signal to the imaging mode switching unit 5. Unlike the first embodiment, the control unit 6 will be described in detail later. However, based on the imaging data included in the frame, a potential designation signal related to the discharge start potential of the accumulated charge in the WDR mode is determined, and the potential designation signal is transmitted to the imaging mode switching unit 5. In this embodiment, the number of output bits of the potential designation signal is 8 bits.

  As in the first embodiment, the imaging mode switching unit 5 sends clear control signals RO, RO_1, RO_2,... To the imaging device 2 based on the mode designation signal from the control unit 6, and thereby each pixel circuit 211. On the other hand, in addition to the configuration described in the first embodiment, the potential designation signal from the control unit 6 is converted from a digital signal to an analog signal, and the electric charge converted into the analog signal is changed to the NDR mode or the WDR mode. A port for outputting the discharge designation potential V_ref to the control potential generation unit 212e is provided.

  The control potential generation unit 212e is configured by adding an adder to the inverting amplifier 212d in the first embodiment. The control potential generation unit 212e includes a potential at the fifth node N5 (hereinafter also referred to as a fifth node potential) V_n5 and the imaging mode switching unit 5. The control potential Vg is generated by calculating an analog signal based on the charge discharge designated potential V_ref input from the and is output.

  Here, the characteristics of the control potential Vg in the WDR mode will be described with reference to FIG.

  FIG. 18 shows the characteristic of the control potential Vg with respect to the overflow charge amount de discharged from the first node N1 to the fifth node N5 through the charge discharge gate 212a. In this figure, V_ref0, V_ref1, V_ref2, V_ref3 and V_ref4 correspond to charge discharge designated potentials when the potential designation signal (8 bits) output from the control unit 6 is 0, 63, 127, 191 and 255, respectively. To do.

  When the overflow charge amount de is 0, the control potential Vg is equal to the charge discharge designated potential V_ref, and when the overflow charge amount de is larger than 0, the control potential Vg increases in proportion to the increase of the overflow charge amount de. . Therefore, the control potential Vg can be expressed by the following formula (1).

Vg = V_ref + k × de (1)
Here, k represents the increment of the control potential Vg with respect to the increment of the overflow charge amount de, that is, the inclination.

  This equation (1) can be transformed into the following equation (2).

Vg = V_ref + k * VDD-k * (VDD-de) (2)
In the formula (2), VDD is a power supply potential supplied from the power supply potential node N3, so VDD-de is equivalent to the fifth node potential V_n5. Therefore, it can be expressed as the following formula (3).

Vg = V_ref + k × VDD−k × V_n5 (3)
This equation (3) can be transformed into the following equation (4).

Vg = V_ref + k (VDD−V_n5) (4)
Since VDD-V_n5 in the second term of the equation (4) is subtraction, it can be constituted by a subtractor or an inverting amplifier. Further, the slope k can be set by the amplification factor for the output after subtraction or the amplification factor of the inverting amplifier. Further, the addition of the first term and the second term in the equation (4) can be performed by an adder. Therefore, in the present embodiment, the control potential generator 212e is configured with an inverting amplifier (or subtractor) and an adder so that the control potential Vg has a characteristic represented by the expression (1).

  That is, the control potential Vg before the charge is discharged from the first node N1 to the fifth node N5 is equivalent to the charge discharge specified potential V_ref, and this charge discharge specified potential V_ref is the initial potential barrier of the charge discharge gate 212a. Therefore, in the second embodiment, the imaging mode switching unit 5 outputs the charge discharge designation potential V_ref corresponding to the potential designation signal from the control unit 6 to the control potential generation unit 212e. The initial height of the potential barrier of the discharge gate 212a can be changed and set. Then, after the charge of the first node N1 is accumulated in the overflow drain 212b beyond the potential barrier of the charge discharge gate 212a, the fifth node potential V_n5 is lowered and controlled by the same as in the first embodiment. Since the potential Vg increases and the height of the potential barrier of the charge discharging gate 212a decreases, the discharging of charges from the first node N1 to the fifth node N5 is promoted.

  Here, in the WDR mode in which the accumulated charge discharging operation is performed, a change in the charge discharging operation accompanying a change in the initial height of the potential barrier of the charge discharging gate 212a will be described. In the following description, there is a part overlapping with the description in the first embodiment.

  First, when the control unit 6 transmits “63” as a potential designation signal to the imaging mode switching unit 5 (corresponding to V_ref1 in FIG. 18), a relatively small charge discharge designated potential V_ref is supplied to the control potential generation unit 212e. The movement of charges when inputting will be described with reference to FIG.

  FIG. 19 is a diagram schematically showing the state of the potential barrier of the charge discharging gate 212a and the movement of charges. In this figure, the vertical axis represents the height of the potential barrier, while on the horizontal axis, W1 is an arrangement region of the fifth node N5 (overflow drain capacitance) (hereinafter also referred to as an overflow drain region), and W2 is a charge discharge gate. 212a is an arrangement region (hereinafter also referred to as a charge discharge gate region), W3 is an arrangement region of a photodiode 211a (hereinafter also referred to as a photodiode region), and W4 is an arrangement region of a transfer transistor 211b (hereinafter also referred to as a transfer transistor region). W5 corresponds to an arrangement region (hereinafter also referred to as a floating diffusion region) of the second node N2 (so-called floating diffusion).

  As shown in FIG. 19, the origin of the control potential Vg is above the vertical axis, and as the value of the control potential Vg increases, the potential barrier peak moves downward, that is, the potential barrier height is set lower.

  Before the charge is discharged from the first node N1 to the fifth node N5 shown in FIG. 19A, the control potential Vg is equivalent to the charge discharge specified potential V_ref as described above. Like the potential V_ref, it is relatively small, and the difference from the substrate potential (that is, 0 V) is relatively small. Therefore, the initial potential barrier of the charge discharging gate 212a is set to be relatively high. In this state, the charge generated by receiving light by the photodiode 211a is accumulated in the potential well (that is, the photodiode region W3) of the photodiode 211a.

  As the amount of light received by the photodiode 211a increases, as charge is further accumulated in the photodiode region W3, as shown in FIG. 19B, the charge exceeds the potential barrier of the charge discharge gate region W2, It overflows from the diode region W3 to the overflow drain region W1.

  Thus, the charges overflowing from the photodiode region W3 to the overflow drain region W1 are accumulated in the overflow drain region W1 (the clear switch 212c is off to perform the charge discharging operation). By accumulating charges in the overflow drain region W1, the fifth node potential V_n5 decreases and the control potential Vg increases as can be understood from the equation (4). As shown, the height of the potential barrier in the charge discharge gate region W2 decreases. As a result, charges further overflow from the photodiode region W3 to the overflow drain region W1. In this way, the accumulated charge discharging operation is promoted.

  At this time, the amount of charge accumulated in the photodiode region W3 is determined by the balance between the amount of charge newly generated in the photodiode 211a and the amount of charge discharged from the photodiode region W3 to the overflow drain region W1.

  If the potential barrier height of the charge discharge gate region W2 is constant without decreasing, the amount of charge discharged from the photodiode region W3 to the overflow drain region W1 is the amount of charge generated in the photodiode 211a. Proportional to quantity. This is the same mechanism as that when a voltage is applied to a resistor having a certain resistance value, the current increases proportionally when the voltage is increased. Therefore, even if charge is discharged from the photodiode region W3 to the overflow drain region W1, the amount of charge accumulated in the photodiode region W3 is lower than that when charge is not discharged. It increases linearly as the amount of charge generated in the diode 211a increases.

  On the other hand, in the imaging device of the present invention, the height of the potential barrier of the charge discharge gate region W2 decreases as the amount of accumulated charge in the overflow drain region W1 increases, and therefore overflows from the photodiode region W3. The amount of charge discharged to the drain region W1 does not increase in proportion to the amount of charge generated in the photodiode 211a, but increases in an accelerated manner. Similarly to the first embodiment, the imaging apparatus 101 adjusts the gate width and gate length (or impurity concentration) of the charge discharge gate 212a to discharge charge from the first node N1 to the fifth node N5. Thus, the amount of charge newly generated in the photodiode 211a is configured to exceed the amount of charge discharged from the first node N1 to the fifth node N5. From this, when the charge is discharged from the photodiode region W3 to the overflow drain region W1, the amount of charge remaining in the photodiode region W3 is a balance between the amount of newly generated charge and the amount of discharged charge. As the amount of electric charge generated in the photodiode 211a increases, it does not increase linearly but increases logarithmically. Here, since the value of the imaging data (the output value of the imaging device) is based on the amount of charge accumulated in the photodiode region W3, as shown in FIG. 22, the amount of light received by the photodiode 211a is not relatively large. In other words, when the amount of light received by the photodiode 211a is small to medium and the accumulated charge does not overflow beyond the charge discharge gate, the amount of light received by the photodiode 211a increases linearly. When the amount of light received by the photodiode 211a becomes relatively large, discharge of electric charge is started, and thereafter, the amount of light received by the photodiode 211a increases logarithmically. That is, the value of the imaging data indicates a hybrid output characteristic in which a linear characteristic and a logarithmic characteristic are connected. For this reason, the dynamic range is expanded as compared with imaging data having linear characteristics. Furthermore, since it has an area that has a linear characteristic, it has a substantial number of gradations compared to a simple logarithmic characteristic in which the output suddenly increases with an increase in the amount of received light in the low luminance range (in which the amount of received light is small). Can be secured.

  Next, when the control unit 6 transmits “191” as the potential designation signal to the imaging mode switching unit 5 (corresponding to V_ref3 in FIG. 18), the relatively large charge discharge designated potential V_ref is set to the control potential generation unit 212e. The movement of the charge when it is input to will be described with reference to FIG.

  Before the charge is discharged from the first node N1 to the fifth node N5 shown in FIG. 20A, the control potential Vg is equivalent to the charge discharge designated potential V_ref as described above. Similar to the specified potential V_ref, it is relatively large, and the difference from the substrate potential (that is, 0 V) is relatively large. Therefore, the height of the initial potential barrier of the charge discharge gate 212a is set lower than that in FIG.

  When charges are accumulated in the photodiode region W3 from the state of FIG. 20A, the received light quantity of the photodiode 211a is reduced by the amount of the initial height of the potential barrier, as shown in FIG. Even when the charge is relatively small, the charge overflows the potential barrier of the charge discharge gate region W2 and overflows from the photodiode region W3 to the overflow drain region W1.

  Thus, the charges overflowing from the photodiode region W3 to the overflow drain region W1 are accumulated in the overflow drain region W1, and the potential barrier height of the charge discharge gate region W2 is further lowered ((c in FIG. 20). )reference). As a result, charges further overflow from the photodiode region W3 to the overflow drain region W1.

  Therefore, after the charge discharge is started, the charge is discharged to the photodiode region W3 when the charge is discharged from the photodiode region W3 to the overflow drain region W1, as in the case where the initial height of the potential barrier is high. The amount of accumulated charge does not increase linearly with respect to the amount of charge generated in the photodiode 211a, but increases logarithmically. Therefore, as shown in FIG. 23, the value of the imaging data shows a hybrid characteristic in which a linear characteristic and a logarithmic characteristic are connected, while the photodiode 211a is compared with a case where the initial potential barrier is high. Since the charge discharge is started when the amount of received light is relatively small, the range showing the logarithmic characteristic becomes relatively wide. Therefore, when the initial potential barrier height is low (FIG. 23), the dynamic range is expanded compared to when the initial potential barrier height is high (FIG. 22).

  As described above, by changing the charge discharge designated potential V_ref input to the control potential generation unit 212e, the height of the initial potential barrier of the charge discharge gate 212a is changed, and the dynamic range of the image sensor 2 can be changed.

  Next, for reference, the movement of charge when the imaging mode is the NDR mode will be described with reference to FIG. In the NDR mode, it is preferable to increase the initial potential barrier height by setting the potential designation signal to “8”, for example, and inputting a small charge discharge designation potential V_ref to the control potential generator 212e. Note that the potential designation signal may be appropriately set to a value that is larger than 0 and relatively small.

  Before the charge is discharged from the first node N1 to the fifth node N5 shown in (a) of FIG. 21, the control potential Vg is equivalent to the charge discharge specified potential V_ref as described above. The signal and the charge discharge designated potential V_ref are comparatively small and substantially the same as the substrate potential (that is, 0 V). Therefore, the potential barrier of the charge discharge gate 212a is relatively high.

  When charge is accumulated in the photodiode region W3 from the state of FIG. 21A and the charge of the photodiode 211a exceeds the potential barrier of the charge discharge gate region W2, as shown in FIG. It overflows from the region W3 to the overflow drain region W1. At this time, since the clear switch 212c is on (that is, the fifth node N5 and the power supply potential node N3 are connected), the charge that overflows from the photodiode region W3 to the overflow drain region W1 It is attracted to the potential node N3 and disappears without being accumulated in the overflow drain region W1. Therefore, the control potential Vg does not change, and the height of the potential barrier in the charge discharge gate region W2 remains constant without decreasing.

  Then, the transfer control signal TG is set to Hi at the transfer timing of the accumulated charge, thereby connecting the first node N1 and the second node N2, and as shown in FIG. The charge accumulated in the diode region W3 is transferred to the floating diffusion region W5. The charges transferred to the floating diffusion region W5 are output as image data from the image sensor 2 through the amplifier 211d and the output signal processing unit 22.

  For this reason, in the NDR mode, the charge discharging operation for reducing the height of the potential barrier of the charge discharging gate 212a is not performed. Therefore, the amount of charge accumulated in the photodiode region W3 is the amount of charge generated in the photodiode 211a. It increases linearly with the increase of. For this reason, as shown in FIG. 24, the output increases linearly as the amount of light received by the photodiode 211a increases, and reaches a value corresponding to the charge storage capacity of the photodiode 211a with a predetermined amount of received light. To do.

  As described above, in the second embodiment, control is performed to optimize the dynamic range according to the subject by changing the dynamic range in the WDR mode.

  Next, determination processing of the potential designation signal by the control unit 6 relating to optimization of the dynamic range executed in the WDR mode will be described.

  When the imaging mode is the NDR mode, the control unit 6 sets the potential designation signal to a constant value of “8”, for example, as described above.

  On the other hand, when the imaging mode is the WDR mode, the control unit 6 sets the potential designation signal so that the maximum value of the imaging data included in the frame (hereinafter also simply referred to as the maximum value of the imaging data) becomes a predetermined value. To do. As described above, in accordance with the change of the potential designation signal, the charge discharge designation potential V_ref input to the control potential generation unit 212e is changed, so that the output characteristics with respect to the received light amount of the photodiode 211a are linear characteristic regions and logarithmic characteristic regions. As a result, the output value for the same received light quantity changes. Therefore, the control unit 6 determines the potential designation signal so that the maximum value of the imaging data included in the frame is a predetermined value, as shown in FIG. By doing so, the output of the image sensor 2 rises to the maximum output value, and the output is not unnecessarily compressed, and the number of gradations can be secured most as a whole within one frame.

  FIG. 25 illustrates a process in which the control unit 6 determines the potential designation signal so that the maximum value of the imaging data becomes the maximum output value as a predetermined value when the imaging data is output in 8 bits. It is a figure to do. In FIG. 25, the maximum output value is set to 250 so as to be different from the NDR mode, but the maximum output value may be changed as appropriate.

  First, as described in the first embodiment, immediately after the imaging mode is switched from the NDR mode to the WDR mode, since the imaging in the NDR mode was possible until then, the received light amount of the photodiode 211a is the NDR mode. It is often near the amount of received light corresponding to saturation, and it can be estimated that the amount of charge discharged is not so large. On the other hand, as the output characteristics of the image sensor 2, it is desirable that the range of the logarithmic characteristics in which the output is compressed by narrowing the range of the linear characteristics as narrow as possible. For this reason, the potential designation signal is set to a relatively small value in order to make the initial potential barrier relatively high. In the present embodiment, immediately after the imaging mode is switched from the NDR mode to the WDR mode, the control unit 6 sets the 8-bit potential designation signal to “32”. This “32” also means that 256 is divided into eight equal parts in order to set subsequent potential designation signals in 32 increments.

  Imaging is performed in a state where the potential designation signal is “32”, and the control unit 6 extracts the maximum value of the imaging data included in the output frame. As shown in FIG. 25, when the maximum value of the imaging data reaches, for example, saturation (255) and exceeds the maximum output value, the control unit 6 charges from the first node N1 to the fifth node N5. It is determined that the discharge amount is insufficient, and “32” is added to the potential designation signal so as to further increase the charge discharge amount, and the potential designation signal is determined to be “64”. Thus, imaging is performed with the initial potential barrier of the charge discharging gate 212a further lowered. When the maximum value of the imaging data does not exceed the maximum output value, a binary search method described later is executed.

  Assume that the maximum value of the imaging data of the frame output by imaging in the state where the potential designation signal is “64” is, for example, 150 as shown in FIG. 25 and does not exceed the maximum output value. The controller 6 determines that the charge discharge amount from the first node N1 to the fifth node N5 is excessive, and reduces the charge discharge amount. Subsequent potential designation signals are determined by a so-called binary search method. Specifically, the control unit 6 determines the potential designation signal to be “48” which is an intermediate value between “32” and “64”.

  Then, imaging is performed in a state where the potential designation signal is “48”, and the maximum value and the maximum output value of the imaging data of the frame output thereby are compared. For example, in the example of FIG. 25, the maximum value of the imaging data is 250 by setting the potential designation signal to “48”. Therefore, in this example, the search is terminated, and thereafter, imaging is continued with the potential designation signal set to “48”.

  That is, the control unit 6 adds the potential designation signal by 32 until the maximum value of the imaging data becomes smaller than the maximum output value, and after the maximum value of the imaging data becomes smaller than the maximum output value, the potential designation is performed. The signal is determined by a binary search method. In this case, the maximum number of searches is 13 times obtained by adding 8 times that is a number obtained by dividing 256 into 32 and 5 times the maximum number of searches when searching for the step size 32 by the binary search method. For example, when the potential designation signal when the maximum value of the imaging data becomes the maximum output value (hereinafter also referred to as the optimum potential designation signal) is 229, the potential designation signal is 32, 64, 96, 128, 160, 192. , 224, 255, 240, 232, 228, 230, and 229 in this order, the search can be performed 13 times. In this way, by using the binary search method for determining the potential designation signal, even if the amount of light received by the photodiode 211a changes abruptly, follow-up control (so-called linear search) that searches for the optimum potential designation signal in order is performed. ) Can be searched very quickly. However, by using the binary search method for determining the potential designation signal, the value of the imaging data may repeatedly increase and decrease. For example, when the optimum potential designation signal is 229, the value of the imaging data increases or decreases as in the case where the potential designation signal changes like 232, 228, 230, 229, and converges. There is a case. When a person sees an output image captured in this state, it may feel uncomfortable. Therefore, in the present embodiment, once the optimum potential designation signal is obtained, the amount of change in the maximum value of the imaging data is checked every time a frame is output, and the amount of change is a predetermined value (for example, 5). When the change amount is smaller, the follow-up control is performed to change the value of the potential designation signal by ± 1. On the other hand, when the change amount is equal to or greater than a predetermined value, the binary search method is used.

  Here, in the imaging device 1 according to the first embodiment, the charge discharge for turning off the clear switch 212c in the charge accumulation period in one raster period, that is, in at least a part of the period before the accumulated charge transfer period. Although a period is provided (see, for example, the alternate long and short dash line in FIG. 26), in the imaging apparatus 101 according to the second embodiment, as described above, by optimizing the potential designation signal, for example, as illustrated in FIG. Since the maximum value of the imaging data can be set to the maximum output value, the charge discharge period may be set to the entire period of the charge accumulation period (all periods before the accumulated charge transfer period (extended charge discharge period)). (See FIG. 26). By doing so, the charge discharge operation is started when the charge stored in the first node N1 exceeds the potential barrier of the charge discharge gate 212a in the charge storage period. The charge discharge period may be a part of the charge accumulation period instead of the entire period, and in that case, the end of the charge discharge period may correspond to the start of the accumulated charge transfer period.

  As described above, in the imaging apparatus 101 according to the second embodiment, the value of the imaging data shows a hybrid characteristic in which the linear characteristic and the logarithmic characteristic are connected as shown in FIG. Thus, the maximum value of the imaging data is a predetermined value (250) while changing the amount of received light at which the linear characteristic and the logarithmic characteristic are switched (that is, changing the amount of received light at which charge discharge is started). The characteristics approaching. Therefore, in the imaging device 1 of the first embodiment, when imaging in the WDR mode, the dynamic range is expanded as compared with the NDR mode, but the dynamic range is fixed to a predetermined range width. In contrast, the output is compressed unnecessarily below a predetermined amount of received light that does not reach saturation, and this may cause a decrease in contrast and a decrease in the number of gradations. The imaging apparatus 101 according to the second embodiment has an advantage that the dynamic range is optimized, so that the contrast and the number of gradations are not reduced.

  Therefore, the imaging apparatus 101 according to the second embodiment of the present invention can output a high-quality image while avoiding overexposure.

  In the imaging apparatus 101 according to the second embodiment, in the WDR mode, if the image is captured with a relatively small potential designation signal comparable to that in the NDR mode and no saturation occurs, the charge discharging operation is executed. Therefore, the output characteristic exhibits a linear characteristic that is substantially the same as that of the NDR mode. For this reason, the WDR mode may be always set without performing the operation for switching the imaging mode between the NDR mode and the WDR mode using the determination frame described in the first embodiment. However, in the WDR mode, as described above, in order to optimize the dynamic range, a process for searching for an optimal potential designation signal is necessary, whereas the switching operation between the NDR mode and the WDR mode is performed. In such a case, since such processing is not required during imaging in the NDR mode, the switching operation between the NDR mode and the WDR mode is advantageous from the viewpoint of processing load.

  As described above, the image pickup apparatus of the present invention can output an image with high contrast and high gradation while avoiding overexposure or the like. For example, the image pickup apparatus can be applied to various apparatuses including a television door phone and a network camera. This is useful in that a high-quality image can always be obtained when used.

1 is a schematic configuration diagram of an imaging apparatus according to the present invention. It is a figure explaining an example of arrangement | positioning relationship between a pixel circuit and a stored charge discharge circuit. 1 is a circuit diagram of an imaging apparatus according to Embodiment 1 of the present invention. It is a circuit diagram concerning the modification of an imaging device. It is a block diagram of a color filter of a Bayer arrangement. It is a figure explaining the timing which inserts an imaging mode determination frame. It is a flowchart for determining the imaging mode by a control part. It is a figure explaining the method to determine the flame | frame output to an image process part. It is another flowchart of the saturation determination by a control part. It is another flowchart of the saturation determination by a control part. It is a figure which shows the other structure of an imaging mode determination frame. It is a figure which shows the other structure of a flame | frame. It is explanatory drawing about the method of synthesize | combining the flame | frame output to an image process part using the flame | frame shown in FIG. It is a figure explaining an example of arrangement | positioning relationship between a pixel circuit and a stored charge discharge circuit. It is a figure explaining an example of arrangement | positioning relationship between a pixel circuit and a stored charge discharge circuit. It is a figure explaining an example of arrangement | positioning relationship between a pixel circuit and a stored charge discharge circuit. It is a circuit diagram of the imaging device concerning Embodiment 2 of the present invention. It is a graph which shows the characteristic of a control potential. FIG. 6 is a diagram schematically illustrating a potential barrier and a charge state when a relatively small charge discharge designated potential is input to a control potential generation unit. FIG. 6 is a diagram schematically illustrating a potential barrier and a charge state when a relatively large charge discharge designated potential is input to a control potential generation unit. It is the figure which represented typically the potential barrier and the state of an electric charge when imaged in NDR mode. It is a figure which shows the characteristic of the imaging data when a comparatively small charge discharge designation | designated electric potential is input into the control electric potential generation | occurrence | production part. It is a figure which shows the characteristic of the imaging data when a comparatively big electric charge discharge | release designation | designated electric potential is input into the control electric potential generation | occurrence | production part. It is a figure which shows the characteristic of the imaging data when imaged in NDR mode. It is a figure explaining the determination process of the electric potential designation | designated signal by a control part. 6 is a timing chart illustrating an operation of one raster in the imaging apparatus.

1,101 imaging device 2 imaging device 21 sensor area 211 pixel circuit 211a photodiode (light receiving device)
212 Accumulated charge discharging circuit 212a Charge discharging gate 212b Overflow drain 212c Clear switch 212e Control potential generating unit 22 Output signal processing unit 3 Buffer memory 4 Image processing unit 5 Imaging mode switching unit 6 Control unit Vg Control potential V_n5 Fifth node potential V_ref Charge Discharge specified potential

Claims (5)

A charge is generated by receiving each light, and a plurality of light receiving elements arranged in a lattice pattern so as to form a plurality of rasters, and a charge generated by the light receiving elements are provided corresponding to each of the light receiving elements. And a plurality of pixel circuits that output an output signal corresponding to the amount of accumulated charge, and that continuously output frames configured based on the output signal in time series With
The imaging device includes an overflow drain that accumulates charges discharged from the pixel circuits, a control potential generator that outputs a control potential according to the amount of charges accumulated in the overflow drains, and the pixel circuits. A charge that forms a potential barrier with the overflow drain and changes the discharge mode of the charge from the pixel circuit to the overflow drain by changing the height of the potential barrier according to the control potential. A discharge gate, and an accumulation charge discharge operation capable of discharging a part of the charge accumulated in each pixel circuit to the overflow drain before outputting an output signal of each pixel circuit. A charge discharging circuit;
A control unit that sets an initial height of a potential barrier of the charge discharge gate corresponding to the charge discharge start potential by controlling the control potential through the control potential generation unit according to an output signal of the pixel circuit. An image pickup apparatus further comprising: The imaging device according to claim 1,
An output signal processing unit for outputting the frame including imaging data obtained by converting the output signal of each pixel circuit into a digital signal;
The control unit controls an initial height of the potential barrier so that a maximum value of the imaging data included in the frame becomes a preset maximum output value of the imaging element. Imaging device. The imaging device according to claim 2,
The control unit is configured to output a potential designation signal corresponding to the output signal,
The control potential generation unit receives an accumulated charge amount of the overflow drain as an input, and outputs an inverting amplifier that outputs a higher control potential as the accumulated charge amount increases, and a charge corresponding to a potential designation signal from the control unit An adder for adding the discharge designated potential to the control potential,
The charge discharge gate is configured such that the higher the control potential, the lower the potential barrier height,
The control unit further outputs the potential designation signal so that the charge discharge designated potential is increased when the maximum value of the imaging data included in the frame exceeds the maximum output value of the imaging element, while the imaging An image pickup apparatus configured to output the potential designation signal so that the charge discharge designated potential is lowered when a maximum value of data is lower than the maximum output value. The imaging apparatus according to any one of claims 1 to 3,
The imaging mode of each of the pixel circuits for the imaging device is a first imaging mode in which the accumulated charge discharging operation by the accumulated charge discharging circuit is not executed, and a second imaging mode in which the accumulated charge discharging operation is executed. An imaging mode switching unit that switches between
The imaging device includes at least a part of a first imaging mode raster configured based on the output signal captured by the pixel circuit in the first imaging mode every time a predetermined number of frames are output. Output an imaging mode judgment frame consisting of
The control unit further controls the imaging mode of each pixel circuit through the imaging mode switching unit based on the output signal constituting the first imaging mode raster in the imaging mode determination frame. An imaging apparatus characterized by specifying a mode. The imaging apparatus according to claim 4,
The imaging device further includes a clear switch for clearing the accumulated charge in the overflow drain,
The imaging mode switching unit switches to the first imaging mode in which the accumulated charge discharging circuit does not execute the accumulated charge discharging operation by turning on the clear switch, while turning off the clear switch. An imaging apparatus, wherein the discharging circuit is configured to be in a second imaging mode in which the accumulated charge discharging operation is executed.

Images