JP2010171867A - Imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus for outputting a high-quality image while avoiding over-exposure or the like in accordance with the situation of a subject. <P>SOLUTION: The imaging apparatus 1 includes: an imaging mode switching section 5 and a control section 6. The imaging mode switching section 5 switches an imaging mode of each of pixel circuits 211 between a narrow dynamic range (NDR) mode and a wide dynamic range (WDR) mode. During the NDR mode, when the quantity of received light of a photodiode 211a is less than a predetermined quantity of light, an output signal is linearly increased with increase in the quantity of received light and when the quantity of received light of the photodiode 211a is equal to or more than the predetermined quantity of light, an output signal is saturated. During the WDR mode, when the quantity of received light of the photodiode 211a is equal to or more than the predetermined quantity of light, an output signal is not saturated. On the basis of an output signal at a time when the pixel circuit 211 captures an image in the NDR mode, for the imaging mode switching section 5, the control section 6 designates the imaging mode of each of the pixel circuits 211 to the NDR mode or to the WDR mode. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an imaging apparatus capable of imaging by switching output characteristics.

  2. Description of the Related Art Conventionally, an image pickup apparatus including a semiconductor image pickup device (hereinafter also simply referred to as an image pickup device) such as a CCD or a CMOS image sensor has been used in various devices, and examples thereof include a TV door phone and a network camera. . This image pickup apparatus includes a plurality of light receiving elements arranged in a grid pattern in the image pickup element, and converts the received light into an electric signal by each light receiving element, thereby outputting an image based on the electric signal. Specifically, the image sensor accumulates an amount of charge corresponding to the amount of light received by each light receiving element during a preset charge accumulation period, and the amount of the accumulated charge (hereinafter also referred to as accumulated charge). The electric signal according to is output. The imaging device forms a frame as a plane image based on the electrical signal output from the imaging element, and outputs a moving image by continuing the frame in time series. Here, the image sensor generally has a characteristic of accumulating an amount of electric charge proportional to the amount of received light (hereinafter also referred to as the amount of received light). Therefore, an electrical signal output from the image sensor is Although it shows a so-called linear characteristic that increases linearly with the increase in the amount of received light, if charge is accumulated until it reaches a capacity (hereinafter also referred to as charge storage capacity) that can store the charge of the image sensor. An electrical signal output from the image sensor (hereinafter also referred to as a saturated state) becomes constant at a predetermined saturation potential. Since the dynamic range of the image sensor is defined in this way, when a relatively dark subject is imaged so as not to reach saturation, the gradation of brightness according to the magnitude of the electrical signal output from each image sensor Appears in the image, but when a relatively bright subject that is in a saturated state is imaged, the image is completely white, that is, whites are blown out. In view of this, there is known an imaging apparatus that avoids whiteout of an image even in a relatively narrow dynamic range by changing the charge accumulation period according to the brightness of the subject. That is, in this imaging apparatus, when a bright subject is imaged, the charge accumulation period is shortened so that the accumulation of charges does not reach a saturation state to avoid overexposure.

  However, when imaging a subject that includes a part with a large difference in brightness in one field of view, such as when imaging a person with the sun behind, if the charge accumulation period is shortened according to the bright part, the dark part is completely black. In other words, the black portion is crushed, but if the charge accumulation period is lengthened in accordance with the dark portion, the bright portion will be blown out.

  Therefore, attempts have been made to expand the dynamic range of the image sensor (hereinafter also referred to as a wide dynamic range) by obtaining output characteristics different from the linear characteristics by various methods.

  For example, in the image sensor disclosed in Patent Document 1, when the amount of charge generated and accumulated by the light receiving element is large (more specifically, the charge generated and accumulated by the photodiode is set in advance). In the case of overflowing to the outside beyond the potential barrier), at a predetermined intermediate timing of the charge accumulation period, by performing a so-called charge discharging operation that discharges all of the accumulated charges accumulated so far, Saturation at the signal output timing is avoided, thereby realizing a wide dynamic range.

Further, for example, Patent Document 2 includes a MOS transistor that inputs a photocurrent generated according to the amount of received light, and a bias unit that biases the MOS transistor to a state that is equal to or lower than a threshold voltage and allows a subthreshold current to flow. A technique for realizing a wide dynamic range by logarithmically compressing a photocurrent with the MOS transistor is disclosed.
Japanese Patent No. 3996618 Japanese Patent Laid-Open No. 03-192864

  By the way, in the imaging device disclosed in Patent Document 1, when the amount of received light exceeds a predetermined amount of light, the accumulated charge is increased at a predetermined intermediate timing of the charge accumulation period (the larger the amount of received light). (Accelerated), because the output potential is determined based on the charge that has been discharged once and then re-accumulated during the remaining period, the magnitude of the amount of received light, the beginning and end of the charge discharge timing, and the length of the charge discharge period For example, there is a concern that gradation inversion occurs in the final output signal. For example, consider a case where one output frame is composed of gradation gradations having a large contrast. At this time, the charge is not discharged in the dark part, and the output gradually increases linearly (linear characteristics) as the amount of received light increases. Is discharged at an accelerated rate, so the output that was monotonically increasing up to the middle of the gradation gradation turns to zero. As a result, even if an attempt is made to correct the amount of re-accumulated charge based on the ratio of the re-accumulation period set after discharging the charge and the entire charge accumulation period, gradation gradation (as long as there is a point where the output becomes zero) In this case, tone reversal can occur in any part of the dark to bright tone change.

On the other hand, the output of the image sensor disclosed in Patent Document 2 exhibits logarithmic characteristics that asymptotically approach a saturation potential. This image sensor can theoretically realize a very large dynamic range, but in order to reliably avoid saturation even in a wide dynamic range such as a light amount ratio of 1:10 ⁶ (about 120 [dB]), for example, Since the output is compressed as the value becomes larger, the output becomes smaller than the electric signal output from the linear imaging device even if the amount of received light is the same. For this reason, in an imaging apparatus equipped with this imaging device, when a relatively dark subject that does not reach saturation is imaged, the contrast is lower than that of an imaging device having linear characteristics, and the output compression described above is performed. As a result, the substantial number of gradations also decreases.

  In response to issues such as a decrease in contrast and a decrease in the number of gradations, it is conceivable to amplify the output with an analog amplifier. However, in this way, the noise component is also amplified at the same time, and the output differs from the linear characteristic. An image sensor having characteristics may have a larger noise than an image sensor having linear output characteristics.

  As described above, an image pickup apparatus provided with an image pickup element having a wide dynamic range has an advantage of a wide dynamic range, but suffers from disadvantages such as a decrease in gradation and an increase in noise.

  The present invention has been made in view of the above points, and an object of the present invention is to provide an imaging device capable of high-quality imaging while avoiding overexposure depending on the state of the subject. It is in.

  The inventors of the present application, for example, by creating an image pickup device of Patent Document 1 by appropriately setting a manufacturing process, even if a charge discharging operation is executed, all of the accumulated charges accumulated so far are discharged. Without charge generation and charge discharge based on the amount of received light, and an output characteristic similar to a logarithmic characteristic can be developed, and on / off of a switch element specifying execution / non-execution of the charge discharge operation ( Furthermore, by appropriately controlling the gate potential of the switch element, it is easy to operate with a relatively narrow dynamic range with linear output characteristics and with a relatively wide dynamic range with output characteristics similar to logarithmic characteristics. I noticed that I could switch to Therefore, in an imaging apparatus that can operate by switching output characteristics, attention was paid to performing imaging while appropriately switching these operations.

  In the first invention, the imaging device generates charges by receiving light, and is provided corresponding to each of the plurality of light receiving elements arranged in a lattice pattern so as to form a plurality of rasters. And a plurality of pixel circuits for accumulating charges generated in the light receiving element and outputting an output signal corresponding to the accumulated amount of the charges, and configured based on the output signal, For the image sensor that continuously outputs in time series, and for the image sensor, the imaging mode of each pixel circuit is set to increase the amount of received light when the amount of received light of the light receiving element is smaller than a predetermined amount of light. While the output signal increases linearly, when the received light amount of the light receiving element is equal to or greater than the predetermined light amount, the first imaging mode in which the output signal is saturated, and the received light amount of the light receiving element is the predetermined light amount. The imaging mode switching unit that switches between the second imaging mode in which the output signal is not saturated when the amount of light is greater than the light amount, and the imaging mode based on the output signal when the pixel circuit images in the first imaging mode And a control unit that designates the imaging mode of each of the pixel circuits as the first or second imaging mode with respect to the switching unit.

  In the first imaging mode, when the amount of light received by the light receiving element is smaller than a predetermined amount of light, the output signal increases linearly as the amount of received light increases. Therefore, when a relatively dark subject whose output signal does not reach saturation is imaged in the first imaging mode, the imaging device outputs an image with high contrast and high gradation. On the other hand, in the first imaging mode, the output signal is saturated when the amount of light received by the light receiving element is greater than or equal to a predetermined amount of light. For this reason, when a relatively bright subject is imaged in the first imaging mode, the image may be blown out. That is, the first imaging mode corresponds to an operation state in which the dynamic range is relatively narrow.

  On the other hand, the second imaging mode is an imaging mode that does not saturate when the amount of light received by the light receiving element is greater than or equal to a predetermined amount of light. Therefore, a relatively bright subject in which an image is blown out when captured in the first imaging mode. However, when the image is captured in the second imaging mode, the image is not overexposed. That is, the second imaging mode corresponds to an operation state in which the dynamic range is relatively wide.

  According to the imaging device of the first invention, the control unit designates the imaging mode of each pixel circuit to the imaging mode switching unit based on the output signal when imaging is performed in the first imaging mode, and thereby the imaging device The imaging mode is switched between the first imaging mode and the second imaging mode. Thus, when no problem occurs in the output image even if the first imaging mode with a relatively narrow dynamic range is used, imaging is performed in the first imaging mode. By doing so, inconveniences (for example, low contrast, a decrease in the number of gradations, an increase in noise, etc.) that may occur when imaging in the second imaging mode are avoided. On the other hand, when a problem such as overexposure may occur when the image is captured in the first image capture mode, by switching to the second image capture mode with a relatively wide dynamic range and performing image capture, a clear image that avoids overexposure etc. Can output a simple image. Thus, the imaging device of the first invention takes advantage of the advantages of each operation by appropriately switching between the operation state with a relatively narrow dynamic range and the operation state with a relatively wide dynamic range, The shortcomings of each operation can be compensated.

  According to a second aspect, in the first aspect, the imaging element outputs an imaging mode determination frame based on the output signal captured by the pixel circuit in the first imaging mode every time a predetermined number of frames are output. The control unit makes a predetermined determination using the output signal included in the imaging mode determination frame as an imaging mode determination target, and determines the imaging mode of each pixel circuit thereafter based on the determination result. specify.

  By doing so, the imaging device outputs an imaging mode determination frame based on imaging in the first imaging mode every time a predetermined number of frames are output. At this time, when the imaging mode of the imaging device is the first imaging mode, an imaging mode determination frame based on imaging in the first imaging mode may be output as it is. In contrast, when the imaging mode of the imaging device is the second imaging mode, the imaging mode determination frame is output by temporarily switching the imaging mode of the imaging device to the first imaging mode at the output timing of the imaging mode determination frame. do it. Thus, the control unit can specify the imaging mode of each pixel circuit after the imaging mode determination frame based on the output signal of the first imaging mode included in the imaging mode determination frame. That is, by periodically determining which of the first and second imaging modes is appropriate, even when the state of the subject changes, the imaging mode of the image sensor is appropriately set according to the change. It becomes possible to switch.

  In a third aspect based on the first aspect, the imaging device is configured based on the output signal captured by the pixel circuit in the first imaging mode every time a predetermined number of frames are output. An imaging mode determination frame configured to include an imaging mode raster and a second imaging mode raster configured based on the output signal captured by the pixel circuit in the second imaging mode, and the control The unit performs a predetermined determination using the output signal constituting the first imaging mode raster in the imaging mode determination frame as a determination target, and specifies an imaging mode of each pixel circuit thereafter based on the determination result To do.

  By doing so, an appropriate imaging mode is not determined based on all output signals included in the imaging mode determination frame, but an appropriate imaging mode is determined based on a part of output signals included in the imaging mode determination frame. In order to make a determination, the number of output signals to be determined is reduced by that amount, and the processing related to the determination of the imaging mode is reduced. This is advantageous in terms of processing time and processing resources. In particular, the higher the resolution of the image sensor, the more effective.

  According to a fourth invention, in the second or third invention, further comprising an output signal processing unit that outputs the frame configured to include imaging data obtained by converting the output signal of each pixel circuit into a digital signal, The control unit determines whether the maximum value of the imaging data corresponding to the determination target of the imaging mode determination frame output from the output signal processing unit exceeds a predetermined threshold, and the maximum value is When the predetermined threshold value is exceeded, the imaging mode switching unit designates the imaging mode of each pixel circuit as the second imaging mode.

  According to this configuration, when the maximum value of the imaging data corresponding to the determination target of the imaging mode determination frame is equal to or less than the predetermined threshold, the control unit sets the imaging mode of each pixel circuit to the imaging mode switching unit. When the imaging mode is designated and the maximum value of the imaging data corresponding to the determination target of the imaging mode determination frame exceeds a predetermined threshold, the imaging mode of each pixel circuit is set to the second imaging mode with respect to the imaging mode switching unit. Is specified. That is, when the imaging mode determination frame is relatively bright and includes a portion that is overexposed in imaging in the first imaging mode, the subsequent imaging is performed in the second imaging mode. As a result, it is possible to reliably prevent white spots from being included in the image.

  In a fifth aspect based on the fourth aspect, the control unit calculates an average value of the imaging data corresponding to the determination target of the imaging mode determination frame, and whether the average value is greater than a predetermined value. When the average value is equal to or less than the predetermined value, the image pickup mode of each pixel circuit is set to the image pickup mode switching unit even when the maximum value exceeds the predetermined threshold. The first imaging mode is designated.

  According to this configuration, the control unit determines an appropriate imaging mode not only based on the maximum value of the imaging data corresponding to the determination target but also considering the average value of the imaging data corresponding to the determination target. That is, only when the maximum value exceeds a predetermined threshold and the average value is larger than the predetermined value, the imaging mode of each pixel circuit is designated as the second imaging mode for the imaging mode switching unit, In other cases, even when the maximum value exceeds a predetermined threshold, when the average value is equal to or less than the predetermined value, the imaging mode of each pixel circuit is designated as the first imaging mode. Therefore, the imaging apparatus captures an image in the second imaging mode when the white spot is spread over a comparatively wide range in the frame, while the white spot is included even if the white spot is included in the frame. If the location is local, imaging is performed in the first imaging mode. As a result, an image closer to human vision can be output.

  In a sixth aspect based on the fourth aspect, the control unit counts the occurrence frequency of the imaging data that is equal to or less than a predetermined determination value for the imaging data corresponding to the determination target of the imaging mode determination frame, and Further determining whether or not the occurrence frequency is less than a predetermined reference value, and when the occurrence frequency is greater than or equal to a predetermined reference value, even when the maximum value of the imaging data exceeds the predetermined threshold, The imaging mode switching unit designates the imaging mode of each pixel circuit as the first imaging mode.

  According to this configuration, in addition to the maximum value of the imaging data corresponding to the determination target, the control unit appropriately captures the imaging data corresponding to the determination target based on the occurrence frequency of the imaging data equal to or less than the predetermined determination value. A proper imaging mode is determined. That is, only when the maximum value exceeds a predetermined threshold and the occurrence frequency is smaller than a predetermined reference value, the imaging mode is designated as the second imaging mode for the imaging mode switching unit, and otherwise At this time, even if the maximum value exceeds a predetermined threshold value, when the occurrence frequency is equal to or higher than a predetermined reference value, the imaging mode of each pixel circuit is designated as the first imaging mode. Accordingly, as in the fifth aspect of the invention, the imaging device performs imaging in the second imaging mode when the white spot is spread over a relatively wide range in the frame, while the white spot is present in the frame. Even if it is included, when the white spot is local and the relatively dark part occupies the majority, the first imaging mode is used to capture an image closer to human vision. Can be output.

  According to a seventh invention, in the second or third invention, further comprising an output signal processing unit that outputs the frame configured to include imaging data obtained by converting the output signal of each pixel circuit into a digital signal, The control unit includes, for the imaging data corresponding to the determination target of the imaging mode determination frame output from the output signal processing unit, whether the imaging data that has reached the upper limit value corresponding to the saturation is included in the frame When the imaging data that has reached the upper limit value is included in the frame, the imaging mode of each pixel circuit is set to the second imaging mode with respect to the imaging mode switching unit. specify.

  According to this configuration, the control unit designates the imaging mode of each pixel circuit as the first imaging mode with respect to the imaging mode switching unit when imaging data that has reached the upper limit value corresponding to saturation is not included in the frame. On the other hand, when the imaging data that has reached the upper limit value is included in the frame, the imaging mode switching unit designates the imaging mode of each pixel circuit as the second imaging mode. In other words, in imaging in the first imaging mode, when a bright portion where the output signal is saturated is included in the frame, the image is captured in the second imaging mode, and thus an overexposed portion is included in the image. Is reliably prevented.

  In an eighth aspect based on the seventh aspect, the control unit calculates an average value of the imaging data corresponding to the determination target of the imaging mode determination frame, and whether the average value is greater than a predetermined value. Further, when the average value is equal to or less than the predetermined value, each of the image capturing mode switching unit is configured to perform each of the image capturing mode switching units even when the image capturing data reaching the upper limit value is included in the frame. The imaging mode of the pixel circuit is designated as the first imaging mode.

  According to this configuration, the control unit considers not only whether the imaging data that has reached the upper limit value is included in the frame, but also an appropriate imaging mode in consideration of the average value of imaging data corresponding to the determination target. Determine. That is, when the image data that has reached the upper limit value is included in the frame and the average value of the image data included in the frame is greater than a predetermined value, the image capturing of each pixel circuit is performed with respect to the image capturing mode switching unit. While the mode is designated as the second imaging mode, the imaging mode of each pixel circuit is set when the average value is equal to or less than the predetermined value even if the other imaging data that has reached the upper limit value is included in the frame. The first imaging mode is designated. Therefore, the imaging apparatus captures an image in the second imaging mode when the white spot is spread over a comparatively wide range in the frame, while the white spot is included even if the white spot is included in the frame. When the location is local, imaging is performed in the first imaging mode, and an image closer to human vision can be output.

  In a ninth aspect based on the seventh aspect, the control unit counts the frequency of occurrence of the imaging data equal to or less than a predetermined determination value for the imaging data corresponding to the determination target of the imaging mode determination frame, and It is further determined whether or not the occurrence frequency is lower than a predetermined reference value. When the occurrence frequency is equal to or higher than the predetermined reference value, the image data that has reached the upper limit value is included in the frame. Sometimes, the imaging mode switching unit designates the imaging mode of each pixel circuit as the first imaging mode.

  According to this configuration, in addition to whether or not the imaging data that has reached the upper limit value is included in the frame, the control unit also adds the occurrence frequency of imaging data equal to or less than a predetermined determination value to the determination condition, Determine an appropriate imaging mode. That is, the imaging mode is designated as the second imaging mode for the imaging mode switching unit only when imaging data that has reached the upper limit value is included in the frame and the occurrence frequency is smaller than a predetermined reference value. On the other hand, even when other imaging data that has reached the upper limit value is included in the frame, if the occurrence frequency is equal to or higher than a predetermined reference value, the imaging mode of each pixel circuit is designated as the first imaging mode. To do. Accordingly, as in the eighth aspect, the imaging device performs imaging in the second imaging mode when the white spot is spread over a relatively wide range in the frame, while the white spot is present in the frame. Even if it is included, if the white spot is local, the image is captured in the first imaging mode, which makes it possible to output an image closer to human vision.

  In the tenth invention, the imaging device generates charges by receiving light, and is provided corresponding to each of the plurality of light receiving elements arranged in a lattice pattern so as to form a plurality of rasters. A plurality of pixel circuits for accumulating charges generated in the light receiving element and outputting an output signal corresponding to the accumulated amount of the charges, and each pixel circuit for the image sensor When the received light amount of the light receiving element is smaller than a predetermined light amount, the output signal increases linearly as the received light amount increases, while the received light amount of the light receiving element is equal to or greater than the predetermined light amount. Between the first imaging mode in which the output signal is saturated and the second imaging mode in which the output signal is not saturated when the amount of received light of the light receiving element is equal to or greater than the predetermined amount of light. An imaging mode switching unit that switches in units of rasters, and the imaging element includes a first imaging mode raster based on the output signal imaged in the first imaging mode and the output imaged in the second imaging mode. A frame including the second imaging mode raster based on the signal is continuously output in time series, and one of the first imaging mode raster and the second imaging mode raster included in the frame is output. The imaging mode to be selected is determined based on the output signal included in the raster, and based on the determination result, the first imaging mode raster or the first imaging mode is determined from the frame and the frame imaged immediately before. The image processing apparatus further includes a control unit that selects the two imaging mode rasters and creates a frame obtained by combining the rasters. Here, the positions of the first imaging mode raster and the second imaging mode raster may be switched between the frames adjacent in time series.

  According to the imaging device of the tenth aspect of the invention, since each frame includes both the first imaging mode raster and the second imaging mode raster, the control unit includes the first imaging mode raster or the first imaging mode raster included in each frame. Based on the output signal included in the two imaging mode raster, it can be determined for each frame which of the first and second imaging modes is appropriate. When it is determined that the first imaging mode is an appropriate imaging mode, the first imaging mode raster is extracted from the current frame and the frame output immediately before it, and the extracted first imaging mode is extracted. When the second imaging mode is determined to be an appropriate imaging mode while the first imaging mode frame is synthesized by the raster, the second imaging mode raster is extracted from the current frame and the frame output immediately before it. Then, the second imaging mode frame is synthesized by the extracted second imaging mode raster. In this way, the imaging apparatus appropriately switches and outputs the frame captured in the first imaging mode and the frame captured in the second imaging mode according to the state of the subject, so that the imaging mode determination frame is output. It is possible to output a clear image avoiding overexposure without having to do anything.

  In an eleventh aspect based on the tenth aspect, the imaging mode switching unit alternately arranges the first imaging mode raster and the second imaging mode raster so as to be adjacent to each other in each frame.

  In this way, the frame is configured with the first imaging mode raster and the second imaging mode raster being equalized, so that the output signal of the entire frame is not the state of the output signal at a specific part of the frame. The imaging mode can be determined based on the state.

  In a twelfth aspect based on the eleventh aspect, the imaging mode switching unit swaps the positions of the first imaging mode raster and the second imaging mode raster between the frames adjacent in time series.

  Since the position of the raster that is the imaging mode determination target (first imaging mode raster or second imaging mode raster) is changed with the passage of time, it is avoided to continue determination based on the situation of the specific position in the frame, The imaging mode can be appropriately determined in response to a change in the situation of the subject.

  According to a thirteenth aspect, in any one of the first to twelfth aspects, the image pickup device discharges a part of the charge accumulated in each pixel circuit before discharging the output signal. And a storage charge discharging circuit capable of executing an operation, wherein the imaging mode switching unit performs execution and non-execution of the storage charge discharging operation of the storage charge discharging circuit based on an imaging mode designated by the control unit. Switch.

  According to this configuration, when the stored charge discharging operation by the stored charge discharging circuit is not executed, the charge is linearly stored in each pixel circuit, and the charge storage is saturated when the amount of received light is large. Become. Therefore, when the accumulated charge discharging operation is not executed, the output signal increases linearly when the amount of light received by the light receiving element is smaller than the predetermined amount of light, while when the amount of received light of the light receiving element is equal to or greater than the predetermined amount of light. Operation in the first imaging mode in which the output signal is saturated can be performed.

  On the other hand, when the stored charge discharging operation of the stored charge discharging circuit is executed, a part of the charge stored in each pixel circuit is discharged, so that it is avoided that the stored charge is saturated in each pixel circuit. Even when the amount of received light is greater than or equal to the predetermined amount, the output signal does not saturate and the operation in the second imaging mode can be performed.

  In a fourteenth aspect based on the thirteenth aspect, the accumulated charge discharging circuit is provided in the middle of a path for discharging accumulated charges, and includes a charge discharging gate configured by a transistor, In the second imaging mode, the discharge is controlled to start discharging when the amount of light received by the light receiving element is smaller than in the first imaging mode.

  According to this configuration, the accumulated charge discharging circuit discharges the charges accumulated in the pixel circuit through the charge discharging gate. At this time, by controlling the charge discharge gate to start discharging charges from a stage where the received light amount is smaller than that in the first imaging mode, the received light amount is such that the output signal is saturated when the image is captured in the first imaging mode. However, it is possible to prevent the output signal from being saturated. Thereby, the second imaging mode can be realized.

  As described above, in the imaging apparatus of the present invention, it is possible to appropriately switch between an operation state with a relatively narrow dynamic range and an operation state with a relatively wide dynamic range, and thereby each operation While taking advantage of the state, each defect can be compensated.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. The following description of the preferred embodiment is merely exemplary in nature.

  As shown in FIG. 1, the imaging apparatus 1 of the present invention temporarily stores the imaging element 2 that converts incident light into an electrical signal and outputs it as imaging data, and imaging data output from the imaging element 2. The buffer memory 3, the imaging data transferred from the buffer memory 3, and the image processing unit 4 for constructing the image data based on the imaging data, and the imaging mode of the imaging device 2 will be described in detail later. An imaging mode switching unit 5 that switches between a narrow dynamic range (NDR) mode and a wide dynamic range (WDR) mode with a relatively wide dynamic range, and imaging based on imaging data stored in the buffer memory 3 The imaging mode is designated to the mode switching unit 5 and the overall operation of the imaging device 2 is controlled. A control unit 6 which comprises a.

  As shown in FIG. 1, the image pickup device 2 is arranged in a matrix by being arranged side by side in the main scanning direction (vertical direction in FIG. 1) and the sub-scanning direction (horizontal direction in FIG. 1) orthogonal thereto. A sensor area 21 having a plurality of pixel circuits 211 arranged in the pixel circuit 211 and an accumulated charge discharging circuit 212 connected to the pixel circuit 211, and the sensor area 21 arranged on the right side in FIG. The output signal processing unit 22 and the peripheral circuit 23 disposed below the sensor area 21 in FIG. 1 are provided.

  As shown in FIGS. 1 and 2 (where FIG. 2 shows a state in which the pixel circuit 211 is rotated 90 ° clockwise with respect to FIG. 1), the pixel circuits 211 are equally spaced (for example, at an interval of 7.8 μm). For example, 640 pixels are arranged in the main scanning direction and 480 pixels are arranged in the sub-scanning direction, so that a total of 640 × 480 pixel circuits 211 are provided in one image sensor 2. . As a result, the imaging device 2 constitutes a so-called 1/3 inch imaging device, and the imaging device 1 can output an image having a resolution equivalent to VGA.

  As shown in FIG. 3, each pixel circuit 211 has a photodiode 211a as a light receiving element and a transfer for transferring the charge generated by the photodiode 211a and accumulated in the first node N1 to the second node N2. A gate 211b, a reset switch 211c that can clear charges accumulated in the second node N2, an amplifier 211d that amplifies and outputs a signal corresponding to the potential of the second node N2, an amplifier 211d and an output node NO And a pixel selection switch 211e that outputs a signal amplified by the amplifier 211d from the pixel circuit 211 at a predetermined timing.

  The photodiode 211a is composed of a semiconductor pn junction, and receives incident light by a so-called photovoltaic effect and generates an amount of electric charge corresponding to the amount of received light. The cathode side of the photodiode 211a is connected to the first node N1, while the anode side is connected to the ground node N4 that supplies the ground potential VSS and is biased to the ground potential VSS. Accordingly, the charge generated in the photodiode 211a becomes a negative charge, and this negative charge is stored in the first node N1 as a capacitor capable of storing the charge of the photodiode 211a.

  The transfer gate 211b is a switch element that is interposed between the first node N1 and the second node N2 and is configured by, for example, a transistor, and its on / off state is controlled by the transfer control signal TG. As a result, the transfer gate 211b has a function of controlling the transfer timing of the charge accumulated in the first node N1 from the first node N1 to the second node N2.

  The reset switch 211c is a switch element that is interposed between the second node N2 and the power supply potential node N3 that supplies the power supply potential VDD, and is configured by, for example, a transistor. The reset switch 211c is turned on and off by a reset control signal RF. Be controlled. Accordingly, the reset switch 211c has a function of removing the charge at the second node N2 at a predetermined timing (that is, before transferring the charge from the first node N1 to the second node N2). The power supply potential VDD is set to a positive potential because the charge at the second node N2 is a negative charge.

  As described above, the amplifier 211d amplifies and outputs a signal corresponding to the electric charge of the second node N2, the input node is connected to the second node N2, and the output node is connected to the pixel selection. It is connected to the output node NO via the switch 211e, and is constituted by, for example, a source follower circuit.

  The pixel selection switch 211e is a switch element that is interposed between the amplifier 211d and the output node NO and is configured by, for example, a transistor, and the ON / OFF of the pixel selection switch 211e is controlled by the pixel selection control signal PS. Thus, the pixel selection switch 211e has a function of controlling the output timing of the signal amplified by the amplifier 211d from the amplifier 211d to the output node NO, and hence to the output signal processing unit 22.

  As shown in FIGS. 1 and 2, one accumulated charge discharging circuit 212 is provided for one pixel circuit 211 in this embodiment.

  The accumulated charge discharging circuit 212 is a circuit for discharging the charges accumulated in the first node N1 as necessary, and as shown in FIG. 3, the ground potential is supplied by the ground node N4 that supplies the ground potential VSS. The overflow drain 212b biased to VSS, the charge drain gate 212a interposed between the fifth node N5 as the overflow drain capacitance and the first node N1 of the pixel circuit 211, the fifth node N5 and the power supply potential A clear switch 212c interposed between the power supply potential node N6 for supplying VDD2 and an inverting amplifier 212d whose input node is connected to the fifth node N5 and whose output node is connected to the charge discharge gate 212a. And.

  Here, the power supply potential VDD2 is a switchable power supply potential, and is switched by the imaging mode switching unit 5 to either the power supply potential VDD or a power supply potential (however, a positive potential) as an intermediate potential lower than the power supply potential VDD.

  The charge discharge gate 212a has a function of controlling transfer (that is, discharge) of charges accumulated in the first node N1 from the first node N1 to the fifth node N5. In the present embodiment, the charge discharging gate 212a is configured by a transistor equivalently functioning as a variable resistor. As will be described in detail later, the charge discharging gate 212a is controlled based on the output potential Vg of the inverting amplifier 212d. The height of the potential barrier constituted by 212a changes and the charge discharging operation is changed and controlled.

  The overflow drain 212b accumulates the charge transferred from the first node N1 to the fifth node N5 through the charge discharge gate 212a.

  The clear switch 212c is a switch element constituted by a transistor, and its on / off state is controlled by a clear control signal RO. When the clear switch 212c is on, the power supply potential VDD2 supplied from the power supply potential node N6 causes the charge of the fifth node N5 to disappear, whereas when the clear switch 212c is off, the charge of the fifth node N5 does not disappear. As electric charges (electrons) are discharged from the first node N1 and accumulated in the overflow drain 212b, the potential of the fifth node N5 decreases. Although described in detail later, the imaging mode of the imaging device 2 (each pixel circuit 211) is changed between the NDR mode and the WDR mode by the switching control of the power supply potential of the power supply potential VDD2 and the control of the clear switch 212c. Can be switched.

  The clear control signal RO output from the imaging mode switching unit 5 is commonly input to the clear switch 212c of each accumulated charge discharging circuit 212 connected in the main scanning direction (the vertical direction in FIG. 3) in the imaging element 2. The accumulated charge discharging circuits 212 connected in the main scanning direction simultaneously execute the charge discharging operation by controlling the clear control signal RO to the Hi level. The image sensor 2 has a plurality of such structural units in the sub-scanning direction (left-right direction in the drawing in FIG. 3) (not shown), and the charge discharging operation in each row in the sub-scanning direction is performed in the imaging mode. It is independently controlled by RO_1, RO_2,.

  That is, in the present embodiment, the image pickup mode can be switched between the NDR mode and the WDR mode with each accumulated charge discharging circuit 212 connected in the main scanning direction as one unit, and in the sub scanning direction in one frame described later. In addition, it is possible to mix a row imaged in the NDR mode and a row imaged in the WDR mode (see FIG. 11 and the like).

  The inverting amplifier 212d is configured such that the output potential Vg is low when the potential of the fifth node N5 as the input potential is high, and conversely, the output potential Vg is high when the potential of the fifth node N5 is low. This output potential Vg is applied to the gate of the transistor constituting the charge discharge gate 212a. As a result, the resistance value (potential barrier) of the charge discharging gate 212a changes according to the potential of the fifth node N5.

  Here, the operation of the sensor area 21 will be described. When light enters the sensor area 21, a charge is generated in the photodiode 211a, and the charge is accumulated in the first node N1. The charges at the first node N1 are continuously accumulated while light is incident on the photodiode 211a in a predetermined charge accumulation period set during one frame period. Thus, when the accumulated amount of charge at the first node N1 exceeds the potential barrier as the potential of the charge discharge gate 212a, the charge overflows from the first node N1 to the fifth node N5.

  Here, the power supply potential VDD2 is a power supply potential configured to be switchable. In the present embodiment, two power sources (the potentials are, for example, VDD and VDD × 3/4) are prepared in advance. Based on a signal (not shown) output by the imaging mode switching unit 5, the potential of VDD2 is set to one by switching these systems. Note that this system switching is possible in units of rasters.

  Regarding the switching of the power supply potential VDD2, the system may be set to VDD and VDD × α, and the latter may be generated by a DA converter provided in the imaging mode switching unit 5.

  When the potential of the power supply potential VDD2 is switched to the same potential as the power supply potential VDD and the clear switch 212c is on, the potential of the fifth node N5 is substantially the same as the power supply potential VDD. The electric charge overflowing to the fifth node N5 is attracted to the fifth node N5 and disappears (that is, VDD becomes a clear potential). Accordingly, since the potential of the fifth node N5 remains unchanged at the power supply potential VDD, the output potential Vg of the inverting amplifier 212d remains low, and the potential barrier of the charge discharge gate 212a remains high and does not change. Therefore, when the clear switch 212c is on, the amount of accumulated charge at the first node N1 increases linearly as the amount of light received by the photodiode 211a increases, and the amount of charge accumulated at the photodiode 211a at a predetermined amount of received light is obtained. It reaches capacity and saturates. Therefore, the output signal output from the pixel circuit 211 also increases linearly with the increase in the amount of light received by the photodiode 211a and becomes saturated at a predetermined amount of received light (in this way, reaches saturation). The potential of the output signal is also referred to as “saturation potential”). Although this output characteristic has a drawback that the dynamic range is relatively narrow, there is an advantage that a linear output characteristic can be obtained below a predetermined received light quantity that does not lead to saturation. Thus, in a state where the clear switch 212c is turned on, the imaging mode of the imaging device 2 is the NDR mode as the first imaging mode.

  On the other hand, when the clear switch 212c is turned off, the electric charge discharged from the first node N1 to the fifth node N5 is accumulated in the overflow drain 212b, and according to the discharge amount, the fifth node N5 is stored. The potential of becomes gradually lower. For this reason, the output potential Vg of the inverting amplifier 212d gradually increases, and accordingly, the potential barrier of the charge discharge gate 212a gradually decreases. This facilitates discharge of charges from the first node N1 to the fifth node N5. As the charge discharge is promoted, the potential of the fifth node N5 is further lowered, and the potential barrier height of the charge discharge gate 212a is further lowered. Thus, the discharge of charges from the first node N1 to the fifth node N5 is accelerated at an accelerated rate.

  In this embodiment, in a predetermined period before the clear switch 212c is turned off, the power supply potential VDD2 is switched to a potential lower than the power supply potential VDD (for example, the clear potential is VDD × 3/4), and the clear switch 212c is turned on. As a result, the discharge charges accumulated in the overflow drain 212b are once cleared. Thus, the potential of the fifth node N5 is set to a potential lower than the power supply potential VDD, and the output potential Vg output through the inverting amplifier 212d in the predetermined period becomes at least higher than that in the above-described NDR mode. . Therefore, the height of the potential barrier formed by the charge discharging gate 212a is lower than that in the NDR mode. As a result, the charge overflows from the first node N1 to the fifth node N5 before reaching the saturation potential. That is, in terms of function, the charge discharge gate 212a is controlled to start discharging charges when the amount of received light is smaller than in the NDR mode.

  Furthermore, in the present embodiment, the gate width and gate length of the charge discharging gate 212a are adjusted so as to limit the charge transfer (that is, the gate width is narrowed or the gate length is lengthened). Even when the charge discharge gate 212a is opened (a state in which the output Vg of the inverting amplifier 212d is increased and the charge can be discharged completely, that is, a state where the height of the potential barrier is minimum), the charge discharge gate 212a is predetermined. The charge discharge amount (or charge discharge speed) is suppressed to such an extent that the charge accumulated in the first node N1 is not discharged to the overflow drain 212b during the charge discharge period. That is, in the imaging device 1 of the present embodiment, even if the charge discharging operation is performed, not all of the charges accumulated in the first node N1 so far are discharged, and only a part thereof is discharged.

  As a method for limiting the charge transfer, in addition to adjusting the physical shape of the gate portion described above, the impurity concentration of the transistor constituting the charge discharge gate 212a may be adjusted low.

As described above, by introducing the control that once clears the overflow drain 212b at the intermediate potential and the configuration that suppresses the discharge of the electric charge, the imaging device of the present embodiment
i) When the amount of received light is small and the charge generated by the photodiode 211a does not overflow beyond the potential barrier of the charge discharge gate 212a (as described above, the barrier height is lower than that of the NDR mode), the output characteristics Indicates a linear characteristic,
ii) When the amount of received light increases and the charge generated by the photodiode 211a overflows beyond the potential barrier of the charge discharge gate 212a, the charge discharge amount depends on the amount of charge newly generated by the photodiode 211a. Is dynamically adjusted (and autonomously)
This brings about a new action.

  Accordingly, by turning off the clear switch 212c during at least a part of the charge accumulation period, the amount of charge accumulated at the first node N1 increases linearly with an increase in the amount of light received by the photodiode 211a. Without saturation, saturation is suppressed even if the amount of received light increases. At this time, the output signal output from the pixel circuit 211 exhibits a characteristic similar to the logarithmic characteristic that gradually approaches the upper limit value corresponding to the saturation as the amount of light received by the photodiode 211a increases. . As described above, in the image pickup device of the present embodiment, the output when the amount of received light is small exhibits a linear characteristic, but when the amount of received light increases, the output exhibits a logarithmic characteristic. It has a hybrid output characteristic in which the generated charges are connected at a light amount that starts to overflow beyond the potential barrier of the charge discharging gate 212a. As a result, a wide dynamic range can be secured in the same way as an image pickup device having a normal logarithmic characteristic, and the degree of compression in a high-luminance region can be reduced by the presence of a linear characteristic portion as compared to an image pickup device having a logarithmic characteristic. As a result, the gradation becomes smaller and the gradation in the high luminance region is ensured.

  Further, as described above, if the potential for clearing the overflow drain 212b can be adjusted (VDD × α), the height of the potential barrier of the charge discharging gate 212a can be arbitrarily set, so the linear characteristic is switched to the logarithmic characteristic. The degree of freedom of point setting increases, and it becomes possible to cope with all imaging conditions.

  Thus, by providing a period during which the clear switch 212c is turned off, the imaging mode of the imaging device 2 becomes the WDR mode as the second imaging mode.

  However, although this output characteristic has the advantage of a relatively wide dynamic range, it does not change in that the output is compressed below a predetermined received light quantity that does not lead to saturation. The problem of incurring a decrease in the temperature remains.

  After a predetermined charge accumulation period elapses, the transfer control signal TG turns on the transfer gate 211b to transfer the accumulated charge at the first node N1 to the second node N2. The signal corresponding to the charge transferred to the second node N2 is amplified by the amplifier 211d at a predetermined timing and output as an output signal from the output node NO.

  Next, components other than the sensor area 21 of the image sensor 2 will be described.

  The output signal processing unit 22 includes a CDS (not shown) and an AD converter.

  640 CDSs and AD converters are provided in the main scanning direction corresponding to each pixel circuit 211.

  The CDS removes noise included in the output signal output from the output node NO by so-called correlated double samples.

  The AD converter converts the output signal output from the output node NO from an analog signal to a digital signal.

  The peripheral circuit 23 outputs a timing signal for synchronization when the pixel circuit 211 captures an image. This timing signal also includes the transfer control signal TG, the reset control signal RF, and the pixel selection control signal PS.

  Here, the operation of the image sensor 2 will be described. First, a timing signal is output from the peripheral circuit 23 for each pixel row (hereinafter also referred to as a raster) extending in the main scanning direction (the vertical direction in FIG. 3). The timing signal output from the peripheral circuit 23 is output to each pixel circuit 211 and each accumulated charge discharging circuit 212 through a control signal line (not shown). Next, imaging is performed in each pixel circuit 211 based on the timing signal. At this time, an output signal is output from each pixel circuit 211 as described above. The image is scanned in 480 columns while being shifted in the sub-scanning direction in units of rasters composed of a plurality of pixel circuits 211 and the like that are continuous in the main scanning direction, whereby an output signal of 640 × 480 pixels for one frame can be obtained. . This output signal is output to the output signal processing unit 22 through the readout signal line, and the output signal processing unit 22 performs noise removal and digitization. Then, this digital output signal (hereinafter also referred to as imaging data) is transferred to the buffer memory 3 through the imaging data path. The output signal processing unit 22 is shared by all the rasters (480), and 640 output signal processing units 22 are mounted on the image sensor 2 of the present embodiment.

  In this embodiment, each pixel circuit 211 is provided with one accumulated charge discharging circuit 212. However, as shown in FIG. 14, one pixel is formed by four pixel circuits 211 adjacent in the main scanning direction and the sub-scanning direction. A group may be configured, and one accumulated charge discharging circuit 212 may be shared by the pixel group. Here, the charge accumulation period for accumulating charges in the first node N1 is determined by the timing at which the so-called electronic shutter is released. If a so-called rolling shutter operation is used as a method for releasing this electronic shutter, the shutter is released in units of rasters. Therefore, the timing of discharging charges is superimposed on the same accumulated charge discharging circuit 212 between the raster pixel circuits 211 adjacent in the sub-scanning direction, which may adversely affect the charge discharging operation from the pixel circuit 211. . In order to avoid this, it is preferable to perform a rolling shutter operation in which the shutter is released in units of two rasters that the accumulated charge discharging circuit 212 straddles. Further, since the charges discharged from the four pixel circuits 211 are accumulated in one overflow drain 212b, there is a possibility that the capacity that can accumulate the charges in the overflow drain 212b may be exceeded. In this case, the charge overflows from the overflow drain 212b to the surrounding photodiode 211a, and the image quality of the image output from the imaging device 1 deteriorates. In this regard, as shown in FIG. 2, the configuration in which one accumulated charge discharging circuit 212 is provided for each pixel circuit 211 can secure a sufficient overflow drain capacity for one pixel circuit 211. Is advantageous. In addition, as shown in FIG. 2, providing the one accumulated charge discharging circuit 212 for each pixel circuit 211 without sharing the accumulated charge discharging circuit 212 with the plurality of pixel circuits 211 causes the accumulated charge discharging circuit 212 to discharge. Since the generated charge does not affect the surrounding pixel circuit 211, it is advantageous in that a rolling shutter operation can be performed in units of one raster.

  Further, as shown in FIG. 15, the accumulated charge discharging circuits 212 may be arranged in a staggered manner with respect to the pixel circuit 211, and substantially two accumulated charge discharging circuits 212 may be assigned by the four pixel circuits 211. By doing so, it is possible to increase the overflow drain capacity for one pixel circuit 211 compared to the configuration of FIG. However, when the rolling shutter operation is used, since the shutter is released in units of rasters, the effect of shifting the timing at which charges are discharged from the pixel circuit 211 of the adjacent raster to the same accumulated charge discharging circuit 212 is affected in the sub-scanning direction. There is a risk of spreading not only to adjacent pixel circuits 211 but also to all pixel circuits 211. For this reason, it is preferable to use a so-called global shutter operation in which the electronic shutter operation is performed at the same timing for all the pixel circuits 211 mounted on the image sensor 2.

  Further, as shown in FIG. 16, substantially four accumulated charge discharging circuits 212 may be assigned by four pixel circuits 211. In this way, the overflow drain capacity for one pixel circuit 211 can be further increased. However, the global shutter operation is preferably used because the influence of the timing at which charges are discharged from the pixel circuit 211 of the adjacent raster to the same accumulated charge discharging circuit 212 is spread to all the pixel circuits 211 as described above.

  14 to 16, when one accumulated charge discharging circuit 212 is shared by a plurality of pixel circuits 211, each pixel circuit 211 and the accumulated charge discharging circuit 212 are connected as shown in FIG. A charge discharging gate 212a may be provided between them.

  14 to 16, it is preferable that the charge discharging capability of the charge discharging gate 212a is suppressed as described above.

  Next, components other than the imaging device 2 of the imaging device 1 will be described. As described above, the buffer memory 3 is a storage device that temporarily stores the imaging data output from the imaging device 2, and is a dual port memory having a storage capacity of two frames in this embodiment.

  The image processing unit 4 has a function of converting imaging data transferred from the buffer memory 3 into image data by performing image processing. In the image processing performed by the image processing unit 4, for example, as illustrated in FIG. 5, before the light enters the image sensor 2, green filters are arranged in a staggered manner so as to correspond to the pixel circuit 211. The image data having red, green, and blue color information obtained by passing through a so-called Bayer color filter in which red and blue filters are alternately arranged between the green filters is obtained. There is a so-called interpolation process in which the actual color of target imaging data is estimated from the color of surrounding imaging data. The image processing unit 4 has a storage device for storing imaging data (so-called RAW data) in order to perform image processing as described above. Note that this storage device may be used as the buffer memory 3.

  The imaging mode switching unit 5 sends a clear control signal RO for turning on and off the clear switch 212c to the imaging device 2 based on a control signal from the control unit 6, thereby changing the imaging mode of each imaging device 2 to NDR. It has a function to set the mode or WDR mode. However, as described above, switching of the imaging mode is performed in units of columns with each charge discharging circuit 212 connected in the main scanning direction as one unit.

  Although not shown, the control unit 6 includes a CPU and a calculation unit configured by so-called wired logic. The control unit 6 controls the output signal processing unit 22 and the peripheral circuit 23 to control the imaging device 2. Control the whole operation. Further, as a characteristic point in the imaging apparatus 1 according to the present embodiment, the control unit 6 determines an appropriate imaging mode for each pixel circuit 211 and sets the imaging mode determined for the imaging mode switching unit 5. Has the function of commanding. In the illustrated example, the function block of the imaging mode switching unit 5 is provided separately from the control unit 6, but the function of the imaging mode switching unit 5 may be included in the control unit 6.

  Next, imaging mode determination processing executed by the control unit 6 will be described with reference to the flowchart shown in FIG.

  As described above, the imaging data output from the imaging device 2 constitutes a frame of 640 (pixels) × 480 (raster). Thus, the imaging device 1 provides a moving image by outputting 30 frames per second. The control unit 6 determines an appropriate imaging mode based on the imaging mode determination frame. Therefore, as shown in FIG. 6, the control unit 6 instructs to insert an imaging mode determination frame every time a predetermined number of frames are output. In the present embodiment, while the imaging apparatus 1 is set to 30 frames / second, the control unit 6 outputs 14 frames in order to determine an appropriate imaging mode every 0.5 seconds. It is instructed to insert an imaging mode determination frame every time. The imaging mode determination frame is configured by imaging data when each pixel circuit 211 performs imaging in the first imaging mode, and the control unit 6 basically includes imaging data in a saturated state within the imaging mode determination frame. An appropriate imaging mode is determined on the basis of whether or not is included.

  First, when each variable in the flowchart shown in FIG. 7 is defined, FC is a frame counter for counting the number of captured frames, INSERT is an insertion flag indicating whether or not an imaging mode determination frame is inserted, and INSERT_R is an imaging mode determination frame. This is a control constant for controlling the timing, and is set to 15 here. PM is a parameter indicating the imaging mode, and TEMP is a save variable for saving the parameter PM.

  In the flowchart, first, in step S1, variables used in the following steps are initialized. Specifically, 0 is input to the frame counter FC, 0 is input to the insertion flag INSERT, and the NDR mode is input to the parameter PM.

  In the subsequent step S2, it is determined whether or not an instruction for imaging has been given by the user. If the user has instructed imaging, the process proceeds to step S3. On the other hand, if the user has not instructed imaging, the process ends. When the imaging apparatus 1 is equipped with an operating system such as a runtime monitor, a loop that returns to step S1 or step S2 after a predetermined break period may be configured.

  In step S3, it is determined whether or not an imaging mode determination frame is to be inserted. Specifically, when the insertion flag INSERT is 0, the process proceeds to step S4 to insert an imaging mode determination frame, whereas when the insertion flag INSERT is not 0, the process proceeds to step S12 and the imaging mode determination frame is inserted. Do not insert.

  First, when the description is continued assuming that an imaging mode determination frame is inserted (that is, when the insertion flag INSERT is 0), in step S4, the parameter PM is saved in the save variable TEMP.

  In subsequent step S5, the imaging mode switching unit 5 is instructed to set the imaging mode to the NDR mode, and imaging is performed. For this reason, when the image is captured in the NDR mode until now, the image is captured as it is in the NDR mode, whereas when the image is captured in the WDR mode so far, the mode is switched to the NDR mode. . An NDR mode frame obtained as a result is an imaging mode determination frame.

  In subsequent step S6, the imaging mode determination frame obtained in step S5 is stored in the buffer memory 3.

  In subsequent step S7, it is determined whether or not the imaging mode determination frame stored in the buffer memory 3 includes imaging data in a saturated state (hereinafter also referred to as saturation determination). At this time, the fact that the imaging data is in a saturated state may be, for example, the upper limit value of 255 if the imaging data is output in 8 bits. Further, when a predetermined value (for example, 250) is exceeded, it may be determined that the imaging data is in a saturated state. By doing so, it is possible to avoid the influence of an error that may occur in the output signal in the amplifier 211d and the AD converter of the output signal processing unit 22. Note that the imaging data is in a saturated state, in other words, when the imaging data is output as an image, whiteout occurs.

  The saturation determination may be performed on all the imaging data included in the imaging mode determination frame, but may be performed on only a part of the imaging data. For example, determination may be made only for imaging data having green color information among imaging data having red, green, and blue color information obtained by the Bayer array color filter described above. Green color information has the highest correlation with luminance in human visual characteristics, and is arranged in a staggered manner within one frame, so the number of image data to be determined is halved, and the processing time and processing resources required for determination are reduced. This is advantageous in terms. Further, when calculating the strictness, the luminance component (Y component) may be calculated (YC separation) from the red, green, and blue color information of the imaging data, and the saturation determination may be performed based on the luminance component. Further, the saturation determination may be performed by extracting imaging data having the largest value from the imaging mode determination frame and determining whether or not the imaging data exceeds a predetermined value. Furthermore, when there is a predetermined number or more of saturated imaging data, it may be determined that the imaging mode determination frame includes saturated imaging data.

  If it is determined that the imaging mode determination frame includes saturated imaging data, the process proceeds to step S8, while the imaging mode determination frame does not include saturation imaging data. If so, the process proceeds to step S9.

  In Step S8, since the imaging mode determination frame includes the imaging data in the saturated state, more specifically, the imaging data that has a large amount of light received by the imaging device 2 and is saturated when imaging in the NDR mode is output. Therefore, the WDR mode is input to the parameter PM. Then, the process proceeds to step S10.

  In step S9, since the imaging mode determination frame does not include saturated imaging data, more specifically, the amount of light received by the imaging device 2 is small, and saturated imaging data is output even when imaging is performed in the NDR mode. Therefore, the NDR mode is input to the parameter PM. Then, the process proceeds to step S14.

  In step S10, it is determined whether or not the parameter PM saved in the save variable TEMP in step S4 is in the WDR mode. Thus, when the save variable TEMP is in the WDR mode, the process proceeds to step S11. On the other hand, when the save variable TEMP is not in the WDR mode, the process proceeds to step S14.

  In step S 11, the frame stored immediately before the imaging mode determination frame stored in step S 6 is extracted from the buffer memory 3 and output to the image processing unit 4. Then, the process proceeds to step S15. On the other hand, in step S14, the most recently stored frame in the buffer memory 3 is output to the image processing unit 4. That is, the imaging mode determination frame stored in the buffer memory 3 in step S6 is output to the image processing unit 4. Then, the process proceeds to step S15.

  Here, the frame stored in the buffer memory 3 immediately before the imaging mode determination frame in step S11 is a frame captured in the WDR mode indicated by the save variable TEMP. That is, the frame output to the image processing unit 4 before the imaging mode determination frame is inserted is output twice to the image processing unit 4. That is, as shown in FIG. 8A, the imaging mode of the frame output to the image processing unit 4 before inserting the imaging mode determination frame is the WDR mode, and the image processing unit after inserting the imaging mode determination frame When the imaging mode of the frame output to 4 is also in the WDR mode, if the frame is output to the image processing unit 4 in this state, the NDR mode including the imaging data in the saturated state between the frames of the WDR mode that are continuously output Are output to the image processing unit 4. If a state in which the imaging mode is switched for a moment is output as a moving image, a person watching the moving image will see a white-out image for a moment and feel uncomfortable. In order to prevent this, instead of outputting the NDR mode imaging mode determination frame to the image processing unit 4, the WDR mode frame output to the image processing unit 4 before inserting the imaging mode determination frame is continued twice. Output.

  On the other hand, the imaging mode of the frame output to the image processing unit 4 before inserting the imaging mode determination frame shown in FIG. 8B is the NDR mode, and the image processing unit after inserting the imaging mode determination frame When the imaging mode of the frame to be output to 4 is also the NDR mode, or the imaging mode of the frame output to the image processing unit 4 before inserting the imaging mode determination frame shown in FIG. When the imaging mode of the frame output to the image processing unit 4 after the imaging mode determination frame is inserted is the WDR mode, or before the imaging mode determination frame is inserted as shown in FIG. The imaging mode of the selected frame is the WDR mode, and the imaging mode of the frame output to the image processing unit 4 after inserting the imaging mode determination frame is the NDR mode. The, since not occur the state in which imaging mode described above is switched a moment, in step S14, and outputs the image pickup mode determining frame as it is to the image processing unit 4.

  In the present embodiment, as described above, a frame imaged in the NDR mode is used as the imaging mode determination frame. However, a frame imaged in the WDR mode can be used instead. This is because it is possible to specify the value of imaging data in the WDR mode when saturation occurs in the NDR mode by imaging in the WDR mode when the output is just saturated when imaging in the NDR mode. .

  Now, when a frame imaged in the WDR mode is used as the imaging mode determination frame, the problem is that the imaging mode determination frame in the WDR mode is inserted in the state where the image is captured in the NDR mode, and the subsequent imaging is performed here. This is a case where it is determined that an image is captured in the NDR mode as a mode. In this case, it is only necessary to perform control so that the frame imaged in the NDR mode is output twice in step S11 described above. To generalize this, if frames that are continuous before and after the imaging mode determination frame are captured in an imaging mode that is different from the imaging mode used in the imaging mode determination frame, the imaging mode determination frame is discarded and the imaging mode determination is performed. That is, control is performed so that the frame immediately before the frame is output again.

  Furthermore, as described above, since the frame captured in the WDR mode as well as the NDR mode can be adopted as the imaging mode determination frame, the current (that is, the frame captured immediately before the imaging mode determination frame is inserted). When the imaging mode is the WDR mode, the imaging mode determination frame is set to the WDR mode. When the current imaging mode is the NDR mode, the imaging mode determination frame is inserted as the NDR mode. It is possible to eliminate the problem that the imaging mode is switched between the determination frame and before and after the determination frame. When this control is employed, it is possible to avoid outputting the same frame twice as described above.

  Note that the imaging mode determination frame is used only for determination of the imaging mode, and is not output to the image processing unit 4. The number of frames per unit time output from the pixel circuit 2 is determined per unit time output from the imaging device 1. It may be more than the number of frames. For example, by configuring the pixel circuit 2 to output 32 frames per second, the imaging mode determination frame (two imaging mode determination frames are output per second) is output to the image processing unit 4. Even without this, the imaging apparatus 1 can output 30 frames per second. In this case, output of the same frame twice as in step S11 in the flowchart described above is avoided.

  In step S15 which has shifted from step S11 and step S14, 1 is added to the frame counter FC.

  In subsequent step S16, the frame counter FC is divided by the control constant INSERT_R, and the remainder is input to the insertion flag INSERT. Then, the process returns to step S2. In this embodiment, INSERT_R is set to 15, and an imaging mode determination frame can be inserted every time 14 frames are output. Note that the control constant INSERT_R may be changed as appropriate. That is, a mode setting by user designation is separately provided in the image pickup apparatus 1. For example, for a subject moving at high speed, the control constant INSERT_R is decreased, and control for increasing the switching opportunity between the NDR mode and the WDR mode is performed. It is preferable to do.

  Next, when the insertion flag INSERT is not 0 and the imaging mode determination frame is not inserted, the process proceeds from step S3 to step S12.

  In step S12, the imaging mode switching unit 5 is instructed to set the imaging mode to the value of the parameter PM, and imaging is performed. Therefore, the image is captured in the NDR mode as it is while the image is captured in the NDR mode, and the image is captured in the WDR mode as it is while the image is captured in the WDR mode.

  In the subsequent step S13, the frame obtained in step S12 is stored in the buffer memory 3. Then, the process proceeds to step S14.

  In step S14, as described above, the most recently stored frame in the buffer memory 3 is output to the image processing unit 4. That is, the frame stored in the buffer memory 3 in step S12 is output to the image processing unit 4. Then, it returns to step S2 through step S15 and step S16.

  In this manner, during imaging by the imaging apparatus 1, an appropriate imaging mode is the NDR mode or the WDR mode based on the imaging mode determination frame configured by imaging data captured in the NDR mode. Is determined, and the imaging mode of the imaging device 2 is set to an appropriate imaging mode based on the determination result. In other words, when imaging in the NDR mode with a relatively narrow dynamic range is possible, imaging is performed in the NDR mode. Imaging is performed. In this way, the respective disadvantages can be compensated while utilizing the advantages of the NDR mode and the WDR mode.

  In the present embodiment, the imaging mode of a frame to be output after that is determined based only on whether or not the imaging mode determination frame includes saturated imaging data. As shown in FIG. 9, when it is determined in step S7 that the imaging mode determination frame includes saturated imaging data, the average value of all the imaging data included in the imaging mode determination frame is a predetermined value. You may make it transfer to step S7A which determines whether it is larger than this.

  In step S7A, when the average value of all the imaging data included in the imaging mode determination frame is larger than a predetermined value, the process proceeds to step S8, and the subsequent imaging mode is set to the WDR mode, while the imaging mode determination frame is set. When the average value of all the imaging data included in is less than or equal to a predetermined value, the process proceeds to step S9, and the subsequent imaging mode is set to the NDR mode. As a result, the imaging mode is set to the WDR mode only when the imaging mode determination frame includes saturated imaging data and the average value of all the imaging data included in the imaging mode determination frame is greater than a predetermined value. In other cases, even when saturated imaging data is included in the imaging mode determination frame, the imaging mode is set to the NDR mode. As a result, when the imaging data in a saturated state (that is, high brightness) is spread over a relatively wide range (when the average value is larger than the predetermined value) in the imaging mode determination frame, the overexposed area is wide. Therefore, when the imaging mode is set to the WDR mode to avoid over-exposure of the image, and when the imaging data in the saturated state is only local (when the average value is equal to or less than a predetermined value), the over-exposed region is localized. Therefore, the imaging mode is set to the NDR mode. By doing so, an image closer to human vision can be obtained.

  Note that since humans generally tend to gaze at the center of the image, in step S7A, not the average value of all the imaging data included in the imaging mode determination frame, but the imaging data included in the center of the imaging mode determination frame. It may be determined whether the average value of is greater than a predetermined value. In this way, it is possible to determine the imaging mode that matches the human sense.

  Moreover, as long as it is the control of the same content as the control shown in FIG. 9, you may change suitably, for example, you may replace the order of step S7 and step S7A.

  Furthermore, as shown in FIG. 10, when it is determined in step S7 that the imaging mode determination frame includes saturated imaging data, the frequency of occurrence of imaging data below a predetermined value is greater than a predetermined reference value. Alternatively, the process may proceed to step S7B for determining whether the value is smaller.

  In step S7B, when the occurrence frequency of imaging data below a predetermined value in the imaging mode determination frame is smaller than a predetermined reference value, the process proceeds to step S8, and the subsequent imaging mode is set to the WDR mode. If the occurrence frequency of the imaging data is greater than or equal to a predetermined reference value, the process proceeds to step S9, and the subsequent imaging mode is set to the NDR mode. Thus, the imaging mode is changed only when the imaging mode determination frame includes saturated imaging data and the frequency of occurrence of imaging data below a predetermined value in the imaging mode determination frame is smaller than a predetermined reference value. In the WDR mode, otherwise, the imaging mode is set to the NDR mode even when the imaging mode determination frame includes saturated imaging data. That is, as described above, when the imaging data in the saturation state in the imaging mode determination frame is spread over a relatively wide range (when the occurrence frequency is smaller than the reference value), the whiteout region becomes wide. When the imaging mode is set to the WDR mode to avoid overexposure of the image, and when the imaging data in the imaging mode determination frame is only in a saturated state (when the occurrence frequency is equal to or higher than the reference value), the overexposed region is Since it is determined to be local and the entire image is dark, the imaging mode is set to the NDR mode.

  In step S7B as well, only the imaging data included in the central portion of the imaging mode determination frame may be subject to frequency measurement in the same manner as described above.

  Further, the control may be appropriately changed as long as the control has the same content as the control shown in FIG. Then, when the occurrence frequency of imaging data greater than or equal to a predetermined value is greater than the predetermined value, the imaging mode may be changed to the WDR mode, or the order of step S7 and step S7B may be switched.

  Further, in the present embodiment, the imaging mode of a frame subsequent to the imaging mode determination frame is determined based on the saturation determination result, and the imaging mode is changed in real time. Frames that are output in the imaging mode determined by the saturation determination only when the result determined in is held for a predetermined time (for example, 0.5 seconds), and then the saturation determination is performed again and both results match. May be imaged. In this way, the amount of incident light normally changes gradually, but the result of the saturation determination is that the result of the saturation determination before and after the saturation of the imaging data due to the noise included in the imaging data, for example. Even if they are changed every time, it is possible to prevent the actual imaging mode from being changed frequently.

  Furthermore, in the present embodiment, the entire pixels of the imaging mode determination frame are configured by imaging data in the NDR mode. However, the imaging device 2 includes a pixel circuit 211 and an accumulated charge discharging circuit 212 as shown in FIG. Is compatible with 1: 1, and an NDR mode and a WDR mode can be individually set for each pixel circuit 211, for example, as shown in FIG. The mode determination frame may be configured such that rasters configured with imaging data in the NDR mode and rasters configured in the WDR mode are alternated. In this way, when performing saturation determination, saturation determination can be performed using half of the imaging data in the imaging mode determination frame as a determination target. In addition, it is also possible to further extract image data having green color information, and perform saturation determination using the extracted image data as a determination target. As a result, processing time can be shortened and processing resources can be reduced. This is advantageous in reducing the cost of the imaging apparatus. Note that the sampling for reducing the determination target of saturation determination is effective even in an image sensor equivalent to a VGA of 640 × 480 pixels as in the present embodiment, and for example, 1280 × 1024 pixels (so-called SXGA) or 1600 × 1200 pixels. This is particularly effective when a high-resolution image sensor such as (so-called UXGA) is used.

  As shown in FIG. 14, when one accumulated charge discharging circuit 212 is shared between two pixel circuits 211 adjacent in the sub-scanning direction, there is a problem due to the shutter being released in units of one raster as described above. Since it is assumed, as shown in FIG. 11B, the imaging mode determination frame may be configured so that the imaging data in the NDR mode and the imaging data in the WDR mode are alternated in units of two rasters. At this time, one pixel group shown in FIG. 14 needs to be completely included in the raster imaged in the WDR mode or the NDR mode, and one pixel group extends over both the WDR mode and the NDR mode. Should not be set.

  In the above-described embodiment, the imaging mode determination frame is periodically inserted and the optimal imaging mode is periodically determined. However, as a modification, each frame output in time series from the imaging device 2 is determined. The NDR mode imaging data (raster) and the WDR mode imaging data (raster) are configured, and the raster is composed of two temporally continuous frames, thereby equivalently configuring the imaging mode determination frame. Depending on the determination result based on the imaging mode determination frame, rasters may be appropriately combined between two temporally continuous frames and output from the imaging apparatus 1.

  That is, in this modification, based on the imaging mode switched by the imaging mode switching unit 5, the imaging device 2 has a raster (first imaging mode raster) based on an output signal imaged in the NDR mode constituting the frame, A raster (second imaging mode raster) based on an output signal imaged in the WDR mode is output. Then, the control unit 6 should be selected based on an output signal (imaging data stored in the buffer memory 3) included in one of the first imaging mode raster and the second imaging mode raster. The imaging mode is determined, and based on the determination result, the first imaging mode raster or the second imaging mode raster is selected from the frame and the frame captured immediately before this frame, and is combined. is there.

  Specifically, when the frame counter FC is an even number, as shown in FIG. 12A, the odd-numbered row (number of rows is counted from 1) in the sub-scanning direction (vertical direction in the drawing). ) Raster (hereinafter also referred to as odd-numbered raster) is constituted by imaging data in NDR mode, while a raster in even-numbered rows (hereinafter also referred to as even-numbered raster) is constituted by imaging data in WDR mode. That is, in one frame, rasters captured in the NDR mode and rasters captured in the WDR mode are alternately arranged so as to be adjacent to each other.

  On the other hand, when the frame counter FC is an odd number, as shown in FIG. 12 (b), contrary to FIG. 12 (a), the odd raster is composed of imaging data in the WDR mode, while the even raster is in the NDR mode. It is comprised by the imaging data of. That is, the positions of rasters captured in the NDR mode and rasters captured in the WDR mode are exchanged between adjacent frames.

  The raster imaged in the NDR mode and the raster imaged in the WDR mode may be reversed. Then, as shown in FIG. 13, when the frame counter FC is 1, the control unit 6 stores the frame stored in the buffer memory 3 when the frame counter FC is 0 (hereinafter also referred to as the 0th frame). The odd-numbered raster imaging data (that is, the NDR mode imaging data) is extracted from the frame, and the even-numbered raster imaging data from the frame (hereinafter also referred to as the first frame) stored in the buffer memory 3 when the frame counter FC is 1. (Ie, NDR mode imaging data) is extracted. In this way, an imaging mode determination frame constituted by only the imaging data of the NDR mode is equivalently created by the 0th and first frames. Then, saturation determination is performed by extracting imaging data having further green color information from the equivalent imaging mode determination frame. For the saturation determination, the above-described various determination methods can be employed.

  Here, when the imaging data determination frame includes saturated imaging data, the control unit 6 sets the even-numbered raster imaging data (that is, WDR mode imaging data) of the 0th frame and the first frame. The odd-numbered raster image data (that is, WDR mode image data) is combined and output to the image processing unit 4 (DMA transfer).

  On the other hand, when the imaging data determination frame does not include saturated imaging data, the control unit 6 uses the odd-numbered raster imaging data (that is, NDR mode imaging data) of the 0th frame and the first frame. Are combined with the even raster image data (that is, the NDR mode image data) and output to the image processing unit 4 (DMA transfer).

  In parallel with the synthesis of the 0th frame and the 1st frame and output to the image processing unit 4, the imaging device 2 uses the frame memory FC 2 for the frame counter FC (hereinafter also referred to as the second frame) to the buffer memory 3. Output to.

  Then, the saturation determination is performed in the same manner as described above based on the even-numbered raster image data of the first frame and the odd-numbered raster image data of the second frame. At this time, since the control unit 6 has already performed the saturation determination for the first frame, it is only necessary to perform the saturation determination for the odd-numbered raster imaging data (that is, the NDR mode imaging data) in the second frame. At this time, if the result of the saturation determination of the first frame already performed is different from the result of the saturation determination of the second frame newly performed, the determination result based on the second frame having a later chronological order is obtained. You may make it give priority.

  In this embodiment, a raster imaged in the NDR mode is extracted from two consecutive frames, and these are combined to form an equivalent imaging mode determination frame. However, the raster imaged in the NDR mode is used instead. A raster imaged in the WDR mode may be used.

  In addition, since the phase between images is generally very high between adjacent rasters, an image of 640 × 480 pixels is not constructed as an imaging mode determination frame, but an image is captured in the NDR mode obtained by one imaging. Saturation determination or the like may be performed based on only the raster (or the raster imaged in the WDR mode).

  Thereafter, by repeating the same control as described above, the synthesized frames are continuously output to the image processing unit 4. Even in this way, since the frame based on the imaging data in the NDR mode and the frame based on the imaging data in the WDR mode are output while being switched, the contrast is unnecessarily lowered while avoiding overexposure or the like. It is possible to obtain a moving image that avoids a decrease in gradation. Further, by performing this control, the same effect as that of so-called progressive scan can be obtained. Therefore, an incidental effect that a moving image with less image blur can be obtained even for a subject moving at high speed can be obtained.

  Note that the technology disclosed herein is not limited to an image sensor that can perform a so-called charge discharging operation as shown in FIG. 3 and the like, and its output characteristics are different between a plurality of different output characteristics. It can be widely applied to an image sensor that can be switched with the.

  As described above, the imaging apparatus according to the present invention is an imaging apparatus that can operate by switching output characteristics so that the dynamic range is changed. However, since the drawbacks of the respective operations can be compensated, it is useful in that an appropriate and high-quality image can always be obtained when used in various devices including a television door phone and a network camera.

1 is a schematic configuration diagram of an imaging apparatus according to the present invention. It is a figure explaining an example of arrangement | positioning relationship between a pixel circuit and a stored charge discharge circuit. It is a circuit diagram of an imaging device concerning the present invention. It is a circuit diagram concerning the modification of an imaging device. It is a block diagram of a color filter of a Bayer arrangement. It is a figure explaining the timing which inserts an imaging mode determination frame. It is a flowchart for determining the imaging mode by a control part. It is a figure explaining the method to determine the flame | frame output to an image process part. It is another flowchart of the saturation determination by a control part. It is another flowchart of the saturation determination by a control part. It is a figure which shows the other structure of an imaging mode determination frame. It is a figure which shows the other structure of a flame | frame. It is explanatory drawing about the method of synthesize | combining the flame | frame output to an image process part using the flame | frame shown in FIG. It is a figure explaining an example of arrangement | positioning relationship between a pixel circuit and a stored charge discharge circuit. It is a figure explaining an example of arrangement | positioning relationship between a pixel circuit and a stored charge discharge circuit. It is a figure explaining an example of arrangement | positioning relationship between a pixel circuit and a stored charge discharge circuit.

DESCRIPTION OF SYMBOLS 1 Imaging device 2 Imaging element 21 Sensor area 211 Pixel circuit 211a Photodiode (light receiving element)
212 accumulated charge discharging circuit 212a charge discharging gate 212b overflow drain 212c clear switch 22 output signal processing unit 3 buffer memory 4 image processing unit 5 imaging mode switching unit 6 control unit

Claims (14)

A charge is generated by receiving each light, and a plurality of light receiving elements arranged in a lattice form so as to form a plurality of rasters, and a charge generated by the light receiving elements are provided corresponding to each of the light receiving elements. And a plurality of pixel circuits that output an output signal corresponding to the amount of stored charge, and that sequentially output frames configured based on the output signal in time series When,
With respect to the imaging device, the imaging mode of each pixel circuit is set such that when the received light amount of the light receiving element is smaller than a predetermined light amount, the output signal increases linearly as the received light amount increases. A first imaging mode in which the output signal is saturated when the amount of received light of the element is equal to or greater than the predetermined amount of light, and a second imaging in which the output signal is not saturated when the amount of received light of the light receiving element is greater than or equal to the predetermined amount of light. An imaging mode switching unit that switches between the modes,
Control for designating the imaging mode of each pixel circuit to the first or second imaging mode with respect to the imaging mode switching unit based on the output signal when the pixel circuit has imaged in the first imaging mode. And an imaging device. The imaging device according to claim 1,
Each time the imaging device outputs a predetermined number of frames, each pixel circuit outputs an imaging mode determination frame based on the output signal imaged in the first imaging mode,
The control unit performs a predetermined determination using the output signal included in the imaging mode determination frame as an imaging mode determination target, and designates an imaging mode of each of the pixel circuits thereafter based on the determination result. An imaging device that is characterized. The imaging device according to claim 1,
The imaging device outputs a first imaging mode raster configured based on the output signal captured by the pixel circuit in the first imaging mode every time a predetermined number of frames are output, and the pixel circuit An imaging mode determination frame configured to include a second imaging mode raster configured based on the output signal captured in the 2 imaging mode;
The control unit performs a predetermined determination using the output signal constituting the first imaging mode raster in the imaging mode determination frame as a determination target, and based on the determination result, the imaging mode of each pixel circuit thereafter An imaging apparatus characterized by designating an image. In the imaging device according to claim 2 or 3,
An output signal processing unit for outputting the frame including imaging data obtained by converting the output signal of each pixel circuit into a digital signal;
The control unit determines whether the maximum value of the imaging data corresponding to the determination target of the imaging mode determination frame output from the output signal processing unit exceeds a predetermined threshold, and the maximum value is When the predetermined threshold value is exceeded, the imaging mode switching unit designates the imaging mode of each pixel circuit as the second imaging mode. The imaging apparatus according to claim 4,
The control unit calculates an average value of the imaging data corresponding to the determination target of the imaging mode determination frame, further determines whether the average value is larger than a predetermined value, and the average value is The imaging mode of each pixel circuit is designated as the first imaging mode for the imaging mode switching unit even when the maximum value exceeds the predetermined threshold when the value is equal to or less than a predetermined value. An imaging device. The imaging apparatus according to claim 4,
The control unit counts the occurrence frequency of the imaging data below a predetermined determination value for the imaging data corresponding to the determination target of the imaging mode determination frame, and whether the occurrence frequency is smaller than a predetermined reference value. In addition, when the occurrence frequency is equal to or higher than a predetermined reference value, each pixel is detected with respect to the imaging mode switching unit even when the maximum value of the imaging data exceeds the predetermined threshold. An imaging apparatus, wherein an imaging mode of a circuit is designated as the first imaging mode. In the imaging device according to claim 2 or 3,
An output signal processing unit for outputting the frame including imaging data obtained by converting the output signal of each pixel circuit into a digital signal;
The control unit includes, in the frame, the imaging data that has reached an upper limit value corresponding to the saturation for the imaging data corresponding to the determination target of the imaging mode determination frame output from the output signal processing unit. And when the imaging data that has reached the upper limit value is included in the frame, the imaging mode switching unit sets the imaging mode of each pixel circuit to the second imaging mode. An imaging apparatus characterized by being specified as follows. The imaging apparatus according to claim 7,
The control unit calculates an average value of the imaging data corresponding to the determination target of the imaging mode determination frame, further determines whether the average value is larger than a predetermined value, and the average value is When the value is equal to or less than a predetermined value, the imaging mode of each pixel circuit is set to the first imaging mode for the imaging mode switching unit even when the imaging data that has reached the upper limit value is included in the frame. An imaging apparatus characterized by specifying. The imaging apparatus according to claim 7,
The control unit counts the occurrence frequency of the imaging data below a predetermined determination value for the imaging data corresponding to the determination target of the imaging mode determination frame, and whether the occurrence frequency is smaller than a predetermined reference value. And further determining whether or not, when the frequency of occurrence is a predetermined reference value or more, even when the imaging data that has reached the upper limit value is included in the frame, An imaging apparatus, wherein an imaging mode of each pixel circuit is designated as the first imaging mode. A charge is generated by receiving each light, and a plurality of light receiving elements arranged in a lattice form so as to form a plurality of rasters, and a charge generated by the light receiving elements are provided corresponding to each of the light receiving elements. And a plurality of pixel circuits that output an output signal corresponding to the amount of accumulated charge, and an imaging device,
With respect to the imaging device, the imaging mode of each pixel circuit is set such that when the received light amount of the light receiving element is smaller than a predetermined light amount, the output signal increases linearly as the received light amount increases. A first imaging mode in which the output signal is saturated when the amount of received light of the element is equal to or greater than the predetermined amount of light, and a second imaging in which the output signal is not saturated when the amount of received light of the light receiving element is greater than or equal to the predetermined amount of light. An imaging mode switching unit that switches between the modes in units of the raster,
The imaging element includes a first imaging mode raster based on the output signal imaged in the first imaging mode and a second imaging mode raster based on the output signal imaged in the second imaging mode. Output continuous frames in time series,
An imaging mode to be selected is determined based on the output signal included in one of the first imaging mode raster and the second imaging mode raster included in the frame, and based on the determination result And a controller that selects the first imaging mode raster or the second imaging mode raster from the frame and the frame imaged immediately before the frame and creates a frame obtained by combining the first imaging mode raster and the second imaging mode raster. An imaging device. The imaging device according to claim 10.
The imaging apparatus is characterized in that the imaging mode switching unit alternately arranges the first imaging mode raster and the second imaging mode raster so as to be adjacent to each other in each frame. The imaging device according to claim 11,
The imaging apparatus according to claim 1, wherein the imaging mode switching unit interchanges positions of the first imaging mode raster and the second imaging mode raster between the frames adjacent in time series. The imaging apparatus according to any one of claims 1 to 12,
The imaging device further includes a stored charge discharging circuit capable of executing a stored charge discharging operation for discharging a part of the charge stored in each pixel circuit before outputting the output signal,
The imaging apparatus, wherein the imaging mode switching unit switches between execution and non-execution of an accumulated charge discharging operation of the accumulated charge discharging circuit based on an imaging mode designated by the control unit. The imaging device according to claim 13.
The accumulated charge discharging circuit is provided in the middle of a path for discharging accumulated charges, and includes a charge discharging gate constituted by a transistor,
In the second imaging mode, the charge discharging gate is controlled to start discharging charges when the amount of light received by the light receiving element is smaller than that in the first imaging mode.

Images