JP2018101937A - Imaging apparatus, control method of the same, and control program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain a good ND effect by reducing a change in color balance and unevenness when light amount control is performed using an EC element.SOLUTION: An imaging device 3 includes a plurality of pixels arranged in a two-dimensional matrix. Each of the pixels includes: a photoelectric conversion unit that converts the optical image into electric charges; an accumulation unit that accumulates charges of the photoelectric conversion unit; a first transfer section for transferring electric charges from the photoelectric conversion unit to the accumulation unit; and a second transfer section for transferring electric charge of the photoelectric conversion unit to a charge discharging unit. The exposure amount of the imaging device is adjusted according to the coloring of an EC element. When an imaging device drive control unit 4 selectively performs the transfer by the first transfer section and the second transfer section and adjusts the exposure amount of the imaging device, the exposure amount is adjusted on the basis of the coloring state of the EC element and an image signal.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image pickup apparatus, a control method thereof, and a control program, and more particularly to an image pickup apparatus that controls the transmittance of light quantity at the time of shooting.

  In general, in an imaging apparatus such as a digital camera, a technique is known in which imaging is performed while reducing the transmittance of the amount of imaging light when shooting in an extremely bright environment. This method can be used to express a subject with a shallow depth of field by opening the aperture even in a bright environment or when the subject moves without causing saturation even when long-second exposure is performed (for example, a waterfall). Used when expressing.

  Conventionally, when adjusting the photographing light quantity, the light is reduced using an optical filter such as an ND filter. On the other hand, in recent years, a technique for obtaining the same effect as an ND filter by using a coloring and decoloring technique using an electrochromic (hereinafter referred to as EC) element is known (see Patent Documents 1 and 2).

JP 2001-519922 A Japanese Patent Laid-Open No. 6-301065

  However, when the EC element described in Patent Document 1 is used as the ND filter described in Patent Document 2 by coloring and decoloring by energization drive control, there are the following problems.

  In the EC element, the color balance at the time of coloring may not be uniform in the EC layer due to differences in redox potential and diffusion coefficient of EC molecules in the EC layer. Furthermore, the wavelength dependence of the light transmittance may change depending on the degree of coloring. When such a situation occurs, transmittance unevenness occurs in the photographing range, and inconveniences such as a color balance being lost according to the number of ND effect stages occur.

  Accordingly, an object of the present invention is to provide an imaging apparatus, a control method thereof, and a control program capable of obtaining a good ND effect by reducing color balance change and unevenness when performing light quantity control using an EC element. It is to provide.

  In order to achieve the above object, an imaging device according to the present invention is an imaging device including an imaging device that outputs an image signal corresponding to an optical image, and the imaging device includes a plurality of pixels arranged in a two-dimensional matrix. Each of the pixels includes a photoelectric conversion unit that converts the optical image into an electric charge, an accumulation unit that accumulates the electric charge of the photoelectric conversion unit, and transfers the electric charge from the photoelectric conversion unit to the accumulation unit A first transfer unit; and a second transfer unit configured to transfer the charge of the photoelectric conversion unit to a charge discharging unit, and selectively performing transfer by the first transfer unit and the second transfer unit. And first control means for adjusting the exposure amount of the image sensor, and second control means for adjusting the exposure amount of the image sensor in accordance with the coloring of the electrochromic element, The control means includes the electrochromic element. And performing adjustment of the exposure amount on the basis the state of coloration and the said image signal.

  According to the present invention, when performing light amount control using an electrochromic element, it is possible to obtain a favorable ND effect by reducing changes in color balance and unevenness.

It is a block diagram which shows the structure about an example of the imaging device by embodiment of this invention. It is a figure which shows the circuit structure about a part of imaging device shown in FIG. 3 is a timing chart for explaining an imager ND function performed by the image sensor shown in FIG. 2. It is a figure which shows the potential state of the pixel until the accumulation | storage time of the image pick-up element shown in FIG. 2 is complete | finished. FIG. 4 is a timing chart showing in detail a state from time t3 to time t5 with respect to pulses φTX1A (n) and φTX1A (n + 1) shown in FIG. 3. It is sectional drawing which shows the example about the structure of the ECND part shown in FIG. It is a figure which shows the relationship between the light wavelength and light transmittance in the ECND part shown in FIG. 2 is a flowchart for explaining ECND control and image sensor drive control in the camera shown in FIG. 1. 3 is a timing chart for explaining an example of charge accumulation and reading performed in the image sensor 3 shown in FIG. 2.

  Hereinafter, an example of an imaging apparatus according to an embodiment of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration of an example of an imaging apparatus according to an embodiment of the present invention.

  The illustrated image pickup apparatus is, for example, a digital camera (hereinafter simply referred to as a camera) 1000, and an optical image is formed on the image pickup device 3 via the photographing optical system 1. As will be described later, the image sensor 3 functions as a first ND unit (imager ND unit), and adjusts the exposure amount according to the control of the accumulation time and the number of accumulations. The exposure amount adjustment by the image sensor 3 is performed by the image sensor drive control unit 4.

  As shown in the figure, an ECND section 2 that is a second ND section is disposed between the photographing optical system 1 and the image sensor 3. The ECND unit 2 adjusts the exposure amount of the optical image formed on the image pickup device 3 by coloring drive control of an electrochromic device (EC device) incorporated in the ECND unit 2. As will be described later, exposure amount control by the ECND unit 2 is performed by an ECND control unit (not shown).

  The imaging element drive control unit 4 performs imaging based on the detection result by the EC coloring detection unit 5 that detects the coloring state of the EC element provided in the ECND unit 2 and the characteristic value of the ECND unit 2 recorded in the storage unit 6. The element 3 is controlled.

  The imaging device 3 outputs an image signal corresponding to the optical image by photoelectric conversion. The image processing unit 7 performs predetermined image processing such as development processing on the image signal, and stores the image data obtained as a result in the memory 8.

  FIG. 2 is a diagram showing a circuit configuration of a part of the image sensor shown in FIG.

  The image sensor 3 is, for example, a CMOS image sensor. In the illustrated example, the image sensor 3 has a plurality of pixels arranged in a two-dimensional matrix. Here, the pixel in the first row and the first column (1, 1) and the last row, the nth row and the first column ( n, 1) pixels are shown (n is an integer of 2 or more). The configuration of the pixel in the first row and first column (1,1) and the pixel in the nth row and first column (n, 1) are the same, and therefore the same reference numerals are given here.

  The pixel has two signal holding units (storage units) 507A and 507B with respect to the photodiode (PD: photoelectric conversion unit) 500. Note that a CMOS image sensor including a signal holding unit is described in, for example, Japanese Patent Application Laid-Open No. 2013-172210, and thus description thereof is omitted here.

  In FIG. 2, the pixel includes a PD 500, a first transfer transistor 501A, a first signal holding unit 507A, a second transfer transistor 502A, a third transfer transistor 501B, a second signal holding unit 507B, and a fourth signal holding unit. A transfer transistor 502B is included. Further, the pixel includes a fifth transfer transistor 503, a floating diffusion (FD) region (floating diffusion region) 508, a reset transistor 504, an amplification transistor (amplification unit) 505, and a selection transistor 506.

  The first transfer transistor (transfer unit) 501A is ON / OFF controlled by a transfer pulse φTX1A, and the second transfer transistor 502A is ON / OFF controlled by a transfer pulse φTX2A. The third transfer transistor 501B is on / off controlled by the transfer pulse φTX1B, and the fourth transfer transistor 502B is on / off controlled by the transfer pulse φTX2B.

  Further, the reset transistor 504 is ON / OFF controlled by a reset pulse φRES, and the selection transistor 506 is ON / OFF controlled by a selection pulse φSEL. The fifth transfer transistor 503 is ON / OFF controlled by the transfer pulse φTX3. That is, the first to fifth transfer transistors are selectively turned on / off by the transfer pulse.

  Each pulse described above is sent from a vertical scanning circuit (not shown). Further, as shown in the figure, the image pickup device 3 includes power supply lines 520 and 521 and a signal output line 523.

  As described above, since the image pickup device 3 includes the two signal holding units 507A and 507B for one PD 500, for example, a still image and a moving image can be taken simultaneously. As will be described later, the charge is transferred from the PD 500 to the signal holding unit 507 by the transfer transistor 502. At this time, the total accumulation time depends on the width of the transfer pulse and the number of transfer pulses related to the transfer transistor 502. Can be adjusted. That is, the image pickup device 1 functions as the first ND unit by adjusting the total accumulation time.

  In the illustrated example, the image sensor 3 has two signal holding units, but the number of signal holding units is not limited, and in any case, the image sensor 3 has a function as an imager ND unit. Just do it.

  FIG. 3 is a timing chart for explaining the imager ND function performed by the image sensor shown in FIG.

  FIG. 4 is a diagram for explaining the potential state of the pixel until the accumulation time of the image sensor shown in FIG. 2 ends. FIG. 4A shows the first potential state, and FIG. 4B shows the second potential state. FIG. 4C shows the third potential state, and FIG. 4D shows the fourth potential state. Further, FIG. 4E shows the fifth potential state, and FIG. 4F shows the sixth potential state.

  3 and 4, here, for convenience of explanation, the PD 500, the first transfer transistor 501A, the first signal holding unit 507A, and the second transfer transistor 502A are used in the pixel shown in FIG. The charge accumulation and charge to the signal holding unit will be described. Further, here, operations up to reading are performed using the fifth transfer transistor 503, the FD region 508, the reset transistor 504, the amplification transistor 505, and the selection transistor 506.

  FIG. 3 shows changes in the pulses φTX1A to φTX3 given to the control electrodes (bases) of the transfer transistors 501A, 502A and 503 and the pulse φRES given to the control electrode of the reset transistor 504. The subscripts n, n + 1, and n + 2 indicate pixel row numbers. For example, φTX1 (n) indicates a pulse applied to the first transfer transistor 501A of the n-th row pixel.

  In the initial state before time t0, transfer pulses φTX1 and φTX2 are at a low level (L level), and transfer pulse φTX3 and reset pulse φRES are at a high level (H level). The potential state of the pixel at this time is shown in FIG.

  In the initial state, there is a potential barrier formed in the first transfer transistor 501A against the charge accumulated in the first signal holding unit 507A (MEM). On the other hand, the fifth transfer transistor 503 has no potential barrier. For this reason, the electric charge (indicated by a black circle in FIG. 4A) generated in the PD 500 does not move to the first signal holding unit 507A (MEM). Then, the electric charge is discharged to OFD (overflow drain: charge discharging unit) through the fifth transfer transistor 503 (TX3).

  Here, the potential barrier formed in the first transfer transistor 501A (TX1A) is lower than the potential barrier formed in the second transfer transistor 502A (TX2A). That is, when the PD 500, the first transfer transistor 501A (TX1A), and the first signal holding unit 507A (MEM) are buried channel types, the potential barrier has the above relationship.

  In the period from time t0 to time t1, transfer pulses φTX2A (n) to φTX2A (n + 2) are at the H level. Therefore, the potential barrier is eliminated in the second transfer transistor 502A (TX2A) located between the first signal holding unit 507A (MEM) and the FD region 508. Therefore, the charge held in the first signal holding unit 507A before time t0 is transferred to the FD region 508. FIG. 4B shows the potential state of the pixel during this period.

  During this period, transfer pulses φTX1A (n) to φTX1A (n + 2) are at the L level, and transfer pulses φTX3 (n) to φTX3 (n + 2) are at the H level. Therefore, the charge generated in the PD 500 is discharged to the OFD through the fifth transfer transistor 503 (TX3). Therefore, at this time, the charge generated in the PD 500 ideally does not exist in the first signal holding unit 507.

  At time t1, when the transfer pulses φTX2A (n) to φTX2A (n + 2) become L level, the potential state of the pixel becomes the state shown in FIG. The state shown in FIG. 4C is the same as the state shown in FIG. Even during this period, there is a potential barrier formed in the first transfer transistor 501A (TX1A). On the other hand, the fifth transfer transistor 503 (TX3) has no potential barrier. Therefore, the charge generated in the photodiode 500 is discharged to the OFD through the fifth transfer transistor 503 (TX3) without moving to the first signal holding unit 507.

  Next, when the transfer pulses φTX3 (n) to φTX3 (n + 2) transition to the L level at time t3, the potential state of the pixel becomes the state shown in FIG. In this period, the fifth transfer transistor 503 (TX3) has a higher potential barrier against the charge accumulated in the first signal holding unit 507 than the first transfer transistor 501A (TX1A). Since the transfer pulses φTX2A (n) to φTX2A (n + 2) are at the L level, the charges generated in the PD 500 that exceed the potential barrier of the first transfer transistor 501A are the PD 500 or the first signal holding unit 507. Stay on. Accordingly, the accumulation time of each pixel starts from the timing at which the transfer pulses φTX3 (n) to φTX3 (n + 2) transition to the L level at time t3.

  In the period from time t3 to time t4, the transfer pulses φTX1A (n) to φTX1A (n + 2) are at the H level. As a result, the potential barrier formed in the first transfer transistor 501A disappears, and the charge generated in the PD 500 is transferred to the first signal holding unit 507 (see FIG. 4E).

  Thereafter, the period in which the transfer pulses φTX1A (n) to φTX1A (n + 2) are at the L level and the period at which the transfer pulses are at the H level are repeated a plurality of times by time t5. Further, between time t3 and time t5, the transfer pulses φTX3A (n) to φTX3A (n + 2) are inverted in H and L levels from the transfer pulses φTX1A (n) to φTX1A (n + 2). As a result, charges generated in the PD 500 during a period other than the period when the transfer pulses φTX1A (n) to φTX1A (n + 2) are at the H level are discharged to the OFD through the fifth transfer transistor 503 (TX3). The number of transfers is not particularly limited.

  The charge accumulated in the PD 500 can be periodically transferred to the first signal holding unit 507 by the above-described driving method. In addition, in the case of a normal image sensor, accumulation for a predetermined time is continuously performed, but in the image sensor 3, there are a plurality of accumulation times and non-accumulation times in the predetermined time. Using the time relationship between accumulation and non-accumulation, the image sensor 3 can obtain a desired ratio by chopping in the time direction the light amount obtained in the exposure time from time t3 to time t5. That is, such an operation is an exposure adjustment operation using the imager ND function performed by the image sensor 3.

  At time t5, the transfer pulses φTX1A (n) to φTX1A (n + 2) transition to the L level, and at the same time, the transfer pulses φTX3 (n) to φTX3 (n + 2) become the H level. As a result, the potential state of the pixel becomes the state shown in FIG. Since the charge generated in the PD 500 after time t5 is discharged to the OFD through the fifth transfer transistor 503, the accumulation time for all the pixels ends at time t5.

  In this way, by simultaneously transferring charges from the PD 500 to the first signal holding unit 507 for all the pixels, the accumulation start and end times for all the pixels can be matched. As a result, an in-plane synchronized electronic shutter operation can be performed.

  Next, from time t6 to time t8, the transfer pulse φTX2A (n) goes high at time t6 during the period when the reset pulse φRES1 (n) is at L level. As a result, the charge held in the first signal holding unit 507 of each pixel in the n-th row is transferred to the FD 508 via the second transfer transistor 502A (TX2A). At least at this timing, the selection transistor 506 is on. Therefore, a voltage of a level corresponding to the amount of charge transferred to the FD region 508 appears on the signal output line 523 by the source follower circuit formed by the amplification transistor 505 and the constant current source. A signal corresponding to the voltage level appearing on the signal output line 523 is output to the outside from an output circuit (not shown).

  The same operation is performed for the pixels in the (n + 1) th row and the (n + 2) th row, and a signal corresponding to the pixel in each row is output from the output circuit. Thus, the operation for one frame (that is, one frame time) is completed.

  Here, OFD is used as the charge discharge region, but it is not limited to this. For example, the fifth transfer transistor 503 may be connected to the FD 508, and the second transfer transistor 502A may discharge the charge from the first signal holding unit 507 to the power supply line before transferring the charge to the FD region 508. Even in such a configuration, the charge photoelectrically converted by the PD 500 is discharged to the first signal holding unit 507 and discharged through the FD region 508, so that the exposure amount adjustment operation can be performed.

  FIG. 5 is a timing chart showing in detail the state from time t3 to time t5 for the pulses φTX1A (n) and φTX1A (n + 1) shown in FIG.

  As described above, in the period from time t3 to t5, the transfer pulse φTX1A repeats the L level and the H level a plurality of times. As a result, the image pickup device 3 exhibits an imager ND function. Here, this imager ND function further changes according to the line. That is, the pulse widths at the H level of the transfer pulse φTX1A for driving the first transfer transistor 501A provided in the pixels in the nth row and the (n + 1) th row are set to be different from each other. Such control is performed by changing the pulse control program of the vertical scanning circuit by the image sensor drive control unit 4.

  In FIG. 5, the transfer pulse is controlled so that the total accumulation time of the pixels in the n-th row is shorter than the total accumulation time of the pixels in the (n + 1) -th row. The illustrated image pickup device 3 has a color filter with a so-called Bayer array, and the n-th row is an RG row having an RG (red-green) color filter. In this case, the (n + 1) th row is a GB row having a GB (greenish blue) color filter.

  When the difference in the pulse width shown in FIG. 5 is applied to the entire imaging device 3, an image in which the spectral transmittance of R (red) is reduced with respect to B (blue) is obtained. Also, by assigning the charge accumulation time evenly to one frame, accumulation blur in one frame can be naturally obtained rather than simply placing a short charge accumulation time in one frame at the head of the frame.

  FIG. 6 is a cross-sectional view showing an example of the configuration of the ECND portion shown in FIG.

  The ECND unit 2 includes an EC element 2020, and electrodes 2000 and 2010 are disposed on the main surface (front surface and back surface) of the EC element 2020, respectively. Further, the EC element 2020 is held isolated from the outside by a sealing material 2030. The pair of electrodes 2000 and 2010 are transparent electrodes and transmit the wavelength of light that the camera 1000 is to shoot. For example, it is desirable to use known ITO electrodes as the electrodes 2000 and 2010.

  As shown in the figure, the pair of electrodes 2000 and 2010 are arranged with their electrode surfaces facing each other, and are joined to each other by a sealing material 2030. An EC element 2020 is formed between the pair of electrodes 2000 and 2010 by EC molecules. As the EC molecule, for example, the EC molecules described in Patent Documents 1 and 2 or known EC molecules are used.

  Here, when a voltage is applied to the pair of electrodes 2000 and 2010, an oxidation-reduction reaction occurs, and the light absorptivity of the EC molecules changes according to the characteristics of the EC molecules. In the camera 1000, exposure adjustment is performed by using the change in the light absorption rate due to voltage application in the same manner as the variable ND filter. The ECND unit 2 is disposed on the subject side of the image sensor 3 so that the electrodes 2000 and 2010 are substantially perpendicular to the imaging optical axis.

  FIG. 7 is a diagram showing the relationship between the light wavelength and the light transmittance in the ECND unit 2 shown in FIG.

  In FIG. 7, the solid line indicates the transmittance for each light wavelength when the ECND unit 2 (EC) is used. Moreover, a broken line shows the transmittance | permeability for every optical wavelength at the time of using an ND filter (ND filter). Here, there are a plurality of curves for both the solid and broken lines because of the difference in the number of ND effect stages.

  As can be easily understood from FIG. 7, when the ECND unit 2 is used, it can be seen that the flatness is low in the transmittance for each light wavelength as compared with the ND filter. That is, since the effect of dimming by the ECND unit 2 changes according to the light wavelength, it is difficult to perform uniform dimming over the entire visible light band.

  Further, in the ECND unit 2, the change in transmittance for each light wavelength according to the number of ND effect stages is not similar, and therefore the ratio of the light wavelength contained in the light transmitted by the number of ND effect stages may not be constant. I understand. That is, for example, there is a concern that a difference in color temperature may occur between an image shot with the two-stage ND stage number effect and an image shot with the three-stage ND stage number effect.

  As described above, the ECND unit 2 has an EC element, and when a voltage is applied to the EC element, the light absorption rate of each of the anode material and the cathode material is changed by an oxidation-reduction reaction. That is, coloring / decoloring occurs. The coloring state of the EC element by the anode material and the cathode material is due to the absorption characteristics of the anode material and the cathode material. For this reason, although various materials have been proposed, it is difficult to obtain flat transmittance characteristics for each light wavelength as in the case of the ND filter. This is a cause of low transmittance flatness in the ECND portion 2 described above.

  On the other hand, the distribution of the absorption wavelength of the anode material and the cathode material may change depending on the applied voltage. This is a cause of a change (color shift) in the transmission wavelength ratio according to the ND stage number effect in the ECND section 2 described above. Further, the electrode reaction becomes active or decreases depending on the environmental temperature at which the ECND unit 2 operates. As a result, the distribution of the absorption wavelength similarly changes, and coloring unevenness may occur in the plane perpendicular to the optical axis of the EC element due to a difference in distance from the electrode.

  FIG. 8 is a flowchart for explaining ECND control and image sensor drive control in the camera shown in FIG.

  When the camera 1000 is turned on, the image sensor drive control unit 4 calls the characteristic value of the ECND unit 2 stored in the storage unit 6 (step S801). The characteristic value of the ECND unit 2 is information on the unevenness of coloration in the EC element plane due to the transmittance distribution for each light wavelength and the arrangement of the EC electrodes according to the designed EC coloring characteristic of the ECND unit 2. Note that the characteristic value of the ECND unit 2 may include not only the design but also the transmittance distribution and coloring unevenness information for each ECND unit 2 due to manufacturing errors and the like.

  These transmittance distribution and coloring unevenness information are stored in the storage unit 6 for each ND effect stage number of the ECND unit 2. Further, the coloring characteristics for each EC element may be stored in the storage unit 6 as individual adjustment values in the manufacturing process, for example, and the aged individual obtained by calibration including the secular change by the user or customer service. You may make it update with an adjustment value.

  As a calibration method, for example, a method of photographing a uniform luminance surface according to the operation state of the ECND unit 2 and correcting unevenness based on unevenness information of an image obtained as a result is used. Then, the camera 1000 may have a built-in calibration function, or the individual adjustment values stored in the storage unit 6 may be automatically updated sequentially.

  The image sensor drive control unit 4 changes the above-described pulse control program of the vertical scanning circuit based on the above characteristic value. Therefore, the characteristic value of the ECND unit 2 stored in the storage unit 6 may be the coloring state itself of the ECND unit 2 or may be a parameter of the pulse control program.

  Subsequently, the image sensor drive control unit 4 determines the total accumulation time for each color filter based on the relationship between the coloring information and the color filter characteristics (spectral transmission characteristics) of the image sensor 3, and the ND single that the captured image is flatter. Set to obtain according to the effect. Specifically, when the ECND unit 2 has a lower transmittance in the wavelength range of the R pixel of the image sensor 3 than the wavelength range of the B pixel, the total accumulation time of the R pixel is longer than the total accumulation time of the B pixel. Change the pulse control program to

  Next, the image sensor drive control unit 4 determines the presence or absence of an exposure change instruction described below (step S802). The exposure change instruction may be, for example, an operation button (not shown) provided on the camera 1000 or a manual exposure operation performed by operating a GUI. The exposure control unit (not shown) controls the subject brightness. It may be an exposure change operation by detection.

  If there is no exposure change instruction (NO in step S802), the image sensor drive control unit 4 stands by. When there is an instruction to change exposure (YES in step S802), the image sensor drive control unit 4 determines whether or not the ND function needs to be used under the exposure conditions determined by the program diagram (step S803).

  The program diagram is used when determining exposure conditions based on subject brightness and the like. The exposure condition is determined based on the Av value determined by the aperture of the aperture device, the Tv value determined by the speed of the shutter device, and the Sv value determined by the sensitivity of the image sensor 3.

  Since the camera 1000 has an imager ND function (hereinafter collectively referred to as an ND function) by the ECND unit 2 and the image pickup device 3, for example, under an extremely bright exposure condition, the automatic exposure control unit needs the ND function. A flag is issued based on the program diagram. When the flag is issued, the image sensor drive control unit 4 determines that the ND function needs to be used.

  If it is determined that it is not necessary to use the ND function (NO in step S803), the image sensor drive control unit 4 returns to the process in step S801. On the other hand, if it is determined that the ND function needs to be used (YES in step S803), the image sensor drive control unit 4 notifies the ECND control unit accordingly. Thus, the ECND control unit turns on and drives the ECND unit 2 (step S804).

  On the other hand, the image sensor drive control unit 4 obtains the detection result of the coloring state of the ECND unit 2 detected by the EC coloring detection unit 5 (step S805). The EC coloring detection unit 5 is a light detection unit having a light projecting / receiving system, and detects the transmittance of each RGB by the ECND unit 2.

  Next, the image sensor drive control unit 4 updates the pulse control program based on the EC element color state detection result by the EC color detection unit 5 (step S806). Then, the image sensor drive control unit 4 turns on the imager ND function of the image sensor 3 based on the updated pulse control program (step S807). When the imager ND function is already turned on, the image sensor drive control unit 4 continuously operates the imager ND function according to the latest pulse control program.

  When the imager ND function is turned ON, the image sensor drive control unit 4 controls the transfer pulse φTX1A described above according to the pulse control program updated based on the coloring information obtained so far, and total accumulation for each row is performed. Change the time. Thereby, the dispersion of the transmittance for each light wavelength of the ECND unit 2 is alleviated, and the flatness of the transmittance for each light wavelength can be improved as compared with the dimming when the ECND unit 2 is used alone. .

  Subsequently, the image sensor drive control unit 4 determines whether or not the pulse control program needs to be changed based on the coloring state detection result by the EC coloring detection unit 5. That is, the image sensor drive control unit 4 determines whether or not the coloring state detection result by the EC coloring detection unit 5 has changed as much as necessary to change the pulse control program (step S808).

  If it is determined that a change is required (YES in step S808), the image sensor drive control unit 4 returns to the process in step S806 and changes the pulse control program (that is, EC coloring characteristics). On the other hand, if it is determined that no change is required (NO in step S808), the image sensor drive control unit 4 maintains the imager ND function without changing the pulse control program. Then, the image sensor drive control unit 4 determines whether or not there is an exposure re-change instruction (step S809).

  As described above, after the imager ND function is turned ON, the imager ND function is feedback-controlled by sequentially referring to the EC color state detection results detected in real time. As a result, it is possible to use the ND function without variations in transmittance and color unevenness for each light wavelength. On the other hand, the above-described calibration also corresponds to the EC color detection unit 5, and similar feedback control may be performed based on the information on the color unevenness obtained by the calibration.

  When there is an exposure re-change instruction (YES in step S809), the image sensor drive control unit 4 returns to the process of step S803. On the other hand, if there is no exposure rechange instruction (NO in step S809), the image sensor drive control unit 4 determines whether or not there is an instruction to stop the ND function (step S810). Note that the stop instruction for the ND function is, for example, a shooting end instruction for the user to stop shooting a moving image or a still image. Further, it may be an automatic exposure end instruction or a sleep start instruction that is performed by the camera 1000 when a predetermined time has passed without a predetermined operation in live view display or the like. Further, the instruction to stop the ND function may be that the user selects a mode that does not use the ND function.

  If there is an instruction to stop the ND function (YES in step S810), the image sensor drive control unit 4 stops the ND function and ends the ECND control and the image sensor drive control. On the other hand, if there is no instruction to stop the ND function (NO in step S810), the image sensor drive control unit 4 returns to the process in step S809.

  As described above, in the above-described embodiment, the flatness of the transmittance for each light wavelength is improved as compared with the ECND function performed by the ECND unit 2 alone. As a result, a good captured image is obtained. That is, it is possible to eliminate the variation in the light beam wavelength ratio of the transmitted light due to the low flatness of the transmittance for each light wavelength of the ECND section 2 and the number of ND effect stages. Accordingly, the camera 1000 has an ND function with improved wavelength flatness. Furthermore, by using the ECND unit 2 and the imager ND function at the same time, an ND function having a wide number of ND effect stages can be obtained.

  In the above-described embodiment, the transfer pulse φTX1A is varied for each row, and the total accumulation time of the image sensor 3 is varied for each row, thereby changing the RGB output characteristics in the captured image. On the other hand, the total accumulation time may be different for each pixel.

  Next, a modification example of charge accumulation and readout performed by the image sensor 3 shown in FIG. 2 will be described.

  FIG. 9 is a timing chart for explaining a modification example of charge accumulation and readout performed in the image sensor 3 shown in FIG.

  Here, the imaging device 3 includes a plurality of signal holding units 507A and 507B as described above. Charge accumulation in the PD 500 and charge transfer (transfer A) to the signal holding unit 507A are performed by a transfer pulse φTX1A. In FIG. 9, a cross hatch portion in “accumulation” and a rectangular portion in “transfer A” are shown.

  Thereafter, accumulation in the PD 500 and charge transfer (transfer B) to the signal holding unit 507B are performed by the transfer pulse φTX1B. In FIG. 9, a left-down hatched portion in “Accumulation” and a rectangular portion in “Transfer B” are shown.

  The above-described operation is repeated 16 times in one frame (here, the Nth frame), and charges are held in the signal holding units 507A and 507B. After any 16th accumulation and transfer, charges are transferred to the FD 508 by transfer pulses φTX2A and φTX2B.

  Normally, voltage signals obtained by both the signal holding units 507A and 507B appear on the signal output line 523 and are read by the vertical scanning circuit. In FIG. 9, the black portions in the readouts A and B are shown.

  On the other hand, in the illustrated image sensor 3, whether or not to read out the charge of the FD 508 is selected for each column by the pulse control program. That is, one of reading A and reading B shown in FIG. 9 is performed depending on the column. As a result, two accumulation patterns can be selected for the accumulation time for each column.

  In the example shown in FIG. 9, the total accumulation time of the signal holding unit 507A is set to be shorter than that of the signal holding unit 507B, and as a result, the spectral sensitivity of the pixel from which the signal holding unit 507B has been read is lowered. It will be. In the pulse control program, for example, the charge of the signal holding unit 507A is transferred for the column including the R pixel, and the charge of the signal holding unit 507B is transferred for the column including the G pixel. As a result, an R pixel signal having a relatively short accumulation time and a G pixel signal having a relatively long accumulation time are obtained.

  Furthermore, if both reading A and reading B are performed, and addition and subtraction are performed in the subsequent stage, it is possible to increase the patterns of total accumulation times that differ in the column direction. In the image sensor 3 of this example, the RGB output characteristics in the entire captured image can be controlled more flexibly.

  Further, if the image pickup device 3 in this example is used, the RGB output characteristics of the photographed image can be changed not only for each row but also for each column. As a result, the uneven coloring in the optical axis vertical plane of the EC device is also two-dimensional. It becomes possible to reduce it.

  As described above, in the modification, the total accumulation time for each pixel is varied using the signal holding units 507A and 507B and the pulse control program. On the other hand, charge accumulation and readout are not limited to this example, and an image sensor that can control the total accumulation time for each pixel and an image sensor drive control unit that controls the image sensor may be provided.

  In the above-described embodiment and modification, two signal holding units are provided. However, the present invention is not limited to this, and a plurality of signal holding units are provided to increase the gradation of the accumulation pattern of each pixel. Can do. Furthermore, even if one frame is divided into sub-frames and read out, it is possible to increase the gradation of the accumulated pattern.

  Furthermore, in the above-described example, the Bayer arrangement is performed using the primary color filter as the color filter in the image sensor 3, but the present invention is not limited to this, and a complementary color filter may be used. An array different from the Bayer array may be used.

  As described above, according to the embodiment of the present invention, when the light amount control is performed using the EC element, it is possible to obtain a favorable ND effect by reducing the change in color balance and unevenness.

  As mentioned above, although this invention was demonstrated based on embodiment, this invention is not limited to these embodiment, Various forms of the range which does not deviate from the summary of this invention are also included in this invention. .

  For example, the function of the above embodiment may be used as a control method, and this control method may be executed by the imaging apparatus. Further, a program having the functions of the above-described embodiments may be used as a control program, and the control program may be executed by a computer included in the imaging apparatus. The control program is recorded on a computer-readable recording medium, for example.

[Other Embodiments]
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in the computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

DESCRIPTION OF SYMBOLS 1 Imaging optical system 2 ECND part 3 Imaging element 4 Imaging element drive control part 5 EC coloring detection part 6 Memory | storage part 7 Image processing part 8 Memory 1000 Imaging device (camera)

Claims (10)

An imaging apparatus including an imaging element that outputs an image signal corresponding to an optical image,
The imaging device includes a plurality of pixels arranged in a two-dimensional matrix,
Each of the pixels includes a photoelectric conversion unit that converts the optical image into a charge, a storage unit that stores the charge of the photoelectric conversion unit, and a first transfer unit that transfers the charge from the photoelectric conversion unit to the storage unit. And a second transfer unit that transfers the charge of the photoelectric conversion unit to the charge discharge unit,
First control means for selectively performing transfer by the first transfer unit and the second transfer unit to adjust an exposure amount of the image sensor;
A second control means for adjusting the exposure amount of the imaging element in accordance with the coloring of the electrochromic element,
The imaging apparatus according to claim 1, wherein the first control unit adjusts the exposure amount based on a coloring state of the electrochromic element and the image signal.   The imaging apparatus according to claim 1, wherein the first control unit drives and controls the pixels for each row of the imaging element. The pixel includes a plurality of the storage units, the first transfer unit, and the second transfer unit,
The imaging apparatus according to claim 1, wherein the first control unit changes drive control of the pixels for each column when adding or subtracting outputs of the plurality of accumulation units. Comprising storage means in which coloring information relating to the coloring of the electrochromic element is recorded;
The imaging apparatus according to claim 1, wherein the first control unit determines a coloring state of the electrochromic element using the coloring information.   The imaging apparatus according to claim 1, further comprising a coloring state detection unit that detects a coloring state of the electrochromic element. The photoelectric conversion unit receives the optical image through a red, green, or blue color filter,
6. The imaging according to claim 1, wherein the first control unit adjusts the exposure amount according to output characteristics of red, green, and blue of the image signal. apparatus.   The pixel includes a third transfer unit for transferring the charge accumulated in the accumulation unit to the floating diffusion region, and an amplification unit for amplifying the charge in the floating diffusion region. The imaging device according to any one of claims 1 to 6.   The imaging apparatus according to claim 1, wherein the first control unit adjusts the exposure amount in one frame time. A plurality of pixels arranged in a two-dimensional matrix, each of the pixels is a photoelectric conversion unit that converts an optical image into electric charge, an accumulation unit that accumulates electric charge of the photoelectric conversion unit, and the photoelectric conversion unit An imaging device that has a first transfer unit that transfers the charge to the storage unit and a second transfer unit that transfers the charge of the photoelectric conversion unit to the charge discharge unit, and outputs an image signal corresponding to the optical image A method for controlling an imaging apparatus including an element,
A first control step of selectively performing transfer by the first transfer unit and the second transfer unit to adjust an exposure amount of the image sensor;
A second control step of adjusting the exposure amount of the imaging device according to the coloring of the electrochromic device,
In the first control step, the exposure amount is adjusted based on a coloring state of the electrochromic element and the image signal. A plurality of pixels arranged in a two-dimensional matrix, each of the pixels is a photoelectric conversion unit that converts an optical image into electric charge, an accumulation unit that accumulates electric charge of the photoelectric conversion unit, and the photoelectric conversion unit An imaging device that has a first transfer unit that transfers the charge to the storage unit and a second transfer unit that transfers the charge of the photoelectric conversion unit to the charge discharge unit, and outputs an image signal corresponding to the optical image A control program used in an imaging device including an element,
In the computer provided in the imaging device,
A first control step of selectively performing transfer by the first transfer unit and the second transfer unit to adjust an exposure amount of the image sensor;
Performing a second control step of adjusting an exposure amount of the imaging element in accordance with coloring of the electrochromic element,
In the first control step, the exposure amount is adjusted based on a coloring state of the electrochromic element and the image signal.

Images