JP6057630B2 - Imaging device - Google Patents

Description

  The present invention relates to an imaging apparatus that performs imaging to obtain wide dynamic range image data in which whiteout and blackout are suppressed.

  When a scene with a large brightness difference is photographed with an imaging apparatus such as a digital still camera or a digital video camera, overexposure may occur in bright areas and blackout may occur in dark areas. As one of the techniques for correcting such overexposure and underexposure, a plurality of images with different exposure amounts are photographed, and an area without overexposure and an underexposure area present in each image is synthesized. There is a technique for generating wide dynamic range image data in which bright portions and dark portions are appropriately reproduced. In addition, there is a technique for photographing a plurality of images having different exposure amounts within one field period in order to generate wide dynamic range image data corresponding to a moving image.

  For example, Patent Document 1 discloses a continuous tone image by driving a solid-state imaging device at a normal n-fold speed and supplying n shutter pulses within one field period to finely control the shutter accumulation period. An output is obtained, and these image outputs are stored after being clipped to n field memories after being white clipped, and output in synchronization with the vertical synchronizing pulse VD, and then added by an adder, whereby an image having a wide dynamic range is obtained. Techniques for generating are described.

  Further, in Patent Document 2, by changing the luminance value of a synthesis image necessary for generating a high-quality wide dynamic range image within one field period, not only during still shooting but also during moving image shooting. In addition, a technique for generating a high dynamic range image with high image quality for each field is described.

Japanese Patent Laid-Open No. 9-200261 JP 2012-109849 A

  However, Patent Document 1 has a problem that power consumption increases when the solid-state imaging device is driven at a normal n-fold speed. Further, in Patent Document 2, if a high dynamic range image with high image quality is to be generated for each field even during moving image shooting, it is necessary to read out an image signal at a higher speed than during normal shooting, which increases power consumption. There was a problem.

  The present invention has been made in view of the above-described problems of the related art, and an object of the present invention is to provide an imaging apparatus capable of suppressing an increase in power consumption and performing shooting for obtaining a wide dynamic range image suitable for moving image shooting.

To solve the problems described above, an imaging apparatus according to the invention 1 is based on a plurality of image data having different exposure amounts, in the imaging apparatus for synthesizing image data of a wide dynamic range, it is photoelectrically converted in accordance with an exposure amount An image sensor in which a large number of pixels for generating a pixel signal are arranged, a first exposure amount according to a first exposure time for each predetermined field time, and a time shorter than the first exposure time. An imaging control unit that performs imaging based on a second exposure amount that is smaller than the first exposure amount by a second exposure time, a pixel signal generated from the imaging element, and a reference signal that changes according to time A comparison unit for comparing, and a counter output unit for outputting a count value corresponding to a time until the pixel signal and the reference signal coincide with each other. Convert to value And A / D conversion control unit that includes a image processing unit for combining the image data with a wide dynamic range on the basis of the image data obtained by the photographing, the A / D conversion control unit, the first When shooting is performed based on the exposure amount, the reference signal is set to the first voltage at the end of the A / D conversion and is input to the comparison unit, and the A / D of the pixel signal below the preset first level is set. When only D conversion is performed and shooting is performed based on the second exposure amount, the reference signal is set to the second voltage at the start of A / D conversion, and is input to the comparison unit . Only the A / D conversion of pixel signals of two or more levels is performed, and the first voltage is a pixel output value obtained by photographing with the first exposure time corresponding to the subject brightness corresponding to the first value. , The second voltage has a subject brightness corresponding to a second value, Is the value of pixel output obtained by photographing by light time.

The imaging apparatus according to a second aspect is the imaging apparatus according to the first aspect, wherein the imaging control unit further includes a third exposure time that is a time between the first exposure time and the second exposure time. in it, which performs photographing based on the third exposure amount of the intermediate level of the first exposure and the second exposure, the a / D conversion control unit, the third exposure When shooting based on the above, the reference signal is set to the third voltage at the start of A / D conversion, set to the fourth voltage at the end of A / D conversion, input to the comparison unit, and set in advance. There line a / D conversion only between the fourth-level pixel signal preset third level that is, the third voltage is subject luminance is corresponding to the second value, the third exposure The pixel output value obtained by photographing in time, and the fourth voltage is the first value of the subject brightness. Corresponding to a value of a pixel output obtained by photographing pats during the fourth exposure.

Imaging according to the invention 3 apparatus, the imaging apparatus according to the invention 1 and the present invention 2, the A / D converter control unit and performs A / D conversion of the double integration method.

Imaging according to the present invention 4 apparatus, in the imaging apparatus according to the invention 1, the image data generated by the image processing unit, further Ru comprising a display unit for displaying a moving image to be each the predetermined field time.

  According to the present invention, it is possible to provide an imaging apparatus capable of performing shooting for suppressing an increase in power consumption and obtaining a wide dynamic range image suitable for moving image shooting.

It is a block diagram which shows the structure of the imaging device common to Embodiment 1 and Embodiment 2 of this invention. It is a block diagram which shows schematic structure of the image pick-up element common to Embodiment 1 and Embodiment 2 of this invention. It is a figure which shows the circuit structure of 1 pixel of the image pick-up element common to Embodiment 1 and Embodiment 2 of this invention. It is a block diagram which shows the structure of the A / D converter common to Embodiment 1 and Embodiment 2 of this invention. 5 is a timing chart when a signal of one pixel is read from the image sensor and A / D conversion is performed in normal photographing. 6 is a timing chart when a signal of one pixel is read from the image sensor and A / D conversion is performed by underexposure shooting in the first and second embodiments of the present invention. 6 is a timing chart when a signal of one pixel is read from an image sensor and A / D conversion is performed by overexposure shooting in Embodiments 1 and 2 of the present invention. It is a timing chart when the signal of 1 pixel is read from an image sensor, and A / D conversion is performed by standard exposure photography in Embodiment 2 of the present invention. 6 is a timing chart showing the relationship between a field synchronization signal and a pixel selection signal φS during normal shooting. 6 is a timing chart showing a relationship between a field synchronization signal and a pixel selection signal φS in Embodiments 1 and 2 of the present invention. It is a figure which shows the relationship between the image data image | photographed by the underexposure in Embodiment 1 of this invention, and the image data image | photographed by overexposure, and object brightness | luminance. It is a conceptual diagram of the synthetic | combination process of wide dynamic range image data in Embodiment 1 of this invention. It is a figure which shows the relationship between the image data image | photographed by the image data image | photographed by the underexposure in Embodiment 2 of this invention, the image data image | photographed by the standard exposure, and the overexposure, and object brightness | luminance. It is a conceptual diagram of the synthetic | combination process of wide dynamic range image data in Embodiment 2 of this invention. It is a figure which shows the flow of the whole operation | movement of the imaging device in Embodiment 1 of this invention. It is a figure which shows the flow of the whole operation | movement of the imaging device in Embodiment 2 of this invention.

[Embodiment 1]
FIG. 1 is a block diagram illustrating a configuration of the imaging apparatus. This imaging apparatus includes a lens 101, a motor 102 that drives the lens 101, a focus control unit 103, an aperture mechanism 104, a motor 105, an aperture control unit 106, a shutter mechanism 107, a plunger 108, a plan, Jar control unit 109, image sensor 110, AE processing unit 112, AF processing unit 113, image processing unit 114, LCD driver 115, LCD 116, nonvolatile memory 117, built-in memory 118, and compression / decompression A unit 119, a removable memory 120, a CPU 121, an input unit 122, a power source unit 123, and a data bus 124.

  The lens 101 is for forming an optical image of a subject on the image sensor 110. The motor 102 drives the lens 101. The focus control unit 103 is for driving the lens 104 to the in-focus position by the motor 102. The aperture mechanism 104 limits the aperture diameter of the subject light beam that has passed through the lens 101. The motor 5 drives the diaphragm mechanism 104 so that the diaphragm has a predetermined size. The aperture control unit 106 controls the motor 105. The shutter mechanism 107 is for opening and closing the shutter to allow the subject light to pass or shield the image sensor 110. The plunger 108 is for driving the shutter mechanism 107. The plunger control unit 109 is for driving and controlling the plunger 108.

  The imaging element 110 is for converting an optical image received on the imaging surface into an electrical signal and generating an image signal. The image sensor 110 includes an A / D converter 203 (see FIG. 2) arranged for each pixel column. The analog image signal read from each pixel of the image sensor 110 is converted into a digital signal by the A / D converter 203, and the digital image data (hereinafter, a digital signal corresponding to an analog “image signal”) is output from the image sensor 110. Is simply referred to as “image data”). The AE processing unit 112 calculates an exposure time and an aperture value so that the exposure level is appropriate.

  The AF processing unit 113 detects the focus state of the subject based on the high frequency component of the image data output from the image sensor 110. The image processing unit 114 performs various types of image processing such as synchronization processing of image data read from the image sensor 110, gradation conversion processing, white balance adjustment, edge processing, and wide dynamic range image data synthesis processing. The LCD 116 is for displaying a captured wide dynamic range image and other information. The LCD driver 115 is for driving the LCD 116. The nonvolatile memory 117 is for storing various programs and user setting data. The built-in memory 118 is a memory capable of high-speed writing / reading, and temporarily stores image data read from the image sensor 110. The built-in memory 118 is used as a work memory for various processes in the image processing unit 114.

  The compression / decompression unit 119 is for performing decompression processing for compressing the image data and returning the compressed image data to the image data before compression. The detachable memory 120 is a non-volatile memory such as a card memory for recording image data, and is detachable from the camera. The CPU 121 is for overall control of the entire imaging apparatus. The CPU 121 has a function as an imaging control unit that controls the exposure amount in moving image shooting and still image shooting and controls the imaging timing. Further, the CPU 121 constitutes a part of the A / D conversion control unit. The input unit 122 is for inputting instructions for various operations such as various mode settings and release operations of the imaging apparatus. The power supply unit 123 is for supplying power to the entire imaging apparatus. The data bus 124 is a bus line for transmitting / receiving various data.

  FIG. 2 is a block diagram illustrating a schematic configuration of the image sensor 110. In the imaging unit 201, pixels P11 to Pmn of n rows and m columns are arranged. A column parallel A / D conversion type A / D converter 203 (A / D1 to A / Dm) is arranged corresponding to each pixel column. The vertical scanning circuit 204 is connected to each pixel row, selects pixels sequentially for each row from the pixel row 1 to the pixel row n, and outputs the selected pixel signal to the A / D converter. This is a circuit for outputting scanning signals (φS, φR, φT). The vertical scanning circuit control unit 205 is a control circuit that is connected to the vertical scanning circuit 204 and controls the timing at which a vertical scanning signal is output to each pixel row. Note that the vertical scanning circuit control unit 205 may cause the CPU 121 to have part or all of the functions.

The ramp wave generation circuit 206 is a circuit that outputs a stepped ramp wave necessary for column-parallel digital CDS (Correlated Double Sampling) A / D conversion. The A / D converter 203 is connected to the horizontal readout circuit 207. The horizontal readout circuit 207 is a circuit for converting the image data output from the A / D converter 203 in parallel for each pixel row into a serial signal and outputting the serial signal to the outside of the image sensor 110. The A / D converter 203 and the ramp wave generation circuit 206 constitute a part of the A / D conversion control unit. Details of the A / D conversion will be described later.

  FIG. 3 is a diagram showing a circuit configuration of one pixel in the image sensor 110 shown in FIG. In FIG. 3, PD (Photo Diode) is a photoelectric conversion unit, and FD (Floating Diffusion) is a signal storage unit that temporarily holds photocharges generated by the photoelectric conversion unit PD. Here, the signal storage unit FD is shielded from light so that the signal held in the signal storage unit FD does not change even when light is incident on the pixel unit 202.

  The transistor Tr1 is a transistor that also functions as a reset unit that resets the photoelectric conversion unit PD and a gate unit that transfers charges accumulated in the photoelectric conversion unit PD to the signal storage unit FD, and is controlled by the charge transfer signal φT. .

Tr2 is an amplifying transistor that functions as an amplifying unit, and constitutes a source follower amplifier. Signal storage section FD accumulated signal V _SF in is amplified by the amplifying transistor Tr2, is output to the vertical signal line Lv via the selection transistor Tr3 functioning as a signal readout section. The selection transistor Tr3 is controlled by a pixel selection signal φS.

  Tr4 is a transistor that functions as a reset unit that resets the photoelectric conversion unit PD and the signal storage unit FD, and is controlled by a pixel reset signal φR.

Next, the configuration of the A / D converter 203 will be described with reference to FIG. The basic configuration of the A / D converter 203 is a known as a column parallel digital CD S scheme (e.g., technical literature: CX-PAL71 No., listed on Sony Corporation).

The A / D converter 203 includes a comparator 401, a latch circuit 402, and a counter 403. The input of the comparator 401, and the vertical signal line Lv for transmitting a signal _{V SF} which is accumulated in the FD in FIG. 2, outputs the reference signal _{V RAMP} to be compared with the pixel signal _{V SF} of the vertical signal line _{L V} Connected to the signal line. The latch circuit 402 is connected to a counter output unit for outputting the count value of the counter 403 and an output unit of the comparator 401. The counter 403, a signal line for supplying a counter clock signal phi _{signal CTCK,} counter 403 signal line for supplying a counter reset signal phi _CR for resetting the and the up-counter the counter 403, or any of the down counter whether the signal line for supplying a count direction signal phi _UD to switch to is connected.

The reference signal V _RAMP is generated by the ramp wave generation circuit 206 (see FIG. 2). The counter clock signal φ _CTCK , the counter reset signal φ _CR , and the count direction signal φ _UD are generated by the vertical operation circuit control unit 205 and the vertical scanning circuit 204. The ramp wave generation circuit 206, the vertical operation circuit control unit 205, and the vertical scanning circuit 204 constitute a part of the A / D conversion control unit.

  Next, the operation of the first embodiment will be described.

  In the first embodiment, the same subject is photographed twice while changing the exposure time as a parameter for controlling the exposure amount, and two frames of image data with different exposure amounts are combined to generate wide dynamic range image data.

  First, the overall operation flow of the imaging apparatus will be described with reference to the timing chart of FIG.

  The field synchronization signal in FIG. 15 is a synchronization signal serving as a reference for image signal readout timing. An image signal that has been exposed for one field period (described as exposure time (long) in FIG. 15) is read out by electronic rolling shutter control as an image signal by overexposure. The read image signal is A / D converted and temporarily stored in the built-in memory 118 as overexposure image data.

  Next, all the photoelectric conversion units PD in the first pixel row are reset at a predetermined timing in the middle of one field period. The photoelectric conversion unit PD is reset by simultaneously applying pulses to the pixel reset signal φR and the charge transfer signal φT in FIG. 3 to simultaneously turn on Tr4 and Tr1. Similar operations are sequentially repeated until the last pixel row. The dotted diagonal lines in FIG. 15 indicate this operation. Then, in synchronization with the next field synchronization signal, a pulse is applied to the charge transfer signal φT of the first pixel row to transfer the pixel signal accumulated in the photoelectric conversion unit PD to the FD, and then the selection transistor Tr3 is turned on. The pixel signal is read by turning on. Similar operations are sequentially repeated until the last pixel row. This is indicated by the solid line adjacent to the dotted line in FIG. Here, the time from the oblique line indicated by the dotted line to the oblique line indicated by the adjacent solid line corresponds to the exposure time. The read image signal by the short exposure is A / D converted and temporarily stored in the built-in memory 118 as underexposure image data.

  The pixel readout control as described above is generally referred to as electronic rolling shutter control.

  Of the image data stored in the built-in memory 118, image data obtained by underexposure undergoes a predetermined level conversion described later. Next, the image processing circuit 114 performs image processing for synthesizing wide dynamic range image data, which will be described later, based on the image data by underexposure and the image data by overexposure.

  The wide dynamic range image data synthesized by the image processing circuit 114 is subjected to compression processing by the compression / decompression unit 119 and then recorded in the removable memory 120. An image corresponding to the wide dynamic range image data synthesized by the image processing circuit 114 is displayed on the LCD 116.

  In FIG. 15, one field period is an overexposure exposure time. However, an exposure time other than the one feel period may be set using an electronic rolling shutter in the same manner as the underexposure exposure time. The exposure time shown in FIG. 15 is only an example. In FIG. 15, generation of moving image data has been described, but underexposure image data immediately after the release switch is detected and wide dynamic range image data obtained by combining overexposure image data are used as still image data. It may be recorded.

  Next, this embodiment will be described in detail.

The timing chart of FIG. 5, for comparison with the operation of the A / D variant exchanger 203 in this embodiment described below, shows the operation of the A / D converter 203 at the time of normal photographing.

In FIG. 5, during a period when the pixel selection signal φS is “H”, the signal V _SF accumulated in the signal accumulation unit FD is output to the signal output line LV and supplied to one input of the comparator 401. When the pulse of the pixel reset signal phi _R is applied to the gate of the transistor Tr4, a voltage signal accumulation unit FD is reset to the voltage VDD. V _SF of FIG. 5 is increased to simultaneously reset voltage and the application of phi _R.

Next, the A / D converter 203 to a counter reset signal phi _CR is applied, the counter 403 is reset (count value is 0) is the. At this time, the count direction signal φ _UD is set to the down count. Next, the counter clock signal φ _CTCK is output to the counter 403, and at the same time, a stepped ramp wave as the reference signal V _RAMP is supplied to the other input of the comparator 401. The counter 403 starts down-counting, and digital data (counter output) representing the count number is output to the latch circuit 402. When the reference signal V _RAMP decreases with time and coincides with V _SF , the output φ _COUT of the comparator 401 is inverted. Next, in response to this change in VCOUT, the counter 403 stops counting. Here, the counter 403 holds the count value finally counted. The digital data held in this counter corresponds to the reset signal _VRST .

When Nrst number of pulses are output, the counter clock φ _CTCK is stopped to have a constant steady value. The number of pulses Nrst may if slightly larger analog voltage the number of bits that can be converted into digital data from the reset voltage V _RST.

Next, when the charge transfer signal φT is applied to the gate of the transistor Tr1, the signal charge stored in the photoelectric conversion unit PD is transferred to the signal storage unit FD for a predetermined exposure time. The pixel signal V _SIG shown in FIG. 5 corresponds to the voltage due to the transferred signal charge. The reference signal V _RAMP is reset in synchronization with the pulse output of φT. Further, in synchronization with the output of .phi.T, counting direction phi _UD is switched to the up counter. Further, the output phi _COUT of the comparator 401 becomes substantially synchronous with inverted "H" level to the phi _R. Subsequently, when the reference signal V _RAMP decreases with time and coincides with V _SF , the output φ _COUT of the comparator 401 is inverted. The latch circuit 402 receives the inversion of the output signal of the comparator 401 and latches digital data representing the count number of the counter 403. Further, the counting operation of the counter 402 is stopped. Here, the digital data latched by the latch circuit 402 corresponds to addition data of the pixel signal V _SIG and the reset signal V _RST .

When N (sig + rst) pulses are output, the counter clock φ _CTCK is stopped and becomes a constant steady value. The number of pulses N (sig + rst) may be any number of bits that can convert an analog voltage slightly larger than the voltage obtained by adding the reset voltage V _RST and the pixel signal V _SIG into digital data.

As a result of the A / D conversion operation described above, the count result finally latched in the latch circuit 402 corresponds to the pixel signal V _SIG obtained by subtracting V _RST from the addition data of the pixel signal V _SIG and the reset signal V _RST. Is equal to the digital pixel data. Then, the pixel data latched by the latch circuit 402 is converted into serial data from the image sensor 110 together with pixel data of the same line in other columns via the horizontal readout circuit 207 (see FIG. 1) of the image sensor. Read out. The A / D conversion method described above is generally called a double integration method.

FIG. 5 shows an A / D conversion operation of one pixel belonging to a predetermined pixel row. All the pixels belonging to the predetermined pixel row are simultaneously parallelized by A / D converters arranged for each pixel column. Therefore, the same A / D conversion operation is executed. When the A / D conversion of the first pixel signal V _SIG is completed, the same A / D conversion is executed for the next pixel row. The above operation is performed until the reading of the pixel signal V _SIG of the pixel in the last pixel row is completed.

  FIG. 9 is a timing chart showing the relationship between the field synchronization signal and the pixel selection signal φS during normal shooting.

  The readout time for this image data is set to approximately one field period. The read image data is subjected to image processing such as synchronization processing, gradation processing, and white balance processing by the image processing unit 114, and then displayed on the LCD 116 as a moving image, or compressed by the compression / decompression unit 119. Is recorded in the removable memory 120 as moving image data or still image data.

6, when shooting in the so-called APEX (Additive system of Photographic EXposure) smaller exposure than the standard exposure based on calculation (hereinafter referred to as "under-exposure".), Timing illustrating the operation of the A / D variant exchanger 203 It is a chart. Since the basic operation of FIG. 6 is the same as that of FIG. 5, only the differences from FIG. 5 will be described below.

6 is the same as FIG. 5 _{except that} the value of the reference signal V _RAMP at the start of up-counting is different from that in FIG. In underexposure, the ramp generation circuit 206 sets the voltage of V _RAMP at the start of up-counting to a value lower by V _START1 than the voltage at the time of normal photographing in FIG. Now, it is assumed that the down-counting is completed in the same manner as in FIG. 5, and the counter 403 holds a count value corresponding to the reset signal _VRST . Next, when the reference signal V _RAMP decreases with time and this V _RAMP and V _SF match, the output φ _COUT of the comparator 401 is inverted. In response to the inversion of this signal, the latch circuit 402 latches digital data representing the count number of the counter 403. Further, the counting operation of the counter 402 is stopped.

  The pixel data latched by the latch circuit 402 is converted into serial data and read out from the image sensor 110 together with the pixel data of the same line in other columns through the horizontal readout circuit 207 (see FIG. 1) of the image sensor. .

Above-described A / D conversion result of the operation, finally latched counted result to the latch circuit 402, the pixel signal _{V SIG} is _{V START1} Hereinafter 0, the pixel signal _{V SIG} is _{V START1} or _{more, V} SF (= It is equal to pixel data (= V _SIG -V _START1 ) obtained by subtracting V _START1 + V _RST from V _SIG + V _RST ).

Therefore, in under-exposure shooting, the image composition processing circuit 114 keeps the output data of the A / D converter 203 when it is 0, and _sets the digital value corresponding to V _START1 for the other data. Data conversion processing to be added is performed. Thereby, a correct pixel signal V _SIG of a pixel that outputs V _SF larger than V _START1 can be obtained. A thick solid line (underexposure) in FIG. 11 for explaining an image composition process to be described later indicates the relationship between the image data V _{SIG_AD on} which the level conversion of this data has been performed and the subject brightness. For example, 12-bit MSB of the A / D converter, _1V input analog voltage of the A / D converter corresponding to the _MSB, the the _{V START1} and 0.3V, digital values corresponding to _{V START1} is 2 ^ 12 * 0.3 / 1 = 1229, and the downcount time is shortened by about 30% compared to normal shooting.

FIG. 6 shows an A / D conversion operation for one pixel. Similar A / D conversion operations are simultaneously performed in parallel by an A / D converter arranged for each pixel column for all pixels belonging to each pixel row. Is executed. When the A / D conversion of the pixel signal V _{SIG in} the first row is completed, the same A / D conversion is executed for the next pixel row. The above operation is performed until the reading of the pixel signal V _SIG of the pixel in the last pixel row is completed.

  The image signal of one field imaged and read by under-exposure in this way is temporarily stored in the built-in memory 118.

  FIG. 7 is a timing chart showing the operation of the A / D converter 203 when photographing with exposure larger than the standard exposure based on the APEX calculation (hereinafter referred to as “overexposure”). Since the basic operation of FIG. 7 is the same as that of FIG. 5, only the differences from FIG. 5 will be described below.

7 is the same as FIG. 5 _{except that} the value of the reference signal V _RAMP at the end of the up-count is different from that in FIG. In _overexposure , the maximum value of V _RAMP at the end of the up-count is set to a value V _END1 higher than the voltage at the time of normal photographing in FIG. When the signal V _SF stored in the FD is smaller than V _END1 , the output φ _COUT of the comparator 401 is inverted when the reference signal V _RAMP decreases with time and coincides with V _SF . The latch circuit 402 receives the inversion of the output signal of the comparator 401 and latches digital data representing the count number of the counter 403. Further, the counting operation of the counter 402 is stopped. On the other hand, when the signal _{V SF} which is accumulated in the FD is greater than _{V END1 is,} when _{the V RAMP} reaches _{V END1,} the ramp generating circuit _206, obtained forcibly taken pixel signal _{V SIG} is a _{V RAMP} Set to a value greater than the maximum value. Then, the output φ _COUT of the comparator 401 is forcibly inverted, whereby the counter output of the counter 403 is latched by the latch circuit 402.

  The pixel data latched by the latch circuit 402 is converted into serial data and read out from the image sensor 110 together with the pixel data of the same line in other columns through the horizontal readout circuit 207 (see FIG. 1) of the image sensor. .

As a result of the A / D conversion operation described above, the count result finally latched in the latch circuit 402 is a pixel signal obtained by subtracting V _SF (the sum of the pixel signal V _SIG and the reset signal V _RST ) and V _RST. Equal to V _SIG . _{Incidentally,} when _{V SF} is _{V END1} larger value, forcibly _{V SF} the latch circuit 402 is count value corresponding to a certain _{V END1} is latched.

That is, the output data of the A / D converter 203 in FIG. _7, when _{V SF} is less than _{V END1} is image data corresponding to _{V SIG,} when _{V SF} is greater than _{V END1} is commensurate with _{V END1} Image data. A thin solid line ( _overexposure ) in FIG. 11 for explaining an image composition process to be described later indicates a relationship between the image data V _{SIG_AD subjected} to the A / D conversion as described above and the subject luminance. For example, 12-bit MSB of the A / D converter, _1V input analog voltage of the A / D converter corresponding to the _MSB, the the _{V END1} to 0.6V, digital values corresponding to _{V END1} is 2 ^ 12 * 0.6 / 1 = 2458, and the up-count time is shortened to about 60% of the normal shooting time.

FIG. 7 shows an A / D conversion operation for one pixel. Similar A / D conversion operations are simultaneously performed in parallel by an A / D converter arranged for each pixel column for all pixels belonging to each pixel row. Is executed. When the A / D conversion of the first pixel signal V _SIG is completed, the same A / D conversion is executed for the next pixel row. The above operation is performed until the reading of the pixel signal V _SIG of the last pixel row is completed.

  In this way, the image signal of one field imaged and read by overexposure is temporarily stored in the built-in memory 118. Then, the image processing circuit 114 combines the image data generated by the underexposure shooting stored in the built-in memory 118 and the image data generated by the overexposure shooting to synthesize a wide dynamic range image described below. Processing is performed.

FIG. 11 is a diagram showing the relationship between image data captured by underexposure, image data captured by overexposure, and subject brightness. In the figure, the horizontal axis represents the luminance of the subject, and the vertical axis represents the value of the image data (the value _obtained by _{subjecting the} image signal V _SIG to A / D conversion and level conversion processing, and is referred to as V _{SIG_AD} ). In the case of underexposure shooting, as shown by the thick solid line in FIG. 11, V _{SIG_AD} is 0 when V _SIG is V _START1 or less (subject luminance is BV _STRT1 or less), as described above. Further, when V _SIG is _{equal to} or higher than V _START1 (subject brightness is _{equal to} or higher than BV _START1 ), when the subject brightness is BV _START1 , V _{SIG_AD} = V _START1 is set to be proportional to the subject brightness.

On the other hand, in the case of shooting by overexposure, as shown by a thin solid line in FIG. 11, when V _SIG is V _END1 or less (subject luminance is BVEND or less), V _SIG increases in proportion to the subject luminance, and V _SIG is V _END1. In the case of the above (subject luminance is BV _END1 or more), it is a constant value (= V _END1 ).

FIG. 12 is a conceptual diagram of the synthesis process for explaining the synthesis process of the wide dynamic range image. In the figure, the thick solid line in the straight line (a) is obtained by photographing the value of image data having a value equal to or greater than _VSTART1 among the image data photographed by underexposure under the same exposure conditions as the _overexposure photographing. It is a straight line converted into a value when assumed. For example, when underexposure shooting (shutter speed: T1) and overexposure shooting (shutter speed: T2) are performed while changing only the shutter speed, the image data obtained by underexposure shooting is multiplied by K (K = T2 / T1).

Here, among the image data represented by the straight line (a) in FIG. 12, for the image data corresponding to the subject brightness between BV _START1 and BV _END1 , either the underexposure shooting or the _overexposure shooting is used. Select data.

  As is clear from FIG. 12, the maximum value of the composite image data exceeds the maximum value Max of the image data, so that a knee point is set and converted to a gradation characteristic obtained by compressing the high luminance portion. The image data converted according to the curve (b) in FIG. 12 is compressed by the compression / decompression unit 119 and then recorded in the removable memory 120. The image data converted according to the curve (b) in FIG.

Further, as is apparent from FIG. 12, in underexposure shooting and overexposure shooting, overlapping portions of subject brightness are required. The necessary and sufficient condition is that BV _START1 <BV _END1 . Therefore, V _START1 and V _END1 are set so that the above inequality is satisfied.

In the above description, among the image data represented by the straight line (a) in FIG. 12, the image data corresponding to the subject brightness between BV _START1 and BV _END1 is _subjected to underexposure shooting or _overexposure shooting. Although either one of the image data is selected, an addition average value of the image data obtained by underexposure photography and the image data obtained by overexposure photography may be employed.

  In the above description, when converting the image data obtained by underexposure shooting to image data for overexposure, the image data obtained by underexposure shooting is multiplied by the exposure time ratio, and data conversion is performed. Image quality may be deteriorated due to an error in element or circuit characteristics. In order to solve this problem, an accurate exposure amount ratio is obtained by statistical processing (for example, summation) of pixel data in a portion where the subject brightness of underexposure shooting and overexposure shooting overlaps. The image data may be converted into overexposed image data.

According to this embodiment, in the under-exposure shooting may speed up the A / D conversion so is not performed up count for V _START1 following signal voltage of the signal V _SF which is accumulated in the FD. Further, in the over-exposure shooting, it is possible to speed up the A / D conversion so is not performed up count for V _END1 or more signal voltage of the signal V _SF which is accumulated in the FD. FIG. 10 is a timing chart showing the relationship between the field synchronization signal and the pixel selection signal φS in underexposure shooting or overexposure shooting. The field period can be shortened as compared with the normal photographing in FIG.

Therefore, in the present embodiment, A / D conversion can be performed in a short time without driving the image sensor at high speed, so that power consumption can be reduced. In addition, since wide dynamic range image data can be generated at high speed without driving the image sensor at high speed, shooting for obtaining wide dynamic range image data suitable for moving image shooting can be performed.
[Embodiment 2]
Next, Embodiment 2 of the present invention will be described.

  In the first embodiment, wide dynamic range image data is synthesized based on image data that has been captured and read by underexposure and overexposure. On the other hand, this embodiment synthesizes wide dynamic range image data based on image data read by underexposure shooting, standard exposure shooting, and overexposure shooting.

  In the following description, only differences from the first embodiment will be described.

  FIG. 16 shows the overall operation flow of the imaging apparatus.

  In FIG. 16, standard exposure photography is added between underexposure photography and overexposure photography in FIG. 15, and image data obtained by underexposure photography, overexposure photography, and standard exposure photography are combined. Thus, the second embodiment is the same as the first embodiment except that the wide dynamic range image data is synthesized.

  Next, this embodiment will be described in detail.

FIG. 8 is a timing chart showing the operation of the A / D converter 203 when photographing with standard exposure based on the APEX operation. In the present embodiment, standard exposure shooting is performed in addition to the overexposure shooting and underexposure shooting in the first embodiment, and image data of a portion having an intermediate luminance distribution of the subject is generated in the standard exposure shooting. In a standard-exposure shooting, the signal _{V SF} which is accumulated in the FD _is, counts up the signal only between _{V END2} from _{V START2.} Starting operation of the up count, except in this embodiment is a _{V START2} whereas the initial values of _{V RAMP} in Embodiment 1 was _{V START1,} A in the under-exposure shooting in Embodiment 1 / This is the same as the D conversion operation. Further, the operation at the end of the up-count is the same as that of the first embodiment except that the value of V _RAMP at the end of the up-count is V _END1 in the first embodiment, but is V _END2 in the present embodiment. This is the same as the A / D conversion operation in exposure photography.

FIG. 8 shows an A / D conversion operation for one pixel. Similar A / D conversion operations are simultaneously performed in parallel by an A / D converter arranged for each pixel column for all pixels belonging to each pixel row. Is executed. When the A / D conversion of the first pixel signal V _SIG is completed, the same A / D conversion is executed for the next pixel row. The above operation is performed until the reading of the pixel signal V _SIG of the last pixel row is completed.

  Thus, the image signal of one field imaged and read by the standard exposure is temporarily stored in the built-in memory 118. Then, the image processing circuit 114 uses the image data read by the underexposure shooting stored in the built-in memory 118, the image data read by the standard exposure shooting, and the image data read by the overexposure shooting. The image processing for synthesizing the wide dynamic range image data is performed by the image synthesizing processing described below.

FIG. 13 is a diagram illustrating the relationship between image data captured by underexposure, image data captured by standard exposure, image data captured by overexposure, and subject brightness. It is a figure which shows the relationship between to-be-photographed object brightness | luminance and image data. In the figure, the horizontal axis represents the luminance of the subject, and the vertical axis represents the value of the image data (the value _obtained by _{subjecting the} image signal V _SIG to A / D conversion and level conversion processing, and is referred to as V _{SIG_AD} ). In the case of photographing by underexposure, as shown by the solid line at the middle level in FIG. 13, the level of data of V _{SIG_AD} is converted by the image processing circuit 114 as described in the first embodiment. As a result, it is 0 when V _SIG is V _START1 or less (subject brightness is BV _STRT1 or less). Further, when V _SIG is _{equal to} or higher than V _START1 , when the subject brightness is BV _START1 , V _{SIG_AD} = V _START1 is used as a base point and increases in proportion to the subject brightness.

In the case of shooting by overexposure, as shown by a thin solid line in FIG. 11, when V _SIG is V _END1 or less (subject brightness is BV _END or less), the ratio increases in proportion to the subject brightness, and V _SIG is V _END1 or more. When the subject brightness is equal to or higher than BV _END1 , it is a constant value (BV _END1 ).

For imaging by standard exposure, as indicated by the most thick solid line in FIG. _{13, V SIG_AD is} 0 when the following _{V SIG} is _{V START2} (subject brightness _{BV STRT2 less), V SIG} is _{V START2} more and _{V END2} following (subject brightness _{BV STRT2} or higher) (subject brightness _{BV END2} less) when the subject luminance is increased in proportion to the object luminance as a base point _V SIG_AD _{= V START2} when _{BV STRT2.} When V _SIG is equal to or higher than V _END2 (subject brightness is equal to or higher than BV _END2 ), the value is constant (= BV _END2 ).

FIG. 14 is a conceptual diagram of a synthesis process for explaining the synthesis process of a wide dynamic range image. In the figure, a straight line thick line in the middle level of (a), among the image data captured by the underexposure, the image data having a V _START1 -V _RST or more values, the same exposure conditions as shooting overexposure It is a straight line converted into a value when it is assumed that the image was taken at. This conversion method is performed in the same manner as in the first embodiment.

Here, among the image data represented by a straight line (a) of FIG. _{14, BV START2} and _BV image data corresponding to the object luminance between the _END1, and _{BV START1} and an image corresponding to the subject luminance between _{BV END2} As for the data, either one of the underexposure shooting and the overexposure shooting is selected.

  As is clear from FIG. 14, the maximum value of the composite image data exceeds the maximum value Max of the image data, so that a knee point is set and converted to a gradation characteristic obtained by compressing the high luminance part. The image data converted according to the curve (b) of FIG. 14 is recorded in the removable memory 120 after being compressed by the compression / decompression unit 119. The image data converted according to the curve (b) in FIG. 14 is displayed on the LCD 116 via the LCD driver 115.

As is apparent from FIG. 14, in underexposure shooting, standard exposure shooting, and overexposure shooting, overlapping portions of subject luminance are necessary. Its necessity sufficient condition is that the _BVBV START2 _{<BV END1,} and _{BV START1} <BV _{END2. Therefore, V START1, V END1, V} START2, V END2 is set such that the inequality holds.

In the above description, among the image data represented by a straight line (a) of FIG. _14, the image data corresponding to the object luminance between _{BV START1} and _{BV END2,} and _{BV START2} the subject between _{BV END1} For image data corresponding to the brightness, either one of the image data is selected. However, the image data obtained by underexposure photography and the image data obtained by standard exposure photography are added, and the image data obtained by standard exposure photography and overexposure photography are obtained. The addition average value of the image data may be adopted.

  In the above description, when converting the image data obtained by underexposure photography and the image data obtained by standard exposure photography to overexposure image data, the image data is converted by multiplying the exposure time ratio. The image quality may be deteriorated due to an error of the image sensor or circuit characteristics. In order to solve this problem, the exposure amount ratio is accurately obtained by statistical processing (for example, summation) of the pixel data of the portion where the subject brightness of the underexposure shooting or the standard exposure shooting and the overexposure shooting overlap. You may make it convert the image data by underexposure photography into the image data of overexposure by exposure amount ratio.

  Further, the combinations of exposure times shown in FIG. 16 are examples, and various combinations of exposure times are conceivable.

  In this embodiment, standard exposure shooting is performed in addition to the underexposure shooting and overexposure shooting in the first embodiment, and image data based on these three types of exposure is combined to synthesize a wide dynamic range image. Compared to 1, it is possible to obtain wide dynamic range image data in which whiteout and blackout are suppressed even for a subject having a wider luminance range.

In the present embodiment, in underexposure shooting, the A / D conversion is speeded up because no up-counting is performed on signal voltages below V _START1 -V _RST among the signals V _SF accumulated in the FD. be able to. In the standard-exposure shooting, to speed up the A / D conversion so is not performed up count for _V START2 _{-V RST} less and _V END2 _{-V RST} or more signal voltage of the signal _{V SF} which is accumulated in the FD Can do.

Further, in the shooting of the over exposure, it is possible to speed up the A / D converter since the up count for _V END1 _{-V RST} or more signal voltage of the accumulated signal _{V S F} to the FD is not performed.

  Therefore, in the present embodiment, A / D conversion can be performed in a short time without driving the image sensor at high speed, so that power consumption can be reduced. In addition, since wide dynamic range image data can be generated at high speed without driving the image sensor at high speed, shooting for obtaining a wide dynamic range image suitable for moving image shooting can be performed.

In the first embodiment, a wide dynamic range image is synthesized based on image data captured and read by each of underexposure and overexposure. In the second embodiment, a wide dynamic range image is synthesized based on image data that has been shot and read by underexposure, proper exposure, and overexposure. However, the present invention is not limited to, underexposed, proper exposure, by any combination of over-exposure can be applied to an imaging apparatus you synthesize a wide dynamic range image.

Further, the present invention is not limited to the above-described embodiment, and the constituent elements can be modified and embodied without departing from the spirit of the invention in the implementation stage. Various inventions can also be formed by appropriately combining a plurality of constituent elements disclosed in the embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined. Thus, various modifications and applications are possible without departing from the spirit of the invention.

DESCRIPTION OF SYMBOLS 101 ... Lens 102 ... Motor 103 ... Focus control part 104 ... Aperture mechanism 105 ... Motor 106 ... Aperture control part 107 ... Shutter mechanism 108 ... Plunger 109 ... Plunger control part 110 ... Imaging element 112 ... AE process part 113 ... AF process Part 114: Image processing part 115 ... LCD driver 116 ... LCD
117: Non-volatile memory 118 ... Built-in memory 119 ... Compression / decompression unit 120 ... Detachable memory 121 ... CPU
122 ... Input unit 123 ... Power supply unit 124 ... Data bus

Claims (4)

In an imaging device that combines image data with a wide dynamic range based on a plurality of image data with different exposure amounts,
An imaging device in which a large number of pixels for generating a pixel signal photoelectrically converted according to an exposure amount are arranged;
For each predetermined field time , a second exposure time that is smaller than the first exposure amount due to the first exposure amount according to the first exposure time and the second exposure time that is shorter than the first exposure time . An imaging control unit that performs imaging based on the exposure amount;
A comparison unit that compares the pixel signal generated from the image sensor with a reference signal that changes according to time, and a counter output that outputs a count value according to the time until the pixel signal matches the reference signal An A / D conversion control unit that performs A / D conversion on the pixel signal to convert the pixel signal into a digital value ;
An image processing unit that synthesizes image data of a wide dynamic range based on the image data obtained by the shooting,
Have
The A / D conversion control unit sets the reference signal to the first voltage at the end of the A / D conversion and inputs the first reference signal to the comparison unit when photographing based on the first exposure amount. When only A / D conversion of the pixel signal of the first level or less is performed and shooting is performed based on the second exposure amount, the reference signal is set to the second voltage at the start of A / D conversion. Input to the comparison unit, and perform only A / D conversion of a pixel signal of a second level or higher set in advance,
The first voltage is a pixel output value obtained by photographing with the first exposure time corresponding to the first value of the subject brightness,
The second voltage is a pixel output value obtained by photographing with the second exposure time, in which the subject brightness corresponds to the second value.
An imaging apparatus characterized by that. The imaging control unit further includes a third exposure time that is a time between the first exposure time and the second exposure time, the first exposure amount and the second exposure amount. Shooting is performed based on the third exposure amount at the intermediate level,
The A / D conversion control unit sets the reference signal to the third voltage at the start of A / D conversion, and sets the reference signal at the end of the A / D conversion when shooting based on the third exposure amount . 4 is set to a voltage input to the comparator unit, have rows of a / D conversion only between the fourth-level pixel signal set in advance a preset third level,
The third voltage is a pixel output value obtained by photographing with the third exposure time, in which the subject brightness corresponds to the second value,
The fourth voltage is a value of a pixel output obtained by photographing without exposure at the time of the fourth exposure, in which the subject brightness corresponds to the first value.
The imaging apparatus according to claim 1. The A / D conversion control section imaging apparatus according to claim 1 or claim 2, characterized in that for performing A / D conversion of the double integration method. The imaging apparatus according to claim 1, further comprising a display unit that displays the image data generated by the image processing unit as a moving image for each predetermined field time .

Images