JP2011055327A - Photographing device and method for controlling the same - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To obtain a wide dynamic range image even at the time of flash photographing. <P>SOLUTION: A plurality of pairs of first and second light receiving elements whose exposure periods can be controlled independently are arranged in a photographing device. During an exposure period tL of the first light receiving element, by using a solid-state image sensing device with which the exposure of the second light receiving element for an exposure period tS that is shorter than the exposure period tL can be performed, the exposure timing t1 and t3 of the second light receiving element to a flash light emission period tF (= t0 to t2) are determined so that the first and second light receiving elements receives light whose light amount corresponds to a ratio of the exposure periods tL and tS. Image data having a wide dynamic range corresponding to the ratio of the exposure periods tL and tS can be generated by combining output signals of the first and second light receiving elements. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an imaging apparatus including a solid-state imaging device such as a CCD or a CMOS sensor and a control method thereof, and more particularly to an imaging apparatus capable of widening an image and a control method thereof.

  An imaging device such as a digital camera incorporating a solid-state imaging device such as a CCD or a CMOS sensor has become widespread. Typical demands for solid-state image sensors include higher pixels and a wider dynamic range. Regarding the increase in the number of pixels, the number of pixels of 10 million pixels or more has been achieved by miniaturization of the light receiving element, and the demand is almost satisfied. On the other hand, the wide dynamic range is accompanied by a decrease in the amount of saturation charge due to the miniaturization of the light receiving element, and therefore there is a limit to the wide dynamic range in the structural improvement of the light receiving element. For this reason, an additional technique is required for wide dynamic range.

  As an additional technique, in a solid-state imaging device in which a plurality of first light receiving elements and second light receiving elements whose exposure periods can be controlled independently are arranged in pairs, the first light receiving elements are exposed for a long time (high Sensitivity), the second light receiving element is exposed for a short period (low sensitivity), the second light receiving element is exposed during the exposure period of the first light receiving element, and the output signals of the first light receiving element and the second light receiving element are synthesized. A technique for obtaining an image having a wide dynamic range and ensuring simultaneity by performing (addition with uniform saturation voltages) has been proposed by the present applicant (see Patent Document 1).

  According to this technique, a desired dynamic range can be obtained by selecting a ratio between the exposure period of the first light receiving element and the exposure period of the second light receiving element. On the other hand, when the wide dynamic range is not required, the exposure periods of the first and second light receiving elements are made the same, and the output signals of the first and second light receiving elements are made into individual pixel data without being synthesized. Thus, a high resolution image can be obtained.

  Further, in Patent Document 1, in order to realize a wide dynamic range at the time of flash photography, flash light emission (light emission of an auxiliary light source) is performed during a period outside the short-term exposure period and within the long-term exposure period. It has been proposed. It is stated that by performing flash emission at this timing, the amount of light (integrated exposure energy) for long-term exposure of the first light receiving element can be increased without affecting the short-term exposure of the second light receiving element. ing.

JP 2007-235656 A

  However, as described in Patent Document 1, when flash emission is performed during a period outside the short-term exposure period and within the long-term exposure period, the amount of flash emission is the first light-receiving element for the long-term exposure. It is determined based on the exposure period. In this case, the first light receiving element that is exposed for a long period of time can obtain an appropriate amount of light, but the second light receiving element that is exposed for a short period is insufficient. Therefore, the technique described in Patent Document 1 has a problem that a desired dynamic range cannot be obtained during flash photography.

  The present invention has been made in view of the above-described problems, and an object of the present invention is to provide a photographing apparatus capable of obtaining an image with a wide dynamic range even during flash photographing and a control method therefor.

  In order to achieve the above object, a photographing apparatus according to the present invention includes a plurality of pairs of flash light emitting means for emitting flash light, and a first light receiving element and a second light receiving element capable of independently controlling the exposure period. Imaging means that enables exposure of the second light receiving element in a second exposure period shorter than the first exposure period during the first exposure period of the first light receiving element; The relative timing relationship between the flash emission period of the flash emission unit and the second exposure period is controlled so that the two light receiving elements respectively receive the light amounts corresponding to the ratio between the first and second exposure periods. Control means and signal processing means for synthesizing the output signals of the first and second light receiving elements to generate image data are provided.

  The control unit synchronizes the flash light emission period with the first exposure period, and the first and second light receiving elements respectively receive light amounts corresponding to a ratio of the first and second exposure periods. In addition, it is preferable to control the timing of the second exposure period with respect to the flash emission period. In this case, it is preferable that the control unit simultaneously sets the start time of the flash light emission period and the first exposure period.

  Further, the control means synchronizes the second exposure period with the first exposure period, and the first and second light receiving elements respectively receive light amounts corresponding to a ratio of the first and second exposure periods. Thus, it is also preferable to control the timing of the flash emission period with respect to the second exposure period. In this case, it is preferable that the control means simultaneously sets the end times of the first exposure period and the second exposure period.

  Further, the control means causes the flash light emitting means to perform preliminary light emission, calculates a light emission amount for main photographing based on reflection luminance information of the flash light detected by the imaging means, and based on the calculated light emission amount. It is preferable to determine the emission time of the flash light.

  In addition, the control means includes a first table that stores a light emission time with respect to the light emission amount for a plurality of the light emission amounts, and determines the light emission time by referring to the first table, It is preferable to determine a relative timing relationship between the flash emission period and the second exposure period based on a ratio of the second exposure period. Further, it is preferable that the control means calculates a light emission time with respect to a light emission amount not stored in the first table by an interpolation calculation.

  In addition, the control means may determine the timing relationship in which the ratio of the light emission amount during the second exposure period to the light emission amount during the first exposure period is a ratio of the first and second exposure periods. It is preferable to further include a second table that stores the ratio, and to determine the timing relationship corresponding to the ratio by referring to the second table. Furthermore, it is preferable that the control means calculates the timing relationship with respect to the ratio that is not stored in the second table by interpolation calculation.

  The control means determines the light emission time based on the reflected luminance information obtained from the first light receiving element, and determines the timing relationship based on the reflected luminance information obtained from the second light receiving element. It is also preferable.

  In addition, it is preferable that a recording unit that records the flash emission period and the determined value of the timing relationship in association with the image data is provided.

  The first light receiving elements are arranged in a square lattice pattern in a row direction and a column direction orthogonal thereto, and the second light receiving elements are arranged in a square lattice pattern in a row direction and a column direction orthogonal thereto. It is preferable that the first light receiving element and the second light receiving element are arranged at the same arrangement pitch and shifted from each other in the row direction and the column direction by a half of the arrangement pitch.

  Further, the imaging means has a Bayer array of red, green, and blue color filters corresponding to the first light receiving elements, and red, green, and blue color filters corresponding to the second light receiving elements. A Bayer arrangement is preferred.

  Further, the image pickup means is arranged along the column in which the first light receiving elements are arranged and the column in which the second light receiving elements are arranged, and the signal charges accumulated in the first and second light receiving elements. The CCD type solid-state imaging device is preferably provided with a vertical charge transfer unit that reads out and transfers in the column direction and a horizontal charge transfer unit that transfers signal charges from the vertical charge transfer unit in the row direction.

  Further, in the control method of the photographing apparatus of the present invention, a plurality of flash light emitting means for emitting flash light and a plurality of first light receiving elements and second light receiving elements capable of independently controlling the exposure period are arranged in pairs. An imaging unit capable of performing exposure of the second light receiving element in a second exposure period shorter than the first exposure period during the first exposure period of the first light receiving element; and the first and second light receiving elements And a signal processing unit that generates image data by synthesizing the output signals of the first and second light receiving elements, wherein the first and second light receiving elements correspond to a ratio of the first and second exposure periods. The relative timing relationship between the flash light emission period of the flash light emission means and the second exposure period is controlled so as to receive light respectively.

  According to the present invention, the relative light emission period and the second exposure period of the flash light emitting unit are such that the first and second light receiving elements respectively receive the light amounts corresponding to the ratio of the first and second exposure periods. Therefore, a wide dynamic range image can be obtained even during flash photography.

1 is a block diagram illustrating a configuration of a digital camera according to a first embodiment of the present invention. It is a plane schematic diagram of a solid-state image sensor. It is a timing chart explaining the drive method of a solid-state image sensor and a flash light emission part. It is a schematic diagram of image data explaining an image composition process. It is a flowchart explaining the effect | action of a digital camera. It is a flowchart explaining a light control process. It is a figure which shows an example of the area division | segmentation of image data. It is a figure which shows an example of the weighting coefficient of each division area. It is a figure which shows the table which matched various light emission amounts and light emission time. It is a figure which shows the waveform of the flash light with respect to a flash drive pulse. It is a table concerning a 2nd embodiment of the present invention. It is a timing chart explaining the drive method of the solid-state image sensor and flash light emission part which concern on 4th Embodiment of this invention. It is a flowchart explaining the light control process which concerns on 4th Embodiment of this invention. It is a block diagram which shows the structure of the digital camera which concerns on 5th Embodiment of this invention. It is a timing chart explaining the drive method of the solid-state image sensor, flash light emission part, and mechanical shutter which concern on 5th Embodiment of this invention.

(First embodiment)
In FIG. 1, the imaging optical system of the digital camera 10 according to the first embodiment of the present invention includes a photographing lens 11, a CCD type solid-state imaging device 12, and a diaphragm 13 provided therebetween. A lens driving unit 14 is connected to the photographing lens 11. An image sensor driving unit 15 is connected to the solid-state image sensor 12. A diaphragm driving unit 16 is connected to the diaphragm 13.

  The CPU 17 performs overall control of the entire electric control system of the digital camera 10 based on an operation signal from the operation unit 18. Further, the CPU 17 includes a light emission control unit 17a that controls the light emission timing and light emission amount of the flash light emission unit 19, a focus adjustment unit 17b that instructs the lens driving unit 14 to move the photographing lens 11, and performs focusing. An exposure adjustment unit 17c that determines a shutter speed (exposure period) and sends an instruction signal to the aperture driving unit 16 and the image sensor driving unit 15 is configured. The image sensor driving unit 15 drives the solid-state image sensor 12 based on the instruction signal, and outputs a subject image captured through the photographing lens 11 as an image signal.

  The electric control system of the digital camera 10 includes an analog signal processing unit 20 that performs analog signal processing such as noise removal on the output signal of the solid-state imaging device 12, and an output signal (RGB color signal) of the analog signal processing unit 20. And an A / D converter 21 for converting the signal into a digital signal, which are controlled by the CPU 17.

  Further, the electric control system of the digital camera 10 includes a memory control unit 23 connected to the frame memory 22 and a digital signal processing unit that performs color interpolation, gamma correction, RGB / YC conversion processing, etc. in addition to image synthesis processing described later. 24, a compression / expansion processing unit 25 that compresses a captured image into a JPEG image, or expands the compressed image, and an external memory control unit 27 that controls recording and reading of data with respect to a removable recording medium 26 A display control unit 29 that performs display control of a liquid crystal display unit (LCD) 28 mounted on the back of the camera body. These units of the electric control system are connected to each other by a control bus 30 and a data bus 31 and controlled by the CPU 17.

  The operation unit 18 includes a release button for performing a shutter operation, a mode dial for switching an operation mode, a menu button for displaying a shooting menu (setting item) on the LCD 28, and a selection determination button for selecting and determining a setting item. Etc. are included. The CPU 17 receives an instruction from the user through the operation unit 18 as an operation signal, and performs various controls based on the input operation signal.

  The release button is a two-stage push button switch. In shooting using an automatic focus adjustment function (AF: Auto Focus) and an automatic exposure adjustment function (AE: Auto Exposure), when a first pressing operation (also referred to as half-pressing) in which the release button is lightly pressed is performed, AF processing and AE processing are performed by the focus adjustment unit 17b and the exposure adjustment unit 17c, and preparations for shooting such as focusing and exposure adjustment are performed. In this state, when a second pressing operation (also referred to as full pressing) in which the release button is strongly pressed is performed, a subject image is captured by the solid-state imaging element 12.

  The mode dial is used to switch between various operation modes. The “wide dynamic range mode” captures images with a wide dynamic range, and the “high resolution mode” captures high-resolution images without expanding the dynamic range. Etc. can be selected.

  Further, the operation unit 18 can set the expansion rate of the dynamic range in the wide dynamic range mode and can set the flash emission on / off by the flash emission unit 19.

  When shooting with flash emission (flash shooting), the light emission control unit 17a causes the flash light emitting unit 19 to emit light twice in conjunction with the second pressing operation of the release button. The first light emission is preliminary light emission (pre-light emission) for measuring the amount of reflected light from the subject, and the light emission amount is smaller than that of normal light emission. The second light emission is a main light emission (to obtain an appropriate exposure amount) for supplying sufficient light to the subject for photographing. If the light emission amount of the main light emission is appropriate, an appropriate brightness is obtained. An image can be obtained.

  In FIG. 2, the solid-state imaging device 12 is arranged in a vertical direction on the semiconductor substrate and in a horizontal direction orthogonal thereto, and a plurality of light receiving elements (photoelectric conversion elements) 40 that photoelectrically convert subject light to generate charges, A plurality of vertical charge transfer units (VCCDs) 41 that transfer charges generated in each of the plurality of light receiving elements 40 in the vertical direction, and connected to the ends of the VCCDs 41, the charges transferred through the VCCDs 41 are horizontally converted. A horizontal charge transfer unit (HCCD) 42 that transfers in the direction and an output unit 43 that converts the charge transferred from the HCCD 42 into a voltage signal and outputs the voltage signal are provided. Hereinafter, the vertical arrangement of the light receiving elements 40 is referred to as a column, and the horizontal arrangement is referred to as a row.

  The plurality of light receiving elements 40 are divided into a first light receiving element group denoted by reference characters R1, G1, and B1 and a second light receiving element group denoted by reference characters R2, G2, and B2. The first light receiving element group and the second light receiving element group are composed of light receiving elements 40 arranged in a square lattice pattern in the row direction and the column direction, respectively, and R, G, and B included in the above symbols are on the light receiving element 40. Correspond to the types of color filters of the three primary colors of red R, green G, and blue B, which are arranged in the same manner. The first light receiving element group and the second light receiving element group are arranged at the same arrangement pitch in the row direction and the column direction, and are shifted from each other in the row direction and the column direction by ½ of the arrangement pitch. The light receiving element 40 has a so-called honeycomb arrangement as a whole.

  The arrangement of the color filters of the first and second light receiving element groups is a Bayer arrangement (a color arrangement in which “G” and “B” are repeated in one direction and “G” and “R” are repeated in a direction orthogonal thereto). It has become. As described above, the color filter array is a color array in which two Bayer arrays are combined as a whole, and is also referred to as a double Bayer array. Each light receiving element 40 is made of a photodiode and basically has the same structure (the same element area, the same opening area, the same junction depth, the same saturation amount, etc.) except for the type of the color filter.

  One VCCD 41 is provided for each row of the light receiving elements 40. Between each column of the light receiving elements 40 and the charge transfer channel (not shown) of the VCCD 41 corresponding thereto, a charge readout gate schematically shown by an arrow in the figure is formed. The signal charges generated and accumulated in each light receiving element 40 during the exposure period are read out to the charge transfer channel of the VCCD 41 via the charge reading gate.

  The charge transfer channel of the VCCD 41 is formed to meander in the vertical direction so as to avoid the light receiving element 40 on the surface layer of the semiconductor substrate. On the surface of the semiconductor substrate, vertical transfer electrodes V1, V2,..., V8 meandering so as to cross the charge transfer channel in the horizontal direction and avoid the light receiving element 40 are laid. The VCCD 41 is driven to transfer by vertical transfer pulses φV1 to φV8 that are output from the image sensor driving unit 15 and supplied to the vertical transfer electrodes V1 to V8, respectively.

  The charge readout gate of the first light receiving element group is provided adjacent to the vertical transfer electrodes V3 and V7. The charge readout gate of one second light receiving element group is provided adjacent to the vertical transfer electrodes V1 and V5. Therefore, charge reading from the light receiving element 40 of the first light receiving element group to the charge transfer channel of the VCCD 41 is performed by applying a read pulse to the vertical transfer electrodes V3 and V7. Similarly, charge reading from the light receiving element 40 of the second light receiving element group to the charge transfer channel of the VCCD 41 is performed by applying a read pulse to the vertical transfer electrodes V1 and V5. As described above, the charge readout of the first and second light receiving element groups is performed by supplying the readout pulse to the different vertical transfer electrodes, and therefore can be performed independently at different timings.

  The HCCD 42 includes a charge transfer channel (not shown) and a plurality of horizontal transfer electrodes (not shown) provided thereon. The HCCD 42 is driven in two phases by horizontal transfer pulses φH 1 and φH 2 output from the image sensor driving unit 15. The output unit 43 connected to the end of the HCCD 42 includes an FD amplifier including a floating diffusion (FD) unit that converts charges into voltage and a source follower circuit.

  Further, the solid-state imaging device 12 has a vertical overflow drain (VOD) structure for sweeping unnecessary charges accumulated in the light receiving element 40 to a semiconductor substrate. The charge discharging operation by the VOD structure is an operation called an electronic shutter. When a predetermined voltage (hereinafter referred to as an electronic shutter pulse φSUB) is applied from the image sensor driving unit 15 to the semiconductor substrate, the potential barrier formed at the bottom of all the light receiving elements 40 is lowered, and all the light receiving elements 40 The electric charges accumulated in are simultaneously swept to the semiconductor substrate.

  Next, a method of driving the solid-state imaging device 12 and the flash light emitting unit 19 when performing flash photography in the wide dynamic range mode will be described. In FIG. 3, before the timing t0, the electronic shutter pulse φSUB is periodically applied to sweep out the accumulated charges in the light receiving element 40 to the semiconductor substrate. At timing t0, the application of the electronic shutter pulse φSUB is stopped, the application of the flash driving pulse to the flash light emitting unit 19 is started, and the flash light emitting unit 19 is applied for a period tF until the timing t2 at which the application of the flash driving pulse is stopped. To output a predetermined amount of flash light.

  The first and second light receiving element groups of the solid-state imaging element 12 start to accumulate charges from timing t0. For the first light receiving element group, after the flash emission period tF has elapsed, at time t4, a read pulse is applied to the vertical transfer electrodes V3 and V7, so that the accumulated charge in the exposure period tL from timing t0 to t4 is transferred to the VCCD 41. read out.

  For one second light receiving element group, a read pulse is applied to the vertical transfer electrodes V1 and V5 at the timing t1 within the flash light emission period tF, and the accumulated charge from the timing t0 to t1 is read to the VCCD 41. Thereafter, high-frequency vertical transfer pulses φV1 to φV8 and horizontal transfer pulses φH1 and φH2 are applied to sweep out the read charges as unnecessary charges at high speed (that is, the second light receiving element group emits flash light). Exposure starts from the time point t1 within the period tF). Then, after sweeping away unnecessary charges, a read pulse is applied to the vertical transfer electrodes V1 and V5 at timing t3 to read out the accumulated charges in the exposure period tS from timing t1 to t3 to the VCCD 41.

  After the timing t4, the accumulated charges (signal charges) of the first and second light receiving element groups read out to the VCCD 41 are transferred vertically and horizontally, respectively, and output from the output unit 43 as imaging signals. Thus, the first light receiving element group is exposed in the exposure period tL, and the entire period (timing t0 to t2) of the flash light emission period tF contributes to the exposure. One second light receiving element group is exposed in the exposure period tS, and only a part of the flash emission period tF, tFP (timing t1 to t2), contributes to the exposure.

  Therefore, it is possible to individually adjust the received light amounts of the first and second light receiving element groups by flash emission by setting the timings t1 and t2, and to adjust the exposure amounts of the first and second light receiving element groups, respectively. Can be optimized. The timings t1 and t2 are determined by the light emission control unit 17a described above.

  FIG. 4 schematically shows signals (image data) output from the solid-state imaging device 12 by the above driving method. High sensitivity image data is obtained from the first light receiving element because of long-term exposure, and low sensitivity image data is obtained from the second light receiving element because of short period exposure. The high-sensitivity image data and the low-sensitivity image data are combined in the digital signal processing unit 24 with corresponding pairs of the same color pixels as indicated by broken lines, thereby generating image data with a wide dynamic range. Is done. As disclosed in Japanese Patent Application Laid-Open No. 2007-235656, this synthesis processing is performed by aligning the saturation voltage of high-sensitivity image data with the saturation voltage of low-sensitivity image data by signal slicing processing, and then matching corresponding pixels of the same color. This is done by adding

  In the high resolution mode, the CPU 17 drives the second light receiving element group at the same timing as the first light receiving element group, and makes the exposure periods of both the same (in this case, the high-speed V transfer shown in FIG. Is unnecessary). In the high resolution mode, the digital signal processing unit 24 does not execute the above-described combining process, and generates high-resolution image data using each as individual pixel data.

  Next, the operation of the digital camera 10 will be described along the flowchart of FIG. When a mode other than the wide dynamic range mode is selected by the mode dial (step S1: NO), the CPU 17 executes processing according to the selected mode (step S2), and the wide dynamic range mode is selected. If yes (step S1: YES), the following processing is executed.

  When detecting the first pressing operation (half pressing) of the release button (step S3: YES), the CPU 17 notifies the focus adjusting unit 17b and the exposure adjusting unit 17c that the first pressing operation has been detected. Upon receiving this notification, the exposure adjustment unit 17c performs AE processing, and the focus adjustment unit 17b performs AF processing (step S4). The CPU 17 sets an aperture value and a shutter speed (exposure value) based on the result of the AE process (step S5). Here, the shutter speed corresponds to the exposure period tL of the first light receiving element group, and the exposure period tS of the second light receiving element group is set to a value corresponding to the expansion rate of the dynamic range. For example, “tS = tL / 2” when the expansion ratio of the dynamic range is 200%, “tS = tL / 4” when 400%, and “tS = tL / 8” when 800%. Is set. Here, the exposure start timing t1 of the second light receiving element group is set to an arbitrary initial value. For example, t1 = t0 is set.

  Subsequently, when the CPU 17 detects the second pressing operation (full pressing) of the release button (step S6: YES determination), the CPU 17 determines whether or not the flash light emission by the flash light emitting unit 19 is necessary (step S7). If necessary (step S7: YES), a light control process is performed (step S8). On the other hand, when the flash emission is unnecessary (step S7: NO), the actual photographing is executed (step S9).

  In FIG. 6 for explaining the dimming process in step S8, when the CPU 17 sends an instruction signal instructing the start of the dimming process to the light emission control part 17a, the light emission control part 17a takes a picture by non-light-emission exposure without flash emission. Then, the image data (image data at the time of non-light emission) stored in the frame memory 22 is acquired (step S11). Subsequently, the light emission control unit 17a performs preliminary light emission by the flash light emission unit 19, and obtains image data (image data at the time of preliminary light emission) photographed by preliminary light emission exposure and stored in the frame memory 22 (step S12). In steps S11 and S12, the light emission control unit 17a obtains a luminance value Ya during non-light emission and a luminance value Yb during preliminary light emission for the first light receiving element group subjected to long-term exposure.

  Next, the light emission control unit 17a divides the acquired non-light emission image data and preliminary light emission image data into a plurality of areas, and calculates a difference value of luminance values for each divided area (step S13). FIG. 7 shows an example of area division of image data. An image represented by image data (non-light emission image data or preliminary light emission image data) is represented by i × j (here, 8 × 8) areas. It is divided into. Hereinafter, the divided blocks are distinguished by codes such as (1, 1) and (i, j).

  Next, the light emission control unit 17a acquires brightness values Ya (1, 1) to Ya (i, j) at the time of non-light emission for each divided area from the image data at the time of non-light emission. For example, RGB / YC conversion is performed on the non-light emitting image data to obtain the luminance value of each pixel, and the average value of each pixel luminance value for each region is set as the luminance value Ya at the time of non-light emitting. For the preliminary light emission image data, the luminance values Yb (1, 1) to Yb (i, j) at the time of preliminary light emission of each divided area are acquired in the same procedure.

  Next, the light emission control unit 17a, for each divided area, a difference value Yd (x, y) (=) between the luminance value Ya (x, y) at the time of non-light emission and the luminance value Yb (x, y) at the time of preliminary light emission. Yb (x, y) −Ya (x, y)) is calculated. This difference value Yd (x, y) represents the reflection component (reflection luminance information) of the flash light from the subject.

  Next, based on the difference values Yd (1,1) to Yd (i, j) of all the divided areas, the light emission control unit 17a determines how many times the flash light emission amount during the main photographing is the light emission amount of the preliminary light emission. (Light emission magnification Hm) is calculated (step S17). For example, a simple average value of the difference values Yd (1,1) to Yd (i, j), or a weighted average Ydh, which is averaged by increasing the weight of a divided area near the center of the image as shown in FIG. Yah is calculated, and the light emission magnification Hm is calculated by the following equation (1) using the target luminance Yo.

  Hm = (Yo-Yah) / Ydh (1)

  Next, the light emission control unit 17a calculates the flash light emission amount Ev at the time of actual photographing based on the light emission magnification Hm by the following equation (2) (step S15).

Ev = log ₂ Hm (2)

  Next, the light emission control unit 17a determines the light emission time (the above-described period tF (value of timing t2 with reference to timing t0)) with respect to the calculated light emission amount Ev (step S16). The timing t2 is determined using, for example, a table LUT1 that stores light emission times for various light emission amounts Ev as shown in FIG. For example, when the light emission amount Ev is 2.0, the timing t2 is determined to be 32.0 μs with reference to the table LUT1.

  Next, the light emission control unit 17a determines the exposure timing of the second light receiving element group (at timings t1 and t3 with reference to the timing t0) based on the ratio of the exposure periods of the first and second light receiving element groups (tS / tL). Value) is determined (step S17).

  In step S17, the timing t1 is determined so that the ratio of the light emission amount of the period tFP to the light emission amount of the period tF matches the exposure period ratio tS / tL. Specifically, referring to the table LUT1, the light emission time for the light amount (1-tS / tL) times the light emission amount Ev obtained in step S15 is obtained, and this light emission time is set as timing t1. For example, when the exposure period ratio tS / tL is “1/2” and the light emission amount Ev obtained in step S15 is “2.0”, the light emission with respect to the light amount 1.0 that is 1/2 the light emission amount Ev is performed. The time is referred to from the table LUT1, and the timing t1 is determined to be 18.0 μs. A value obtained by adding the exposure period tS to the timing t1 is defined as a timing t3.

  Note that in steps S16 and S17, when obtaining the light emission time for the light emission amount Ev not stored in the table LUT1, the light emission time for the closest light emission amount Ev may be referred to, but more accurate processing is possible. Is desired, the light emission control unit 17a is preferably provided with an interpolation processing function, and the light emission time for the light emission amount Ev not stored in the table LUT1 is obtained by interpolation calculation.

  Returning to FIG. 5, when performing flash shooting, the CPU 17 drives the flash light emitting unit 19 and the solid-state imaging device 12 based on the timings t1 to t4 determined by the above processing, and performs main shooting (step S9). ). As a result, since the amount of light received by the first and second light receiving element groups is optimized, the image data after the synthesis processing by the digital signal processing unit 24 has a wide dynamic range corresponding to the expansion rate of the dynamic range. It becomes image data.

  Further, the CPU 17 controls the external memory control unit 27 to record the image data obtained as a result of the main shooting on the recording medium 26, and uses the values of timings t1 to t4 used in the main shooting as additional information. The image data is recorded on the recording medium 26 in association with the image data (step S10). For example, the CPU 17 records the values of timings t1 to t4 as Exif data added to JPEG format image data.

(Second Embodiment)
Next, as a second embodiment of the present invention, a modification of the dimming process taking into account the afterglow of flash emission will be shown. The flash light emitted from the flash light emitting unit 19 does not disappear instantaneously with the end of the flash drive pulse applied to the flash light emitting unit 19, and as shown in FIG. 10A, the flash drive pulse stops at the timing t2. Afterglow occurs afterwards.

  For this reason, the timing determined in step S16 of FIG. 6 is defined in the table LUT1 shown in FIG. 9 by defining the relationship between the light emission amount and the light emission time (timing t2) in consideration of the afterglow component of flash light emission. t2 is highly accurate.

  On the other hand, in step S17, the timing t1 is determined so that the amount of light in the region A shown in FIG. 10B is a ratio of tS / tL of the light emission amount Ev (total light emission amount). The amount of light in the region A corresponds to a value obtained by subtracting the light emission amount Ev when the light emission end time is timing t1 from the light emission amount Ev when the light emission end time is timing t2. However, the light amount related to the exposure of the second light receiving element group is actually the light emission amount after the timing t1 (the light amount obtained by adding the region A and the region B). Therefore, in the first embodiment, the second light receiving element group is subjected to extra exposure by the amount of light corresponding to the region B (afterglow component when the emission end time is timing t1) due to the effect of afterglow. It will be.

  The second embodiment has the same configuration as that of the first embodiment except that the processing content of step S17 of the dimming process shown in FIG. 6 is different. In the second embodiment, the light emission control unit 17a determines the timings t1 and t3 using the table LUT2 illustrated in FIG. The table LUT2 indicates that the total light emission amount (light emission amount Ev) is tS with respect to the total light emission amount (light emission amount Ev) for various exposure period ratios tS / tL (3/4, 1/2, 1/4, etc.). Each of the timings t1 having a ratio of / tL is stored.

  As in the first embodiment, the timing t3 is a value obtained by adding the exposure period tS to the timing t1. When obtaining the timing t1 for the light emission amount Ev or the ratio tS / tL that is not stored in the table LUT2, an interpolation calculation may be performed by the interpolation processing function of the light emission control unit 17a.

  Thus, according to the second embodiment of the present invention, the influence of afterglow of flash emission is prevented, and image data with a wide dynamic range that accurately corresponds to the expansion rate of the dynamic range is generated.

(Third embodiment)
Next, as a third embodiment of the present invention, another modification of the light control processing capable of preventing the influence of afterglow of flash light emission will be described. In the first embodiment, in the dimming process, the processes of steps S11 to S15 are performed only for the first light receiving element group, and the light emission time (timing) is calculated based on the flash emission amount Ev for main photographing calculated as a result. After determining t2), the exposure timing (timing t1, t3) of the second light receiving element group is determined based on the exposure period ratio tS / tL corresponding to the expansion rate of the dynamic range. Thus, in the third embodiment, the processes of steps S11 to S15 are performed for each of the first and second light receiving element groups, and the light emission amount Ev corresponding to the reflected luminance information is set for the first and second light receiving element groups. By obtaining each separately, the light emission time (timing t2) and the exposure timing (timing t1, t3) of the second light receiving element group are independently determined. That is, in the third embodiment, the timing t2 is determined based on the reflected luminance information obtained from the first light receiving element group, and independently of this, the timing t1 is based on the reflected luminance information obtained from the second light receiving element group. , T3. Since other points are the same as those of the first embodiment, the description thereof is omitted.

  As described above, according to the third embodiment of the present invention, the exposure amounts of the first and second light receiving element groups are individually optimized by determining the timings t1 and t2 independently. The effect of such afterglow is prevented.

(Fourth embodiment)
Next, a fourth embodiment of the present invention will be described. In the first embodiment, the exposure amount of the second light receiving element group is optimized by controlling the exposure period of the second light receiving element group for short-term exposure with respect to the flash light emission period. It is also possible to optimize the exposure amount of the second light receiving element group by controlling the flash light emission period with respect to the exposure period of the second light receiving element group.

  FIG. 12 shows a driving method of the solid-state imaging device 12 and the flash light emitting unit 19 when performing flash photography in the wide dynamic range mode in the fourth embodiment. The CPU 17 controls the driving of the solid-state imaging device 12 so as to end the exposure period tL of the first light receiving element group and the exposure period tS of the second light receiving element group at the same time (timing t4), and performs flash light emission. The timings t1 and t3 are adjusted with respect to the exposure period tS (timing t2 to t4) on the short exposure side.

  FIG. 13 shows the light control processing of the fourth embodiment. Steps S21 to S25 are the same as steps S11 to S15 described in the first embodiment, and thus description thereof is omitted. In step S26, the light emission control unit 17a determines the light emission time tF for the light emission amount Ev calculated in step S25. The determination of the light emission time tF is performed using the above-described table LUT1.

  Next, the light emission control unit 17a, based on the ratio (tS / tL) of the exposure periods of the first and second light receiving element groups, in the period tFP (timing t2 to t3) with respect to the light emission amount in the period tF (timing t1 to t3). The flash emission timings t1 and t3 with respect to the exposure start timing t2 of the second light receiving element group are determined so that the ratio of the light emission amount matches the exposure period ratio tS / tL (step S27). The timings t1 and t3 can be determined by the same method as in the first embodiment. Note that the timings t1 and t3 can be determined using the method of the second embodiment or the third embodiment.

(Fifth embodiment)
Next, a fifth embodiment of the present invention will be described. In the fifth embodiment, as shown in FIG. 14, a mechanical shutter 50 is disposed in front of the photographic lens 11, a shutter drive unit 51 that drives the mechanical shutter 50 is provided, and the CPU 17 mechanically passes through the shutter drive unit 51. Driving the shutter 50 differs from the fourth embodiment in that the end timing t4 of the exposure periods tL and tS of the first and second light receiving element groups is defined.

  Specifically, as shown in FIG. 15, with the mechanical shutter 50 opened, the application of the electronic shutter pulse φSUB is stopped to start exposure of the first light receiving element group (timing t0). Then, by applying a read pulse to the vertical transfer electrodes V1 and V5, exposure of the second light receiving element group is started (timing t2). Then, the mechanical shutter 50 is closed when the period tS has elapsed, thereby terminating the exposure of the first and second light receiving element groups (timing t4). Note that flash emission is performed at the timings t1 and t3 described in the fourth embodiment.

  After the mechanical shutter 50 is closed, the high-frequency vertical transfer pulses φV1 to φV8 and the horizontal transfer pulses φH1 and φH2 are applied to quickly sweep away the accumulated charges (unnecessary charges) in the VCCD 41 (high-speed V transfer). ). Thereafter, a read pulse is applied to the vertical transfer electrodes V3 and V7 and the vertical transfer electrodes V1 and V5, and the accumulated charges (signal charges) of the first and second light receiving element groups are read to the VCCD 41. Then, the signal charges read out to the VCCD 41 are transferred vertically and horizontally and output from the output unit 43 as an imaging signal.

  As described above, in the fifth embodiment, since unnecessary charges in the VCCD 41 are swept away immediately before the signal charges are read out to the VCCD 41, the occurrence of smear is prevented. Further, since the high-speed V transfer does not intervene during the short exposure period tS as in the fourth embodiment, the short exposure period tS can be made shorter than the time required for the high-speed V transfer.

  In the first to fifth embodiments, a CCD image sensor is used as the solid-state image sensor 12. However, as the solid-state image sensor, a first light-receiving element and a second light-receiving element that can control the exposure period independently. Any device may be used as long as a plurality of elements are arranged as a pair, and a CMOS image sensor or the like can also be used.

10 Digital Camera 12 Solid-State Image Sensor 17 CPU
17a Light emission control unit 17b Focus adjustment unit 17c Exposure adjustment unit 19 Flash light emission unit 24 Digital signal processing unit 26 Recording medium 27 External memory control unit 40 Light receiving element 41 Vertical charge transfer unit 42 Horizontal charge transfer unit 50 Mechanical shutter V1 to V8 Vertical transfer Electrodes R1, G1, B1 First light receiving element group R2, G2, B2 Second light receiving element group

Claims (16)

Flash light emitting means for emitting flash light;
A plurality of first light receiving elements and second light receiving elements whose exposure periods can be controlled independently are arranged in pairs, and the second light receiving element is shorter than the first exposure period during the first exposure period of the first light receiving elements. Imaging means that enables exposure of the second light receiving element in an exposure period;
The first and second light receiving elements receive a light amount corresponding to the ratio of the first and second exposure periods, respectively, so that the flash light emission period of the flash light emitting unit and the second exposure period are relative to each other. Control means for controlling the timing relationship;
Signal processing means for combining the output signals of the first and second light receiving elements to generate image data;
An imaging apparatus comprising:   The control means synchronizes the flash light emission period with the first exposure period, and the first and second light receiving elements respectively receive light amounts corresponding to a ratio of the first and second exposure periods. The photographing apparatus according to claim 1, wherein the timing of the second exposure period with respect to the flash light emission period is controlled.   The photographing apparatus according to claim 2, wherein the control unit simultaneously sets a start time of the flash light emission period and the first exposure period.   The control unit synchronizes the second exposure period with the first exposure period, and the first and second light receiving elements respectively receive light amounts corresponding to a ratio of the first and second exposure periods. 2. The photographing apparatus according to claim 1, wherein timing of the flash light emission period with respect to the second exposure period is controlled.   The photographing apparatus according to claim 4, wherein the control unit sets an end time of the first exposure period and the second exposure period at the same time.   The control means causes the flash light emitting means to perform preliminary light emission, calculates a light emission amount for main photographing based on reflection luminance information of the flash light detected by the imaging means, and based on the calculated light emission amount, 6. The photographing apparatus according to claim 1, wherein a flash light emission time is determined. The control means includes a first table that stores a light emission time for the light emission amount for a plurality of the light emission amounts,
The light emission time is determined by referring to the first table, and the relative timing relationship between the flash light emission period and the second exposure period is determined based on the ratio of the first and second exposure periods. The imaging device according to claim 6.   The photographing apparatus according to claim 7, wherein the control unit calculates a light emission time with respect to a light emission amount not stored in the first table by interpolation calculation. The control means sets the timing relationship in which the ratio of the light emission amount during the second exposure period to the light emission amount during the first exposure period becomes a ratio of the first and second exposure periods, for a plurality of the ratios. A second table stored;
9. The photographing apparatus according to claim 7, wherein the timing relationship corresponding to the ratio is determined by referring to the second table.   The photographing apparatus according to claim 9, wherein the control unit calculates the timing relationship with respect to the ratio that is not stored in the second table by an interpolation operation.   The control means determines the light emission time based on the reflected luminance information obtained from the first light receiving element, and determines the timing relationship based on the reflected luminance information obtained from the second light receiving element. The imaging device according to claim 6, characterized in that:   12. The photographing apparatus according to claim 1, further comprising a recording unit configured to record the flash emission period and the determined value of the timing relationship in association with the image data.   The first light receiving elements are arranged in a square lattice pattern in a row direction and a column direction perpendicular thereto, and the second light receiving elements are arranged in a square lattice pattern in a row direction and a column direction perpendicular thereto. The first light receiving element and the second light receiving element are arranged at the same arrangement pitch and are shifted from each other in the row direction and the column direction by a half of the arrangement pitch. Item 13. The photographing apparatus according to any one of Items 1 to 12.   In the image pickup means, red, green, and blue color filters corresponding to the first light receiving element are arranged in a Bayer array, and red, green, and blue color filters corresponding to the second light receiving element are arranged in a Bayer array. The imaging apparatus according to claim 13, wherein the imaging apparatus is configured.   The imaging means reads signal charges stored in the first and second light receiving elements, arranged along a column in which the first light receiving elements are arranged and a column in which the second light receiving elements are arranged. 2. A CCD type solid-state imaging device, comprising: a vertical charge transfer unit configured to transfer in a column direction; and a horizontal charge transfer unit configured to transfer a signal charge from the vertical charge transfer unit in a row direction. 14. The photographing apparatus according to 14. A plurality of flash light emitting means for emitting flash light and a first light receiving element and a second light receiving element that can control the exposure period independently are arranged in pairs, and during the first exposure period of the first light receiving element, Image data is generated by synthesizing the imaging means capable of exposing the second light receiving element in a second exposure period shorter than the first exposure period, and the output signals of the first and second light receiving elements. In a method for controlling an imaging apparatus comprising a signal processing means,
The first and second light receiving elements receive a light amount corresponding to the ratio of the first and second exposure periods, respectively, so that the flash light emission period of the flash light emitting unit and the second exposure period are relative to each other. A method for controlling a photographing apparatus, characterized by controlling a timing relationship.

Images