JP2006352825A - Imaging apparatus and imaging system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus and an imaging system capable of shortening processing time without causing dispersion in the lightness and the contrast between images of frames resulting from a consecutive shot and without losing the contrast of a principal object in the case of carrying out the consecutive shot. <P>SOLUTION: The imaging apparatus capable of variably controlling the dynamic range of an output image and the imaging system are provided, wherein a group of consecutive shot images the dynamic range of which is made constant during the consecutive shot independently of a luminance change in the object by constantly controlling the dynamic range of the output images in the consecutive shot. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an imaging device, and more particularly, to an imaging device having a continuous imaging mode and an imaging system including the imaging device and an image processing device.

In recent years, in an imaging apparatus such as a digital camera, in response to a demand for higher image quality, it is one of the major things to expand the luminance range of a subject that can be handled by the imaging element, that is, the dynamic range (hereinafter referred to as D range). It has become a theme.

  Regarding the expansion of the D range, by adding a logarithmic conversion circuit equipped with a MOSFET or the like to a solid-state imaging device in which photoelectric conversion elements such as photodiodes are arranged in a matrix, and utilizing the subthreshold characteristics of the MOSFET, There is known a logarithmic conversion type image pickup device in which an electrical signal is logarithmically converted with respect to an incident light amount as an output characteristic of a solid-state image pickup device (see, for example, Patent Document 1).

  Further, in the logarithmic conversion type image pickup device, by applying a specific reset voltage to the MOSFET, a linear operation state in which an electrical signal is linearly converted and output according to the original output characteristics of the solid state image pickup device, that is, the amount of incident light, and There is known a logarithmic conversion type imaging device (hereinafter referred to as a linear log sensor) that can automatically switch between the above-described logarithmic operation states (hereinafter referred to as a linear log characteristic) (for example, Patent Documents). 2). Furthermore, an imaging apparatus is disclosed that can automatically switch from a linear operation state to a logarithmic operation state, and that can adjust the potential state of the MOSFET by adjusting the reset time of the MOSFET (for example, Patent Document 3). reference).

  By the way, when the linear log sensor is used in a linear operation state, an output proportional to the amount of electric charge generated by the photoelectric conversion element can be obtained, so that even a low-luminance object has a high contrast (high gradation) image. While there is an advantage that a signal can be obtained, there is a disadvantage that the D range becomes narrow. On the other hand, when the logarithmic log sensor is used in a logarithmic operation state, an output that is naturally logarithmically converted with respect to the amount of incident light is obtained. Therefore, there is a disadvantage that the contrast is deteriorated.

  Furthermore, in addition to the linear log sensor, an imaging device that synthesizes images of a wider D range than individual screens by selecting and synthesizing screen portions of appropriate levels from a plurality of screens with different exposure amounts (for example, patents) Reference 4) and a charge-voltage converter that converts photoelectric conversion signal charges of the image sensor into signal voltages. The charge-voltage converter is composed of a plurality of capacitors having different voltage dependencies, and the D range is variable. An image sensor (see, for example, Patent Document 5) and the like have also been proposed.

  However, since the image sensor proposed in Patent Document 4 synthesizes images from images with different exposure amounts by image processing, a simple image synthesis results in an unnatural and extremely difficult to see image. Very complex image composition is essential, which makes the cost very high. Further, the image pickup device proposed in Patent Document 5 has a complicated device structure and a high manufacturing cost, and it is difficult to control the voltage dependency of the capacitance, so that adjustment for each device is required, which causes an increase in cost.

  The imaging device and the imaging device according to Patent Document 3 only disclose that the imaging device can be automatically switched from the linear operation state to the logarithmic operation state. However, in view of the advantages and disadvantages of the linear operation state and the logarithmic operation state described above, the image capturing operation is performed not only by automatic switching but also by actively utilizing the advantages of each operation state. Is desirable. For example, even in the case of performing automatic exposure control, if the control is performed by associating the luminance of the target subject with the switching point from the linear operation state to the logarithmic operation state of the imaging sensor, the advantages provided by each operation state There is a possibility that the optimum automatic exposure control using the can be performed.

  Therefore, the applicant of the present application, in Japanese Patent Application No. 2004-159760, performs exposure control of the imaging device in association with the photoelectric conversion characteristics of the imaging sensor included in the imaging device, so that the light amount from the subject (subject luminance) is satisfied. Thus, an imaging apparatus and an imaging method capable of imaging a subject in an optimal exposure state and with a predetermined D range secured have been proposed.

  On the other hand, a digital camera that is a typical example of an imaging apparatus has a continuous imaging mode (see, for example, Non-Patent Document 1). Here, the continuous imaging is a technique for continuously capturing a plurality of photographs in order to obtain a continuous image of a moving subject or the like.

  On the other hand, the wide D range of the display system such as a monitor has not progressed as much as the imaging system at the present time, contrary to the wide D range of the imaging system as in the linear log sensor. Even if the D range is achieved, the effect cannot be fully exhibited in the display system. Therefore, it is necessary to compress the D range of the captured image so that the captured image of the wide D range fits in the D range of the display system (that is, as viewed by the observer as an image of a wide luminance range). .

  By the way, in “D range compression”, the relationship between the meaning of adjusting the contrast of the image locally, that is, improving the contrast (gradation) by compressing the illumination component of the image, and the contrast of the entire image as it is. There are two meanings of literally compressing the band (D range) while maintaining the state (compressing the image uniformly regardless of local contrast adjustment). In order to distinguish these, “Dodge processing” will be referred to as “D-range compression”. As will be described later, it can be said that the illumination component is substantially a low-frequency component, and a reflectance component described later is a high-frequency component.

  Conventionally, in the dodging process, for example, an illumination component is extracted from an image (at this time, a reflectance component is also extracted), this illumination component is D-range compressed, and the above-described illumination component and reflectance component after the D-range compression are used as described above. A new image in which the contrast of the image is adjusted locally is generated.

  In this regard, for example, as shown in FIG. 19A, an image having a photoelectric conversion characteristic of a linear log characteristic obtained by a linear log sensor (hereinafter referred to as a linear log image. This linear log image corresponds to an original image I described later. ) Is divided and extracted into a logarithmic image I1 and a linear image I2, a dodging process is performed on each image, and then these images are combined (see, for example, Patent Document 6). In Patent Document 6, in order to prevent the deterioration of the S / N ratio at the time of image composition, as shown in FIG. 19B, the image composition means performs pixel values of the image I ′ synthesized after the dodging process. Is larger than the pixel value of the original image I (image I ′> image I; part 901 in FIG. 19B), not the image I ′ (part 901 in FIG. 19B) but the original image I ( The portion 902 in FIG. 19B is selected.

Also, when compressing an image, the image signal is classified into large and small signals using a certain reference value so as not to lower the contrast of the image where sufficient contrast was originally obtained. A technique for performing D-range compression using correction data for D-range compression is disclosed (for example, see Patent Document 7).
JP 11-298798 A JP 2002-77733 A JP 2002-300476 A Japanese Patent Publication No. 7-97841 JP 2000-165755 A Japanese Patent Application No. 2004-377875 Japanese Patent No. 2509503 Konica Minolta Photo Imaging Co., Ltd .: α Suite DIGITAL catalog (http://konicaminolta.jp/products/consumer/digital_camera/catalogue/pdf/a-sweetdigital.pdf) “Sequential shooting” on page 1/8 “Shooting function” Column; Searched on September 29, 2005)

  However, in the above prior art and Japanese Patent Application No. 2004-159760, continuous imaging is not considered when the imaging system is widened or the D range is compressed so as to be within the D range of the display system. Therefore, it is not possible to acquire a continuous imaging frame group in which the D range is constant between the imaging frames regardless of the luminance change of the subject during continuous imaging.

  In the technique shown in Patent Document 6, since the compression rate is set according to the D range of the input image, the illumination component of the main subject (main subject luminance) may be compressed in some cases. Compared with an image (photographed image), for example, the gradation of a person's face, which is generally an important part in a photograph, is crushed (for example, a bright face becomes brighter and becomes so-called whiteout). That is, as shown in FIG. 20A, when the main subject luminance enters the luminance range 911, there is a problem that the contrast is lowered due to compression.

  As shown in FIG. 20B, this problem may be avoided by not using the technique shown in Patent Document 7 and performing dodging processing on an area of the original image 921 that is less than the predetermined value θ. For example, when the reflectance component 922 is dodged in a region equal to or larger than the predetermined value θ, the image I ′> the original image I may be obtained as described above, and the S / N ratio is deteriorated ( If you brighten an image in a dark area, noise will become more noticeable and the image quality will deteriorate.) Further, in the technique of Patent Document 6, since the image I ′ and the original image I are compared after performing the dodging process on the entire image, the processing time becomes long. In general, much of the processing time in the dodging process is required for extraction processing of illumination components using an edge maintaining filter such as a median filter or an epsilon filter.

  Thus, the conventional technique has not been considered for the control of the D range during continuous imaging.

  The present invention has been made in view of the above circumstances, and there is no variation in image brightness or contrast between frames of continuous imaging during continuous imaging using a plurality of imaging elements having different photoelectric conversion characteristics. An object of the present invention is to provide an imaging apparatus and an imaging system capable of shortening the processing time without impairing the contrast.

The object of the present invention can be achieved by the following constitution.

1. In an imaging apparatus provided with mode setting means for setting a continuous imaging mode,
Comprising output dynamic range control means for controlling the dynamic range of the output image of the imaging device;
When the continuous imaging mode is set by the mode setting means,
The said output dynamic range control means controls the dynamic range of the output image of the said imaging device during a continuous imaging uniformly, The imaging device characterized by the above-mentioned.

2. In an imaging apparatus provided with mode setting means for setting a continuous imaging mode,
An image sensor having a plurality of different photoelectric conversion characteristics;
Imaging dynamic range control means for controlling the dynamic range of the imaging device,
When the continuous imaging mode is set by the mode setting means,
The imaging dynamic range control means controls the dynamic range of the imaging element during continuous imaging to be constant.

3. In an imaging apparatus provided with mode setting means for setting a continuous imaging mode,
An image sensor having a plurality of different photoelectric conversion characteristics;
Processing dynamic range control means for controlling the dynamic range of the imaging output of the imaging device,
When the continuous imaging mode is set by the mode setting means,
The processing dynamic range control means controls the dynamic range of the imaging output of the imaging device during continuous imaging to be constant.

4). The processing dynamic range control means includes
Compression processing means for compressing the dynamic range of the imaging output of the imaging device;
When the continuous imaging mode is set by the mode setting means,
4. The imaging apparatus according to 3, further comprising a compression rate control unit that controls a compression rate of the compression processing unit during continuous imaging to be constant.

5). Exposure control means for controlling the exposure at the time of imaging the subject to be constant;
An image sensor having a plurality of different photoelectric conversion characteristics;
Imaging dynamic range control means for controlling the dynamic range of the imaging device;
Compression processing means for compressing a dynamic range of a captured image of the image sensor;
Compression rate control means for controlling the compression rate of the compression processing means,
By making the dynamic range of the imaging device constant by the imaging dynamic range control means and by making the compression rate of the compression processing means constant by the compression rate control means, the output image of the imaging device during continuous imaging 2. The imaging apparatus according to 1, wherein the dynamic range is controlled to be constant.

6). Exposure control means for variably controlling the exposure when imaging a subject;
An image sensor having a plurality of different photoelectric conversion characteristics;
Imaging dynamic range control means for controlling the dynamic range of the imaging device;
Compression processing means for compressing a dynamic range of a captured image of the image sensor;
Compression rate control means for controlling the compression rate of the compression processing means,
The dynamic range of the image sensor is variably controlled by the imaging dynamic range control unit, and the compression rate of the compression processing unit is variably controlled by the compression rate control unit, so that the imaging device during continuous imaging can be controlled. 2. The imaging apparatus according to 1, wherein a dynamic range of an output image is controlled to be constant.

7). An imaging apparatus comprising mode setting means for setting a continuous imaging mode;
In an imaging system comprising an image processing device that processes a captured image of the imaging device,
The image processing apparatus includes:
Compression processing means for compressing a dynamic range of a captured image of the imaging device;
When the imaging device is set to the continuous imaging mode by the mode setting means,
An image pickup system comprising: a compression ratio control means for controlling a compression ratio to be constant when compressing a dynamic range of continuously picked up images of the image pickup apparatus by the compression processing means.

  According to the present invention, in an imaging device in which the dynamic range of an output image can be variably controlled, control is performed so that the dynamic range of an output image in continuous imaging is constant, thereby continuously It is possible to provide an imaging apparatus and an imaging system capable of obtaining a group of continuously captured images having a constant dynamic range during imaging. According to the inventions of claims 2, 3 and 7, the processing time during continuous imaging can be shortened by fixing the dynamic range of the imaging device or imaging output. Furthermore, according to the third and seventh aspects of the invention, it is possible to provide an imaging apparatus and an imaging system in which there is no variation in image brightness or contrast between frames in continuous imaging.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  1A and 1B are schematic external views of a digital camera which is an example of an imaging apparatus according to the present invention. FIG. 1A is a front view and FIG. 1B is a rear view.

  An interchangeable lens 20 is attached to the front surface of the body 10 of the digital camera 1.

  A release button 101, which is an operation member for imaging, is installed on the upper surface of the body 10, and an AF that operates in the first stage of pressing the release button 101 inside the body 10 is below the release button 101. A / AE switch 101a and a two-stage switch constituting the release switch 101b that operates in the second stage of pressing the release button are arranged. A flash 102 is built in the upper part of the body 10 and a mode setting dial 112 for setting the operation mode of the digital camera 1 is arranged. The mode setting dial 112 functions as mode setting means in the present invention.

  On the back of the body 10, there are a power switch 111 for turning on / off the power of the digital camera 1 and five switches, up, down, left, right, and center, and a jog dial for performing various settings in each operation mode of the digital camera 1. 115, a viewfinder eyepiece 121a, and an image display device 131 for displaying a recorded image are arranged.

  FIG. 2 is a block diagram showing an example of a circuit of the digital camera 1 shown in FIG. In the figure, the same parts as those in FIG.

  A camera control circuit 150 that is a control unit of the digital camera 1 includes a CPU (central processing unit) 151, a work memory 152, a storage unit 153, and the like, and reads a program stored in the storage unit 153 to the work memory 152. Each part of the digital camera 1 is centrally controlled according to the program. The camera control circuit 150 receives inputs from the power switch 111, the mode setting dial 112, the jog dial 115, the AF / AE switch 101a, the release switch 101b, and the like, and communicates with the photometry module 121b on the optical viewfinder 121. Controls the photometric operation, controls the focusing operation by communicating with the AF module 144, drives the reflex mirror 141 and the sub mirror 142 via the mirror driving means 143, and controls the shutter 145 via the shutter driving means 146 (Here, the camera control circuit 150 functions as an exposure control unit in the present invention), controls the flash 102 via the flash control circuit 147, and controls the imaging operation by communicating with the imaging control circuit 161. Output image data and various information images Is displayed on the Display device 131 displays various information on the in-finder display means 132.

  Further, the camera control circuit 150 functions as a communication unit between the body 10 and the interchangeable lens 20, and a BL communication unit (body side) 172 provided on the mount (body side) 171 and a mount (lens side) 271. A lens control circuit 241 for controlling the focus and zoom of the lens 211 and a diaphragm control circuit for controlling the diaphragm 221 via the BL interface (lens side) 272 provided above and the lens interface 251 of the interchangeable lens 20. 222, controls the entire interchangeable lens 20 by communicating with the lens information storage unit 231 storing the unique information of the interchangeable lens 20 (here, the camera control circuit 150 functions as an exposure control means in the present invention). To do).

  An image formed by the lens 211 of the interchangeable lens 20 is photoelectrically converted by the image sensor 162, amplified by the amplifier 163, converted to digital data by the analog / digital (A / D) converter 164, and temporarily After being recorded in the memory 181, it is converted into digital image data subjected to predetermined image processing by the image processing device 165, and finally recorded in the memory card 182. These operations are controlled by the imaging control circuit 161 under the control of the camera control circuit 150.

  Here, the camera control circuit 150 and the imaging control circuit 161 function as imaging dynamic range control means in the present invention, and the image processing device 165 functions as processing dynamic range control means in the present invention. The control circuit 161 and the image processing device 165 function as output dynamic range control means in the present invention.

  Next, continuous imaging in the digital camera 1 will be described. 3 to 7 are flowcharts showing the flow of the continuous imaging operation in this embodiment. FIG. 3 is a main routine, and FIGS. 4 to 7 are subroutines showing each continuous imaging mode. In each figure, the same step is given the same number.

  In FIG. 3, when the power switch 111 is operated in step S101 to turn on the camera power, the operation mode of the digital camera 1 set by the mode setting dial 112 is confirmed in step S111. When the camera mode is set (step S111; YES), the process proceeds to step S121. If it is set to a mode other than the camera mode (for example, the image reproduction mode) (step S111; NO), the control proceeds to each mode according to the setting. Description is omitted.

  In step S121, it is confirmed whether the imaging mode in the camera mode of the digital camera 1 is set to the continuous imaging mode. When the continuous imaging mode is set (step S121; YES), the process proceeds to step S131. When an imaging mode other than the continuous imaging mode (for example, one-shot imaging mode, bracket imaging mode, etc.) is set (step S121; NO), the control proceeds to each mode according to the setting. Description is omitted.

  In step S131, it is confirmed whether or not the setting for making the D range of the image sensor (hereinafter referred to as the imaging D range) constant in the continuous imaging mode is made. When the setting for making the imaging D range constant is made (step S131; YES), the process proceeds to step S132, and it is confirmed whether or not the setting for making the exposure constant is made in the continuous imaging mode. If the exposure is set to be constant (step S132; YES), the process proceeds to step S231 to set the imaging output D range (hereinafter referred to as the processing D range) of the imaging device to be constant in the continuous imaging mode. It is confirmed whether or not If the processing D range is set to be constant (step S231; YES), the process proceeds to step S133, the “continuous shooting A1 mode” subroutine of FIG. 4A is executed, the process returns to step 111, and thereafter. The above operation flow is repeated.

  If the setting for making the processing D range constant is not made in step S231 (step S231; NO), the process proceeds to step S232, the “continuous shooting A2 mode” subroutine of FIG. 4B is executed, and the process proceeds to step S111. After that, the above operation flow is repeated.

  If the setting for making the exposure constant is not made in step S132 (step S132; NO), the process proceeds to step S241, and it is confirmed whether or not the setting for making the processing D range constant is made in the continuous imaging mode. If the processing D range is set to be constant (step S241; YES), the process proceeds to step S141, the “continuous shooting B1 mode” subroutine in FIG. 5A is executed, the process returns to step 111, and the digital It is confirmed whether or not the operation mode of the camera 1 has been changed from the camera mode. Thereafter, the above-described operation flow is repeated.

  If the setting for making the processing D range constant is not made in step S241 (step S241; NO), the process proceeds to step S242, the “continuous shooting B2 mode” subroutine of FIG. 5B is executed, and the process proceeds to step S111. After that, the above operation flow is repeated.

  If the setting for making the D range constant is not made in step S131 (step S131; NO), it is confirmed in step S151 whether the setting for making the exposure constant is made. When the setting for making the exposure constant is made (step S151; YES), the process proceeds to step S251, and it is confirmed whether or not the setting for making the processing D range constant is made in the continuous imaging mode. If the processing D range is set to be constant (step S251; YES), the process proceeds to step S152, the “continuous shooting C1 mode” subroutine of FIG. 6A is executed, the process returns to step 111, and thereafter. The above operation flow is repeated.

  If the setting to make the processing D range constant is not made in step S251 (step S251; NO), the process proceeds to step S252, the “continuous shooting C2 mode” subroutine of FIG. 6B is executed, and the process proceeds to step S111. After that, the above operation flow is repeated.

  If it is determined in step S151 that the exposure is not set to be constant (step S151; NO), the process proceeds to step S261, and it is confirmed whether or not the process D range is set to be constant in the continuous imaging mode. If the processing D range is set to be constant (step S261; YES), the process proceeds to step S161, the “continuous shooting D1 mode” subroutine of FIG. 7A is executed, and the process returns to step 111. The above operation flow is repeated.

  If the setting for making the processing D range constant is not made in step S261 (step S261; NO), the process proceeds to step S262, the “continuous shooting D2 mode” subroutine of FIG. 7B is executed, and the process proceeds to step S111. After that, the above operation flow is repeated.

  In each of the above processes, step S133 “continuous shooting A1 mode”, step S141 “continuous shooting B1 mode”, step S252 “continuous shooting C2 mode”, step S161 “continuous shooting D1 mode” and step S262 “continuous shooting D2 mode”. In this case, the dynamic range of the output image of the digital camera 1 can be controlled to be constant, there is no variation in the brightness and contrast of the images between the frames of continuous imaging, the main subject contrast is not impaired, and the processing time is reduced. It is possible to shorten it.

  FIG. 4A shows a “continuous shooting A1 mode” in step S133 of FIG. 3, that is, a mode subroutine for performing continuous imaging in which the imaging D range, exposure, and processing D range are all constant.

  In step S301, it is confirmed whether or not the release button 101 is operated and the AF / AE switch 101a is turned on. It waits in step S301 until it is turned on. If it is turned on (step S301; YES), an AF operation is performed in step S311 to focus. In step S312, photometry is performed by the photometry module 121b. In step S313, the exposure, that is, the aperture value of the aperture 221 of the interchangeable lens 20 and the shutter speed of the shutter 145 are set from the photometry result.

  In step S314, the imaging D range for the first frame of continuous imaging is set in accordance with the exposure set in step S313. In step S321, it is confirmed whether the release button 101 is operated and the release switch 101b is turned on. Until it is turned on, the operation flow from step S301 to step S321 is repeated. When turned on (step S321; YES), in step S322, imaging is performed with the aperture value and shutter speed set in step S313 and the imaging D range set in step S314, and the image captured in step S323. And image attached information (such as a flag indicating that images were captured in the continuous shooting A1 mode) are temporarily recorded in the image memory 181. The exposure, that is, the aperture value, the shutter speed, and the imaging D range are held at the same setting until the continuous imaging is completed.

  In step S331, it is confirmed whether the release button 101 is released and the release switch 101b is turned off. If it is not turned off (step S331; NO), the process returns to step S322, and imaging is performed with the held exposure and the imaging D range. In step S323, the captured image and information attached to the image are stored in the image memory 181. Once recorded. When the release switch 101b is turned off (step S331; YES), the “process D range constant dodging process subroutine” shown in FIG. 17 is performed on the continuously captured image recorded in the image memory 181 in step S341. In step S332, the continuously captured output image subjected to the dodging process in step S341 and the attached information of the output image are recorded in the memory card 182, and the process returns to the main routine.

  FIG. 4B is a subroutine of the “continuous shooting A2 mode” in step S232 of FIG. 3, that is, a mode for performing continuous imaging in which the imaging D range and exposure are constant and the processing D range is variably set.

  Steps S301 to S331 are the same as those in FIG. However, the attached information of the image recorded in the image memory 181 together with the image in step S323 is “a flag indicating that the image was captured in the continuous shooting A2 mode”. When the release switch 101b is turned off (step S331; YES), the “process D range variable dodging processing subroutine” shown in FIG. 18 is performed on the continuously captured image recorded in the image memory 181 in step S342. In step S332, the continuously captured output image subjected to the dodging process and the attached information of the output image are recorded in the memory card 182, and the process returns to the main routine.

  In the above-described “continuous shooting A1 mode” and “continuous shooting A2 mode”, the “dodge processing subroutine” in step S341 and step S342 is performed after the release switch 101b is turned off in step S331, that is, after the continuous imaging ends. When the image picked up in step S323 and the attached information of the image are recorded in the image memory 181, they may be recorded while performing a dodging process. However, in this case, dodging processing is performed within a range that does not reduce the speed of continuous imaging, and if the processing cannot be completely performed due to processing speed limitations of the CPU 151, the image before processing is temporarily stored in the image memory 181. It is desirable to take measures such as buffering and processing again after the CPU 151 is idle. The same applies to each continuous shooting mode below the “continuous shooting B1 mode”.

  FIG. 5A shows a “continuous shooting B1 mode” in step S141 of FIG. 3, that is, a subroutine for a mode in which continuous imaging is performed with the imaging D range and the processing D range fixed.

  In the “continuous shooting B1 mode”, the flow from step S301 to step S321 is the same as the “continuous shooting A1 mode” in FIG. 4A. In step S422, the aperture value and shutter speed set in step S313, and step Imaging is performed in the D range set in S314, and the imaging D range is held at the same setting until continuous imaging ends. In step S323, the captured image and the attached information of the image (such as a flag indicating that the image is captured in the continuous shooting B1 mode) are temporarily recorded in the image memory 181.

  In step S331, it is confirmed whether the release button 101 is released and the release switch 101b is turned off. If not turned off (step S331; NO), photometry is performed by the photometry module 121b in step S441. In step S442, the currently set exposure, that is, the combination of the aperture value and the shutter speed, is obtained by the photometry result in step S441. In step S443, a process for holding the imaging D range constant in accordance with the exposure reset in step S442 is performed, and the process returns to step S422 to re-establish the stored imaging D range. Imaging is performed with the set exposure.

  When the release switch 101b is turned off (step S331; YES), in step S341, dodging processing is performed on the continuously captured image recorded in the image memory 181 to keep the processing D range constant, and in step S332. Then, the output image of the continuous imaging subjected to the dodging process and the attached information of the output image are recorded in the memory card 182 and the process returns to the main routine.

  FIG. 5B is a “continuous shooting B2 mode” in step S242 of FIG. 3, that is, a mode subroutine for performing continuous imaging in which the imaging D range is constant and the exposure and processing D range are variably set. .

  Steps S301 to S331 and steps S441 to S443 are the same as in FIG. However, the attached information of the image recorded in the image memory 181 together with the image in step S323 is “a flag indicating that the image is captured in the continuous shooting B2 mode”. When the release switch 101b is turned off (step S331; YES), in step S342, dodging processing is performed on the continuously captured image recorded in the image memory 181 with the processing D range being variable, and in step S332. Then, the output image of the continuous imaging subjected to the dodging process and the attached information of the output image are recorded in the memory card 182 and the process returns to the main routine.

  FIG. 6A is a subroutine of the “continuous shooting C1 mode” in step S152 of FIG. 3, that is, a mode for performing continuous imaging with constant exposure.

  In the “continuous shooting C1 mode”, the flow from step S301 to step S321 is the same as the “continuous shooting A1 mode” in FIG. 4A. In step S522, the aperture value and shutter speed set in step S313, and step Imaging is performed in the D range set in S314, and the exposure is held at the same setting until continuous imaging ends. In step S323, the captured image and the attached information of the image (such as a flag indicating that the image is captured in the continuous shooting C1 mode) are temporarily recorded in the image memory 181.

  In step S331, it is confirmed whether the release button 101 is released and the release switch 101b is turned off. If not turned off (step S331; NO), in step S541, photometry is performed by the photometry module 121b. In step S542, the imaging D range is reset based on the photometry result in step S441, and the process returns to step S522. Imaging is performed with the held exposure and the reset imaging D range.

  When the release switch 101b is turned off (step S331; YES), in step S341, dodging processing is performed on the continuously captured image recorded in the image memory 181 to keep the processing D range constant, and in step S332. Then, the output image of the continuous imaging subjected to the dodging process and the attached information of the output image are recorded in the memory card 182 and the process returns to the main routine.

  FIG. 6B is a sub-routine of the “continuous shooting C2 mode” in step S252 of FIG. 3, that is, a mode for performing continuous imaging in which the exposure is constant and the imaging D range and the processing D range are variably set. .

  Steps S301 to S331 and steps S541 to S542 are the same as in FIG. However, the attached information of the image recorded in the image memory 181 together with the image in step S323 is “a flag indicating that the image is captured in the continuous shooting C2 mode”. When the release switch 101b is turned off (step S331; YES), in step S342, dodging processing is performed on the continuously captured image recorded in the image memory 181 with the processing D range being variable, and in step S332. Then, the output image of the continuous imaging subjected to the dodging process and the attached information of the output image are recorded in the memory card 182 and the process returns to the main routine.

  FIG. 7A shows a “continuous shooting D1 mode” in step S161 of FIG. 3, that is, a subroutine for a mode in which exposure and imaging D range are variable and processing D range is constant.

  In the “continuous shooting D1 mode”, the flow from step S301 to step S321 is the same as the “continuous shooting A1 mode” in FIG. 4A. In step S622, the aperture value and shutter speed set in step S313, and the step Imaging is performed in the D range set in S314, but neither exposure nor imaging D range is made constant during continuous imaging. In step S323, the captured image and the attached information of the image (such as a flag indicating that the image is captured in the continuous shooting D1 mode) are temporarily recorded in the image memory 181.

  In step S331, it is confirmed whether the release button 101 is released and the release switch 101b is turned off. If not turned off (step S331; NO), the photometry is performed by the photometry module 121b in step S641, and the exposure, that is, the combination of the aperture value and the shutter speed is reset according to the photometry result in step S641. In step S643, the currently set imaging D range is reset to a default value that matches the exposure reset in step S642, and the process returns to step S622 to reset the reset exposure and the reset D. Imaging is performed in the range.

  FIG. 7B is a “continuous shooting D2 mode” in step S262 of FIG. 3, that is, a subroutine for a mode for performing continuous imaging in which all of the exposure, imaging D range, and processing D range are variably set.

  Steps S301 to S331 and steps S641 to S643 are the same as those in FIG. However, the attached information of the image recorded in the image memory 181 together with the image in step S323 is “a flag indicating that the image is captured in the continuous shooting D2 mode”. When the release switch 101b is turned off (step S331; YES), in step S342, dodging processing is performed on the continuously captured image recorded in the image memory 181 with the processing D range being variable, and in step S332. Then, the output image of the continuous imaging subjected to the dodging process and the attached information of the output image are recorded in the memory card 182 and the process returns to the main routine.

  In the description of FIG. 3 to FIG. 7B described above, the resetting of the exposure is performed by resetting the combination of the aperture value and the shutter speed in accordance with the photometry result. Alternatively, only the shutter speed may be changed.

  Next, a first example of the image sensor 162 having photoelectric conversion characteristics including a linear characteristic area and a logarithmic characteristic area in the present invention will be described with reference to FIGS.

  FIG. 8 is a block diagram showing the internal configuration of the image sensor 162. In the figure, the same parts as those in FIG. Pixels 162 a are two-dimensionally arranged on the image sensor 162. The photoelectric conversion output VP of the horizontal pixel 162a selected by the vertical scanning circuit 162b is output to the vertical signal line 306, simultaneously held for one row in the sample hold circuit 162c, and scanned by the horizontal scanning circuit 162e to output circuit The image is sequentially output from 162 d as an image output 307 and input to the amplifier 163. Each operation of the image sensor 162 is controlled by a timing generator (TG) 162f in accordance with an imaging control signal 305 from the imaging control circuit 161.

  FIG. 9 is a circuit diagram illustrating a first example of a circuit of a pixel 162a that constitutes the imaging element 162 and has a photoelectric conversion characteristic including a linear characteristic region and a logarithmic characteristic region.

  The pixel 162a includes a photodiode PD, transistors T1 to T6 as MOSFETs (Metal Oxide Semiconductor Field Effect Transistors), and a capacitor C as a capacitor for integration. In this example, P-channel MOSFETs are used for the transistors T1 to T6. φVD, φV, φVPS, φRST, φS, and RSB indicate signals (voltages) to the respective transistors and capacitors C, and GND indicates ground.

  The photodiode PD is a photoelectric conversion unit and outputs a photocurrent IPD corresponding to the amount of incident light from the subject.

  The transistor T1 is a switch used when a pixel variation signal indicating an error component between pixels generated due to the manufacturing variation of the transistor T2 is taken out. The transistor T1 is normally turned on, and is between the transistor T2 and the photodiode PD. The photocurrent IPD flows through the first electrode. When taking out the pixel variation signal, the transistor T1 is turned off, the photocurrent IPD of the photodiode PD is cut off, and only the pixel variation signal is taken out.

  The transistor T2 has a gate and a drain connected to each other, and utilizes a subthreshold characteristic in the MOSFET (a characteristic in which a minute current called a subthreshold current flows when the gate voltage is equal to or lower than a threshold value). It serves to generate a linearly or logarithmically converted voltage for the current IPD.

  Specifically, when the subject to be imaged is dark, that is, when the incident light quantity incident on the photodiode PD is small, the gate potential of the transistor T2 is higher than the source potential of the transistor, so that the transistor T2 is a so-called cut-off state, the subthreshold current does not flow in the transistor T2, the photocurrent IPD generated in the photodiode PD flows in the parasitic capacitance CPD of the photodiode PD, the photocurrent IPD is integrated, and the integrated charge amount is obtained. A corresponding voltage is generated.

  At this time, since T1 is turned on, a voltage corresponding to the photocurrent IPD integrated into the parasitic capacitance CPD is generated as a voltage VG at the gates of the transistors T2 and T3. The voltage VG causes a current to flow through the transistor T3, and an amount of charge proportional to the voltage VG is accumulated in the capacitor C (the transistor T3 and the capacitor C constitute an integrating circuit). A voltage linearly proportional to the integral value of the photocurrent IPD appears at the connection node a between the transistor T3 and the capacitor C, that is, the output VOUT. This is the operation in the linear characteristic region.

  On the other hand, when the subject to be imaged is bright and the amount of incident light incident on the photodiode PD is large, the gate potential of the transistor T2 becomes lower than the source potential of the transistor, and the transistor T2 operates in the subthreshold region. A threshold current flows, and a voltage VG having a value obtained by natural logarithm conversion of the photocurrent IPD is generated at the gates of the transistors T2 and T3. The voltage VG causes a current to flow through the transistor T3, and a charge equivalent to a value obtained by natural logarithmically converting the integrated value of the photocurrent IPD is accumulated in the capacitor C. As a result, a voltage proportional to a value obtained by natural-logarithmically converting the integral value of the photocurrent IPD is generated at the connection node a (output VOUT) between the capacitor C and the transistor T3. This is the operation of the imaging sensor 30 in the logarithmic characteristic region.

  As described above, whether the operation of the pixel 162a has a linear characteristic or a logarithmic characteristic is determined by the relationship between the gate potential VG and the source potential φVPS of the transistor T2. Since the gate potential VG is determined by the photocurrent IPD as described above, the point (inflection point) at which the linear characteristic and the logarithmic characteristic are switched can be controlled by controlling the source potential φVPS.

  As described above, a voltage linearly or naturally logarithmically proportional to the integrated value of the photocurrent IPD is generated for each pixel in accordance with the brightness of the subject, that is, the amount of incident light.

  The transistor T4 is a transistor for resetting the capacitor C, and operates as a switch that is turned on / off according to the voltage φRST applied to the gate of the transistor T4. When the transistor T4 is turned on, the reset voltage RSB is applied to the capacitor C, and the accumulated charge is returned to the state before the start of integration.

  The transistor T5 constitutes a source follower amplifier circuit, and functions to lower the output impedance by performing current amplification on the output VOUT described above.

  The transistor T6 is a signal reading transistor, and operates as a switch that is turned on and off in accordance with the voltage φV applied to the gate. The source of the transistor T6 is connected to the output signal line 306. When the transistor T6 is turned on, the photoelectric conversion output VP that has been subjected to current amplification by the transistor T5 and reduced in impedance is led to the output signal line 306.

  Next, a photoelectric conversion characteristic of the image sensor 162 using the pixel 162a illustrated in FIG. 9 and a resetting method of exposure control and imaging D range control during continuous imaging based on the photoelectric conversion characteristics will be described in detail. FIG. 10 is a schematic diagram showing the photoelectric conversion characteristics of the pixel 162a of the image sensor 162 when both the exposure and the imaging D range are held constant, and FIG. 11 shows only the exposure while holding the imaging D range constant. FIG. 12 is a schematic diagram showing changes in the photoelectric conversion characteristics of the pixel 162a of the image sensor 162 when the image sensor 162 is operated. FIG. 12 shows the photoelectric conversion characteristics of the pixel 162a of the image sensor 162 when controlling only the imaging D range while keeping exposure constant. FIG. 13 is a schematic diagram showing a change in photoelectric conversion characteristics of the pixel 162a of the image sensor 162 when both the imaging D range and exposure are controlled. 10 to 13, the horizontal axis represents the logarithmic coordinates of the subject brightness B (logarithm value of the subject brightness), and the vertical axis represents the photoelectric conversion output VP.

  First, continuous shooting A1 mode and A2 mode imaging (corresponding to the flowcharts of FIGS. 4A and 4B) in which both the exposure and the D range are kept constant will be described with reference to FIG.

  The photoelectric conversion characteristic 401 is a photoelectric conversion indicating an appropriate D range determined by, for example, the method described in Japanese Patent Application No. 2004-159760 described above from the result of the photometric operation in step S312 of FIGS. 4 (a) and 4 (b). It is a characteristic. The photoelectric conversion characteristic 401 is divided into a linear characteristic region (the dark side of the subject luminance) and a logarithmic characteristic region (the same, the bright side) with the inflection point 411 (the photoelectric conversion output at this time is Vth41) as a boundary.

  In the continuous shooting A1 mode and A2 mode, in step S313 until the release switch 101b is confirmed in step S331 after the release switch 101b is confirmed in step S321 in FIGS. 4A and 4B. The set exposure, that is, the aperture value and the shutter speed, and the imaging D range DR41 set in step S314, that is, the photoelectric conversion characteristic 401 are held, and continuous imaging is performed. In this case, it is not necessary to calculate and control the exposure and the imaging D range during the continuous imaging from step S321 to step S331, so that the continuous imaging speed can be increased.

  Next, continuous shooting B1 mode and B2 mode imaging (corresponding to the flowcharts of FIGS. 5A and 5B) in which only the exposure is controlled while keeping the imaging D range constant will be described with reference to FIG. .

  The photoelectric conversion characteristic 502 is a photoelectric conversion characteristic at a proper exposure obtained by the photometric operation in step S312 of FIGS. 5A and 5B. The photoelectric conversion characteristic 502 is divided into a linear characteristic area (the dark side of the subject brightness) and a logarithmic characteristic area (the same bright side) with an inflection point 512 (the photoelectric conversion output value at this time is Vth52) as a boundary.

  With respect to this photoelectric conversion characteristic 502, in the continuous shooting B1 mode and the B2 mode, when the subject brightness changes during continuous imaging, from the result of photometry in step S441 in FIGS. Appropriate exposure determined by the method described in Japanese Patent Application No. 2004-159760, that is, the aperture and shutter speed are calculated, and the aperture and shutter speed are reset.

  At this time, if the control value of the source potential φVPS of the transistor T2 in FIG. 9 is kept at Vth52, only the slope of the curve in the linear characteristic region changes, and the switching point between the linear characteristic region and the logarithmic characteristic region and the logarithmic characteristic region are changed. Since the imaging D range is changed in parallel with the horizontal direction in FIG. 11, the transistor in FIG. 9 is matched with the reset aperture value and shutter speed in order to keep the imaging D range DR52 constant. The control value of the source potential φVPS of T2 is also reset.

  The photoelectric conversion characteristics 501 in FIG. 11 are characteristics when the subject brightness is higher than the subject brightness in the photoelectric conversion characteristics 502, and the photoelectric conversion characteristics 503 are characteristics when the subject brightness is dark. The inflection point is from Vth52. They are changed to Vth51 and Vth53, respectively.

  As a result, compared to the continuous shooting A1 mode and the A2 mode, calculation time is required, and the continuous imaging speed is slightly slower. However, the subject brightness changes because the imaging D range is kept constant. In addition, the subject luminance range that can be imaged does not change, i.e., it is possible to perform imaging that is not crushed on the high luminance side.

  In actual continuous imaging, based on the result of photometry in step S312, first, in step S313, an aperture value and a shutter speed corresponding to the photoelectric conversion characteristics 502 are set, and in step S314, the imaging D range is set to DR52. Then, the control value of the source potential φVPS of the transistor T2 is set to set the imaging D range, and imaging is performed in step S422. Next, in step S441, photometry is performed again. If the exposure calculated from the result is, for example, the photoelectric conversion characteristic 501, the aperture value and shutter speed corresponding to the photoelectric conversion characteristic 501 are reset in step S442, and In step S443, the control value of the source potential φVPS of the transistor T2 is reset so that the imaging D range is kept constant at DR52, and imaging is performed in step S422. Until the release switch 101b is confirmed to be off in step S331, continuous imaging is continued while rotating through the above loop.

  Next, continuous shooting C1 mode and C2 mode imaging (corresponding to the flowcharts of FIGS. 6A and 6B) in which exposure is held constant and only the imaging D range is controlled will be described with reference to FIG. .

  The photoelectric conversion characteristic 602 is a photoelectric characteristic indicating an appropriate imaging D range determined by, for example, the method described in Japanese Patent Application No. 2004-159760 described above from the result of the photometric operation in step S312 of FIGS. 6 (a) and 6 (b). It is a conversion characteristic. The photoelectric conversion characteristic 602 is divided into a linear characteristic region (the dark side of the subject luminance) and a logarithmic characteristic region (the same, the bright side) with an inflection point 612 (the photoelectric conversion output at this time is Vth62) as a boundary.

  In contrast to this photoelectric conversion characteristic 602, in the continuous shooting C1 mode and the C2 mode, when the subject brightness changes during continuous imaging, for example, the above-mentioned Japanese Patent Application No. 2004-159760 is obtained from the result of photometry in step S541 in FIG. The appropriate imaging D range determined by the method described in (1) is calculated, and the control value of the source potential φVPS of the transistor T2 in FIG. 9 is reset. At this time, the exposure, that is, the aperture value and the shutter speed are kept the same as the photoelectric conversion characteristic 602.

  The photoelectric conversion characteristic 601 is a characteristic when the subject luminance is dark, and the photoelectric conversion characteristic 603 is a characteristic when the subject luminance is high. The photoelectric conversion output at the inflection point is Vth61 with respect to the photoelectric conversion characteristic 402, respectively. Are shifted to DR61 narrower than DR62 and wide DR63, respectively.

  This makes it possible to always set an appropriate imaging D range with respect to changes in subject luminance during continuous imaging, even with constant exposure.

  In actual continuous imaging, based on the result of the photometric operation in step S312, in step S314, a control value of the source potential φVPS of the transistor T2 that sets the imaging D range DR62 of the photoelectric conversion characteristic 602 is set, and in step S522. Thus, imaging is performed using the control value. Next, in step S541, photometry is performed again. If the imaging D range calculated from the result is, for example, the photoelectric conversion characteristic 601, the imaging D range DR61 corresponding to the photoelectric conversion characteristic 601 is obtained in step S542. The control value of the source potential φVPS of the transistor T2 is reset, and imaging is performed in step S522. Until the release switch 101b is confirmed to be off in step S331, continuous imaging is continued while rotating through the above loop.

  Finally, continuous shooting D1 mode and D2 mode imaging (corresponding to the flowcharts of FIGS. 7A and 7B) in which the imaging D range and exposure are variably controlled will be described with reference to FIG.

  The photoelectric conversion characteristic 702 is a photoelectric conversion characteristic with appropriate exposure obtained by the photometric operation in step S312 of FIGS. 7A and 7B. The photoelectric conversion characteristic 702 is divided into a linear characteristic region (the dark side of the subject luminance) and a logarithmic characteristic region (the same, the bright side) with an inflection point 712 (the photoelectric conversion output value at this time is Vth72) as a boundary.

  With respect to this photoelectric conversion characteristic 702, in the continuous shooting D1 mode and the D2 mode, when the subject brightness changes during continuous imaging, from the result of photometry in step S641 in FIGS. Appropriate exposure determined by the method described in Japanese Patent Application No. 2004-159760, that is, the aperture and shutter speed are calculated, and the aperture and shutter speed are reset.

  Furthermore, the imaging D range is also reset to the control value of the source potential φVPS of the transistor T2 of FIG. 9 corresponding to the imaging D range shifted by a predetermined amount corresponding to the reset aperture value and shutter speed. The photoelectric conversion characteristics 701 in FIG. 13 are characteristics when the subject brightness is higher than the subject brightness in the photoelectric conversion characteristics 702, and the photoelectric conversion characteristics 703 are characteristics when the subject brightness is dark, and the inflection point is also from Vth72. They are changed to Vth71 and Vth73, respectively.

  As a result, even in an imaging device using an imaging device having a photoelectric conversion characteristic composed of a linear characteristic region and a logarithmic characteristic region, the brightness and contrast of images continuously captured by a normal silver salt film camera or a normal digital camera can be reduced. Close continuous captured images are obtained.

  In actual continuous imaging, based on the result of the photometric operation in step S312, first, in step S313, an aperture value and shutter speed corresponding to the photoelectric conversion characteristics 702 are set, and in step S314, the imaging D range becomes DR72. Thus, the control value of the source potential φVPS of the transistor T2 is set, and imaging is performed in step S622. Next, in step S641, photometry is performed again. If the imaging D range calculated from the result is, for example, the photoelectric conversion characteristic 701, the aperture value and shutter speed corresponding to the photoelectric conversion characteristic 701 are reset in step S642. In step S643, the control value of the source potential φVPS of the transistor T2 is reset so that the imaging D range DR71 is obtained, and imaging is performed in step S622. Until the release switch 101b is confirmed to be off in step S331, continuous imaging is continued while rotating through the above loop.

  Next, a second example of the image sensor 162 having a photoelectric conversion characteristic composed of a linear characteristic region and a logarithmic characteristic region in the present invention will be described with reference to FIG. FIG. 14 is a circuit diagram illustrating a second example of a circuit of a pixel 162a that constitutes the imaging element 162 and has a photoelectric conversion characteristic including a linear characteristic region and a logarithmic characteristic region.

  The pixel 162a includes a buried photodiode PD10 and transistors T11 to T14 as N-channel MOSFETs. A connection portion between the drain of the transistor T11 and the source of T12 is configured by a floating diffusion FD. φRSB, φRST, φTX, and φV indicate signals (voltages) for the respective transistors, VDD indicates a power supply, and GND indicates ground.

  The embedded photodiode PD10 is a photoelectric conversion unit, and generates a photocurrent IPD corresponding to the amount of incident light from the subject. Since the embedded photodiode cannot directly extract the photoelectrically converted photocurrent, it outputs it through a transfer gate and an extraction portion called a floating diffusion.

  The transistor T11 is called a transfer gate, and is an element for transferring the photoelectric charge photoelectrically converted by the embedded photodiode PD10 to the floating diffusion FD. The transistor T11 is normally turned on, and is between the transistor T12 and the photodiode PD10. The photocurrent IPD flows through the first electrode.

  The transistor T12 is called a reset gate, and resets the buried photodiode PD10 and the floating diffusion FD to a high potential, and a subthreshold characteristic in the MOSFET (a small current called a subthreshold current is generated when the gate voltage is equal to or lower than a threshold value). Using the flow characteristic), the source of the transistor functions to generate a voltage that is linearly or logarithmically converted with respect to the photocurrent IPD.

  Specifically, when the subject to be imaged is dark, that is, when the amount of incident light incident on the photodiode PD10 is small, the reset operation causes the gate potential of the transistor T12 to be lower than the source potential of the transistor. In addition, the transistor T12 is in a so-called cut-off state, the subthreshold current does not flow through the transistor T12, and the photocurrent IPD generated in the embedded photodiode PD10 is reset to a high potential, and the parasitic capacitance CPD10 of the embedded photodiode PD10 is reset. , The photocurrent IPD is integrated, and a voltage VFD corresponding to the integrated charge amount is generated in the floating diffusion FD. This is the operation in the linear characteristic region.

  On the other hand, when the subject to be imaged is bright and the amount of incident light incident on the photodiode PD10 is large, the gate potential of the transistor T12 becomes equal to or higher than the source potential of the transistor, and the transistor T12 operates in the subthreshold region. A threshold current flows, and a voltage VFD obtained by natural logarithm conversion of the photocurrent IPD is generated in the floating diffusion FD. This is the operation in the logarithmic characteristic region. Since the second example does not have the integrating capacitor C as in the first example shown in FIG. 8, the output VFD in the logarithmic characteristic region is not obtained by logarithmically converting the integral value of the photocurrent IPD. It is not a logarithmic conversion value of the photocurrent IPD at that moment.

  As described above, whether the operation of the pixel 162a has a linear characteristic or a logarithmic characteristic depends on the relationship between the gate potential and the source potential of the transistor T12. Therefore, the point (inflection point) at which the linear characteristic and the logarithmic characteristic are switched can be controlled by appropriately controlling the gate potential φRST. Further, the inflection point between the linear characteristic and the logarithmic characteristic can be controlled by controlling the gate potential φTX of the transistor T11 instead of the transistor T12.

  The transistor T13 constitutes a source follower amplifier circuit, and functions to lower the output impedance by performing current amplification on the output VFD described above.

  The transistor T14 is a signal reading transistor, and operates as a switch that is turned on and off in accordance with the voltage φV applied to the gate. The source of the transistor T14 is connected to the output signal line 306. When the transistor T14 is turned on, the photoelectric conversion output VP that has been amplified by the transistor T13 and reduced in impedance is led to the output signal line 306.

  When continuous imaging is performed using the pixel circuit of the second example, the imaging D range may be set by appropriately controlling φRST in the flowcharts shown in FIGS. 4A to 7B. . However, when the control shown in the flowcharts of FIGS. 5A and 5B is performed, if the imaging D range is set in step S314, as described above, in the logarithmic characteristic region, the photoelectric conversion output is Since it is a logarithmically converted value of the instantaneous photocurrent IPD, changing exposure (aperture value and shutter speed) in step S442 does not require changing φRST to make the imaging D range constant in step S443. , Vth automatically changes to keep the imaging D range constant, so that the photoelectric conversion characteristics shown in FIG. 11 can be obtained.

  In addition, as shown in Patent Document 4, an imaging device that synthesizes an image of a wider D range than individual screens by selecting and synthesizing a screen portion of an appropriate level from a plurality of screens having different exposure amounts. The image sensor shown in Patent Document 5 has a charge-voltage converter that converts photoelectric conversion signal charges into signal voltages, and the charge-voltage converter comprises a plurality of capacitors having different voltage dependencies, and the D range is variable. The present invention is also applicable to an imaging apparatus using a wide D range imaging element other than a logarithmic conversion type such as an imaging element.

  Next, a first embodiment of the dodging process will be described with reference to FIGS. FIG. 15 is a functional block diagram for explaining each function of the first embodiment related to the dodging process of the image processing apparatus 165 of FIG. As shown in the figure, the first embodiment of the image processing apparatus 165 includes a main subject luminance acquisition unit 1651, a characteristic conversion unit 1652, an illumination component extraction unit 1653, a compression start point setting unit 1654, an illumination component compression unit 1655, A reflectance component calculation unit 1656 and an image generation unit 1657 are provided. The image processing device 165 functions as a compression processing unit and a compression rate control unit in the present invention.

  The main subject luminance acquisition unit 1651 acquires (calculates) the main subject luminance of an image (original image I) obtained by imaging with the image sensor 162. The main subject luminance acquisition unit 1651 is a central region (referred to as a main subject region) in which an imaging region is divided into a plurality of (for example, 36) detection blocks by the split photometry (multi-pattern photometry) method using the photometry module 121b of FIG. ) And a plurality of (for example, 16) detection blocks divided into a plurality of sections (peripheral subject areas) and divided into a plurality of sections, and from the brightness information detected from the images of the sections, for example, average brightness The main subject brightness is calculated as follows.

  In this case, for example, a main subject brightness histogram (distribution) for each block of the main subject area is calculated, and a main subject overall brightness histogram in the entire main subject area is calculated from the main subject brightness histogram. The average luminance for the main subject area may be calculated from

  In calculating the average luminance, for example, luminance data “cut off” processing may be performed using a certain predetermined threshold, or luminance information of a peripheral subject area (a peripheral subject luminance histogram or a peripheral subject overall luminance histogram). ) May be used. Note that the method for calculating the main subject luminance is not limited to the above-described method, and various methods can be employed. Further, the main subject brightness may not be calculated as the average brightness, and may be calculated as, for example, the maximum brightness or the minimum brightness.

  Next, FIG. 16 is a graph for explaining the first embodiment of the dodging process described above. As shown in FIG. 16, the characteristic conversion unit 1652 performs a process for unifying the characteristics (characteristic unification process) for the original image I having the photoelectric conversion characteristics including the linear characteristics 301 and the logarithmic characteristics 302. This original image I is an input image from the image sensor 162 and has a relational expression (photoelectric conversion characteristic) of the pixel value y with respect to the input luminance x as shown in the following expressions (1) and (2).

  However, the coordinate point (Xth, Yth) in the figure shows the inflection point 303, the expression (1) shows the image I2 having the linear characteristic 301, and the expression (2) shows the image I1 having the logarithmic characteristic 302. Show. However, the symbol “*” in the expression indicates multiplication, and a, b, α, and β indicate predetermined coefficients (the same applies to the following).

y = a * x + b (0 ≦ x ≦ Xth) (1)
y = α * log (x) + β (x> Xth) (2)
Here, the characteristic conversion unit 1652 performs a process of converting the logarithmic characteristic 302 into a linear characteristic 304 that is the same characteristic as the linear characteristic 301. In this case, assuming that the pixel value of the image sensor 162 is i, an image (image It shown by reference numeral 310) having a photoelectric conversion characteristic made up of the linear characteristic 301 and the linear characteristic 304 obtained by the characteristic unification process is as follows: 3) given by the transformation based on the equation. However, the symbol “/” in the formula indicates division (and so on).

if (i> Yth)
It = a * exp ((i−β) / α) + b
else
It = i (3)
The characteristic conversion unit 1652 performs conversion from the logarithmic characteristic 302 to the linear characteristic 304 using a predetermined LUT (referred to as a conversion LUT). The conversion LUT may be stored in, for example, an LUT storage unit (not shown) provided in the characteristic conversion unit 1652.

  The illumination component extraction unit 1653 extracts an illumination component from the image It (linear image) obtained by the characteristic unification process. According to the so-called Retinex theory, this image It is represented by the following equation (4), where the illumination component in the image It is an illumination component L and the reflectance component is a reflectance component R. Note that the straight line graph shown in the image It is appropriately handled as representing the illumination component L extracted from the image It.

It = L * R (4)
The extraction process of the illumination component L from the image It is performed using a so-called edge maintaining filter (nonlinear filter) such as a median filter or an epsilon (ε) filter. This is expressed by the following equation (5).

L = F (It) (5)
However, “F” in the equation (5) indicates the edge maintaining filter. Note that a normal LPF (low-pass filter; linear filter) that simply passes a low-frequency component in a narrow sense cannot accurately extract the change at the illumination boundary (edge) of the image, that is, a linear filter is used. If the D-range compression is performed based on the signal extracted in this way, for example, artifacts (dark and sunk portions of the image) appear, so that edge maintenance can be performed to avoid such problems and to accurately extract edges. A filter is used.

  The compression start point setting unit 1654 sets the compression start point S indicated by reference numeral 305, specifically, the compression start level Ys when performing the compression process on the illumination component L extracted from the image It (see the input luminance). The compression start level Xs may be set). In this case, the compression start level Ys is set to be a value equal to or higher than the main subject luminance given by the above average luminance or the like. As a method for setting the compression start level Ys that is equal to or higher than the main subject luminance, for example, a value obtained by multiplying the main subject luminance value by a predetermined number may be used as the compression start level Ys.

  For example, when the main subject is a human face, the specific value (magnification) of the predetermined number of times is about 2 to 3 times at which appropriate brightness can be obtained by performing gamma correction. Alternatively, when the main subject is a landscape, it may be about double. However, the magnification is not limited to these, and any value can be adopted. Further, this magnification may be set as one fixed value calculated according to the subject assumed in advance, or may be set differently depending on the subject.

  In this case, for example, one selected from a plurality of fixed values based on an instruction input from the operation unit 10 by the user may be set, or the main subject luminance obtained from the photometric result in the main subject luminance acquisition unit 1651. Depending on the value, a value automatically selected from a plurality of fixed values may be set, or a photographing mode (for example, “portrait (person)) obtained from the photometric result and photographing magnification (focal length) “Mode” or “landscape” mode) may be set according to the type of the main subject identified. In any case, the compression start level Ys is appropriately set according to the type and luminance information of the main subject (subject), and is not the main subject luminance level (or the latest level) but a level higher than the main subject luminance by a predetermined value. Therefore, it is possible to more reliably prevent the influence of the D range compression on the main subject regardless of the difference between the various subjects.

  Further, the method of setting the compression start level Ys that is a value equal to or higher than the main subject luminance is not limited to the above-described method, and for example, a predetermined luminance histogram based on luminance information of the main subject and / or surrounding subjects is calculated. Alternatively, a predetermined level in a region higher than the main subject luminance, for example, a luminance value (luminance level) with a low frequency may be set as the compression start level Ys. Specifically, if the luminance histogram distribution has a so-called “mountain” shape and the vicinity of the peak of the peak is the main subject luminance, the frequency of the luminance histogram equal to or higher than the main subject luminance is small. Set so that the compression start level is around the foot of the mountain.

  Thus, by performing the compression with the luminance value having a low frequency as the compression start level Ys (because it is possible to avoid the compression of the high frequency portion in the luminance histogram), the change in the gradation characteristics can be made inconspicuous. . Note that the setting of the compression start level Ys based on this luminance histogram may be performed each time a captured image is obtained by the image sensor 162.

  Further, when the imaging element 162 is a linear log sensor as shown in the present embodiment, the compression start level Ys is obtained as, for example, Ys = Yth * P (P; a predetermined multiple value smaller than 1), etc. A value linked to the photoelectric conversion output value Yth at the inflection point may be set. This is because the photoelectric conversion output value Yth at the inflection point is obtained from the main subject luminance level at the stage where automatic exposure control (AE control) is performed by the digital camera 1, that is, before the dodging process is performed. Is also set to a large value.

  The AE control is performed using an evaluation value for AE control (AE evaluation value) obtained from the captured image in the image processing device 165. In any case, in short, the compression start level Ys may be set to a level at which the main subject luminance is compressed in the D range and the gradation of the main subject is not crushed.

  The illumination component compression unit 1655 performs D range compression on the illumination component L extracted from the image It by the illumination component extraction unit 1653. By this D-range compression by the illumination component compression unit 1655, the illumination component L indicated by reference numeral 310 becomes a compressed illumination component L ′ indicated by reference numeral 320. This compressed illumination component L ′ is given by the following equation (6).

L ′ = exp (log (L) * c) * n (6)
However, “c” is a D-range compression rate, and “n” is a normalization term.

  As illustrated in FIG. 16, the illumination component image having the D range of 0 (zero) to Ymax (maximum linear characteristic) is compressed into the D range of 0 (zero) to Omax (imaging D) by the D range compression of the illumination component. The D range is compressed so as to be within the range. This Omax is the maximum output value of the predetermined image output system (may be the maximum output value or maximum pixel value of the image sensor 162), and takes a gradation value of “255” in an 8-bit image, for example.

  The function L ′ indicating the illumination component after D range compression shown in the above equation (6) is a compression level indicated by reference numeral 306 at the compression start point S (Xs, Ys) and the output maximum value Omax (or the maximum input luminance Xmax). In order to pass through two points M ′ (Xmax, Ymax), the two unknowns c and n in the equation (6) are obtained from simultaneous equations obtained by substituting the coordinate values of these points S and M ′, respectively. Can be calculated. During the continuous imaging, the dynamic range of the imaging output of the imaging element 162 can be kept constant by keeping the above-described D range compression ratio c, normalization term n, and compression start level Ys constant.

  In addition, in the D range compression here, the parameter “n” is introduced in addition to the compression parameter “c”, for example, the D range (8-bit image) in which the captured image of the image sensor 162 has 0 to 255 gradations. If the compressed illumination component L ′ after D range compression has a gradation of, for example, 0 to 100, the value of “n” is set to 2.5, for example, to match this with a gradation of 0 to 255. This is to enable adjustment (normalization of the compressed illumination component L ′) such that the entire gradation value is multiplied by 2.5.

  The reason why the compression start point S is set in the D range compression for the illumination component L is to prevent the main subject luminance from being compressed in the D range as described above. Although the compression characteristic 320 having the compression characteristic 307 is used in the D range compression for L, the compression illumination component L ′ obtained by the compression characteristic 320 is not used as an output value in the region where the illumination component L is less than the compression start level Ys. Then, the process of using the original image I as an output value is performed.

  In order to obtain the compressed illumination component L ′, a lookup table (hereinafter referred to as LUT) that receives the illumination component L extracted from the image It as an input and outputs the compressed illumination component L ′ after compression in the equation (6). ) Saves time and is efficient. At this time, the contents of the LUT during continuous imaging are fixed.

  The reflectance component calculation unit 1656 calculates (extracts) the reflectance component R from the image It. The reflectance component R is calculated by the following equation (7) using the illumination component L.

R = It / L (7) (derived from the above equation (4))
The image generation unit 1657 calculates the original image I from the compressed illumination component L ′ obtained by the illumination component compression unit 1655 and the reflectance component R obtained by the reflectance component calculation unit 1656 by the following equation (8). A new image I ′ (image I ′ after the dodging process) is generated.

I '= L' * R (8)
In FIG. 16, an image I ′ is obtained by multiplying the entire compressed illumination component L ′ indicated by reference numeral 320 by the reflectance component R.

  FIG. 17 is a flowchart of step S341 “Dodge processing subroutine with constant processing D range” used in FIG. 4A and others in the first embodiment of the dodging process.

  First, in step S701, the characteristic conversion unit 1652 performs characteristic unification processing to linear characteristics to obtain an image It. Next, in step S702, the illumination component extraction unit 1653 extracts the illumination component L from the image It based on the edge maintenance filter (nonlinear filter) processing. However, this illumination component extraction processing is performed simultaneously on all the pixels of the image It.

  In step S703, the compression start level setting unit 1654 sets the compression start level Ys, and the illumination component compression unit 1655 determines whether the illumination component L is equal to or higher than the set compression start level Ys. . When it is determined that the compression start level is equal to or higher than the compression start level Ys (step S703; YES), the reflectance component calculation unit 1656 calculates the reflectance component R in step S704.

  In step S705, the illumination component compression unit 1655 compresses the illumination component L in the D range based on the information of the compression start point S (compression start level Ys) and the compression level point M ′ to obtain a compressed illumination component L ′. . At this time, in order to keep the processing D range constant, parameters for determining the D range compression characteristics (D range compression rate c, normalization term n, and compression start level Ys) are one frame of continuous imaging during continuous imaging. It is fixed to a value determined at the time of eye imaging. As a result, it is possible to obtain a continuously captured image without variations in brightness and contrast between frames.

  In step S706, an image I ′ is generated and output from the reflectance component R and the compressed illumination component L ′ obtained in steps S704 and 705.

  If it is determined in step S703 that the illumination component L is less than the compression start level Ys (step S703; NO), the original image I is selected and output in step S707. However, the operations in steps S703 to S707 are sequentially executed for each pixel of the illumination component L (for one pixel). In step S708, it is confirmed whether or not the operations in steps S703 to S707 have been completed for all pixels. If completed (step S708; YES), the dodging process ends. If not completed (step S708; NO), the process returns to step S703, and the processes of steps S703 to S707 are repeated until the processing of all pixels is completed.

  Next, FIG. 18 is a flowchart of step S342 “Process D range variable dodging process subroutine” used in FIG. 4B and others in the first embodiment of the dodging process. The same steps as those in FIG. 17 are assigned the same numbers.

  Steps S701 to S704 are the same as those in FIG. In step S715, the illumination component compression unit 1655 compresses the illumination component L based on the information of the compression start point S (compression start level Ys) and the compression level point M ′ to obtain a compressed illumination component L ′. . At this time, the parameters (D range compression rate c, normalization term n, and compression start level Ys) that determine the D range compression characteristics are not fixed, and are reset to optimum values for each frame of continuous imaging. As a result, the optimum dodging process is performed on the captured image for each frame of continuous imaging.

  In the example described above, the dodging process has been described as being performed by the image processing device 165 in the digital camera 1, but the image data before the dodging process is read from the digital camera 1 and is read by an external personal computer (PC). A dodging process may be performed. In this case, the image data of continuous imaging before the dodging process is recorded in the memory card 182, and tags for recording various information are prepared for this image data, and the continuous imaging information (“ The dodging process is performed based on the information in which mode from “continuous shooting A1 mode” to “continuous shooting D2 mode”, continuous shooting related information such as information on exposure, information on imaging D range, and the like.

  Furthermore, for example, there is a D-range compression method that obtains a brightness value adjusted by a conversion formula from brightness values at different positions of an image, which is proposed as an image quality improvement technique in “Special Table 2004-530568”. When this method is used as the processing D range control means in the present invention, the coefficient α in the conversion equation according to claim 1 and claim 4 and / or the conversion function F according to claim 1, claim 4, or claim 9. By fixing (I), the processing D range can be made constant.

  As described above, according to the present invention, in an imaging device capable of variably controlling the dynamic range of an output image, control is performed so that the dynamic range of the output image in continuous imaging is constant. It is possible to provide an imaging apparatus and an imaging system capable of obtaining a group of continuously captured images having a constant dynamic range during continuous imaging regardless of changes in luminance. According to the inventions of claims 2, 3 and 7, the processing time during continuous imaging can be shortened by fixing the dynamic range of the imaging device or imaging output. Furthermore, according to the third and seventh aspects of the invention, it is possible to provide an imaging apparatus and an imaging system in which there is no variation in image brightness or contrast between frames in continuous imaging.

  It should be noted that the detailed configuration and detailed operation of each component constituting the imaging apparatus and imaging system according to the present invention can be changed as appropriate without departing from the spirit of the present invention.

1 is a schematic external view of a digital camera that is an example of an imaging apparatus according to the present invention. It is a block diagram which shows an example of the circuit of the digital camera shown in FIG. It is the main routine of the flowchart which shows the flow of operation | movement of the continuous imaging in this invention. (A) is a subroutine (1/8) of the flowchart showing the flow of the continuous imaging operation in the present invention. (B) is a subroutine (2/8) of the flowchart showing the flow of the continuous imaging operation in the present invention. (A) is a sub-routine (3/8) of the flowchart showing the flow of the continuous imaging operation in the present invention. (B) is a sub-routine (4/8) of the flowchart showing the flow of the continuous imaging operation in the present invention. (A) is a sub-routine (5/8) of the flowchart showing the flow of the continuous imaging operation in the present invention. (B) is a subroutine (6/8) of the flowchart showing the flow of the operation of continuous imaging in the present invention. (A) is a subroutine (7/8) of the flowchart showing the flow of the continuous imaging operation in the present invention. (B) is a sub-routine (8/8) of the flowchart showing the flow of the continuous imaging operation in the present invention. It is a block diagram which shows the internal structure of the image pick-up element in this invention. FIG. 3 is a circuit diagram illustrating a first example of a pixel circuit having a photoelectric conversion characteristic including a linear characteristic region and a logarithmic characteristic region. FIG. 5 is a schematic diagram illustrating changes in photoelectric conversion characteristics of pixels of the image sensor according to the control flow illustrated in FIGS. It is a schematic diagram which shows the change of the photoelectric conversion characteristic of the pixel of an image pick-up element by the control flow shown to Fig.5 (a) and (b). It is a schematic diagram which shows the change of the photoelectric conversion characteristic of the pixel of an image pick-up element by the control flow shown to Fig.6 (a) and (b). It is a schematic diagram which shows the change of the photoelectric conversion characteristic of the pixel of an image pick-up element by the control flow shown to Fig.7 (a) and (b). It is a circuit diagram which shows the 2nd example of the pixel circuit which has the photoelectric conversion characteristic which consists of a linear characteristic area | region and a logarithmic characteristic area | region. It is a functional block diagram for demonstrating each function of 1st Embodiment of an image processing apparatus. It is a graph explaining 1st Embodiment of dodging process. It is a flowchart of "a dodging process subroutine with constant processing D range" in the first embodiment of the dodging process. 6 is a flowchart of a “process D range variable dodging process subroutine” according to the first embodiment of the dodging process. It is a schematic graph explaining the prior art example of a dodging process. It is a schematic graph which shows the problem of the prior art example of a dodging process.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Digital camera 10 Body 20 Interchangeable lens 101 Release button 101a AF / AE switch 101b Release switch 102 Flash 111 Power switch 112 Mode setting dial 115 Jog dial 121 Viewfinder 121a Viewfinder eyepiece 121b Photometric module 131 Image display device 132 Infinder display means 141 Release Flex mirror 142 Sub mirror 143 Mirror drive means 144 AF module 145 Shutter 146 Shutter drive means 147 Flash control circuit 150 Camera control circuit 151 CPU (central processing unit)
152 Work Memory 153 Storage Unit 161 Imaging Control Circuit 162 Imaging Device 163 Amplifier 164 Analog / Digital (A / D) Converter 165 Image Processing Device 171 Mount (Body Side)
172 BL communication part (Body side)
181 Image memory 182 Memory card 201 Optical axis 211 Lens 221 Aperture 222 Aperture control circuit 231 Lens information storage unit 241 Lens control circuit 251 Lens interface 271 Mount (lens side)
272 BL communication part (lens side)

Claims (7)

In an imaging apparatus provided with mode setting means for setting a continuous imaging mode,
Comprising output dynamic range control means for controlling the dynamic range of the output image of the imaging device;
When the continuous imaging mode is set by the mode setting means,
The said output dynamic range control means controls the dynamic range of the output image of the said imaging device during a continuous imaging uniformly, The imaging device characterized by the above-mentioned. In an imaging apparatus provided with mode setting means for setting a continuous imaging mode,
An image sensor having a plurality of different photoelectric conversion characteristics;
Imaging dynamic range control means for controlling the dynamic range of the imaging device,
When the continuous imaging mode is set by the mode setting means,
The imaging dynamic range control means controls the dynamic range of the imaging element during continuous imaging to be constant. In an imaging apparatus provided with mode setting means for setting a continuous imaging mode,
An image sensor having a plurality of different photoelectric conversion characteristics;
Processing dynamic range control means for controlling the dynamic range of the imaging output of the imaging device,
When the continuous imaging mode is set by the mode setting means,
The processing dynamic range control means controls the dynamic range of the imaging output of the imaging device during continuous imaging to be constant. The processing dynamic range control means includes
Compression processing means for compressing the dynamic range of the imaging output of the imaging device;
When the continuous imaging mode is set by the mode setting means,
The imaging apparatus according to claim 3, further comprising: a compression rate control unit that controls a compression rate of the compression processing unit during continuous imaging to be constant. Exposure control means for controlling the exposure at the time of imaging the subject to be constant;
An image sensor having a plurality of different photoelectric conversion characteristics;
Imaging dynamic range control means for controlling the dynamic range of the imaging device;
Compression processing means for compressing a dynamic range of a captured image of the image sensor;
Compression rate control means for controlling the compression rate of the compression processing means,
By making the dynamic range of the imaging device constant by the imaging dynamic range control means and by making the compression rate of the compression processing means constant by the compression rate control means, the output image of the imaging device during continuous imaging The imaging apparatus according to claim 1, wherein the dynamic range is controlled to be constant. Exposure control means for variably controlling the exposure when imaging a subject;
An image sensor having a plurality of different photoelectric conversion characteristics;
Imaging dynamic range control means for controlling the dynamic range of the imaging device;
Compression processing means for compressing a dynamic range of a captured image of the image sensor;
Compression rate control means for controlling the compression rate of the compression processing means,
The dynamic range of the image sensor is variably controlled by the imaging dynamic range control unit, and the compression rate of the compression processing unit is variably controlled by the compression rate control unit, so that the imaging device during continuous imaging can be controlled. The imaging apparatus according to claim 1, wherein the dynamic range of the output image is controlled to be constant. An imaging apparatus comprising mode setting means for setting a continuous imaging mode;
In an imaging system comprising an image processing device that processes a captured image of the imaging device,
The image processing apparatus includes:
Compression processing means for compressing a dynamic range of a captured image of the imaging device;
When the imaging device is set to the continuous imaging mode by the mode setting means,
An image pickup system comprising: a compression ratio control means for controlling a compression ratio to be constant when compressing a dynamic range of continuously picked up images of the image pickup apparatus by the compression processing means.

Images