JP2014060578A - Image pick-up device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image pick-up device in which the difference of noise level is reduced, when an image is compounded by HDR technique.SOLUTION: Photography conditions for proper exposure are calculated (S3), and a shutter speed causing over exposure for that photography conditions is calculated (S5). When the ISO sensitivity becoming proper exposure is higher than the lowest ISO sensitivity (S7; Yes), the ISO sensitivity is set to the lowest ISO sensitivity (S11) depending on the difference between the ISO sensitivity for proper exposure and the lowest ISO sensitivity. Alternatively, photography conditions for under exposure are set by supplementing the shortage of the lowest ISO sensitivity with the shutter speed (S13, S15), and different exposure photography is performed under the photography conditions thus set (S19).

Description

  The present invention relates to an imaging apparatus that continuously captures the same subject multiple times with different exposure amounts in order to synthesize image data of a plurality of frames and obtain an image with a wide dynamic range.

  In recent years, “multiple HDR (High Dynamic Range) technology” using a plurality of images has attracted attention. In general, this multiple-sheet HDR captures different exposure continuous shot images by changing the shutter speed, and holds the signal levels of the bright portion of the under image and the dark portion of the over image, respectively, thereby synthesizing the images. An HDR image with improved black crushing can be generated. In addition, when the images are combined, a positioning process is performed to deal with subject blurring and camera shake. (See Patent Document 1)

JP 2003-259200 A

  As described above, when synthesizing an HDR image, alignment processing is performed. At this time, since processing is performed based on the similarity between images, it is necessary to match the brightness of both images. . Even if the brightness is adjusted, if noise is superimposed on the image, there is a problem that it is difficult to determine the similarity at the time of alignment. This problem tends to appear when shooting with high sensitivity.

  In the imaging apparatus disclosed in Patent Document 1, brightness information of a main subject portion (focus position) is acquired in advance, and only one of the shooting conditions for continuous shooting based on the brightness information is set to a high shutter speed. By changing so as to be, blurring of the main subject is reduced. However, in this imaging apparatus, since the difference in noise level is not reduced, the above-described problem cannot be solved.

  The present invention has been made in view of such circumstances, and an object of the present invention is to provide an imaging apparatus in which a difference in noise level is reduced when an image is synthesized by the HDR technology.

  In order to achieve the above object, an image pickup apparatus according to a first aspect of the present invention provides a plurality of image pickup operations for the same subject in succession with different exposure amounts in order to obtain an image with a wide dynamic range by combining image data of a plurality of frames. An exposure amount setting unit that sets a parameter for determining an exposure amount in each of the plurality of times of imaging, an imaging unit that generates image data of a plurality of frames by the plurality of times of imaging, and the exposure Based on the parameters set by the amount setting unit, an exposure amount control unit that controls the exposure amount in the plurality of times of imaging, and a composition having a wider dynamic range than a single image based on the image data of the plurality of frames An image processing unit that generates image data, wherein the exposure amount setting unit is based on one of a plurality of times of imaging and is less than the reference. When shooting in the stomach exposure amount is lower the ISO sensitivity.

  In the imaging apparatus according to a second aspect of the present invention, in the first aspect of the present invention, one of a plurality of images as a reference has a different exposure amount from the exposure amount for proper exposure.

  In the image pickup apparatus according to a third aspect of the present invention, when the ISO sensitivity is lowered in the first aspect of the invention, the gain of the analog image signal is adjusted.

  In the imaging apparatus according to a fourth aspect of the present invention, when the exposure amount is controlled by lowering the ISO sensitivity in the first aspect of the invention, if the adjustment amount of the exposure amount based on the ISO sensitivity is insufficient, Is adjusted by adjusting the shutter speed to adjust the exposure amount.

  In the imaging device according to a fifth aspect based on the first aspect, the image processing unit has the plurality of frames of image data to match the brightness of the images represented by the plurality of frames of image data. After the level adjustment is performed, the image data of the plurality of frames subjected to the level adjustment is aligned, and based on the image data of the plurality of frames subjected to the level adjustment, a wider dynamic range than the single image Is generated.

  According to the present invention, it is possible to provide an imaging apparatus in which a difference in noise level is reduced when an image is synthesized by HDR technology.

1 is a block diagram mainly showing an electrical configuration of a digital camera according to an embodiment of the present invention. FIG. 4 is a diagram for explaining overexposure, blackout, and a dynamic range of a captured image in the digital camera according to the embodiment of the present invention. FIG. 6 is a diagram for explaining expansion of a dynamic range with a plurality of HDRs in a digital camera according to an embodiment of the present invention. FIG. 6 is a diagram for explaining expansion of a dynamic range with a plurality of HDRs in a digital camera according to an embodiment of the present invention. FIG. 6 is a diagram for explaining expansion of a dynamic range with a plurality of HDRs in a digital camera according to an embodiment of the present invention. It is a figure explaining the alignment process in multiple sheets HDR in the digital camera which concerns on one Embodiment of this invention. It is a figure explaining the alignment process in multiple sheets HDR in the digital camera which concerns on one Embodiment of this invention. It is a flowchart which shows the operation | movement of the multiple sheet HDR process of the digital camera which concerns on one Embodiment of this invention. It is a figure explaining the precision of position alignment in the digital camera concerning one embodiment of the present invention. It is a figure explaining the expansion effect of a dynamic range in the digital camera which concerns on one Embodiment of this invention. It is a figure explaining the expansion effect of a dynamic range in the digital camera which concerns on one Embodiment of this invention. It is a figure explaining the expansion effect of a dynamic range in the digital camera which concerns on one Embodiment of this invention. It is a flowchart which shows the operation | movement of the modification of the multiple sheet HDR process of the digital camera which concerns on one Embodiment of this invention. 12 is a flowchart illustrating an operation of calculating shooting conditions in shooting an i-th image in a modification of the multiple-sheet HDR processing of the digital camera according to the embodiment of the present invention.

  Hereinafter, a preferred embodiment will be described using a camera to which the present invention is applied according to the drawings. 1 is a digital camera according to a preferred embodiment of the present invention, which has an imaging unit, converts a subject image into image data by the imaging unit, records the converted image data on a recording medium after image processing. When performing wide dynamic (HDR) shooting, shooting is performed with different exposure amounts such as a reference (proper) exposure amount, an overexposure amount, an underexposure amount, and the like (also referred to as different exposure shooting).

  In different exposure shooting, shooting with an overexposure amount is performed by shifting the shutter speed to the high speed side (see S5 in FIG. 8 and S43 in FIG. 13 described later). On the other hand, in photographing with an underexposure amount, photographing is performed by shifting the shutter speed to the low speed side according to the ISO sensitivity at the time of photographing (see S17 in FIG. 8 and S73 in FIG. 14), and only ISO sensitivity is on the low sensitivity side. (See S11 in FIG. 8 and S67 in FIG. 14), or shooting with both ISO sensitivity and shutter speed shifted (see S13 and S15 in FIG. 8, S69 and S71 in FIG. 14). Do either.

  When a plurality of images are obtained by performing different exposure photography, brightness adjustment processing is performed so that the brightness of the plurality of images is substantially the same (S21 in FIG. 8, S55 in FIGS. 9, 11, and 13). (See FIG. 12, FIG. 7, FIG. 8, S23 in FIG. 8, and S57 in FIG. 13) and combining them to obtain an image with an expanded dynamic range (see FIG. 12, FIG. 13). reference).

  FIG. 1 is a block diagram mainly showing an electrical configuration of a camera according to an embodiment of the present invention. This camera includes a camera body 100 and an interchangeable lens 200 that can be attached to and detached from the camera body 100.

  The interchangeable lens 200 includes a photographing lens 201, an aperture 203, a driver 205, a microcomputer 207, and a flash memory 209. An interface (hereinafter referred to as I / F) 999 is provided between the interchangeable lens 200 and the camera body 100 described later. Have.

  The photographing lens 201 is an optical lens for forming a subject image, and has a single focus lens or a zoom lens. A diaphragm 203 is disposed behind the optical axis of the photographing lens 201. The diaphragm 203 has a variable aperture, and restricts the subject light flux that has passed through the photographing lens 201.

  The photographing lens 201 can be moved in the optical axis direction by a driver 205. The focus position of the photographing lens 201 is controlled based on a control signal from the microcomputer 207. In the case of a zoom lens, the focal length is also controlled. Is done. The driver 205 also controls the aperture of the diaphragm 204.

  The microcomputer 207 connected to the driver 205 is connected to the I / F 999 and the flash memory 209. The microcomputer 207 operates according to a program stored in the flash memory 209, communicates with a microcomputer 121 in the camera body 100 described later, and controls the interchangeable lens 200 based on a control signal from the microcomputer 121. Do.

  The flash memory 209 stores various information such as optical characteristics and adjustment values of the interchangeable lens 200 in addition to the above-described program. The I / F 999 is an interface for performing communication between the microcomputer 207 in the interchangeable lens 200 and the microcomputer 121 in the camera body 100.

  A mechanical shutter 101 is disposed in the camera body 100 on the optical axis of the taking lens 201. The mechanical shutter 101 controls the passage time of the subject luminous flux and employs a known lens shutter or focal plane shutter. An image sensor 103 is disposed behind the mechanical shutter 101 and at a position where a subject image is formed by the photographing lens 201.

  In the image sensor 103, photodiodes constituting each pixel are two-dimensionally arranged in a matrix, and each photodiode generates a photoelectric conversion current corresponding to the amount of received light, and this photoelectric conversion current is applied to each photodiode. Charges are accumulated by the connected capacitor. A Bayer color filter is arranged in front of each pixel. The Bayer array has a line in which R pixels and G (Gr) pixels are alternately arranged in a horizontal direction, and a line in which G (Gb) pixels and B pixels are alternately arranged, and the three lines are vertically arranged. It is configured by alternately arranging in the direction. The imaging element 103 and the like function as an imaging unit that generates image data of a plurality of frames by performing imaging a plurality of times when a plurality of HDRs are set.

  The image sensor 103 is connected to an analog processing unit 105. The analog processing unit 105 performs waveform shaping on the photoelectric conversion signal (analog image signal) read from the image sensor 103 while reducing reset noise and the like. In accordance with a control command from the microcomputer 121, the gain is increased so as to obtain an appropriate luminance. The ISO sensitivity is adjusted by adjusting the gain (amplification factor) of the analog image signal in the analog processing unit 105.

  The analog processing unit 105, the mechanical shutter 101, and the like serve as exposure amount control units that control the exposure amount in the plurality of imaging operations based on the parameters set by the exposure amount setting unit (the microcomputer 121 described later functions). Function. The exposure amount control unit changes the ISO sensitivity and controls the exposure amount in at least one imaging.

  The analog processing unit 105 is connected to an A / D conversion unit 107. The A / D conversion unit 107 performs analog-to-digital conversion on the analog image signal, and sends the digital image signal (hereinafter referred to as image data) to the bus 109. Output.

  The bus 109 is a transfer path for transferring various data read or generated inside the camera body 100 to the camera body 100. In addition to the analog processing unit 105 and the A / D conversion unit 107, the bus 109 includes an image processing unit 111, an AE processing unit 113, an AF processing unit 115, an AWB (Auto White Balance) processing unit 117, a JPEG processing unit 119, A microcomputer 121, an SDRAM (Synchronous Dynamic Random Access Memory) 127, a memory interface (hereinafter referred to as memory I / F) 129, and a liquid crystal (hereinafter referred to as LCD) driver 133 are connected.

  An image processing unit 111 connected to the bus 109 includes a white balance correction unit (hereinafter referred to as WB correction unit) 111a, a synchronization processing unit 111b, a color reproduction processing unit 111c, and a noise reduction processing unit (hereinafter referred to as NR processing unit). The image data temporarily stored in the SDRAM 127 including the data 111d is read out, and various image processes are performed on the image data.

  Further, the image processing unit 111 performs brightness adjustment processing and alignment of the plurality of images so that the brightness of the plurality of images becomes substantially the same when a plurality of images are obtained by different exposure shooting. After that, a wide dynamic range image is synthesized. Therefore, the image processing unit 111 functions as an image processing unit that generates composite image data having a wider dynamic range than a single image based on the image data of a plurality of frames.

  The WB correction unit 111a performs white balance correction on the image data. White balance correction is performed so that white is accurately projected under a light source with various color temperatures. The user selects a light source mode such as clear sky, cloudy, shade, light bulb, or fluorescent light, or an auto white balance mode that automatically calculates the white balance correction amount on the camera side. Perform white balance correction on the data. The auto white balance is performed by an AWB processing unit 117 described later.

  The synchronization processing unit 111b performs synchronization processing on image data acquired under the Bayer array into image data including R, G, and B information per pixel. The color reproduction processing unit 111d performs gamma correction processing and color reproduction processing that changes the color of an image.

  The NR processing unit 111d performs a process of reducing noise in the image data by using a filter that reduces a high frequency, a coring process, or the like. The image processing unit 111 selects each of the units 111 a to 111 d as necessary, performs each process, and the image data subjected to the image processing is temporarily stored in the SDRAM 127 via the bus 109.

  The AE processing unit 113 measures the subject brightness and outputs it to the microcomputer 121 via the bus 109. Although a dedicated photometric sensor may be provided for measuring the subject brightness, in this embodiment, the subject brightness is calculated using image data based on the output of the image sensor 103. The AF processing unit 115 extracts a high-frequency component signal from the image data, acquires a focus evaluation value by integration processing, and outputs the focus evaluation value to the microcomputer 121 via the bus 109. In the present embodiment, the photographing lens 201 is focused by a so-called contrast method. As described above, the AWB processing unit 117 automatically calculates the white balance correction amount on the camera side and adjusts the white balance.

  The JPEG processing unit 117 reads image data from the SDRAM 127 when the image data is recorded on the recording medium 131, compresses the read image data according to the JPEG compression method, and temporarily stores the compressed image data in the SDRAM 127. The microcomputer 121 creates a JPEG file by adding a JPEG header necessary for constructing a JPEG file to the JPEG image data temporarily stored in the SDRAM 127, and stores the created JPEG file in the memory I / F 129. To the recording medium 131.

  In addition, the JPEG processing unit 119 also decompresses JPEG image data for image reproduction display. In decompression, the JPEG file recorded on the recording medium 131 is read out, subjected to decompression processing in the JPEG processing unit 119, and the decompressed image data is temporarily stored in the SDRAM 127. In this embodiment, the JPEG compression method is adopted as the image compression method. However, the compression method is not limited to this, and TIFF, MPEG, H.264, and the like. Of course, other compression methods such as H.264 may be used.

  The microcomputer 121 functions as a control unit of the entire camera, and comprehensively controls various sequences of the camera according to a program stored in a flash memory 125 described later. Further, the microcomputer 121 serves as an exposure amount setting unit for setting parameters (shutter speed, ISO sensitivity, etc.) for determining an exposure amount in each of a plurality of times of imaging when multiple HDR shooting is set. Function. The exposure amount setting unit sets parameters based on an exposure amount that is different from the exposure amount for proper exposure (see S37 in FIG. 13 described later).

  In addition to the I / F 999 described above, an operation unit 123 and a flash memory 125 are connected to the microcomputer 121. The operation unit 123 includes operation members such as various input buttons and various input keys such as a power button, a release button, a moving image button, a playback button, a menu button, a cross button, and an OK button, and detects an operation state of these operation members. The detection result is output to the macro computer 121. The microcomputer 121 executes various sequences according to the user's operation based on the detection result of the operation member from the operation unit 123. The power button is an operation member for instructing to turn on / off the power of the digital camera. When the power button is pressed, the power of the digital camera is turned on. When the power button is pressed again, the power of the digital camera is turned off.

  The release button includes a first release switch that is turned on when the button is half-pressed and a second release switch that is turned on when the button is further depressed after being half-pressed and then fully pressed. When the first release switch is turned on, the microcomputer 121 executes a shooting preparation sequence such as AE processing and AF processing. When the second release switch is turned on, the mechanical shutter 101 and the like are controlled, image data based on the subject image is acquired from the image sensor 103 and the like, and a series of shooting sequences for recording the image data on the recording medium 131 are executed. Take a picture.

  The movie button is a button for starting and ending movie shooting. The reproduction button is an operation button for setting and canceling the reproduction mode. When the reproduction mode is set, the image data of the photographed image is read from the recording medium 131 and the photographed image is reproduced and displayed on the LCD 135.

  The menu button is an operation button for displaying a menu screen on the LCD 135. On the menu screen, various camera modes can be set. For example, wide dynamic range shooting can be set.

  The flash memory 125 stores a program for executing various sequences of the microcomputer 121. As described above, the microcomputer 121 controls the digital camera based on this program.

  The SDRAM 127 is an electrically rewritable volatile memory for temporary storage of image data and the like. The SDRAM 127 temporarily stores the image data output from the A / D conversion unit 107 and the image data processed by the image processing unit 111, the JPEG processing unit 119, and the like.

  The memory I / F 129 is connected to the recording medium 131, and controls writing and reading of image data and data such as a header attached to the image data on the recording medium 131. The recording medium 131 is a memory that can be attached to and detached from the camera body, but is not limited thereto, and may be a memory built in the camera body such as a hard disk.

  The LCD driver 133 is connected to the LCD 135, and displays an image on the LCD 135 based on the image data read from the SDRAM 127 or the recording medium 131 and expanded by the JPEG processing unit 119. The LCD 135 includes a liquid crystal panel disposed on the back surface of the camera body 100 and performs image display. As the image display, there are a REC view display for displaying the recorded image data for a short time immediately after shooting, a playback display of a still image or a moving image file recorded on the recording medium 131, and a moving image display such as a live view display. included. In addition, when displaying the compressed image data, the image data is displayed after being decompressed by the JPEG processing unit 119 as described above. The display unit is not limited to the LCD, and other display panels such as an organic EL may be adopted.

  Next, wide dynamic range (HDR) imaging according to an embodiment of the present invention will be described with reference to FIGS.

  FIG. 2 is a diagram for explaining overexposure, blackout, and a dynamic range of a captured image. FIG. 2A shows a shooting scene, and this shooting scene is shot by a camera (camera body 100, interchangeable lens 200) with appropriate exposure (shutter speed TV: tv, aperture value AV: av, ISO sensitivity: sv). Then, a photographed image as shown in FIG.

  In a scene where a bright place and a dark place such as backlight are mixed like the shooting scene shown in FIG. 2A, the brightness distribution of the scene is very wide, and the brightness obtained with appropriate exposure as shown in FIG. It does not fall within the range D1. Note that Bv_ave shown in FIG. 2D indicates the average luminance of the shooting scene.

  FIG. 2 (e) shows a digital value (upper part of FIG. 2 (e)) and a luminance distribution of the captured image (lower part of FIG. 2 (e)) when the luminance range that can be photographed with appropriate exposure is taken with a digital camera. Indicates.

  At the time of shooting, light is converted into electric charge by the photodiode of the image sensor 103, but in a region brighter than the luminance range that can be shot, no more charge is generated to exceed the conversion capacity of the photodiode, and pixel saturation and Become. The “whiteout” in the bright part of the photographed image corresponds to an area clipped by the saturation of the pixel. In the example shown in FIG. 2C, the region A2 is a region with high luminance, and this region A2 corresponds to the range R2 in the luminance distribution shown in FIG. That is, the range R2 is a bright area clipped due to pixel saturation, and is a “out-of-white” area.

  In addition, “black” in the dark part at the time of shooting corresponds to a digital value “0”, but an actual captured image includes noise (dark part noise) generated in the image sensor 103 or the like. For this reason, in a dark region of the luminance range that can be photographed, a part thereof is buried in dark part noise. The “black crushing” in the dark portion corresponds to a region buried in this noise. In the example shown in FIG. 2C, the area A1 is a low-luminance area, and this area A1 corresponds to the range R1 in the luminance distribution shown in FIG. That is, the range R <b> 1 is a dark area buried in noise, and is a “blackout” area.

  Therefore, the dynamic range of an appropriate image means a region from a dark part that is not buried in noise to pixel saturation. In the luminance distribution shown in FIG. 2E, the dynamic range D2 of the appropriate image corresponds.

  Next, the expansion of the dynamic range with a plurality of HDRs in the present embodiment will be described with reference to FIGS. FIG. 3 shows three images taken with different exposure amounts. That is, FIG. 3A is a photographed image photographed with appropriate exposure, FIG. 3B is a photographed image photographed with underexposure, and FIG. 3C is a photographed image photographed with overexposure. .

  Here, in the case of proper exposure, shooting is performed at a shutter speed Tv: tv, an aperture value Av: av, and ISO sensitivity Sv: sv. In the case of underexposure, images are taken at a shutter speed Tv: tv + 1, an aperture value Av: av, and ISO sensitivity Sv: sv. In the case of overexposure, images are taken at a shutter speed Tv: tv-1, an aperture value Av: av, and ISO sensitivity Sv: sv. Therefore, compared with the shutter speed for proper exposure, the shutter speed is shifted to the high speed side in the case of underexposure, and the shutter speed is shifted to the low speed side in the case of overexposure. Yes.

  FIG. 3D shows a luminance distribution of a photographed image photographed with the proper exposure shown in FIG. This luminance distribution is the same as that shown in FIG. 2 (e). There is a dark part region (= black crushing region) R1 buried in noise on the dark side, and a bright part region clipped by pixel saturation on the bright part side (= = "Out-of-white area") R2, and there is a dynamic range D2 of an appropriate image. Here, let Bv_ave1 be the average luminance of a captured image captured with appropriate exposure.

  FIG. 3E shows a luminance distribution of a photographed image photographed by the underexposure shown in FIG. This luminance distribution has a dark area (= black area) R1 buried in noise on the dark side and an under image dynamic range D2. Here, the range width of the dark area R1 in the underexposure is the same as that in the case of photographing by appropriate exposure in FIG. 3A (the ISO sensitivity Sv is the same as sv), so the dark noise level is the same. Further, since the underexposure is performed, the average luminance Bv_ave2 is shifted to the dark side as a whole, and therefore there is no bright portion region R2.

  FIG. 3F shows the luminance distribution of the captured image captured by the overexposure shown in FIG. In this luminance distribution, there is a dark area (= black area) R1 buried in noise on the dark area side, and a bright area (= “out-of-white” area) R2 ″ clipped by pixel saturation on the bright area side. And there is an over image dynamic range D2. Here, the range width of the dark area R1 in the overexposure is the same as that in the case of photographing by appropriate exposure in FIG. In addition, since the overexposure is performed, the average luminance Bv_ave3 is shifted to the bright side as a whole, and thus the bright portion region R2 ″ is spread to the bright portion side.

  Thus, since the photographing sensitivity (ISO sensitivity, Sv) does not change in the appropriate exposure, underexposure, and overexposure, the dark portion noise amount is the same amount. However, the brightness of the images is different due to different exposure photography (average luminances Bv_ave1 to Bv_ave3 are different).

  In the HDR technology for synthesizing images using a plurality of images that have been taken with different exposures, the image brightness is set to be substantially the same when synthesizing the images. FIG. 4 shows an example in which “brightness adjustment” for adjusting the brightness of the three photographed images shown in FIG. 3 is performed and the brightness of the under image is adjusted.

  FIG. 4 shows the luminance distribution after brightness adjustment. That is, FIG. 4A shows a photographed image taken with proper exposure, FIG. 4B shows a photographed image taken with underexposure, and FIG. 4C shows a photographed image taken with overexposure. Indicates. As shown in FIGS. 4A to 4C, the average luminance of these three images is substantially the same by adjusting the brightness. FIG. 4D shows the luminance distribution during proper exposure, FIG. 4E shows the luminance distribution during underexposure, and FIG. 4F shows the luminance distribution during overexposure.

  By adjusting the brightness, the brightness distribution is adjusted so that the average brightness Bv_ave1 of proper exposure and the average brightness Bv_ave3 of overexposure coincide with the average brightness Bv_ave2 of underexposure. By matching the average brightness of the three images, a difference occurs in the noise level in the dark part. In addition, the dynamic range also changes, and in the proper exposure, the dynamic range D2 is compressed to D21, and in the overexposure, the dynamic range D2 is further compressed to D23.

  When brightness adjustment is performed, positioning is performed in consideration of subject movement and subject brightness. FIG. 5A shows a composite image that has been aligned, and FIG. 5B shows a luminance distribution of the composite image after alignment. As can be seen from FIG. 5B, the bright area of the under image is applied to the bright area to improve the “whiteout”, and the dark area of the over image is applied to the dark area. , "Black crushing" has been improved. Also, a moving subject is subjected to an alignment process, which will be described later with reference to FIGS.

  When the composition processing as shown in FIGS. 5A and 5B is performed, next, as shown in FIGS. 5C and 5D, development processing is performed on the composite image. Since the composite image shown in FIGS. 5A and 5B has the same average brightness Bv_ave2 as the underexposure, the average brightness is equal to that of the appropriate image while leaving the gradation of the bright part by the development process. Development processing is performed at the same time as gradation conversion processing for raising the level (Bv_ave1). As a result, it is possible to obtain a single image with a wide dynamic range that suppresses whiteout in a bright part and improves blackout in a dark part. That is, improvement of pixel saturation by under-shooting is used in the bright part, and noise level difference in the dark part is useful for the dynamic range expansion effect.

  In the multiple-sheet HDR, image composition is performed using a plurality of different-exposure images, but it is necessary to align the different-exposure images prior to image composition. FIG. 6A shows two images to be aligned, and the left image is brighter than the right image. The alignment is performed by block matching. In block matching, the sum of absolute differences (SAD) is generally used. This SAD is obtained by calculating the absolute difference value of each pixel in each pixel between blocks and summing the absolute difference values of all the pixels in the block. Therefore, the lower the SAD value, the smaller the difference and the higher the correlation value.

  Since this method is based on the difference of each block of two images, accurate alignment cannot be performed if the brightness of the images is different. Therefore, the brightness of the two images is made uniform by brightness adjustment processing. There is a difference in noise level during this brightness adjustment. FIG. 6B shows that a difference occurs in the noise level, and the dot in the right image means the amount of noise.

  When the block matching process is performed by dividing the block as shown in FIG. 6C, the SAD value is calculated as the difference between the noise levels in each block as the image difference. In this process, if the ISO sensitivity of the two shot images is low, the original noise itself is low, so that the accuracy of the SAD value necessary for the alignment process is ensured even if there is a slight difference in noise level. it can.

  The example shown in FIG. 7A is a case where the ISO sensitivity of two captured images is low. The image 11a in FIG. 7A is one of the blocks of the first photographed image (corresponding to the rear wheel portion of the image 11), and the image 11b is matched with the two images 11a. It is an area for. In this example, since the accuracy of the SAD value necessary for the alignment process can be ensured, the image 11 after the synthesis process for expanding the dynamic range does not cause a positional shift of the subject.

  However, when the ISO sensitivity of the two photographed images is high, a lot of noise is originally included, and it is difficult to accurately obtain the positional deviation of the subject with the SAD value. For this reason, a correct motion vector cannot be obtained by block matching, and a double image may be obtained in order to produce a plurality of HDR effects.

  The example shown in FIG. 7B is a case where the ISO sensitivity of two captured images is high. The image 12a in FIG. 7B is one of the blocks of the first photographed image (corresponding to the rear wheel portion of the image 12), and the image 12b is matched with the two images 12a. It is an area for. In this example, since the accuracy of the SAD value necessary for the alignment process cannot be ensured, the image 11 after the synthesis process for expanding the dynamic range causes a positional shift of the subject.

  When performing a plurality of HDRs by hand-held shooting, the ISO sensitivity is increased at a shutter speed that is a camera shake limit in order to suppress the occurrence of camera shake. As described with reference to FIGS. 3 to 5, the effect of expanding the dynamic range of the dark portion in the plurality of HDRs is an effect obtained by the difference in the noise level. However, as the alignment accuracy, the difference in the noise level of the dark portion is different. Therefore, there is a problem that the accuracy is deteriorated. Therefore, in the present embodiment, different exposure photography is performed by changing the ISO sensitivity.

  Next, the operation of the multiple-sheet HDR processing in the present embodiment will be described using the flowchart shown in FIG. This operation is executed by the microcomputer 121 according to a program stored in the flash memory 125.

  When the flow of the multiple-sheet HDR processing shown in FIG. 8 is entered, first, exposure step setting is read (S1). In the multiple image HDR, a plurality of images are taken while changing the exposure amount. In this step, a step of the set exposure amount is read. In the present embodiment, the step for under photographing (−Δev ″), the step for proper exposure photographing (± 0 ev), and the step for over photographing (+ Δev ′) are read from the flash memory 125. In addition to storing the exposure step in the flash memory 125 in advance, for example, the user may set and read the set value on a menu screen or the like.

  Once the exposure step setting has been read, the appropriate exposure photographing condition is calculated (S3). Since the luminance value of the subject is detected by the AE processing unit 113, the shutter speed value (Tv_std), the aperture value (Av_std), and the ISO sensitivity value (Sv_std) at which appropriate exposure is performed by apex calculation or the like using this luminance value. calculate. In addition to the apex calculation, the proper exposure photographing condition may be obtained by other methods such as table reference.

  Once the proper exposure shooting conditions are calculated, the overexposure shutter speed is calculated (S5). Here, the shutter speed value for overexposure is calculated based on the overexposure step (+ Δev ′) read in step S1, while maintaining the aperture value and ISO sensitivity value for proper exposure calculated in step S3. .

  Once the overexposure shutter speed has been calculated, it is next determined whether or not the proper exposure ISO sensitivity Sv_std is greater than the minimum ISO sensitivity (S7). The ISO sensitivity for proper exposure is the ISO sensitivity calculated in step S3. The lowest ISO sensitivity is the lowest value among the ISO sensitivities that can be set by the digital camera. The user may be able to set as appropriate.

  If the result of determination in step S7 is that the calculated appropriate exposure ISO sensitivity Sv_std is not greater than the minimum ISO sensitivity, the underexposure shutter speed is calculated (S17). Here, the shutter speed for underexposure corresponding to the exposure step (−Δev ″) read in step S1 is calculated. That is, the aperture value for proper exposure (Av_std), ISO sensitivity (Sv_std), and exposure step (−Δev). The shutter speed is calculated using “)”. In this step, since the exposure amount cannot be controlled lower than the minimum ISO sensitivity in controlling the exposure amount, the shutter speed is shifted to the high speed side for underexposure.

  On the other hand, if the result of determination in step S7 is that the calculated ISO sensitivity Sv_std for proper exposure is greater than the minimum ISO sensitivity, then the difference value between the ISO sensitivity Sv_std for proper exposure and the minimum ISO sensitivity is the exposure step width. It is determined whether it is greater than Δev ″ ((Sv_std−minimum ISO sensitivity)> Δev ″) (S9). That is, it is determined whether or not the difference value between the ISO sensitivity of the appropriate exposure and the minimum ISO sensitivity is larger than the exposure step width of the underexposure image.

  If the result of determination in step S9 is that the ISO sensitivity difference value is larger than the under exposure step width, the under exposure ISO sensitivity corresponding to the under exposure exposure step is calculated (S11). In under photographing in this case, the ISO sensitivity is calculated such that an exposure step (−Δev ″) is obtained while maintaining the aperture value (Av_std) and shutter speed (Tv_std) for proper exposure.

On the other hand, if the result of determination in step S9 is that the difference value of ISO sensitivity is not greater than the exposure step width on the under side, the under exposure ISO sensitivity is calculated (S13). Here, the lowest ISO sensitivity is the ISO sensitivity for underexposure. Subsequently, the shutter speed for underexposure is calculated (S15). In step S13, the ISO sensitivity is set to the minimum ISO sensitivity, but there is an exposure step that is insufficient with this setting. So this shortage, ie
Δev ″ − (Sv_std−minimum ISO sensitivity)
Based on the above, the shutter speed is calculated. Therefore, in this case, under photographing is controlled according to the calculated ISO sensitivity (= lowest ISO sensitivity) and the shutter speed calculated here, using the aperture value (Av_std) for proper exposure.

  When the underexposure ISO sensitivity is calculated in step S11, the underexposure shutter speed is calculated in step S15, or the underexposure shutter speed is calculated in step S17, next exposure shooting is performed (S19). Here, the shooting conditions for obtaining the appropriate exposure calculated in step S3, the shooting conditions for obtaining the overexposure calculated in step S5, and the underexposure calculated in steps S11, S13, S15, and S17 are obtained. According to the shooting conditions, the three images with the exposure steps read in step S1 are sequentially taken.

  Once the different exposure shooting is performed, next, brightness adjustment processing is performed (S21). As described above, in multiple HDR shooting, if the brightness of each image is different, alignment becomes difficult. Therefore, brightness adjustment processing is performed before synthesis to adjust the brightness of each image. Here, as described with reference to FIG. 4, image processing is performed so that the average luminance values of the three images match.

  Once brightness adjustment processing has been performed, alignment composition processing is then performed (S23). When the moving subject is included in the image, if the image is simply synthesized according to the brightness, a double image is generated. For this reason, the generation of double images is suppressed by detecting a moving subject from the image and refraining from synthesis in the vicinity thereof. Therefore, the moving subject area is detected by the alignment process, and the synthesis process is performed based on the information.

  Once the alignment composition process has been performed, the development process is performed (S25). Here, by using the result of the alignment composition processing in step S23, an image with a wide dynamic range without whiteout or blackout is obtained by appropriately raising the overall brightness while leaving the bright part out of focus. (See (c) and (d) of FIG. 5).

  As described above, in the flow of the multiple-sheet HDR processing in the present embodiment, when performing underexposure at the time of different exposure shooting, the exposure amount is controlled with priority given to lowering the ISO sensitivity (see S11 and S13). For this reason, compared with the case where the exposure amount is controlled only by the shutter speed, the difference in the noise level between the captured images can be reduced (see the description of FIG. 9). When the difference in noise level is reduced, alignment can be performed accurately when performing brightness adjustment.

  Further, in the flow of the multiple-sheet HDR processing in the present embodiment, when the exposure amount is controlled by changing the ISO sensitivity, the adjustment amount of the exposure amount based on the ISO sensitivity is insufficient with respect to the setting value of the exposure amount setting unit. (S9) The amount of exposure is adjusted by compensating for the shortage by adjusting the shutter speed (S13, S15).

  Further, in the flow of the multiple-sheet HDR processing in the present embodiment, after the level adjustment of the image data of the plurality of frames is performed in order to match the brightness of each image represented by the image data of the plurality of frames (S21), Position adjustment of the image data of the plurality of frames subjected to the level adjustment is performed (S23), and the image data of the plurality of frames subjected to the level adjustment is selected for each pixel or for each predetermined area according to the brightness of the subject. The image data of the part to be synthesized is selected or weighted addition processing is performed to generate synthesized image data (S23). For example, overexposure image data is selected for dark images with a predetermined brightness or lower, underexposure image data is selected for bright images with a predetermined brightness or higher, and image data with proper exposure is selected for images with other brightness, or weighting thereof. The composite image data is generated by performing the synthesis with a larger value.

  Next, the effect of alignment in this embodiment will be described with reference to FIG. In the present embodiment, when under shooting is performed, the lowest ISO sensitivity is set (S11 and S13 in FIG. 8). FIG. 9A shows a case where the ISO sensitivity is not adjusted, and FIG. 9B shows a case where the ISO sensitivity is adjusted. That is, FIG. 9A shows a case where different exposure shooting is performed only by adjusting the shutter speed, as in the conventional case, and FIG. 9B shows that different exposure shooting is performed by adjusting the shutter speed and ISO sensitivity in the present embodiment. Indicates when to do.

  In the example shown in FIG. 9A, the shutter speed Tv is set to tv-1 for overexposure, tv for proper exposure, tv + 1 for underexposure, and the aperture value Av is set to av for any exposure and ISO sensitivity for any exposure. But sv. Under these photographing conditions, an overexposed image 21a, a proper exposed image 22a, and an underexposed image 23 are obtained.

As described above, the brightness adjustment is performed using the image captured by underexposure as a reference and adjusting the brightness of the image captured by overexposure and appropriate exposure to the underexposure image. By adjusting the brightness, an overexposed image 21b and a proper exposed image 22b are obtained. The noise level difference at this time is as follows.
Difference in noise level between the underexposure image 23 and the proper exposure image 22b:
1-1 / 2 = 1/2
Difference in noise level between underexposed image 23 and overexposed image 21b:
1-1 / 4 = 3/4

  In the example shown in FIG. 9B, the shutter speed Tv is tv-1 for overexposure, tv for proper exposure, and tv for underexposure. The ISO sensitivity Sv is sv for overexposure and proper exposure, sv-1 for underexposure, and the aperture value Av is av for both exposures. Under these photographing conditions, an overexposure image 31a, a proper exposure image 32a, and an underexposure image 33 are obtained.

As in the case of FIG. 9A, the overexposure image 31b and the proper exposure image 32b are obtained by adjusting the brightness with the underexposure image as a reference. The noise level difference at this time is as follows.
Difference in noise level between the underexposure image 33 and the proper exposure image 32b:
1 / 2-1 / 2 = 0
Difference in noise level between underexposed image 23 and overexposed image 21b:
1 / 2-1 / 4 = 1/4

  As described above, when the ISO sensitivity is adjusted as in this embodiment, the difference in noise level is smaller than when the ISO sensitivity is not adjusted. That is, the noise level difference between the underexposure image and the proper exposure image decreases from 1/2 to 0, and the noise level difference between the underexposure image and the overexposure image decreases from 3/4 to 1/4. ing. As a result, the difference in the image noise level when the brightness is adjusted is reduced. Therefore, in this embodiment, the alignment accuracy is improved.

  Next, the dynamic range expansion effect in the present embodiment will be described with reference to FIG. FIG. 10A shows an image shot with appropriate exposure, FIG. 10B shows an image shot with underexposure, and FIG. 10C shows an image shot with overexposure. FIG. 10 (d) shows the luminance distribution of an image taken with appropriate exposure. Similar to FIG. 3 (d), the dark area (blackout area) R1 buried in noise, the dynamic range D2 of the appropriate image, and pixels Is divided into bright areas (out-of-white areas) R2 clipped.

  FIG. 10E shows the luminance distribution of an image captured by underexposure, and is divided into a dark area (blackout area) R1 'buried in noise and a dynamic range D2' of the underimage. In under-exposure shooting, as described above, since the ISO sensitivity Sv is different from that in proper exposure shooting, the noise levels in the dark area R1 and the dark area R1 'are different.

  FIG. 10F shows the luminance distribution of an image captured by overexposure. The dark area (blackout area) R1 buried in noise, the dynamic range D2 of the appropriate image, and the bright area clipped by pixel saturation ( It is divided into whiteout areas R2 ″.

  As described above, in the present embodiment, the ISO sensitivity is different in the underexposure with respect to the proper exposure and overexposure, and therefore the noise level is different. In the example shown in FIG. Is different.

  Next, brightness adjustment in the present embodiment will be described with reference to FIG. FIG. 11A shows an image taken with appropriate exposure, FIG. 11B shows an image taken with underexposure, and FIG. 11C shows an image taken with overexposure. FIGS. 11D to 11F show the luminance distributions of images taken with appropriate exposure, underexposure, and overexposure, respectively.

  As described above, the brightness adjustment is performed so that the average luminances Bv_ave1 and Bv_ave3 of the images photographed by overexposure and the other appropriate exposures coincide with each other based on the average luminance Bv_ave2 of the images photographed by the underexposure. . In the dark area R1 ′ of the brightness distribution of the captured image of the appropriate exposure and the dark area R1 ′ of the brightness distribution of the image of the under exposure, the difference in the dark noise level due to the difference in ISO sensitivity is eliminated or reduced by the brightness adjustment. Become.

  In addition, the dark portion noise level differs between the dark portion region R1 ′ of the luminance distribution of the under-exposure photographed image and the dark portion region R1 ″ of the luminance distribution of the over-exposure picked-up image by brightness adjustment. In the state where the brightness of the image is matched by the adjustment process, the difference in the noise level between the underexposure and the proper exposure and the underimage and the over image is smaller than in the conventional method (multiple HDRs that change only the Tv value). .

  Next, the alignment in this embodiment is demonstrated using FIG. 12 (a) (b). In the present embodiment, alignment is performed in consideration of the movement of the subject and the luminance of the subject. FIG. 12A shows a composite image, and FIG. 12B shows the luminance distribution of the composite image. As described with reference to FIG. 9, in this embodiment, since the noise level difference is reduced, the alignment accuracy is improved. For the bright area of the dynamic range D3 of the composite image, the bright area is improved by applying the bright area of the under image. In addition, for the dark part area on the dark part side of the dynamic range D3 of the composite image, the black crushing is improved by applying the dark part area of the over image.

  Next, the development processing in the present embodiment will be described with reference to FIGS. As described above, since the average brightness is the same as the underexposure as a result of the synthesis, the tone conversion that finally raises the average brightness to the same level as the image of the appropriate exposure while leaving the tone of the bright part by development processing. Development processing is performed simultaneously with processing. Since the dark part of the over image is applied to the dark part of the composite image and the bright part of the under image is applied to the bright part, the dynamic range is expanded in the same manner as in the conventional method (multiple HDRs that change only the Tv value). An effect is obtained. Furthermore, as described above, by reducing the difference in noise level, the alignment accuracy is improved, and a plurality of HDRs with high sensitivity becomes possible.

  That is, in FIG. 12D, the dark portion dynamic range D21 is expanded using the difference in the dark portion noise level due to the brightness adjustment. Further, the effect of improving the saturation of pixels by under-shooting is used to expand to the dynamic range D22 in the bright part.

  As described above, in one embodiment of the present invention, the exposure control unit performs exposure control with priority given to lowering the ISO sensitivity in under shooting. For this reason, when the images are combined by the HDR technique, the difference in noise level can be reduced, and the alignment at the time of combining can be performed accurately.

  Next, a modified example of the multiple-sheet HDR processing in one embodiment of the present invention will be described with reference to FIGS. In one embodiment of the present invention, hand-held shooting with increased sensitivity and reduced camera shake is performed based on shooting conditions for proper exposure. On the other hand, in this modification, sensitization is performed on the basis of the exposure amount shifted from the appropriate exposure, and hand-held shooting with reduced camera shake is performed. In one embodiment of the present invention, three images are taken for different exposure photography with an exposure step. On the other hand, in this modification, the number of shots is variable.

  When the flow of the multiple-sheet HDR process shown in FIG. 13 is entered, the number of shots (N) is read (S31). Here, the number of sheets to be combined is read. This composite number can be set by the user on a menu screen or the like. If there is no setting, the default value is read.

  Once the number of shots (N) is read, exposure step setting reading is performed (S33). For example, when the number of shots N = 5 is set, −2ev, −1ev, ± 0ev, + 1ev, + 2ev is read as the exposure step.

  Once the exposure step is read, the sensitization reference exposure step (ΔEv_base) is read (S35). This sensitization reference step is, for example, an exposure step used as a reference for sensitization among five exposure steps (-2ev, -1ev, ± 0ev, + 1ev, + 2ev). It is assumed that is set. Note that the step may be appropriately set by the user.

  Once the sensitization reference exposure step is read, the photographing condition at the sensitization reference exposure step is calculated (S37). In the embodiment of the present invention described above, the proper exposure is used as a reference. However, in this modification, a reference other than the proper exposure may be used. In this example, +1 ev is the sensitized reference exposure step, the photographing conditions at +1 ev are calculated, and the calculated shutter speed, aperture value, and ISO sensitivity value are Tv_base, Av_base, and Sv_base, respectively.

  Once the shooting conditions at the sensitization reference exposure step are calculated, the counting variable (i) is initialized (i = 1) (S39). Then, it is determined whether or not the i-th exposure step number is larger than the exposure step ΔEv_base (i-th exposure step number> ΔEv_base) (S41).

  In the determination in step S41, for example, if the exposure difference of the first sheet is + 2ev, it is larger than ΔEv_base (= + 1ev), so the process proceeds to step S43. On the other hand, if the exposure difference of the first sheet is −2ev, since it is not larger than ΔEv_base (= + 1ev), the process proceeds to step S45.

  If the result of determination in step S41 is that the i-th exposure step number is greater than ΔEv_base, the shutter speed for i-th shooting is calculated (S43). Here, the imaging condition calculated in step S37 is obtained by changing only the shutter speed according to the exposure difference as the i-th imaging condition.

  On the other hand, if the result of determination in step S41 is that the i-th exposure step number is not greater than ΔEv_base, it is next determined whether or not the i-th exposure step number is smaller than ΔEv_base (S45). For example, if the exposure difference of the first sheet is −2ev, since it is smaller than ΔEv_base (+ 1ev), the process proceeds to step S47. On the other hand, if the exposure difference of the first sheet is +1 ev, it is equal to ΔEv_base (+1 ev), and since the photographing condition has already been calculated in step S37, nothing is done and the process proceeds to step S49.

  The process proceeds to step S47 when the i-th exposure stage number is smaller than ΔEv_base. In this case, as in FIG. 8, the ISO sensitivity and the shutter speed are calculated according to the minimum ISO sensitivity and the difference from the minimum ISO sensitivity. Detailed operation of the shooting condition calculation for the i-th shot in step S47 will be described later with reference to FIG.

  Since the i-th shooting condition has been calculated by the processing so far, the variable (i) is incremented (S49).

  Then, it is determined whether or not the incremented variable i is larger than the number of shots N (S51). If the result of this determination is that the variable i is not greater than the number of shots N, the shooting conditions for all shooting have not been calculated, and the process returns to step S41 to continue calculating shooting conditions.

  If the result of determination in step S51 is that the variable i is greater than the number of shots N, the shooting conditions for all shooting have been calculated, and processing proceeds to step S53 where different exposure shooting is performed. The processing in steps S53 to S59 is the same as steps S19 to S25 in the flow of FIG.

  Next, calculation of shooting conditions for the i-th shot in step S47 will be described with reference to FIG. If this flow is entered, first, a difference step number (Δev_i) between the i-th exposure step and ΔEv_base is calculated (S61). For example, if the i-th exposure step is −2ev, the difference step is + 3ev from “(+1) − (− 2)”.

  Next, it is determined whether or not the ISO sensitivity Sv_base at the sensitization reference exposure step is larger than the lowest ISO sensitivity (S63). This ISO sensitivity Sv_base is the value calculated in step S37.

  If the result of determination in step S63 is that the minimum ISO sensitivity is the same as Sv_base, the shutter speed at the i-th shooting is calculated (S73). Here, the shooting condition calculated in step S37 is obtained by changing only the shutter speed in accordance with the exposure difference as the i-th shooting condition.

  On the other hand, if the result of determination in step S63 is that Sv_base is greater than the minimum sensitivity, it is next determined whether or not the difference between Sv_base and the minimum ISO sensitivity is greater than Δev_i (S65). Δev_i is a difference step between the i-th exposure step calculated in step S61 and ΔEv_base.

  As a result of the determination in step S65, if the difference between Sv_base and the lowest ISO sensitivity is larger than the difference step Δev_i, the ISO sensitivity in the i-th shooting is calculated (S67). Here, the imaging condition calculated in step S37 is obtained by changing only the ISO sensitivity according to the exposure difference as the i-th imaging condition.

  On the other hand, if the result of determination in step S65 is that the difference between Sv_base and the minimum ISO sensitivity is not greater than the difference step Δev_i, the ISO sensitivity is first set to the minimum ISO sensitivity (S69), and the remaining shortage is set to the shutter speed. The photographing conditions are calculated so as to be compensated by the change of (S71).

  When the ISO sensitivity for the i-th shooting is calculated in step S67, the shutter speed for the i-th shooting is calculated in step S71, or the shutter speed for the i-th shooting is calculated in step S73, Return to the original flow.

  As described above, in the modification of the embodiment of the present invention, sensitization is performed based on the sensitization reference exposure step (ΔEv_base), and handheld shooting with reduced camera shake is performed. Even in the case of sensitization based on a reference other than the proper exposure, in this modification, it is possible to perform a composition process that ensures alignment accuracy.

  As described above, in one embodiment or a modification of the present invention, the exposure amount control unit changes the ISO sensitivity and controls the exposure amount to perform imaging at least once. For this reason, when the images are synthesized by the HDR technique, the difference in noise level can be reduced (see FIGS. 9 and 11).

  It should be noted that the numerical values such as the exposure step shown in the embodiment and modification of the present invention are examples, and it is needless to say that numerical values different from this may be set.

  In the present embodiment, the digital camera is used as the photographing device. However, the camera may be a digital single-lens reflex camera or a compact digital camera, and may be used for moving images such as video cameras and movie cameras. It may be a camera, or may be a camera built into a mobile phone, a smart phone, a personal digital assistant (PDA), a game machine, or the like. In any case, in order to obtain a wide dynamic range image by synthesizing a plurality of frames of image data, the present invention is applied to any device that continuously captures images of the same subject at different exposure amounts. can do.

  In addition, regarding the operation flow in the claims, the specification, and the drawings, even if it is described using words expressing the order such as “first”, “next”, etc. It does not mean that it is essential to implement in this order.

  The present invention is not limited to the above-described embodiments as they are, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, you may delete some components of all the components shown by embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

DESCRIPTION OF SYMBOLS 100 ... Camera body, 101 ... Mechanical shutter, 103 ... Image sensor, 105 ... Analog processing part, 107 ... A / D conversion part, 109 ... Bus, 111 ... Image processing , 111a... WB correction unit, 111b... Synchronization processing unit, 111c... Color reproduction processing unit, 111d... NR processing unit, 113. , 117 ... AWB processing unit, 119 ... JPEG processing unit, 121 ... microcomputer, 123 ... operation unit, 125 ... flash memory, 127 ... SDRAM, 129 ... memory I / F, 131 ... recording medium, 133 ... LCD driver, 135 ... LCD, 200 ... interchangeable lens, 201 ... photographing lens, 203 ... diaphragm, 205 ... dora Bas, 207 ... micro computer, 209 ... flash memory, 999 ··· I / F

Claims (5)

In order to obtain an image with a wide dynamic range by synthesizing image data of a plurality of frames, in an imaging apparatus that performs imaging multiple times for the same subject continuously with different exposure amounts,
An exposure amount setting unit for setting a parameter for determining an exposure amount in each of the plurality of times of imaging,
An imaging unit that generates image data of a plurality of frames by the plurality of times of imaging,
An exposure amount control unit that controls an exposure amount in the plurality of times of imaging based on the parameters set by the exposure amount setting unit;
An image processing unit that generates composite image data having a wider dynamic range than a single image based on the image data of the plurality of frames;
With
The imaging apparatus according to claim 1, wherein the exposure amount setting unit lowers the ISO sensitivity when photographing with an exposure amount smaller than the reference with one of a plurality of times of imaging as a reference.   The imaging apparatus according to claim 1, wherein the exposure amount setting unit sets an exposure amount different from an exposure amount for proper exposure to one of a plurality of images as a reference.   The imaging apparatus according to claim 1, wherein the ISO sensitivity is decreased by adjusting a gain of an analog image signal.   When controlling the exposure amount by lowering the ISO sensitivity, if the adjustment amount of the exposure amount based on the ISO sensitivity is insufficient, the insufficient amount is compensated by adjusting the shutter speed to adjust the exposure amount. The imaging device according to claim 1.   The image processing unit performs the level adjustment of the image data of the plurality of frames in order to match the brightness of each image represented by the image data of the plurality of frames, and then performs the level adjustment of the plurality of images. The composite image data having a dynamic range wider than that of a single image is generated on the basis of the image data of a plurality of frames subjected to the level adjustment, by aligning the image data of the frames. The imaging device described.