JP5156991B2 - Imaging apparatus, imaging method, and imaging program - Google Patents

Description

  The present invention relates to a digital camera and the like, and relates to an imaging apparatus, an imaging method, and an imaging program that perform night scene photography.

  Conventionally, a digital camera has a so-called “night scene shooting” method in which a flash is emitted to shoot a person in front with an appropriate exposure, and a night scene in the background is shot at a considerably slow shutter speed, for example. It was.

  FIG. 13 is a timing chart showing an example of operation of a digital camera according to the prior art during night scene shooting. When shooting night scenes, the mechanical shutter is first opened, and when the shutter button is pressed, the flash is emitted for a predetermined time, and then the mechanical shutter is closed after the predetermined time has elapsed. As a result, an image of the person in the foreground is mainly captured during the flash emission period, and an image of the background (night view) is captured during the period from when the shutter button is pressed until the mechanical shutter is closed.

  In this case, it is necessary to slow down the shutter speed in order to obtain appropriate exposure in a background where the flash light is difficult to reach. When shooting the person and background in the foreground in this way, the shutter speed must be slowed down as a whole, so it is necessary to fix the digital camera body, such as a tripod, to prevent camera shake. It was. Also, there was a problem that if there were moving objects in the night view, the film flowed thinly.

In view of this, there has been proposed a technique for performing shooting with a flash and shooting without a flash in response to a single shooting instruction, detecting the correlation between both images, and correcting the shot image based on the correlation. (For example, refer to Patent Document 1).
JP 2000-102022 A

  However, with the above-described conventional technology, an image with a good balance between brightness and noise can be obtained even if the captured image is corrected using an image shot with the flash and an image shot without the flash. There is a problem that can not be. In addition, two complex operations, correlation detection processing for identifying an image portion affected by flash light and an image portion not affected by flash light, and image correction processing for the identified image portion are performed. Since it needs to be performed separately, there is a problem that the circuit configuration or software becomes complicated.

  Therefore, the present invention secures sufficient brightness for a photographed image, particularly in night photography, and obtains an image having a good balance with noise due to sensitivity change and camera shake due to shutter speed change by simple arithmetic processing. An object is to provide an imaging device, an imaging method, and an imaging program.

In order to achieve the above object, an image pickup apparatus according to a first aspect of the present invention has an image pickup means for picking up a subject image, a light emitting means, and an image pickup sensitivity when shooting with the image pickup means without causing the light emitting means to emit light. A shooting sensitivity setting unit that lowers the imaging sensitivity when shooting with the imaging unit by causing the light emitting unit to emit light, and first image data captured by the imaging unit without causing the light emitting unit to emit light , For each pixel determined by the composition ratio determining means, and a composition ratio determining means for determining for each pixel a composition ratio with the second image data captured by the imaging means by causing the light emitting means to emit light A synthesis unit that synthesizes the first image data and the second image data at different synthesis ratios to generate image data for recording; and the synthesis ratio determination unit includes the first image data. And determining the pixel value and not only one of the pixel values of the pixel values of the second image data, a 3-step or more different synthetic ratios according to the combination of both the pixel values for each pixel .

  Further, as a preferred aspect, for example, as in claim 2, in the imaging apparatus according to claim 1, in accordance with one photographing instruction, photographing by the imaging means that does not cause the light emitting means to emit light, and the light emitting means emits light. An imaging control unit that controls the imaging unit, the light emitting unit, and the sensitivity control unit may be provided so that the imaging by the imaging unit is performed continuously.

As a preferred mode, for example, as described in claim 3, the composition ratio determining means compares the pixel value of the first image data with the pixel value of the second image data, and calculates the larger one. Different composition ratios of the three or more stages may be determined so that the composition ratio of the image data of the above is increased.

Further, as a preferred aspect, for example, as in claim 4, in the imaging device according to claim 1 or 2 , the combination ratio determination unit includes a pixel value obtained by photographing without causing the light emitting unit to emit light and the pixel value. An area indicated by a combination with pixel values obtained by emitting light from the light emitting means is divided into three or more areas in advance, and a different composition ratio is assigned to each area, and the first image is assigned. When the data and the second image data are combined, the region to which the combination of the pixel value of the first image data and the pixel value of the second image data belongs is specified. The composition ratio assigned to the area may be determined as a composition ratio corresponding to the combination of the pixel values .

Further, as a preferred aspect, for example, as in claim 5, in the imaging device according to claim 4 , the combination ratio determining means includes a pixel value obtained by photographing without causing the light emitting means to emit light and the light emitting means. The region indicated by the combination with the pixel value obtained by photographing with light emission is divided into three or more regions in advance, and any one of three or more different composition ratios is assigned to each region, When the first image data and the second image data are combined, a region to which a combination of a pixel value of the first image data and a pixel value of the second image data belongs is specified. One of the three or more different composition ratios assigned to the specified area may be determined as a composition ratio corresponding to the combination of the pixel values .

In order to achieve the above object, the image pickup method according to the sixth aspect of the invention continuously performs shooting without flash emission with high imaging sensitivity and shooting with flash emission with low imaging sensitivity according to a single shooting instruction. A composite ratio determining step for determining, for each pixel, a composite ratio between the first image data captured without flashing and the second image data captured without flashing; Synthesizing the first image data and the second image data at different synthesis ratios for each of the determined pixels to generate image data for recording, and the synthesis ratio determination step includes: not only one of the pixel value of the pixel values of the first said pixel values of the image data of the second image data, a 3-step or more different synthetic ratio according to a combination of both pixel values each And determining the Motogoto.

In order to achieve the above object, the imaging program according to the seventh aspect of the present invention provides an image capturing program that does not cause the computer to emit a flash with a high image sensitivity and an image that causes a flash to emit with a low image sensitivity in accordance with a single image capturing instruction. A combination ratio for determining for each pixel a combination ratio of a shooting function that continuously performs the above, first image data shot without flash firing, and second image data shot with flash firing An imaging program that realizes a determination function and a combination function that combines the first image data and the second image data at a different combination ratio for each of the determined pixels to generate image data for recording The composition ratio determining function is not limited to one of the pixel value of the first image data and the pixel value of the second image data, but a combination of both pixel values. And determining for each pixel three stages or more different synthetic ratios depending on the cause.
In order to achieve the above object, an image pickup apparatus according to an eighth aspect of the present invention is an image pickup device for picking up an image of a subject, a light emitting device, and an image pickup device when photographing with the image pickup device without causing the light emitting device to emit light. A sensitivity setting means for lowering imaging sensitivity when shooting with the imaging means by increasing the sensitivity and emitting the light emitting means, and a first image taken by the imaging means without causing the light emitting means to emit light A composition ratio determining means for determining for each pixel a composition ratio between the data and the second image data photographed by the imaging means by causing the light emitting means to emit light, and each pixel determined by the composition ratio determining means A synthesis unit that synthesizes the first image data and the second image data at a different synthesis ratio for each to generate image data for recording, and the synthesis ratio determination unit includes the first image data and the second image data. Picture And determines the combination ratio that varies in a plurality of stages of three or more stages according to the pixel value or the pixel value of the second image data of the data.

According to the first aspect of the present invention, when photographing is performed by the imaging means without causing the light emitting means to emit light by the photographing sensitivity setting means, the imaging sensitivity is increased, and when photographing is performed by the imaging means by causing the light emitting means to emit light. The imaging sensitivity is lowered, and the first image data photographed by the imaging means without causing the light emitting means to emit light by the combination ratio determining means, and the second image photographed by the imaging means by causing the light emitting means to emit light. A synthesis ratio with the image data is determined for each pixel, and the first image data and the second image data are synthesized by the synthesis means at a different synthesis ratio for each determined pixel, and for recording The image data is generated, and the combination ratio determining unit determines not only one of the pixel values of the first image data and the pixel value of the second image data but also a combination of both pixel values. The pixel Since so as to determine a different combination ratio, in such particular night photography, while securing sufficient brightness in the captured image, noise and by the sensitivity change, the image of good balance between the hand shake by the shutter speed changes, easy The advantage that it can be obtained by simple arithmetic processing is obtained.

  According to the second aspect of the present invention, the image pickup means, the image pickup means, so that the shooting control means continuously performs the shooting without causing the light emitting means to emit light and the shooting with which the light emitting means emits light in response to a single shooting instruction. Since the light emission means and the sensitivity control means are controlled, the shooting sensitivity can be lowered without performing special processing for specifying the distance to each subject and the amount of flash light irradiation, especially during night photography. Noise that ensures the brightness of the front subject portion that was shot with flash firing and the back subject portion that was shot with higher shooting sensitivity secured by flash firing. Select an image with a small amount of noise, and select an image that has noise in the back subject area but ensures brightness with high sensitivity. Advantage can be obtained by can be obtained.

According to the invention described in claim 3, the pixel ratio of the first image data and the pixel value of the second image data are compared by the combination ratio determining means, and the combination ratio of the larger image data is compared. Since the composition ratio is determined so that the image becomes higher , especially in night shooting, the subject part in the foreground shot with low shooting sensitivity and flash firing and the depth of the shot taken with high shooting sensitivity The subject part in the front is roughly identified, and the front subject part is selected with a low-noise image that has been brightened by flash emission, while the subject part in the back is noisy but bright due to high sensitivity. An advantage is obtained in that an image equivalent to that obtained by selecting an image in which the thickness is ensured and performing the synthesis can be obtained by simpler arithmetic processing.

According to the fourth aspect of the present invention, the combination of the pixel value obtained by photographing without causing the light emitting means to emit light and the pixel value obtained by photographing while causing the light emitting means to emit light by the combination ratio determining means. When the first image data and the second image data are synthesized, the area indicated by is divided into three or more areas in advance and assigned different synthesis ratios for each area. The region to which the combination of the pixel value of the image data and the pixel value of the second image data belongs is specified, and the combination ratio assigned to the specified region is determined as a combination ratio corresponding to the combination of the pixel values. In particular, when shooting at night, etc., it is possible to roughly distinguish the near subject part that was shot with low shooting sensitivity and flash firing and the deep subject part that was shot with high shooting sensitivity. Shi , Select a low noise image that has been brightened by flash emission for the subject area in the foreground, and select an image that has high sensitivity and brightness for the back subject area. There is an advantage that an image equivalent to that obtained by combining can be obtained by a simpler arithmetic processing.

According to the fifth aspect of the present invention, the combination ratio determining means indicates a combination of a pixel value obtained by photographing without causing the light emitting means to emit light and a pixel value obtained by photographing while causing the light emitting means to emit light. When the area is divided into three or more areas in advance and any one of three or more different combining ratios is assigned to each area, and the first image data and the second image data are combined. , Specifying which region the combination of the pixel value of the first image data and the pixel value of the second image data belongs, and any of three or more different composition ratios assigned to the specified region the synthesis ratio, since determined as the combination ratio corresponding to a combination of the pixel values, particularly in the nighttime photography, and before the object part taken by flash to lower the imaging sensitivity, imaging Approximately identify the back subject part that was shot at a higher degree, select a low-noise image that was brightened by flash emission for the front subject part, and The advantage is that it is possible to obtain an image equivalent to that obtained by selecting and synthesizing an image that has noise but has high brightness due to high sensitivity, by simpler arithmetic processing.

According to the sixth aspect of the present invention, in accordance with one shooting instruction, the step of continuously performing shooting without flash emission with high imaging sensitivity and shooting with flash emission with low imaging sensitivity, A composition ratio determining step for determining, for each pixel, a composition ratio between the first image data photographed without light emission and the second image data photographed with the flash light emitted; And combining the first image data and the second image data at different composition ratios to generate image data for recording, and by combining ratio determination step, the pixel value of the first image data When not only one pixel value of the pixel values of the second image data. Thus to determine the different synthesis ratio for each pixel in accordance with a combination of both pixel values, such as especially at night shooting There are, while securing sufficient brightness in the captured image, noise and by the sensitivity change, the balance of good image with camera shake by the shutter speed changes, the advantage is obtained that can be obtained by simple arithmetic processing.

According to the invention described in claim 7 , in accordance with one shooting instruction, the shooting function for continuously performing shooting without flash emission with high imaging sensitivity and shooting with flash emission with low imaging sensitivity is performed. And a composite ratio determining function for determining a composite ratio between the first image data captured without the flash and the second image data captured with the flash for each pixel. An imaging program for realizing a synthesis function for synthesizing the first image data and the second image data at a different synthesis ratio for each pixel and generating image data for recording, The function determines not only one pixel value of the pixel value of the first image data and the pixel value of the second image data but also a different composition ratio for each pixel according to the combination of both pixel values. Were so as, in, especially at night shooting, while securing sufficient brightness in the captured image, noise and by the sensitivity change, the balance of good image with camera shake by the shutter speed changes, be obtained by a simple calculation process The advantage that it can be obtained.
According to the eighth aspect of the present invention, when photographing is performed by the imaging means without causing the light emitting means to emit light by the photographing sensitivity setting means, the imaging sensitivity is increased, and the light emitting means is caused to emit light and photographing is performed by the imaging means. In this case, the imaging sensitivity is lowered, and the first image data photographed by the imaging means without causing the light emitting means to emit light by the combination ratio determining means, and the second image data photographed by the imaging means by causing the light emitting means to emit light. Is synthesized for each pixel, and the synthesis means synthesizes the first image data and the second image data at different synthesis ratios for each determined pixel, and records image data for recording. Since the composition ratio determining means generates and determines the composition ratio that changes in a plurality of stages of three or more stages according to the pixel value of the first image data or the pixel value of the second image data. For shooting There are, while securing sufficient brightness in the captured image, noise and by the sensitivity change, the balance of good image with camera shake by the shutter speed changes, the advantage is obtained that can be obtained by simple arithmetic processing.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

A. First embodiment A-1. Configuration of First Embodiment FIG. 1 is a block diagram showing a configuration of a digital camera according to a first embodiment of the present invention. In the figure, the image acquisition unit 10 includes a lens 11, a shutter 12, and an LPF 13. The lens 11 is a normal optical lens and includes a lens group in which aspherical lenses are stacked. The shutter 12 is a so-called mechanical shutter that is operated by a driver 14 driven by the control unit 20 when a shutter button is operated. Depending on the digital camera, a mechanical shutter may not be provided. In the case of a model equipped with a retractable lens structure and a mechanical zoom, these drivers are also controlled by the driver 14. The LPF 13 is a crystal low-pass filter and is mounted to prevent the occurrence of moire.

  Next, the analog signal processing unit 15 includes an image sensor (CCD, CMOS) 16, a sampling / signal amplification processing unit 17, and an A / D converter 18. The imaging sensor 16 forms a subject image (image), and converts the intensity of light of each color of RGB into a current value. The sampling / signal amplification processing unit 17 performs correlated double sampling processing and signal amplification processing for suppressing noise and color unevenness. The A / D converter 18 is also called an analog front end, and converts the sampled / amplified analog signal into a digital signal (converts each color of RGB and CMYG into 12-bit data and outputs it to the bus line).

  Next, the control unit (CPU) 20 controls the entire digital camera 1 (imaging device) according to a program stored in a program memory described later. In particular, in the first embodiment, in night scene shooting, a background night scene and a subject (such as a person) in the foreground are shot by two exposures. Although details will be described later, in the first shooting (high-sensitivity shooting), the ISO sensitivity is increased and exposed, and in the second shooting (flash flash shooting), the ISO sensitivity is decreased and the flash is emitted for exposure. Then, the first image and the second image are combined according to a predetermined threshold and a predetermined combining ratio (weight) described later. The control unit 20 controls the operation at the time of night scene shooting, image capture, image composition, and the like.

  The preview engine 22 is stored in the image buffer 26 immediately after detection of digital data input via the image acquisition unit 10 and the analog signal processing unit 15 in the recording mode (also referred to as recording mode or photographing mode) or shutter operation detection. Thinning processing is performed in order to display the digital data and the digital data stored in the image memory 31 on the display unit 25. The D / A converter 23 converts the digital data thinned out by the preview engine 22 and outputs it to the driver 24 at the subsequent stage.

  The driver 24 includes a buffer area for temporarily storing digital data displayed on the display unit 25 at the subsequent stage, and drives the display unit 25 based on a control signal input via the key operation unit 27 and the control unit 20. The display unit 25 includes a color TFT liquid crystal, an STN liquid crystal, or the like, and displays a preview image, image data after shooting, a setting menu, and the like.

  The image buffer 26 temporarily stores digital data immediately after photographing until it is input via the analog signal processing unit 15 or the digital signal processing unit 28 and passed to the digital signal processing unit 28. The key operation unit 27 includes a shutter button, a recording / playback mode selection slide switch, a menu button, a cross key (determined by pressing the center), and the like.

  The digital signal processing unit 28 performs white balance processing, color processing, gradation processing, contour emphasis, conversion from RGB format to YUV format, YUV format for digital data input via the analog signal processing unit 15. To JPEG format. In particular, in the first embodiment, the digital signal processing unit 28 emits a flash by lowering the ISO sensitivity and the image of the first shooting (high-sensitivity shooting), which is exposed with increased ISO sensitivity during night scene shooting. The image obtained by the second shooting (flash flash shooting) is then combined in accordance with a predetermined threshold value and a predetermined combining ratio (weight) described later.

  The image compression / decompression processing unit 29 compresses and encodes the digital data input via the digital signal processing unit 28 into the JPEG format, generates a motion JPEG format movie file, or converts the motion JPEG format movie file to MPEG. It is converted into a moving image file in a format, or in a reproduction mode, a moving image file in JPEG format, motion JPEG format, or MPEG format is expanded.

  The program memory 30 stores various programs loaded in the control unit 20, EV values in the best shot function, color correction information, and the like. The image memory 31 stores image data temporarily stored in the image buffer 26, digital data converted into various file formats, moving image data, and the like. The card I / F 32 controls data exchange between the external recording medium 33 and the imaging apparatus main body.

  The external recording medium 33 is a detachable recording medium composed of a compact flash (registered trademark), a memory stick, an SD card, or the like. The external connection I / F 34 includes a USB connector slot and the like, is connected to a personal computer or the like, and is used for transferring photographed image data. The RAM 35 stores various parameters necessary for control of the control unit (CPU) 20, various parameters at the time of night scene shooting (gain (ISO sensitivity), aperture, shutter speed, threshold for image composition, weight, etc.), and the like. .

  Next, FIG. 2 is a timing chart showing the operation of the digital camera according to the first embodiment during night scene shooting. When shooting a night scene, first, when the mechanical shutter is opened and the shutter button is pressed, first, in the exposure period ET1, the ISO sensitivity is increased to expose the image, the mechanical shutter is closed, and the data is transferred in the transfer period TT1. Read with. This image corresponds to an image of a night view of the background (far view). Thereafter, the mechanical shutter is immediately opened, the ISO sensitivity is lowered, the flash is emitted, the exposure is performed with the exposure time ET2, the data is read out in the transfer period TT2 after the mechanical shutter is closed. The image corresponds to an image of a subject such as a person in front.

  Next, FIG. 3 is a conceptual diagram showing a threshold when combining an image at the time of high-sensitivity shooting with the digital camera of the first embodiment and an image at the time of flash emission shooting. In the first embodiment, for the image data D2 captured at the time of the second flash photography, it is determined for each pixel whether the luminance value of the pixel is larger than the threshold value THA, and the luminance value of the pixel is larger than the threshold value THA. If it is in A, the pixel of the image data D2 (D1 × 0% + D2 × 100%) captured at the time of the second flash photography is used, and if the luminance value of the pixel is equal to or lower than the threshold THA and in the region B, the first time. An image for recording is obtained by synthesizing images using pixels of image data D1 (D1 × 100% + D2 × 0%) during high-sensitivity shooting. The synthesis ratio of the threshold THA and the image data D1 and D2 is stored in the above-described RAM 35 in the form of a data table or a calculation formula.

AB. Operation of the First Embodiment Next, the operation of the first embodiment will be described. Here, FIG. 4 is a flowchart for explaining the operation of the digital camera according to the first embodiment during night scene shooting. FIG. 5 is a schematic diagram showing an image at the time of night scene shooting and an example of image synthesis.

  When night scene shooting is selected, first, as a high sensitivity setting, the aperture is opened, the gain (ISO sensitivity) is set to the maximum, and the shutter speed is set to the maximum time without camera shake (step S10). Next, when the shutter button is pressed, high-sensitivity exposure (exposure period ET1 shown in FIG. 2) is performed, and a night view of a distant view is mainly photographed (step S12), which is transferred to the image buffer 26 as image data D1 and temporarily stored. Save (step S14). As shown in FIG. 5A, the night scene image is taken in a better state of the night scene as a background without camera shake. However, the foreground subject is underexposed.

  Next, as the flash exposure setting, the gain (ISO sensitivity) is set to the minimum, the flash is set to the maximum light emission, and others are set so that the highest quality image can be taken under the above two conditions (step S16). Next, a flash is emitted for exposure (exposure period ET2 shown in FIG. 2), a subject (such as a person) in the foreground is photographed (step S18), transferred to the image buffer 26 as image data D2, and temporarily stored (step S18). Step S20). As shown in FIG. 5B, the close-up image is taken in a better state of the person who is the subject. However, the night view of the distant view is underexposed because the flash light does not reach and the exposure time ET2 is short.

  Next, a loop process for creating data for each pixel is executed in steps S22 to S32. First, the image data D2 at the time of flash photography is read at a predetermined coordinate (step S24), and it is determined whether or not the luminance value of the image data D2 exceeds the threshold value THA (step S26). When the luminance value at the time of flash photography is greater than the threshold value THA, that is, when it is the area A in FIG. 2, the image data D2 at the time of flash photography (D1 × 0% + D2 × 100%). Is stored in the image memory 31 as composite data (step S28).

  On the other hand, when the luminance value at the time of flash photography does not exceed the threshold value THA, that is, in the region B in FIG. 2, the image data D1 (D1 × 100% + D2 × 0) at the time of the first high-sensitivity photography. %) Is stored in the image memory 31 as composite data (step S30). The above processing is performed for all pixels to obtain recording image data.

  As a result, as shown in FIG. 5C, the synthesized image is composed of a night view taken in a better state and a subject in the foreground taken in a better state. The captured image has a good balance of the / N ratio.

B. Second Embodiment Next, a second embodiment of the present invention will be described. The configuration of the digital camera is the same as that in FIG. Also, the operation of the digital camera at the time of night view shooting is the same as the timing chart shown in FIG. In the second embodiment, the method of setting the threshold value when the first image and the second image are combined is changed, and the threshold value THB is set at 1: 1 as shown in FIG. For each pixel, compare the brightness value of the image captured during the first high-sensitivity shooting with the brightness value of the image captured during the second flash shooting, and synthesize the image using the pixel with the higher luminance value. Thus, an image for recording is obtained.

  That is, if the luminance value of the pixel of the image captured at the second flash photography is larger, in other words, if it is in the area A, the image data D2 is the pixel value of the image captured at the second flash photography. (D1 × 0% + D2 × 100%) is used, and if the luminance value of the pixel of the image captured at the time of the first high-sensitivity shooting is larger, in other words, if it is in the region B, at the time of the first high-sensitivity shooting An image for recording is obtained by combining images using image data D1 (D1 × 100% + D2 × 0%) which is a pixel value of the captured image. The synthesis ratio of the threshold THB and the image data D1 and D2 is stored in the RAM 35 in the form of a data table or a calculation formula.

B-2. Operation of the Second Embodiment Next, the operation of the second embodiment described above will be described. Here, FIG. 7 is a flowchart for explaining the operation of the digital camera according to the second embodiment during night scene shooting. When night scene shooting is selected, first, as a high sensitivity setting, the aperture is opened, the gain (ISO sensitivity) is set to the maximum, and the shutter speed is set to the maximum time without camera shake (step S40). Next, when the shutter button is pressed, high-sensitivity exposure (exposure period ET1 shown in FIG. 2) is performed, and a night view of a distant view is mainly photographed (step S42), which is transferred to the image buffer 26 as image data D1 and temporarily stored. Save (step S44). As shown in FIG. 5A, the night scene image is taken in a better state of the night scene as a background without camera shake. However, the foreground subject is underexposed.

  Next, as the flash exposure setting, the gain (ISO sensitivity) is set to the minimum, the flash is set to the maximum light emission, and others are set so that the highest quality image can be taken under the above two conditions (step S46). Next, the flash is fired for exposure (exposure period ET2 shown in FIG. 2), a subject (such as a person) in the foreground is photographed (step S48), transferred to the image buffer 26 as image data D2, and temporarily stored (step S48). Step S50). As shown in FIG. 5B, the close-up image is taken in a better state of the person who is the subject. However, the night view of the distant view is underexposed because the flash light does not reach and the exposure time ET2 is short.

  Next, a loop process for creating data for each pixel is executed in steps S52 to S62. First, the image data D1 at the time of high-sensitivity shooting and the image data D2 at the time of flash shooting are read at predetermined coordinates (step S54), and whether or not the luminance value of the image data D1 is greater than the luminance value of the image data D2. Is determined (step S56). If the luminance value at the time of high-sensitivity shooting is larger, that is, in the region B of FIG. 6, the image data D1 (D1 × 100% + D2 × 0%) at the time of the first high-sensitivity shooting is obtained. The synthesized data is stored in the image memory 31 (step S58).

  On the other hand, when the luminance value at the time of flash photography is larger, that is, in the area A of FIG. 6, the image data D2 (D1 × 0% + D2 × 100%) at the time of the second flash photography is obtained. The synthesized data is stored in the image memory 31 (step S60). The above processing is performed for all pixels to obtain recording image data.

  As a result, the synthesized image is composed of a night view taken in a better state and a subject in the foreground taken in a better state, and has a good balance between exposure and S / N ratio without camera shake. It becomes.

C. Third Embodiment Next, a third embodiment of the present invention will be described. The configuration of the digital camera is the same as that in FIG. In the third embodiment, the threshold value and the composition ratio for combining the first image and the second image are changed, and threshold values THC1 and THC2 are provided as shown in FIG. The area obtained by the relationship between the brightness value of the image and the brightness value at the time of high-sensitivity shooting is subdivided into four areas A to D. Note that the boundary between the region B and the region C is determined by size comparison. Then, for each pixel, the luminance value of the image captured at the first high-sensitivity shooting is compared with the luminance value of the image captured at the second flash shooting, and it is determined whether the luminance value is in which area. The pixel composition ratio to be used as the (image) is determined.

  For example, in the area A, the pixel value of the composite image is set to the image data D1 × 0% at the time of high sensitivity shooting + the image data D2 × 100% at the time of flash shooting, and in the area B, the image data D1 at the time of high sensitivity shooting. X25% + image data D2 × 75% during flash photography, and in area C, image data D1 × 75% during high sensitivity photography + image data D2 × 25% during flash photography, and in area D, D1 × 100% + D2 × 0%. The synthesis ratios of the threshold values THC1 and THC2 and the image data D1 and D2 are stored in the above-described RAM 35 in the form of a data table or a calculation formula.

  Next, the operation of the third embodiment will be described. Here, FIGS. 9 and 10 are flowcharts for explaining the operation of the digital camera according to the third embodiment at the time of night scene shooting. When night scene shooting is selected, first, as a high sensitivity setting, the aperture is opened, the gain (ISO sensitivity) is set to the maximum, and the shutter speed is set to the maximum time without camera shake (step S70). Next, when the shutter button is pressed, high-sensitivity exposure (exposure period ET1 shown in FIG. 2) is performed, and a night view of a distant view is mainly photographed (step S72), which is transferred to the image buffer 26 as image data D1 and temporarily stored. Save (step S74). As shown in FIG. 5A, the night scene image is taken in a better state of the night scene as a background without camera shake. However, the foreground subject is underexposed.

  Next, as the flash exposure setting, the gain (ISO sensitivity) is set to the minimum, the flash is set to the maximum light emission, and others are set so that the highest quality image can be taken under the above two conditions (step S76). Next, a flash is emitted for exposure (exposure period ET2 shown in FIG. 2), a subject (such as a person) in the foreground is photographed (step S78), transferred to the image buffer 26 as image data D2, and temporarily stored (step S78). Step S80). As shown in FIG. 5B, the close-up image is taken in a better state of the person who is the subject. However, the night view of the distant view is underexposed because the flash light does not reach and the exposure time ET2 is short.

  Next, a loop process for creating data for each pixel is executed in steps S82 to S104. First, the image data D1 at the time of high-sensitivity shooting and the image data D2 at the time of flash shooting are read at predetermined coordinates (step S84), and whether or not the luminance value of the image data D1 is larger than the luminance value of the image data D2. Is determined (step S86). When the brightness value of the image data D1 at the time of high-sensitivity shooting is larger, that is, in the region C or the region D in FIG. 8, the brightness value of the image data D1 and the brightness value of the image data D2 are It is determined whether or not the difference is larger than a threshold value THC2 shown in FIG. 8 (step S88). If the difference between the luminance value of the image data D1 and the luminance value of the image data D2 is larger than the threshold value THC2, it is determined that the region is in the region D, and the image data D1 (D1 × 100) at the time of the first high-sensitivity shooting. % + D2 × 0%) is stored in the image memory 31 as composite data (step S90).

  On the other hand, when the difference between the luminance value of the image data D1 and the luminance value of the image data D2 is equal to or less than the threshold value THC2, it is determined that the region is in the region C, and the combined image data Db is image data D1 × 3/4 + image data. D2 × 1/4 is calculated (step S92), and the composite image data Db is stored in the image memory 31 (step S94).

  On the other hand, if the brightness value of the image data D2 at the time of flash photography is larger in step S86, that is, if it is the area A or the area B in FIG. 8, the brightness value of the image data D1 and the image data It is determined whether or not the difference between the luminance value of D2 is larger than the threshold value THC1 shown in FIG. 8 (step S96). If the difference between the luminance value of the image data D1 and the luminance value of the image data D2 is smaller than the threshold value THC1, it is determined that the region is in the region B, and the combined image data Dc is image data D1 × 1/4 + image data. D2 × 3/4 is calculated (step S98), and the composite image data Dc is stored in the image memory 31 (step S100).

  On the other hand, when the difference between the luminance value of the image data D1 and the luminance value of the image data D2 is equal to or greater than the threshold value THC1, it is determined that the area is A, and the image data D2 (D1 × 0) at the time of the second flash photography. % + D2 × 100%) is stored in the image memory 31 as composite data (step S102).

  As a result, the synthesized image is composed of a night view taken in a better state and a subject in the foreground taken in a better state, and has a good balance between exposure and S / N ratio without camera shake. It becomes.

D. Fourth Embodiment Next, a fourth embodiment of the present invention will be described. The configuration of the digital camera is the same as that in FIG. In the fourth embodiment, the threshold value and the composition ratio for combining the first image and the second image are changed, and threshold values THD1 to THD4 are provided as shown in FIG. The area obtained by the relationship between the brightness value of the image and the brightness value at the time of high-sensitivity shooting is subdivided into nine areas A to I. Then, for each pixel, the luminance value of the image captured at the first high-sensitivity shooting is compared with the luminance value of the image captured at the second flash shooting, and it is determined whether the luminance value is in which area. The pixel composition ratio to be used as the (image) is determined.

  For example, in regions A and I, the pixel values of the composite image are set to image data D1 × 50% at the time of high-sensitivity shooting + image data D2 × 50% at the time of flash shooting, and regions B, C, E, G, H The image data D1 × 0% at the time of high-sensitivity shooting and the image data D2 × 100% at the time of flash shooting are set. In the regions D and F, the image data D1 × 100% at the time of high-sensitivity shooting + The image data D2 × 0%. The synthesis ratios of the threshold values THD1 to THD4 and the image data D1 and D2 are stored in the above-described RAM 35 in the form of a data table or a calculation formula. Note that the region determination and the image composition are basically the same as those in the third embodiment described above, and thus the description thereof is omitted.

E. Modified Example Next, a modified example of the present invention will be described. In the first to fourth embodiments described above, different composition ratios are clearly set for each region for determining the pixel composition ratio. That is, in the adjacent regions, the composition ratios are different even when they exist in the vicinity of the boundary. On the other hand, in this modification, as shown in FIG. 12, for example, the boundary portion between the region A and the region B is divided into finer regions C1 to C6, and the composition ratio is changed stepwise. . This makes it possible to obtain a finer composite image.

  As another modification, as described above, the boundary portion between the region A and the region B may not be divided more finely, but the synthesis ratio may actually be gradually changed by an analog circuit. Further, although the pixel value at each position is determined by combining the pixels at the same position, pixels in the peripheral area may be used. Three or more captured images may be used.

  In the above-described embodiment, identification processing for accurately identifying the front subject portion and the background subject portion is performed by simultaneously performing identification and pixel synthesis of the front subject portion and the background subject portion. Although not required, if distance measurement information indicating the distance to the subject can be used, the distance information may be used for identification.

  In the above-described embodiments, different pixel composition methods have been described as different embodiments. However, one pixel composition method is set in advance by mode selection from a plurality of pixel composition methods, and a shooting instruction is issued. At this time, the setting mode may be determined, and pixel synthesis may be executed according to the selected pixel synthesis method. In this case, the setting of the pixel synthesis mode may be arbitrarily specified by the user, may be set automatically by sensing external brightness with a sensor, or based on image information obtained by the image sensor. May be set automatically.

  In the above-described embodiment, only the digital camera has been described. However, the present invention is not limited to this, and the present invention may be applied to an electronic apparatus including an imaging device such as a mobile phone. In addition, the image composition processing and the like according to the above-described embodiment are realized by software control by the control unit 20, and all or part of them is considered in consideration of processing speed, versatility, device size, cost, and the like. May be realized by a circuit.

It is a block diagram which shows the structure of the digital camera by 1st Embodiment of this invention. It is a timing chart which shows the operation | movement at the time of night view imaging | photography of the digital camera by this 1st Embodiment. It is a conceptual diagram which shows the threshold value at the time of synthesize | combining the image at the time of high sensitivity imaging | photography with the digital camera of this 1st Embodiment, and the image at the time of flash light emission imaging | photography. 4 is a flowchart for explaining the operation of the digital camera according to the first embodiment during night scene shooting. It is a schematic diagram which shows the image at the time of night view photography of the digital camera by this 1st Embodiment, and an example of an image synthesis. It is a conceptual diagram which shows the threshold value at the time of synthesize | combining the image at the time of high sensitivity imaging | photography with the digital camera of 2nd Embodiment of this invention, and the image at the time of flash light emission imaging | photography. It is a flowchart for demonstrating operation | movement at the time of night view imaging | photography of the digital camera by this 2nd Embodiment. It is a conceptual diagram which shows the threshold value at the time of synthesize | combining the image at the time of high sensitive imaging | photography with the digital camera of 3rd Embodiment of this invention, and the image at the time of flash light emission imaging | photography. It is a flowchart for demonstrating operation | movement at the time of night view imaging | photography of the digital camera by this 3rd Embodiment. It is a flowchart for demonstrating operation | movement at the time of night view imaging | photography of the digital camera by this 3rd Embodiment. It is a conceptual diagram which shows the threshold value at the time of synthesize | combining the image at the time of high sensitivity imaging | photography with the digital camera of 4th Embodiment of this invention, and the image at the time of flash light emission imaging | photography. It is a conceptual diagram which shows the threshold value at the time of synthesize | combining the image at the time of high sensitivity imaging | photography with the digital camera of the modification of this invention, and the image at the time of flash light emission imaging | photography. It is a timing chart which shows operation | movement of the digital camera at the time of night view photography by a prior art.

Explanation of symbols

1 Digital Camera 10 Image Acquisition Unit 11 Lens 12 Shutter 13 LPF
14 Driver 15 Analog Signal Processing Unit 16 Imaging Sensor (Imaging Means)
17 Sampling / Signal Amplification Processing Unit 18 A / D Converter 19 Flash (Light Emitting Unit)
20 control unit (photographing sensitivity setting means, photographing control means)
22 Preview Engine 23 D / A Converter 24 Driver 25 Display Unit 26 Image Buffer 27 Key Operation Unit 28 Digital Signal Processing Unit (Composition Unit)
29 Image compression / decompression processing unit 30 Program memory 31 Image memory 32 Card I / F
33 External recording medium 34 I / F for external connection
35 RAM

Claims (8)

Imaging means for capturing a subject image;
A light emitting means;
An imaging sensitivity setting unit that increases imaging sensitivity when shooting with the imaging unit without causing the light emitting unit to emit light, and lowers imaging sensitivity when shooting with the imaging unit with the light emitting unit emitting light;
A combination ratio of first image data captured by the imaging unit without causing the light emitting unit to emit light and second image data captured by the imaging unit while causing the light emitting unit to emit light is determined for each pixel. A synthesis ratio determining means to
Synthesizing the first image data and the second image data at a different composition ratio for each pixel determined by the composition ratio determining means, and generating image data for recording,
The combination ratio determining unit is configured to combine not only one pixel value of the first image data and one pixel value of the second image data but also three or more different combinations according to the combination of both pixel values. An imaging apparatus , wherein a ratio is determined for each pixel .   In response to a single shooting instruction, the imaging unit, the light emitting unit, and the sensitivity are configured to continuously perform shooting by the imaging unit that does not cause the light emitting unit to emit light and imaging by the imaging unit that causes the light emitting unit to emit light. The imaging apparatus according to claim 1, further comprising photographing control means for controlling the control means. The synthesis ratio determining means compares the pixel value of the first image data with the pixel value of the second image data and compares the pixel value of the second image data with three or more steps so that the synthesis ratio of the larger image data is increased. The imaging apparatus according to claim 1 , wherein different composition ratios are determined.   The combination ratio determining means includes at least three areas indicated in advance by a combination of a pixel value obtained by photographing without causing the light emitting means to emit light and a pixel value obtained by photographing while causing the light emitting means to emit light. When the first image data and the second image data are combined, the pixel value of the first image data and the first image data are divided. A region to which a combination of pixel values of the second image data belongs is specified, and a combination ratio assigned to the specified region is determined as a combination ratio corresponding to the combination of the pixel values. The imaging apparatus according to claim 1 or 2.   The combination ratio determining means includes at least three areas indicated in advance by a combination of a pixel value obtained by photographing without causing the light emitting means to emit light and a pixel value obtained by photographing while causing the light emitting means to emit light. When the first image data and the second image data are combined, the first image data and the second image data are combined. Identifying which region the combination of the pixel value of the image data and the pixel value of the second image data belongs to, and combining at any one of the three or more different combining ratios assigned to the specified region The imaging apparatus according to claim 4, wherein the ratio is determined as a combination ratio corresponding to the combination of the pixel values. In response to a single shooting instruction, a step of continuously performing shooting without flash emission with high imaging sensitivity and shooting with flash emission with low imaging sensitivity;
A composition ratio determining step for determining, for each pixel, a composition ratio between the first image data captured without the flash being emitted and the second image data captured with the flash emitted;
Combining the first image data and the second image data at a different composition ratio for each of the determined pixels, and generating image data for recording,
The composition ratio determining step includes not only one pixel value of the pixel value of the first image data and the pixel value of the second image data, but also three or more different composition depending on the combination of both pixel values. An imaging method, wherein a ratio is determined for each pixel . On the computer,
In response to a single shooting instruction, a shooting function that continuously performs shooting without flash with high imaging sensitivity and shooting with flash with low imaging sensitivity;
A composition ratio determining function for determining, for each pixel, a composition ratio between the first image data captured without the flash being emitted and the second image data captured with the flash emitted;
An imaging program for realizing a synthesis function for synthesizing the first image data and the second image data at different synthesis ratios for each of the determined pixels and generating image data for recording,
The composition ratio determining function has not only one pixel value of the pixel value of the first image data and the pixel value of the second image data but also three or more different composition depending on the combination of both pixel values. An imaging program , wherein a ratio is determined for each pixel . Imaging means for capturing a subject image;
A light emitting means;
An imaging sensitivity setting unit that increases imaging sensitivity when shooting with the imaging unit without causing the light emitting unit to emit light, and lowers imaging sensitivity when shooting with the imaging unit with the light emitting unit emitting light;
A combination ratio of first image data captured by the imaging unit without causing the light emitting unit to emit light and second image data captured by the imaging unit while causing the light emitting unit to emit light is determined for each pixel. A synthesis ratio determining means to
Synthesizing the first image data and the second image data at a different composition ratio for each pixel determined by the composition ratio determining means, and generating image data for recording,
The composite ratio determining means determines a composite ratio that changes in a plurality of stages of three or more stages according to a pixel value of the first image data or a pixel value of the second image data. .

Images