JP2011078074A - Imaging apparatus and control method therefor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To obtain an image having proper brightness by correcting blur of an image captured using an auxiliary light source. <P>SOLUTION: The imaging apparatus having an auxiliary light source determines a light emitting pattern which shows on/off of light emission in an auxiliary light source by 1/0, as shown in (a), based on imaging conditions so that a corrected image has proper brightness. The imaging apparatus emits light from the auxiliary light source for imaging, according to the light emitting pattern, and obtains a photographic image and information on blur at the time of imaging. Based on the blur information and the light emitting pattern, blur of the image is corrected. Furthermore, the light emitting pattern, as shown in (b), is set so that frequency characteristics in the result of calculating a point spread function may include only non-zero values. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an imaging apparatus that performs shake correction on an image obtained by imaging in an imaging apparatus having an auxiliary light source, and a control method thereof.

  2. Description of the Related Art In recent years, camera shake correction processing at the time of shooting has attracted attention as the resolution of an image pickup element in an image pickup apparatus such as a digital camera increases. As a camera shake correction method, there are a hardware implementation method and a software implementation method.

  First, as a hardware implementation method, a method of reducing camera shake by mounting a gyro sensor on a digital camera and driving a lens or an image sensor so as to cancel vibrations obtained from the gyro sensor during exposure can be considered. For example, an implementation method for reducing camera shake by correcting a lens optical system has been proposed (see, for example, Patent Document 1).

  Further, as a method for realizing camera shake correction processing by software, a technique called “Coded Exposure” as described below has been proposed (for example, see Patent Document 2). In other words, this is a technique that encodes the opening and closing of the shutter at the time of exposure when acquiring one image, and corrects camera shake or subject blur by deconvolution calculation using the information on the shutter opening and closing.

  In general, in a digital camera, an image is taken using an auxiliary light source when the shooting environment is dark. However, when the intensity of light that can be output from the auxiliary light source is small, it is necessary to lengthen the shutter time in order to obtain an optimal amount of light for photographing the subject. As a result, there is a problem that camera shake is likely to occur. . In order to solve this problem, there is a method of acquiring a proper exposure image by photographing a plurality of low-brightness images at a shutter speed at which camera shake does not occur, and synthesizing the obtained plurality of images by software processing. It has been proposed (see, for example, Patent Document 3).

JP 2002-214657 A US Patent Application Publication No. 2007/0258706 JP 2006-66996 A

  The above-described camera shake correction processing using hardware has a problem of increasing the number of parts and manufacturing cost. Therefore, when a similar function is realized by an inexpensive digital camera, it is realistic to perform correction processing by software.

  However, the realization method using software described in Patent Document 3 has a problem that blurring of each low-luminance image itself cannot be corrected. Japanese Patent Application Laid-Open No. 2004-228561 discloses a method of performing image blur correction by encoding and shooting a light emission pattern of an auxiliary light source. However, since a specific method for controlling the auxiliary light source is not described, overexposure, underexposure, or the like may occur in the image after blur correction, and an image with appropriate brightness may not be obtained.

  The present invention has been made to solve the above-described problems, and provides an imaging apparatus that performs shake correction on a captured image using an auxiliary light source to acquire an image with appropriate brightness, and a control method thereof. Objective.

  As a means for achieving the above object, an imaging apparatus of the present invention comprises the following arrangement.

  That is, an imaging apparatus having an auxiliary light source, the light emission condition determining means for determining a light emission condition including a light emission pattern indicating on / off of light emission in the auxiliary light source based on a photographing condition, and the light emission condition according to the light emission condition Based on the light emission photographing means for photographing a subject by emitting an auxiliary light source and obtaining a photographed image, the blur information obtaining means for obtaining blur information at the time of photographing by the light emission photographing means, the blur information and the light emission conditions. And a light emission photographic image correction means for performing blur correction on the photographic image.

  According to the present invention having the above-described configuration, it is possible to perform blur correction on a captured image using an auxiliary light source and obtain an image with appropriate brightness.

The figure which shows the external appearance of the digital camera in each embodiment which concerns on this invention, The block diagram which shows the circuit structure in the digital camera of a present Example, The flowchart which shows the blurring correction process by the encoding light emission photography in this embodiment, A flowchart showing correction function creation processing in the present embodiment, The figure which shows an example of the arrangement | sequence set to zero in this embodiment, The figure which shows the one-dimensional Fourier-transform example of the encoding light emission pattern in this embodiment, The flowchart which shows the image correction process in this embodiment, The figure which shows the fixed light emission pattern example of the auxiliary light source in this embodiment, and its transition blur function, The figure which shows the encoding light emission pattern example of the auxiliary light source in this embodiment, and its point image blur function, The flowchart which shows the light emission condition determination process of the auxiliary light source in this embodiment, The figure which shows an example of the encoding light emission pattern table in this embodiment. The figure which shows the luminous intensity determination example of the auxiliary light source in this embodiment, The figure which shows the example of the encoding light emission pattern which increased / decreased the luminous intensity of the auxiliary light source in this embodiment continuously, and its luminous intensity determination example, The flowchart which shows the switching process of camera-shake correction in 2nd Embodiment, The flowchart which shows the switching process of camera-shake correction in 3rd Embodiment, The flowchart which shows the imaging | photography switching process in 4th Embodiment, The figure which shows the example of the encoding light emission pattern using the some auxiliary light source in 5th Embodiment, The figure which shows the relationship between the light of the auxiliary light source in 6th Embodiment, and the light of ambient light, A flowchart showing blur correction processing by encoded flash photography in the sixth embodiment; The flowchart which shows the ambient light information acquisition process in 6th Embodiment, The flowchart which shows the preparation process of the correction function in 6th Embodiment, and the figure which shows the production | generation method of the light emission pattern in consideration of ambient light, The flowchart which shows the ambient light information acquisition process in the modification of 6th Embodiment, The figure which shows the production | generation method of the light emission pattern in consideration of the time change of ambient light, A flowchart showing blur correction processing by encoded flash photography in the seventh embodiment; The flowchart which shows the imaging | photography process of the image for area determination in 7th Embodiment, and the figure which shows the example of a timing of the image imaging for area | region determination, and encoding light emission imaging | photography It is a flowchart which shows the blurring correction process at the time of combining 6th Embodiment and 7th Embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. The following embodiments do not limit the present invention related to the scope of claims, and all combinations of features described in the present embodiments are essential to the solution means of the present invention. Not necessarily.

Device Configuration First, the configuration of a digital camera (hereinafter simply referred to as a camera) common to each embodiment of the present invention will be described.

  In FIG. 1, (a) is a perspective view showing an appearance of a camera according to the present invention, and (b) is a sectional view thereof.

  In FIG. 1 (a), an eyepiece window 111 for finder observation, an AE (automatic exposure) lock button 112, an AF range-finding point selection button 113, and a release button 114 for performing a shooting operation are provided on the upper part of the camera body 100. Is provided. An electronic dial 411, a shooting mode selection dial 117, an external display device (OLC) 409, and a built-in flash 118 as an auxiliary light source are also provided. The electronic dial 411 is a multi-function signal input device that is used in combination with other operation buttons to input numerical values to the camera and to switch shooting modes. An external display device (OLC) 409 includes a liquid crystal display device, and displays shooting conditions such as shutter speed, aperture, and shooting mode, and other information.

  Also, on the back of the camera body 100, there are an LCD monitor 417 for displaying captured images and various setting screens, a monitor switch 121 for turning on / off the LCD monitor 417, a cross switch 116, and a menu button 124. Is provided. The cross placement switch 116 has four buttons arranged vertically and horizontally, and a SET button arranged in the center. The user instructs the camera to select and execute a menu item displayed on the LCD monitor 417. Used for. The menu button 124 is a button for displaying a menu screen for performing various camera settings on the LCD monitor 417. For example, when selecting and setting the shooting mode, after pressing the menu button 124, the user operates the up / down / left / right buttons of the cross switch 116 to select the desired mode, and the desired mode is selected. Setting is completed by pressing the SET button. The menu button 124 and the cross placement switch 116 are also used for setting an AF mode, which will be described later.

  Here, since the LCD monitor 417 is a transmissive liquid crystal, an image cannot be visually recognized only by driving the LCD monitor, and a backlight illumination 416 is necessarily provided on the back surface thereof as shown in FIG. is there. Thus, the LCD monitor 417 and the backlight illumination 416 constitute an image display device.

  As shown in FIG. 1B, the photographing lens 200 of the imaging optical system can be attached to and detached from the camera body 100 via a lens mount 202. In FIG. 1B, 201 is a photographing optical axis, and 203 is a quick return mirror. The quick return mirror 203 is arranged in the photographing optical path, and a position for guiding subject light from the photographing lens 200 to the finder optical system (a position shown in FIG. 1B, an oblique position) and a position for retreating from the photographing optical path. (Referred to as a retracted position).

  In FIG. 1B, the subject light guided from the quick return mirror 203 to the finder optical system is imaged on the focus plate 204. Reference numeral 205 denotes a condenser lens for improving the visibility of the finder, and 206 denotes a pentagonal roof prism, which guides the subject light that has passed through the focus plate 204 and the condenser lens 205 to an eyepiece lens 208 and a photometric sensor 207 for finder observation.

  The shutter includes a rear curtain 209 and a front curtain 210. By opening the rear curtain 209 and the front curtain 210, the image sensor 418, which is a solid-state image sensor disposed behind, is exposed for a necessary time. Although details will be described later, the data obtained from the image sensor 418 is processed by an A / D converter, an image processing circuit, or the like, and then used in the shake correction circuit for the light emission conditions and motion information of the auxiliary light source (built-in flash 118). A shake correction process is performed and output to the external storage device 419.

  The image sensor 418 is held on a printed circuit board 211, and a display board 215, which is another printed circuit board, is disposed behind the printed circuit board 211. An LCD monitor 417 and a backlight illumination 416 are disposed on the opposite surface of the display substrate 215.

  Reference numeral 419 denotes an external storage device for storing image data, 217 denotes a battery (portable power supply), and these are detachable from the camera body 100.

  FIG. 2 is a block diagram showing a circuit configuration of the camera according to the present invention. In FIG. 2, a microcomputer 402 as a CPU controls the operation of the entire camera, including processing of image data output from an image sensor (CCD in the embodiment) 418 and display control of an LCD monitor 417.

  The switch (SW1) 405 is turned on when the release button 114 (see FIG. 1A) is half-pressed, whereby the digital camera enters a shooting preparation state. The switch (SW2) 406 is turned on when the release button 114 is pressed to the end (fully pressed state), whereby the digital camera starts a shooting operation.

  The lens control circuit 407 performs communication with the photographic lens 200 (see FIG. 1B) and drive control of the photographic lens 200 and drive control of the diaphragm blades during AF (autofocus).

  An external display control circuit 408 controls an external display device (OLC) 409 and a display device (not shown) in the finder. The switch sense circuit 410 transmits signals from a number of switches including an electronic dial 411 provided in the camera to the microcomputer 402.

  The flash light emission dimming control circuit 412 controls the built-in flash 118 as an auxiliary light source in the present invention and the external flash grounded through the X contact 412a.

  A distance measuring (AF) circuit 413 detects a defocus amount for a subject for AF. A photometry (AE) circuit 414 measures the luminance of the subject. A shutter control circuit 415 controls the shutter and performs appropriate exposure on the image sensor. The LCD monitor 417 is paired with the backlight illumination 416 to constitute the image display device as described above. The external storage device 419 is, for example, a hard disk drive or a semiconductor memory card that can be attached to and detached from the camera body 100.

  The microcomputer 402 is connected to an A / D converter 423, a buffer memory 424, and an image processing circuit 425 including a DSP, and processes data obtained by the image sensor 418.

  In addition to the motion information detection circuit 426 and the light emission condition generation circuit 427, a shake correction circuit 428 and a correction function generation circuit 429 are connected to the microcomputer 402.

  Further, although details will be described later, it is possible to connect the photographing environment light amount determination circuit 430, the camera shake determination circuit 431, and the region determination circuit 432 according to the embodiment.

<First Embodiment>
Hereinafter, a first embodiment of the camera of the present invention having the above-described configuration will be described in detail. The present embodiment is characterized in that the shake correction circuit 428 performs camera shake correction by photographing with coded light emission (hereinafter referred to as “Coded Flash”). This camera shake correction technique using coded flash photography is an application of camera shake correction technique using coded aperture photography, and is simply the following technique. That is, during the exposure time when acquiring one image, shooting is performed with a predetermined light emission pattern for turning on / off the light emission of the auxiliary light source (that is, the light emission on / off encoded). Then, blur correction is performed by performing a deconvolution operation using a point spread function based on the light emission pattern information of the auxiliary light source on the captured image obtained by the coded light emission imaging. Note that the auxiliary light source in this embodiment is the built-in flash 118, but is simply referred to as “auxiliary light source” hereinafter.

  In the following, an outline of blur correction processing by coded flash photography will be described by taking as an example the case of correcting blur in the vertical direction of an image, and then the details will be described.

● Outline of shake correction processing by coded flash photography First, an outline of shake correction processing by coded flash photography will be described. Here, the intensity of incident light per unit time at the pixel position (x, y) is i (x, y), the vertical blurring speed of the imaging device (camera) is v, and the emission time of the auxiliary light source is T. . Then, information obtained by moving i (x, y) by vT at time t is incident on the camera by one photographing, that is, exposure. Therefore, the image data i_blur (x, y) to be captured can be expressed by the following equation (1).

  In equation (1), h (t) is a function representing the light emission pattern of the auxiliary light source. h (t) takes a value of 1 or 0. If 1, the auxiliary light source emits light. If 0, the auxiliary light source does not emit light. Here, in order to simplify the description, the following description will be given by taking as an example a case where blurring in the vertical direction (y direction) occurs during shooting. Here, the convolution operation in the real space can be described as a product format on the spatial frequency. Therefore, the following equation (2) is obtained by Fourier transforming both sides of the equation (1) representing the acquisition process of the imaging data.

  In Expression (2), I_blur (u, v), I (u, v), and H (u) are Fourier transforms of i_blur (x, y), i (x, y), and h (t), respectively. .

  Next, in Expression (2), when the intensity of incident light to be obtained is mathematically converted with respect to Fourier transform I (u, v) of i (x, y), the following Expression (3) is obtained.

  In the equation (3), it is assumed that the light emission time T is a known parameter, and the camera shake speed v can be obtained from a gyro sensor provided in the motion information detection circuit 426. Also, I_blur (u, v) and H (u) can be obtained by Fourier transform of the image data i_blur (x, y) to be captured and the light emission pattern h (t) of the auxiliary light source. The function is also known. Therefore, the Fourier transform I (u, v) of the image data i (x, y) to be acquired can be derived by performing the calculation of the equation (3). Finally, by performing inverse Fourier transform on the derived I (u, v), it is possible to obtain input data i (x, y) that does not include a shake by calculation.

  As described above, in principle, it is possible to remove the blur from the image including the blur by the deconvolution calculation according to the equation (3). However, depending on the light emission pattern of the auxiliary light source, the Fourier transform value H (u) of h (t) may hold zero. In that case, a so-called “zero division” in which the denominator is zero occurs on the right side of Equation (3), and I (u, v), which is the solution of Equation (3), cannot be obtained correctly, and blurring in the captured image occurs. Correction is insufficient. Hereinafter, a case where blur correction is insufficient due to “zero percent” will be described with an example.

  FIG. 8A illustrates an example of a constant light emission pattern of a normal auxiliary light source. H (t) = 1 represents a state in which the auxiliary light source emits light, and h (t) = 0 represents an auxiliary light source. This represents a state where no light is emitted. The constant light emission pattern of the auxiliary light source shown in FIG. 8A is expressed by a mathematical formula as follows.

h (t) = 1 (0 ≦ t ≦ T)
h (t) = 0 (other than above)
FIG. 8B is a graph in which the constant light emission pattern h (t) of the auxiliary light source shown in FIG. 8A is Fourier transformed and the intensity is plotted for each frequency. This is a point spread function indicating the blur on the image used in the deconvolution described above. According to FIG. 8 (b), it can be confirmed that the frequency at which the intensity of the PSF becomes 0 appears periodically, and that information corresponding to the frequency has been lost at the time of image acquisition. Yes. It is known that when such a situation occurs, an unnecessary waveform pattern (ringing) appears in a corrected image after deconvolution, and image quality deterioration is induced.

  In this embodiment, in order to prevent such image quality degradation, the emission on / off of the auxiliary light source is patterned, that is, encoded within one exposure time, so that the PSF of the light emission pattern h (t) of the auxiliary light source is obtained. Set so that the frequency characteristics are composed of non-zero values. FIG. 9A shows an example in which the light emission pattern of the auxiliary light source is encoded. According to the figure, the encoded light emission pattern h (t) is a predetermined pattern with a value of 0 indicating light source off or 1 indicating light source on within the light emission time T of the auxiliary light source. FIG. 9B is a PSF in which the light emission pattern h (t) of the auxiliary light source shown in FIG. 9A is Fourier-transformed and the intensity is plotted for each frequency. According to FIG. 9B, it can be seen that the PSF is composed of non-zero values at all frequencies. In this case, since the PSF can obtain image information in all frequency bands, it is possible to perform suitable image correction using Expression (3).

  In the present embodiment, as described above, the blur correction process is performed on the image captured by encoding the light emission pattern of the auxiliary light source based on the light emission pattern of the auxiliary light source and the blur information. In the above example, the example of performing the blur correction process in the vertical direction (vertical direction) has been described. However, the application range of the present invention is not limited to the shift blur correction in the vertical or horizontal direction. For example, the blur correction process in the rotation direction can be performed by converting the blur correction process in the vertical direction into polar coordinates.

Details of Blur Correction by Encoded Flash Shooting Hereinafter, blur correction processing by coded flash shooting in the present embodiment will be described in detail with reference to the flowchart of FIG.

  First, when the photographer presses the release button 114 (see FIG. 1A) halfway, the light emission condition generation circuit 427 shown in FIG. 2 emits light from the auxiliary light source so that the image after blur correction has an appropriate brightness. Conditions are determined (S301). Here, the light emission conditions are an encoded light emission pattern of the auxiliary light source, a light emission time of the auxiliary light source, and a light emission intensity of the auxiliary light source. Details of the light emission condition determination processing will be described later.

  Next, when the photographer fully presses the release button 114, the flash emission dimming control circuit 412 shown in FIG. 2 performs emission control of the auxiliary light source, and imaging is performed under the emission conditions determined in S301 (S302). The captured image obtained here is stored in the buffer memory 424.

  Next, the correction function generation circuit 429 generates a correction function based on the encoded light emission condition of the auxiliary light source determined in S301 (S303). Details of this correction function generation processing will be described later.

  Next, blur information detected by the motion information detection circuit 426 at the time of shooting is acquired (S304). Note that the shake information detected in the shake information acquisition process is the shake speed v at the time of shooting.

  Finally, in the shake correction circuit 428, the image captured in S302 is corrected using the correction function created in S303 and the shake information acquired in S304 (S305). Details of the image correction processing will be described later.

  Hereinafter, the processing in each of S301, S303, and S305 will be described in detail.

● Auxiliary light source emission condition determination processing (S301)
As described above, it is possible to perform blur correction by encoding an emission pattern of the auxiliary light source, capturing an image, and performing a deconvolution operation on the obtained captured image using the emission pattern information. However, for example, as shown in FIG. 9A, when shooting with the coded light emission pattern of the auxiliary light source (encoded flash shooting), as compared with shooting with the auxiliary light source emitting constant light as shown in FIG. As a result, the amount of light as a whole decreases. Therefore, when performing coded flash photography, unless the light emission conditions of the auxiliary light source at the time of photography (encoded light pattern, light emission time, light emission intensity, etc.) are appropriately determined, the image after blur correction has an appropriate brightness. Must not. Therefore, in the present embodiment, an appropriate light emission condition of the auxiliary light source at the time of shooting is determined from the shooting conditions (illuminance of the shooting environment, luminance of the subject, distance between the camera and the subject, aperture value, ISO sensitivity, shutter speed, etc.). Hereinafter, the light emission condition determination process in S301 will be described with reference to the flowchart of FIG.

  First, the light quantity Q [cd · sec] required for photographing is calculated based on the photographing conditions (S1001). For the light quantity Q, the following formulas (4), (5), and (6) are used from the aperture value F specified by the photographer, the ISO sensitivity S, and the distance D to the subject obtained by the distance measuring (AF) circuit 413. To calculate. In equations (4) and (5), Gno is a guide number for ISO sensitivity S.

Gno = F × D × √ (100 / S) (4)
Gno = 0.3 × √ (4π × 0.0045 × S × Q) (5)
From the equations (4) and (5), Q = {(F ² × D ² ) / S ² } × {10 ⁷ / (162π)} (6)
Note that the light quantity Q necessary for photographing can also be calculated from the illuminance of the photographing environment and the luminance of the subject (TTL photometric value).

  Next, based on the obtained light quantity Q, the light emission time T and the light emission pattern h (t) of the auxiliary light source suitable for photographing are determined (S1002). The light emission time T and the light emission pattern h (t) are determined with reference to an encoded light emission pattern table created in advance by a method described later. FIG. 11 shows an example of the encoded light emission pattern table. As shown in the figure, the encoded light emission pattern table has information on the encoded light emission pattern h (t) and the maximum light quantity Qmax for each light emission time T. In the present embodiment, the table is sequentially referred to from the shorter light emission time T, and the light emission time T and the encoding thereof are such that the maximum light amount Qmax is equal to or greater than the light amount Q required for photographing obtained in S1001 (Q ≦ Qmax). A light emission pattern h (t) is employed. Here, Qmax is the maximum light amount that can be output by the encoded light emission pattern in the light emission time T. For example, when QmaxT2 <Q ≦ QmaxT3, the light emission time T of the auxiliary light source is determined as T = T3. If the amount of light Q required for shooting is larger than any Qmax (Q> Qmax), a corresponding error message is output indicating that shooting is impossible, and the process is terminated.

  Next, the following formula (7) is used by using the maximum light quantity Qmax larger than the light quantity Q necessary for photographing, which is determined in S1002 as the luminous intensity I [cd] of the auxiliary light source suitable for photographing. (S1003). In Expression (7), Imax is the maximum luminous intensity that the auxiliary light source can output.

I = Imax × Q / Qmax (7)
For example, when QmaxT2 <Q ≦ QmaxT3, the luminous intensity I of the auxiliary light source is determined as I = Imax × Q / QmaxT3 from Equation (7). As a result, as shown in FIG. 12, the light intensity I of the auxiliary light source is determined to be an optimum value for photographing that is smaller than the maximum light intensity Imax.

  In the present embodiment, as described above, the encoded light emission pattern h (t) of the auxiliary light source, the light emission time T of the auxiliary light source, and the luminous intensity I of the auxiliary light source are determined as the light emission conditions of the auxiliary light source at the time of shooting.

Encoded Light Emitting Pattern Table Creation Processing Here, the encoded light emitting pattern table creating processing for each light emission time T shown in FIG. 11 will be described. First, an encoded light emission pattern h (t) as shown in FIG. 9A is generated every light emission time T of the auxiliary light source. When the encoded light emission pattern h (t) is Fourier transformed, a PSF is formed with non-zero values at all frequencies as shown in FIG. 9B.

  Next, the maximum light quantity Qmax in each encoded light emission pattern h (t) for each generated light emission time T is calculated. Here, the maximum light quantity Qmax represents the maximum light quantity that the encoded light emission pattern h (t) can output in the light emission time T. Assuming that the maximum luminous intensity that can be output by the auxiliary light source is Imax [cd], the maximum light quantity Qmax obtained in the encoded light emission pattern h (t) of the light emission time T is the sum of light emission on in the pattern, that is, Calculated by equations (8) and (9).

Qmax = Imax × Qcoded (9)
As described above, when the encoded light emission pattern h (t) and the maximum light quantity Qmax are calculated for each light emission time T of the auxiliary light source, a plurality of sets are stored as a table as shown in FIG. To do. In the table shown in FIG. 11, the light emission times T are arranged in ascending order (T1 <T2 <T3...), And the maximum light quantity Qmax is also correspondingly reduced (QmaxT1 <QmaxT2 <QmaxT3...). line up.

  In the present embodiment, a table of coded light emission patterns for each light emission time T is stored in advance, and the light emission time T is determined by referring to the table. However, Q ≦ Qmax at the time of shooting. It is also possible to generate and use an encoded light emission pattern of the light emission time T.

Correction function generation processing (S303)
Hereinafter, the correction function generation processing in S303 will be described with reference to the flowchart of FIG. First, the encoded light emission pattern h (t) of the auxiliary light source determined in S301 is input (S401). Here, t represents time. The light emission pattern h (t) of the auxiliary light source is a function that takes a value of 1 or 0. If it is 1, the auxiliary light source is emitting light, and if it is 0, the auxiliary light source is not emitting light. Hereinafter, an encoded light emission pattern of the auxiliary light source as described below, in which the light emission time T of the auxiliary light source is T = 15, will be described as an example.

h (t) = [1,0,0,0,1,0,0,1,1,0,1,0,1,1,1]
When the encoded light emission pattern h (t) of the auxiliary light source is input, the encoded light emission pattern h composed of a two-dimensional array is referred to with reference to the size of the captured image defined in advance (N × M pixels in this embodiment). Zero setting for '(x, y) is performed (S402). Specifically, the array h ′ (x, y) at the pixel position (x, y) is created as follows using the one-dimensional encoded light emission pattern h (t).

h ′ (0, y) = h (y) (0 ≦ y ≦ T)
h '(x, y) = 0 (other than above)
Note that the pixel position (x, y) changes in the range of 0 ≦ x <N and 0 ≦ y <M, and in this embodiment, the size of the captured image is set to N × M = 256 × 256 pixels. Here, FIG. 5 schematically shows a two-dimensional array h ′ (x, y) described by the above equation. In FIG. 5, h (t), that is, 0 or 1, is stored in the vertical direction from the origin of the upper left end point, that is, 0 ≦ y ≦ T, and 0 is uniformly stored in the other arrays. In the present embodiment, correction processing for an image including blur in the vertical direction (y direction) is described. Therefore, as shown in FIG. 5, the vertical direction of the two-dimensional array h ′ (x, y) is set. A value other than zero can be set for only a part of the direction (y direction). Similarly, when correction processing is performed on an image including a blur in the horizontal direction (x direction), zero is applied only to a part of the two-dimensional array h ′ (x, y) in the horizontal direction (x direction). Any value other than can be set.

  Next, the two-dimensional encoded light emission pattern h ′ (x, y) set to zero is subjected to a two-dimensional Fourier transform as shown in the following equation (10) (S403).

H (u, v) = FFT2 (h ′ (x, y)) (10)
In equation (10), FFT2 () is a function that performs a two-dimensional Fourier transform. U and v represent frequencies and are normalized to −1 ≦ u ≦ 1 and −1 ≦ v ≦ 1. Here, in FIG. 6, in the two-dimensional encoded light emission pattern h ′ (x, y), h ′ (0, y), which is a place where the one-dimensional encoded light emission pattern h (t) is stored. On the other hand, a result H ′ (u) obtained by performing the one-dimensional Fourier transform is shown. In FIG. 6, the horizontal axis represents the positive frequency u (0 ≦ u ≦ 1). The vertical axis is a logarithmic representation of the response value of each frequency, and each response value is normalized with the value of the DC component (u = 0).

  Finally, a correction function is used to calculate the two-dimensional Fourier transform result H (u, v) of the encoded light emission pattern h ′ (x, y) of the auxiliary light source obtained by the two-dimensional Fourier transform and the light emission time T of the auxiliary light source. (S404). Thus, the correction function generation process is completed.

Image correction processing (S305)
Hereinafter, the image correction processing in S305 will be described with reference to the flowchart of FIG. First, the captured image i_blur (x, y) captured in S302 and stored in the buffer memory 424 is input to the shake correction circuit 428 (S701). Here, it is assumed that the size of the captured image handled in the present embodiment is N × M pixels as described above. Next, camera shake information (camera shake speed v) obtained by the motion information detection circuit 426 is input to the camera shake correction circuit 428 (S702). Subsequently, the correction function created by the correction function generation circuit 429 in S303 is input to the shake correction circuit 428 (S703).

  Next, the blur correction circuit 428 performs a two-dimensional Fourier transform process on the photographed image i_blur (x, y) as shown in the following equation (11) (S704).

I_blur (u, v) = FFT2 (i_blur (x, y)) (11)
Next, deconvolution is performed using the Fourier transform image I_blur (u, v), the camera shake speed v, the two-dimensional Fourier transform H (u, v) of the encoded light emission pattern of the auxiliary light source, and the light emission time T of the auxiliary light source. Processing is performed (S705). Specifically, deconvolution is performed according to the following equation (12).

  Here, since I (u, v) obtained by the equation (12) corresponds to the two-dimensional Fourier transform result of the image i (x, y) that does not include blur, I (u, v) is expressed by the following equation ( It is possible to define as 13).

I (u, v) = FFT2 (i (x, y)) (13)
Therefore, a two-dimensional inverse Fourier transform process shown in the following expression (14) is performed on I (u, v) in the expression (13) (S706). In Expression (14), IFFT2 () is a function for performing a two-dimensional inverse Fourier transform.

i (x, y) = IFFT2 (I (u, v)) (14)
Finally, the image i (x, y) for which the inverse Fourier transform has been completed is output as a corrected image (S707). Thereby, the image correction process is completed.

  In the present embodiment, an example of performing deconvolution by division in the frequency space as shown in Expression (12) has been described, but other deconvolution processing may be performed. For example, a Lucy-Richardson algorithm, an algorithm using a Wiener filter, an algorithm using a regularization filter, or the like may be applied.

  As described above, according to the present embodiment, the light emission condition of the auxiliary light source is appropriately determined, and the light emission condition of the auxiliary light source and the camera shake information are obtained for a captured image obtained by photographing under the light emission condition. Perform shake correction based on. As a result, it is possible to obtain a captured image with appropriate brightness in which camera shake is corrected.

  In the present embodiment, the example using the encoded light emission pattern h (t) represented by [0, 1] as shown in FIG. 9A is shown. However, as shown in FIG. The same effect can be obtained even by using the encoded light emission pattern h (t) in which is continuously increased or decreased. For the encoded light emission pattern h (t) shown in FIG. 13, the Fourier transform result (FIG. 13 (b)) is the same as FIG. 9 (b), and a PSF with non-zero values is formed at all frequencies. In this case, as shown in FIG. 13B, the auxiliary light source is caused to emit light at the light emission period T and the luminous intensity I × h (t) to capture an image. Such photographing using an encoded light emission pattern in which the light intensity continuously changes is effective when a light source that easily controls the light intensity, such as a white LED, is used as an auxiliary light source.

Second Embodiment
Hereinafter, a second embodiment according to the present invention will be described. In the first embodiment described above, image stabilization is performed by appropriately determining the light emission condition of the auxiliary light source, and the obtained captured image is subjected to image blur correction using the light emission condition of the auxiliary light source and the image blur information. An example is shown in which the later image has the optimum brightness. However, when the shooting environment is sufficiently bright, a sufficient correction effect cannot be obtained even if the camera shake correction by the encoded light emission shown in the first embodiment is performed. Therefore, the second embodiment is characterized in that the brightness of the shooting environment is determined, and camera shake correction processing by coded aperture shooting and camera shake correction processing by coded flash shooting are switched. Since the configuration of the digital camera in the second embodiment is the same as that of the first embodiment described above, the description thereof will be omitted by referring to the same reference numerals.

  The camera shake correction switching process according to the second embodiment will be described with reference to the flowchart of FIG. First, the amount of light in the shooting environment is measured using the photometric sensor 207 (S1401). Next, the photographing environment light amount determination circuit 430 determines whether or not the light amount of the photographing environment measured in S1401 is a sufficient amount of light for photographing (S1402). This determination is made based on the aperture value, shutter speed, ISO sensitivity, etc. set by the camera. That is, similarly to S301 in the first embodiment described above, the light quantity Q required for photographing is calculated based on the above parameters and compared with the light quantity measured in S1401.

  If it is determined in S1402 that the amount of light is sufficient for shooting, the process proceeds to a camera shake correction process by coded aperture shooting in S1403. Here, the camera shake correction process by the coded aperture photographing is a correction process by Coded Exposure as described in Patent Document 2 described above. Specifically, first, an opening / closing pattern that encodes the opening / closing of the shutter at the time of exposure when acquiring one image is created, and an image is captured by controlling the opening of the shutter according to the opening / closing pattern. Then, camera shake is corrected by deconvolution calculation using the open / close pattern for the captured image. In the camera shake correction process (aperture control correction process) by the coded aperture photographing, light emission by the auxiliary light source is not performed.

  On the other hand, if it is determined in S1402 that the amount of light is not sufficient for photographing, the process proceeds to the camera shake correction processing by the encoded light emission photographing of the auxiliary light source in S1404. The camera shake correction process by the coded flash photography is the camera shake correction process described in the first embodiment.

  As described above, according to the second embodiment, the camera shake correction processing by the coded aperture photographing and the camera shake correction processing by the coded light emission photographing of the auxiliary light source are switched based on the brightness of the photographing environment. As a result, it is possible to acquire an image in which camera shake is appropriately corrected regardless of whether the shooting environment is bright or dark.

<Third Embodiment>
The third embodiment according to the present invention will be described below. The third embodiment is characterized in that a camera shake correction mode by coded aperture photography and a camera shake compensation mode by coded light emission photography of an auxiliary light source are switched based on a mode selection result by a user. Since the configuration of the digital camera in the third embodiment is the same as that of the first embodiment described above, the description thereof is omitted by referring to the same reference numerals.

  A camera shake correction switching process according to the third embodiment will be described with reference to the flowchart of FIG. First, the user selects either the camera shake correction mode by the coded aperture shooting or the camera shake correction mode by the coded light emission shooting of the auxiliary light source by using the shooting mode selection dial 117 or the electronic dial 411 (S1501).

  Then, it is determined whether or not the camera shake correction mode (aperture control correction) by coded aperture shooting has been selected (S1502). When the camera shake correction mode by the coded aperture photographing is selected, correction processing without light emission of the auxiliary light source by Coded Exposure as described in Patent Document 2 described above is performed (S1503). On the other hand, when the camera shake correction mode by the encoded flash photography is selected, the correction process by the flash photography of the auxiliary light source described in the first embodiment is performed (S1504).

  As described above, according to the third embodiment, the camera shake correction mode by the coded aperture photographing and the camera shake correction mode by the coded light emission photographing of the auxiliary light source are switched based on the mode selection result by the user. Thereby, the user can arbitrarily switch the camera shake correction mode.

<Fourth embodiment>
The fourth embodiment according to the present invention will be described below. The fourth embodiment is characterized by switching between flash photography without camera shake correction and camera shake correction processing by coded light emission photography of an auxiliary light source in accordance with camera settings at the time of photography. Since the configuration of the digital camera according to the fourth embodiment is the same as that of the first embodiment described above, the description thereof will be omitted by referring to the same reference numerals.

  The shooting switching process in the fourth embodiment will be described with reference to the flowchart of FIG. First, the camera shake determination circuit 431 determines whether camera shake is unlikely to occur based on the camera setting at the time of shooting (S1601). If it is determined that camera shake is unlikely to occur, the flow proceeds to S1602, and flash photography without camera shake correction is performed. On the other hand, if it is determined that camera shake is likely to occur, the process advances to step S1603 to perform the camera shake correction process by the coded light emission photographing of the auxiliary light source described in the first embodiment.

  As a determination method in S1601, for example, there is a method of determining that camera shake is unlikely to occur if the set shutter speed is faster than “one focal length” and that camera shake is likely to occur if the shutter speed is slow. It is also possible to detect whether camera shake is difficult to occur by detecting whether the camera is installed on a tripod.

  As described above, according to the fourth embodiment, the flash shooting without camera shake correction and the camera shake correction processing by the coded light emission shooting of the auxiliary light source are switched according to the camera setting at the time of shooting. As a result, when the camera is set so that camera shake is unlikely to occur, loss of the light emission amount per unit time due to encoded light emission can be prevented by performing flash photography without camera shake correction. In addition, when a camera is set so that camera shake is likely to occur, an image with camera shake corrected can be acquired.

  The second to fourth embodiments described above can also be realized by combining them. For example, according to camera settings and user instructions, it is possible to perform control such as appropriately switching between camera shake correction by coded flash photography, camera shake correction by coded aperture photography, and normal flash photography without camera shake compensation. is there.

<Fifth Embodiment>
The fifth embodiment according to the present invention will be described below. In the first embodiment described above, an example in which one auxiliary light source (built-in flash 118) is used has been described. However, in the fifth embodiment, a coded light emission pattern is generated using a plurality of auxiliary light sources. And

  As the encoded light emission pattern in the fifth embodiment, independently controlled n light sources having equivalent functions are caused to emit light by shifting by τ / n. Here, τ is the minimum pulse width [sec] of the light source itself. FIG. 17 shows an example of the encoded light emission pattern in the fifth embodiment. In the example shown in the figure, the three light sources 1 to 3 that are independently controlled are caused to emit light while being shifted by τ / 3. A light emission pattern as shown in the uppermost part of the figure is obtained as a whole by the light emission of each of the three light sources.

  As described above, according to the fifth embodiment, the minimum pulse width of the encoded light emission pattern is reduced by causing the n light sources, which are independently controlled to have the same function, to emit light by shifting by τ / n. It becomes possible to do. In addition, even when using one light source that cannot change the light intensity, the use of n light sources makes it possible to control the encoded light emission pattern with n + 1 gradations, and the maximum light quantity that can be output is n. Doubled. Therefore, more flexible correction control is possible.

<Sixth Embodiment>
The sixth embodiment according to the present invention will be described below. In the first embodiment described above, the example in which the shake correction for the captured image is performed using the light emission condition of the auxiliary light source and the shake information. However, when ambient light is present at the time of coded flash photography as shown in FIG. 18 (a) (for example, photography in the dim evening), the coded light emission described in the first embodiment is used. When camera shake correction is performed, image degradation may occur. This is because the light actually received by the subject at the time of photographing is a combination of the light from the auxiliary light source and the ambient light. For example, when the auxiliary light source emits light with the coded light emission pattern h (t) shown in FIG. 9A in the first embodiment described above, the light actually received by the subject is as shown in FIG. 18B. In addition, the light of the auxiliary light source and the ambient light are combined. In this way, when ambient light is present at the time of coded flash photography, a difference occurs between the light from the auxiliary light source and the light actually received by the subject. Therefore, the shake correction shown in the first embodiment is expected. Such a correction effect cannot be obtained.

  Therefore, in the sixth embodiment, as in the first embodiment, the light emission condition of the auxiliary light source is determined and photographed. At the same time, the ambient light information is acquired, and the shake correction for the obtained captured image is performed, and the light emission condition of the auxiliary light source is obtained. In addition to the shake information, the ambient light information is used. Since the configuration of the digital camera in the sixth embodiment is the same as that of the first embodiment described above, the description thereof will be omitted by referring to the same reference numerals.

  Hereinafter, blur correction processing by encoded flash photography according to the sixth embodiment will be described with reference to the flowchart of FIG. In the flowchart of FIG. 19, the processes other than S1901 and S1902 are the same as those in FIG.

  First, when the light emission condition of the auxiliary light source is determined in S301, ambient light information is acquired in S1901. Here, this ambient light information acquisition processing will be described with reference to the flowchart of FIG. First, the first subject luminance Br1 is acquired using the photometric sensor 207 without causing the auxiliary light source to emit light (S2001). Next, the flash light emission dimming control circuit 412 performs light emission control of the auxiliary light source, preliminarily emits the auxiliary light source, and obtains the second subject luminance Br2 using the photometric sensor 207 (S2002). Here, as the auxiliary light emission of the auxiliary light source, light is emitted at the luminous intensity I determined in S301.

  Returning to FIG. 19, when coded flash photography is performed in S302, a correction function is generated in S1902 based on the light emission conditions determined in S301 and the ambient light information (Br1, Br2) acquired in S1901. .

  Here, the correction function generation processing in the sixth embodiment will be described with reference to the flowchart of FIG. In the flowchart of FIG. 21A, processes other than S2101 are the same as those in FIG. 4 of the first embodiment described above.

  First, in S401, the encoded light emission pattern h (t) of the auxiliary light source is input. Here, as in the first embodiment, the auxiliary light source is configured such that the light emission time T of the auxiliary light source is T = 15 as follows. The encoded light emission pattern of the light source will be described as an example.

h (t) = [1,0,0,0,1,0,0,1,1,0,1,0,1,1,1]
Next, a light emission pattern considering ambient light is generated (S2101). Specifically, by using the ambient light information Br1 and Br2 acquired in S1901, the encoded light emission pattern h (t) is corrected by the following equation, and a new encoded light emission pattern h ′ ( Create t).

h '(t) = Br1 / Br2 + (Br2-Br1) × h (t) / Br2 (15)
FIG. 21 (b) schematically shows a shake correction coded light emission pattern h ′ (t) created based on the above equation (15). According to FIG. 21B, the encoded light emission pattern h ′ (t) for blur correction is a fixed value corresponding to the ambient light information Br1 with respect to the 0 value (light emission off) in the encoded light emission pattern h (t). It can be seen that the pattern reflects this.

  When the shake correction coded light emission pattern h ′ (t) is created as described above, thereafter, similarly to the first embodiment, based on the shake correction coded light emission pattern h ′ (t), S402 is performed. Processes of zero setting, Fourier transform, and correction function output in S404 are performed. As a result, a correction function based on the encoded light emission pattern h ′ (t) for shake correction is created, and in S305, shake correction by the correction function, that is, shake correction considering ambient light at the time of shooting is performed on the captured image. Is done.

  As described above, according to the sixth embodiment, the light emission condition of the auxiliary light source is corrected based on the ambient light information at the time of shooting, and a blur correction function is generated according to the corrected light emission condition. Even when there is ambient light during automatic flash photography, an image with appropriate camera shake correction can be obtained.

  In the sixth embodiment, for the sake of simplicity, an example in which the light intensity during preliminary light emission is set to the same light intensity I as that during coded light emission photographing is shown. However, a different light intensity I ′ may be applied. Is possible. In this case, the encoded light emission pattern h (t) may be further modified in S2101 using the ratio of the light intensity I ′ during the preliminary light emission and the light intensity I during the encoded light emission photographing.

Modified Example In the sixth embodiment, the case where the illuminance of the ambient light is constant has been described as an example. However, the illuminance varies under a general fluorescent lamp using a commercial power source, for example. In order to cope with such fluctuations in illuminance, the following method can be considered as a modification of the sixth embodiment.

  That is, in this case, the above-described ambient light information acquisition process (S1901) and the light emission pattern generation process (S2101) taking ambient light into account are changed as follows.

  First, a modification of the ambient light information acquisition process in S1901 will be described with reference to the flowchart of FIG. First, the first time change Br1 (t) of the subject luminance is acquired using the photometric sensor 207 without causing the auxiliary light source to emit light (S2201). Next, the flash light emission dimming control circuit 412 performs light emission control of the auxiliary light source, causes the auxiliary light source to continuously perform preliminary light emission, and obtains the second subject luminance temporal change Br2 (t) using the photometric sensor 207 ( S2202).

  Next, a modification of the light emission pattern generation process in consideration of ambient light in S2101 will be described. First, the respective maximum values Br1Max and Br2Max are calculated from Br1 (t) and Br2 (t) acquired by the processing shown in FIG. 22 (a), as shown in FIG. 22 (b). Then, by using the Br1 (t), Br1Max, and Br2Max obtained as described above, the encoded light emission pattern h (t) is corrected by the following equation to obtain a new encoded light emission pattern used for blur correction. Create h ′ (t). Note that the Br1 (t) phase information necessary here may be acquired at the time of coded flash photography.

h ′ (t) = Br1 (t) / Br2Max + (Br2Max−Br1Max) × h (t) / Br2Max (16)
FIG. 22 (c) schematically shows a shake correction coded light emission pattern h ′ (t) created based on the above equation (16). According to FIG. 22 (c), the blurring-corrected encoded light emission pattern h ′ (t) is a pattern in which the temporal change Br1 (t) of ambient light is reflected on the encoded light emission pattern h (t). You can see that

  As described above, by correcting the encoded light emission pattern h (t) in consideration of the temporal change of the ambient light information, it is possible to obtain an image in which camera shake is appropriately corrected even when the illuminance of the ambient light varies. it can.

  In addition, it is also possible to execute the sixth embodiment or its modification in combination with the second to fifth embodiments described above.

<Seventh embodiment>
The seventh embodiment according to the present invention will be described below. In the first embodiment described above, the example in which the shake correction for the captured image is performed based on the correction function created based on the light emission condition of the auxiliary light source and the shake information. However, if there is an “area where the light from the auxiliary light source does not reach” in the captured image, the image is degraded when a uniform correction function is applied to the entire captured image as in the first embodiment described above. May happen. This is because the “region where the light from the auxiliary light source does not reach” in the subject receives only ambient light, and thus the region in the photographed image is not encoded and is based on the encoded light emission condition of the auxiliary light source. This is because the correction function generated in this way is inappropriate.

  Therefore, in the seventh embodiment, as in the first embodiment, shooting is performed by determining the light emission condition of the auxiliary light source, and for the pixels in the area where it is determined that the light emission from the auxiliary light source has arrived in the captured image. Only the blur correction similar to that of the first embodiment is performed. Since the configuration of the digital camera in the seventh embodiment is the same as that of the first embodiment described above, the description thereof will be omitted by referring to the same reference numerals.

  Hereinafter, blur correction processing by encoded flash photography according to the seventh embodiment will be described with reference to the flowchart of FIG. In the flowchart of FIG. 23, the processes other than S2301 and S2302 are the same as those in FIG.

  First, when the light emission condition of the auxiliary light source is determined in S301, it is next determined in S2301 whether or not the light emission from the auxiliary light source reaches each pixel of the captured image by the encoded light emission photographing. A region determination image is captured. Here, the imaging process of the area determination image will be described with reference to the flowchart of FIG. First, the first region determination image Foff is photographed without causing the auxiliary light source to emit light, and the obtained image is stored in the buffer memory 424 (S2401). Next, the flash light emission dimming control circuit 412 performs light emission control of the auxiliary light source, causes the auxiliary light source to emit light, captures the second region determination image Fon, and stores the obtained image in the buffer memory 424 (S2402). ). It is assumed that the auxiliary light source emits light at the luminous intensity I determined in S301.

  Returning to FIG. 23, next, in step S302, coded flash photography is performed. Here, FIG. 24B schematically shows the timing of turning on / off the auxiliary light source (flash) and opening / closing the shutter in the area determination image capturing in S2301 and the encoded light emission capturing in S302. According to FIG. 24B, first the first region determination image Foff without auxiliary light source emission is photographed (S2401), and then the second region determination image Fon with auxiliary light source emission is photographed (S2402). ), It can be seen that coded flash photography has started (S302).

  As described above, when shooting of the region determination image and coded flash shooting are completed, a correction function is generated in S303 and blur information is acquired in S304, as in the first embodiment. Then, in step S2302, an area where light emission from the auxiliary light source reaches in the captured image obtained by the encoded light emission imaging is determined. In other words, the area determination circuit 432 calculates the difference between the pixel values of the first area determination image Foff and the second area determination image Fon, and emits the pixels whose difference exceeds a predetermined threshold by the auxiliary light source. Is determined to be an area that has arrived. In the seventh embodiment, image correction based on the correction function is performed in S305 only for the pixel determined to be the region where the light emission from the auxiliary light source reaches in S2302. That is, image correction is not performed on pixels that have not received light emission from the auxiliary light source.

  As described above, according to the seventh embodiment, it is possible to acquire an image in which camera shake is appropriately corrected even when a “region where the light from the auxiliary light source does not reach” exists in the captured image.

  In the seventh embodiment, an example in which an image for area determination is captured in order to determine an area where light emitted from the auxiliary light source has arrived has been described. However, the area determination method is not limited to this. For example, at the time of coded flash photography, by dividing the field of view into a plurality of areas and acquiring information on the distance to the subject in each area, the emitted light arrives based on the luminous intensity of the auxiliary light source. It is also possible to determine the area.

  The sixth and seventh embodiments described above can also be realized by combining them. The processing in that case is realized, for example, as shown in the flowchart of FIG. That is, when the light emission condition of the auxiliary light source is determined in S301, ambient light information is acquired in S1901, and then an area determination image is captured in S2301. In step S302, the coded flash photography is performed, and in step S1902, a correction function is generated based on the light emission condition and the ambient light information. Then, after obtaining blur information in S304, the area where the light emitted by the auxiliary light source has arrived is determined in S2302, and the image correction in S305, that is, the blur correction based on the correction function is performed only for the pixels in the reachable area.

  The seventh embodiment can be executed in combination with the second to fifth embodiments described above.

<Other embodiments>
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

Claims (15)

An imaging device having an auxiliary light source,
A light emission condition determining means for determining a light emission condition including a light emission pattern indicating on / off of light emission in the auxiliary light source based on a photographing condition;
Luminous photographing means for photographing a subject by causing the auxiliary light source to emit light according to the light emission condition, and acquiring a photographed image;
Blur information acquisition means for acquiring blur information at the time of photographing by the light emission photographing means;
Image correction means for performing blur correction on the captured image based on the blur information and the light emission condition;
An imaging device comprising:   The imaging apparatus according to claim 1, wherein the image correction unit performs a deconvolution operation of a point spread function based on the light emission pattern on the captured image.   The said light emission condition determination means further determines the light emission time which shows from the start to the end of the said light emission pattern, and the luminous intensity of the said auxiliary light source as said light emission conditions. Imaging device. Furthermore, it has a holding means for holding a plurality of sets of the light emission time, the light emission pattern, and the maximum light amount by the light emission pattern,
The imaging apparatus according to claim 3, wherein the light emission condition determining unit determines the light emission condition by selecting one of the plurality of sets held by the holding unit.   5. The imaging apparatus according to claim 1, wherein the light emission pattern is set so that a frequency characteristic of a point spread function thereof is configured with a non-zero value. 6.   6. The imaging apparatus according to claim 1, wherein the shake information acquisition unit acquires a movement speed of the imaging apparatus at the time of photographing by the light emission photographing unit as the blur information. Furthermore, it has ambient light information acquisition means for acquiring ambient light information at the time of photographing by the light emission photographing means,
7. The imaging according to claim 1, wherein the image correction unit performs shake correction on the captured image based on the blur information, the light emission condition, and the ambient light information. apparatus.   The ambient light information acquisition means acquires, as the ambient light information, a first subject luminance when the auxiliary light source is not emitted and a second subject luminance when the auxiliary light source is emitted. The imaging apparatus according to claim 7.   9. The imaging according to claim 7, wherein the image correction unit corrects the light emission pattern according to the ambient light information, and performs blur correction on the captured image based on the corrected light emission pattern. apparatus. Furthermore, it has an area determination means for determining whether or not the light emission from the auxiliary light source reaches each pixel of the captured image,
10. The image correction unit according to claim 1, wherein the image correction unit performs blur correction on a pixel that is determined to be a region in which light emission by an auxiliary light source of the photographed image has arrived by the region determination unit. The imaging apparatus of Claim 1.   The region determination means includes: a first region determination image obtained by photographing the subject without causing the auxiliary light source to emit light; and a second region determination image obtained by photographing the subject while causing the auxiliary light source to emit light. The imaging apparatus according to claim 10, wherein it is determined based on a difference between the pixels whether or not each pixel of the captured image is a region where light emission from the auxiliary light source has reached. Furthermore, it has an aperture control correction means for performing blur correction based on the blur information and the open / close pattern for an image photographed by opening control of the shutter according to a predetermined open / close pattern,
The imaging apparatus according to claim 1, wherein the image correction unit and the aperture control correction unit perform switching based on the photographing condition.   The imaging apparatus according to claim 1, wherein the auxiliary light source includes a plurality of light sources. A method for controlling an imaging apparatus having an auxiliary light source,
A light emission condition determining step for determining a light emission condition including a light emission pattern indicating on / off of light emission in the auxiliary light source based on a photographing condition;
A flash shooting step of shooting the subject by emitting the auxiliary light source according to the flash conditions, and acquiring a shot image;
A blur information acquisition step of acquiring blur information of the imaging device at the time of shooting in the flash shooting step;
An image correction step for performing blur correction on the captured image based on the blur information and the light emission condition;
A method for controlling an imaging apparatus, comprising:   The program for making a computer with which an imaging device is provided perform each step of the control method of the imaging device of Claim 14.

Images