JP2015019318A - Imaging apparatus, control method of the same, and computer program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To easily and securely achieve extraction of a smear from an image signal and correction of the smear.SOLUTION: A smear region 2b is detected from a high exposure image signal, and a value of a difference in a pixel unit between the high exposure image signal and a low exposure image signal is detected as a smear component. A region where variation of the smear component is large or a region where a moving amount in the smear region is large is extracted as a moving region. In a region except for the moving region of the smear region 2b, the smear component is smoothed by using the smear component of a value of the region and a smear correction value is derived. On the other hand, the moving region of the smear region 2b is subjected to interpolation processing without using the smear component of the moving region and the smear correction value is derived. The smear correction value which is thus derived is subtracted from a signal of the smear region in the high exposure image signal output from a camera signal processing section 3.

Description

  The present invention relates to an imaging apparatus, an imaging apparatus control method, and a computer program, and is particularly suitable for use in correcting smear in an imaging unit.

In an imaging apparatus having a recent two-dimensional imaging element, a countermeasure against a smear phenomenon that occurs when a high-luminance subject is imaged is an issue.
For example, when the imaging device is a CCD, unnecessary charges are transferred to the vertical transfer path when the amount of incident light is large. As a result, a region having different brightness in the vertical direction of the high-luminance subject portion occurs as a smear phenomenon. Even when the image sensor is a CMOS sensor, the potential fluctuates when resetting a line that captures a high-luminance subject. As a result, the black level changes, and a region with different brightness in the horizontal direction of the high-luminance subject portion occurs as a smear phenomenon.

The smear phenomenon deteriorates the quality of the captured image itself, and at the same time becomes an obstacle to an imaging apparatus that performs a dynamic range expansion process by combining a high-exposure image and a low-exposure image, for example.
In order to deal with such inconveniences, low-exposure imaging is performed so that smear does not occur, and smear components are extracted and corrected from a plurality of image signals having different exposure amounts. There is a method (see Patent Documents 1 and 2).

Japanese Patent Laid-Open No. 2002-305589 Japanese Patent Laid-Open No. 10-294892

However, in the technique described in Patent Document 1, images are picked up by changing the exposure amount in order, and a smear component is extracted based on a difference between a plurality of picked-up images. Therefore, a time difference occurs in the timing of imaging. For this reason, there has been a problem that smear components cannot be correctly extracted when the subject moves.
Further, in the technique described in Patent Document 2, images with different exposure amounts are captured at the same time. Therefore, unlike the technique described in Patent Document 1, a time difference in imaging timing does not occur. However, a special image sensor for capturing images with different exposure amounts at the same time is required. Therefore, it is not a measure for imaging using a general image sensor while sequentially changing the exposure amount.

  The present invention has been made in view of such problems, and an object thereof is to easily and reliably realize smear correction by extracting smear from an image signal.

  The image pickup apparatus of the present invention is an area where smear is generated based on an exposure control unit that controls an exposure amount for the image pickup device and a plurality of image signals picked up by the image pickup device with different exposure amounts. A smear component detecting means for detecting a smear area and a smear component signal; a moving area estimating means for estimating a moving area of a subject in the smear area; and a smear component signal detected by the smear component detecting means; A signal correction unit that smoothes the smear component signal and corrects the smear component signal based on the movement region estimated by the movement region estimation unit; and the smear component corrected by the signal correction unit. And smear correction means for performing smear correction on the image signal in which smear has occurred.

  According to the present invention, it is possible to easily and reliably realize smear correction by extracting smear from an image signal.

It is a figure which shows the 1st example of a structure of an imaging device. It is a figure which shows the smear which generate | occur | produces in an image signal. It is a figure which shows a smear component and a smear correction value. It is a figure which shows the 2nd example of a structure of an imaging device.

Embodiments of the present invention will be described with reference to the drawings.
(First embodiment)
First, the first embodiment will be described. In the present embodiment, a case where a moving area of a subject in a smear area is estimated using a plurality of image signals captured while sequentially changing the exposure amount, and a smear correction value is generated based on the estimated moving area information is used. An example will be described.
FIG. 1 is a block diagram illustrating an example of the configuration of the imaging apparatus.
In FIG. 1, the imaging unit 2 includes an imaging element such as a CCD (Charge Coupled Device) or a CMOS sensor, and an A / D converter that A / D converts an analog signal representing a subject image captured by the imaging element. .
The exposure control unit 1 controls the exposure state for the imaging unit 2.
The camera signal processing unit 3 performs signal processing such as white balance processing, color (luminance / color difference signal) conversion, gain control, and γ correction on the signal output from the imaging unit 2.

The frame memory 4 temporarily holds the image signal output from the camera signal processing unit 3 in units of frames.
The motion amount detection unit 5 compares the image signal output from the camera signal processing unit 3 with the image signal output from the frame memory 4 (one frame before), and based on the comparison result, the moved subject Region information and the amount of movement of the subject are generated and output.
The smear component detection unit 6 compares the image signal output from the camera signal processing unit 3 with the image signal output from the frame memory 4 (one frame before), and based on the comparison result, the smear region Extract and output smear components. Here, the smear region is a region where smear in the image signal is generated. The smear component refers to a smear component signal included in the image signal.
The moving region estimation unit 7 uses the region information output from the motion amount detection unit 5 and the motion amount or the smear region and smear component output from the smear component detection unit 6, and is a region where the subject has moved within the smear region. Guess a certain moving area.

The smear correction value generation unit 8 generates a smear correction value based on the smear region and the smear component output from the smear component detection unit 6 and the movement region output from the movement region estimation unit 7. Here, the smear correction value refers to a smear component signal after correction for reducing noise included in the smear component signal.
The smear correction processing unit 9 performs smear correction processing on the image signal output from the camera signal processing unit 3 based on the smear correction value output from the smear correction value generation unit 8.

FIG. 2 is a diagram schematically illustrating an example of smear generated in an image signal. Specifically, FIG. 2A is a diagram schematically illustrating an example of smear generated in an image signal (high exposure image) captured with a relatively high exposure amount, and FIG. It is a figure which shows typically an example of the smear which generate | occur | produces in the image signal (low-exposure image) imaged with the low exposure amount. It is assumed that the high exposure image and the low exposure image are taken continuously.
In the example shown in FIG. 2A, a high-luminance subject area 2a exists near the center of the high-exposure image, and a smear area 2b having different brightness with respect to the original image signal exists in the left-right direction. On the other hand, in the example shown in FIG. 2B, as in the high-exposure image, the high-luminance subject area 2a exists near the center of the low-exposure image. However, the smear area 2b does not occur around the high brightness subject area 2a. Also, there is a moving body 2c in the smear area 2b of the high exposure image shown in FIG. 2A, and the area (position) of the moving body 2c changes depending on the time difference between the imaging timings of the high exposure image and the low exposure image. It shall be.

Next, an example of the operation of the imaging apparatus according to the present embodiment will be described with reference to FIGS.
First, the exposure control unit 1 captures a low-exposure image that does not generate smear by reducing the amount of exposure to the image capturing unit 2 (for example, compared to that during proper exposure). The camera signal processing unit 3 performs signal processing on the signal output from the imaging unit 2 and holds the signal subjected to signal processing in the frame memory 4 as a low exposure image signal.

Next, the exposure control unit 1 increases the exposure amount with respect to the imaging unit 2 (for example, at the time of proper exposure or low exposure) and captures a high exposure image in which smear occurs. The camera signal processing unit 3 performs signal processing on the signal output from the imaging unit 2, and outputs the signal subjected to signal processing as a high exposure image signal.
In the signal output from the imaging unit 2, a difference in brightness due to a difference in exposure amount occurs between low exposure and high exposure. Thus, in the present embodiment, the camera signal processing unit 3 performs gain processing on the signal output from the imaging unit 2 so as to cancel the difference in exposure amount by gain control. Thereby, the difference in signal level corresponding to the difference in exposure amount between the low exposure image signal and the high exposure image signal output from the camera signal processing unit 3 is canceled. Note that, since the imaged timing differs between the low exposure image signal and the light exposure image signal, even if the difference in signal level corresponding to the difference in exposure amount is canceled due to a change in ambient light or a change in the subject, the same pixel The signals do not necessarily match.

  Next, the motion amount detection unit 5 compares the low exposure image signal (one frame before) output from the frame memory 4 with the high exposure image signal (current frame) output from the camera signal processing unit 3. Processing is performed, and region information and motion amount are extracted and output. In this comparison process, for example, the following process is performed. First, correlation calculation is performed while shifting the phases of two images in units of pixels or divided regions of an image, and phase information having the highest correlation is set as a motion amount. Then, the amount of motion for each pixel or divided region is classified, and coordinate information of regions with similar amounts of motion is used as region information. For example, in FIG. 2, the motion amount detection unit 5 extracts the region of the moving body 2c as region information and extracts the movement amount of the moving body 2c as the motion amount.

Next, the smear component detection unit 6 extracts a smear region and a smear component based on the low exposure image signal output from the frame memory 4 and the high exposure image signal output from the camera signal processing unit 3. Output.
For example, the smear component detection unit 6 extracts a region in the high-exposure image signal where the signal level exceeds the threshold as the high-luminance subject region 2a, and the extracted high-luminance subject region 2a according to the characteristics of the image sensor. Determine the smear area. When smear occurs in the horizontal direction of the image as in the case where a CMOS sensor is used as the image sensor, as shown in FIG. 2A, the horizontal area of the high-brightness subject area 2a is replaced with the smear area 2b. To do. On the other hand, when smear occurs in the vertical direction of the image as in the case where a CCD is used as the image sensor, the vertical area of the high-luminance subject area 2a is set as the smear area.
Further, the smear component detection unit 6 uses a difference value in pixel units between the high exposure image signal and the low exposure image signal in the smear region 2b as a smear component.

FIG. 3 is a diagram conceptually illustrating an example of a smear component (FIG. 3A) and a smear correction value (FIG. 3B). Specifically, FIG. 3 conceptually shows an example of a smear component and a smear correction value at each position of a certain horizontal line 2d of the high exposure image and the low exposure image of FIG.
In FIG. 3A, the horizontal axis represents the coordinates of the horizontal line, and the vertical axis represents the value of the smear component.
The smear phenomenon in the smear region basically changes smoothly. Therefore, in the smear component detected by the smear component detection unit 6, the noise component in the image and the difference component in the moving region are mixed.

Therefore, the movement region estimation unit 7 estimates the movement region based on the smear region 2b and the smear component (see FIG. 3A) detected by the smear component detection unit 6.
When the moving body 2c is present in the smear region 2b, the value of the smear component greatly varies locally. Therefore, it is possible to estimate the moving region by observing the smear component.
For example, in the graph shown in FIG. 3A, an area where the smear component fluctuates with respect to the surroundings can be extracted as the movement area. For example, the moving region estimation unit 7 determines the region when the amount of change between the smear component value of a certain region of the target image signal and the smear component values of surrounding pixels exceeds a predetermined threshold. It can be specified as a moving area. More specifically, for example, a section between two coordinates in which the smear component value is more than a predetermined threshold with respect to the average value of the surrounding smear components and the past smear component values is defined as the moving region. can do. In addition, a section between two coordinates in which the differential value of the smear component with respect to the coordinates of the horizontal line is larger (or smaller) than a predetermined threshold value can be set as a movement region.

Further, the movement area estimation unit 7 can also estimate the movement area using the area information and the movement amount output from the movement amount detection unit 5. For example, the movement area estimation unit 7 can specify area information whose movement amount corresponding to the area information in the smear area is larger than a predetermined threshold as the movement area.
Information on the movement area estimated by the movement area estimation unit 7 as described above is output to the smear correction value generation unit 8.

Next, the smear correction value generation unit 8 performs smear correction on the smear region based on the smear region and smear component output from the smear component detection unit 6 and the moving region information output from the moving region estimation unit 7. Generate a value.
As described above, the smear component is a mixture of the noise component and the difference component of the moving region (see FIG. 3A).
Since the smear phenomenon changes smoothly, the smear correction value generation unit 8 smoothes the smear component detected by the smear component detection unit 6 and generates a smear correction value.

Specifically, the smear correction value generation unit 8 calculates, for example, an average value of a target pixel and its surrounding pixels or performs LPF processing on the smear component for pixels that are not in the moving region. As a result, the smear component can be smoothed to obtain a smear correction value. On the other hand, with respect to the pixels in the moving area, the smear correction value is obtained by interpolating using the smear component values of the pixels around the smear component pixel and not in the moving area without using the smear component value. be able to. In addition, a memory for holding the smear correction value of the past frame may be provided in the smear correction value generation unit 8, and the smear correction value of the pixel in the moving region may be generated using the past smear correction value.
By performing the above processing, the smear correction value generation unit 8 can generate a smear correction value in which the influence of the noise component and the moving region is reduced as shown in FIG.

  Next, the smear correction processing unit 9 subtracts the smear correction value output from the smear correction value generation unit 8 from the smear region signal in the high-exposure image signal output from the camera signal processing unit 3 to perform smear correction processing. Make corrections.

  As described above, in the present embodiment, the smear region 2b is detected from the high-exposure image signal, and the difference value in pixel units between the high-exposure image signal and the low-exposure image signal in the smear region 2b is detected as a smear component. To do. Then, an area where the variation of the smear component is large or an area where the amount of motion in the smear area is large is extracted as a movement area. For a region other than the moving region of the smear region 2b, the smear component is smoothed using the smear component of the value of the region to derive a smear correction value. On the other hand, for the moving area of the smear area 2b, interpolation processing is performed without using the smear component of the moving area to derive a smear correction value. The smear correction value derived in this way is subtracted from the smear region signal in the high-exposure image signal output from the camera signal processing unit 3. Therefore, by using a plurality of image signals captured while sequentially changing the exposure amount, the movement area of the subject in the smear area is estimated, and the smear correction value is generated based on the information on the estimated movement area. Even when moved, smear can be corrected with high accuracy.

(Second Embodiment)
Next, a second embodiment will be described. In the first embodiment, the case of performing smear correction on a high exposure image using a high exposure image and a low exposure image has been described as an example. On the other hand, in the present embodiment, when performing a dynamic range expansion process by combining a high-exposure image and a low-exposure image, appropriate smear correction and image composition processing are performed according to the occurrence of smear. The case will be described. As described above, the present embodiment and the first embodiment are mainly different in configuration and processing by synthesizing a high-exposure image and a low-exposure image and performing dynamic range expansion processing. Therefore, in the description of the present embodiment, the same parts as those in the first embodiment are denoted by the same reference numerals as those in FIGS.

FIG. 4 is a block diagram illustrating an example of the configuration of the imaging apparatus.
In FIG. 4, the motion vector detection unit 10 compares the image signal output from the camera signal processing unit 3 with the image signal output from the frame memory 4 (one frame before), and based on the comparison result. , Detecting a motion vector.
The alignment control unit 11 generates and outputs alignment information for aligning the positions of the high exposure image signal and the low exposure image signal based on the motion vector output from the motion vector detection unit 10.
The alignment processing unit 12 performs image coordinate conversion based on the alignment information output from the alignment control unit, and aligns the high exposure image signal and the low exposure image signal.

The smear correction value generation unit 13 generates a smear correction value based on the smear region and smear component output from the smear component detection unit 6 and the information on the movement region output from the movement region estimation unit 7. As described above, the smear correction value generation unit 13 according to the present embodiment performs signal correction processing to reduce noise included in the smear component signal, similarly to the smear correction value generation unit 8 according to the first embodiment. In addition to this, the smear correction value generation unit 13 of the present embodiment performs a reliability derivation process for deriving a smear correction value reliability according to the size of the moving area in the smear area. Here, the smear correction value reliability refers to the reliability of the smear correction value.
The composition control unit 14 performs a composition ratio derivation process for deriving a composition ratio of the high exposure image signal and the low exposure image signal according to the smear correction value reliability output from the smear correction value generation unit 13.
The composition processing unit 15 performs composition processing of the high exposure image signal and the low exposure image signal according to the composition ratio output from the composition control unit 14.

Next, an example of the operation of the imaging apparatus according to the present embodiment will be described.
First, the exposure control unit 1 reduces the exposure amount for the imaging unit 2 and captures a low-exposure image that does not generate smear. The camera signal processing unit 3 performs signal processing on the signal output from the imaging unit 2 and holds the signal subjected to signal processing in the frame memory 4 as a low exposure image signal.
Next, the exposure control unit 1 increases the exposure amount with respect to the imaging unit 2 and captures a high-exposure image in which smear occurs. The camera signal processing unit 3 performs signal processing on the signal output from the imaging unit 2, and outputs the signal subjected to signal processing as a high exposure image signal.
In this embodiment as well, as described in the first embodiment, it is assumed that the signal levels of the low exposure image signal and the high exposure image signal output from the camera signal processing unit 3 are the same.

  Next, the motion vector detection unit 10 compares the low exposure image signal output from the frame memory 4 and the high exposure image signal output from the camera signal processing unit 3 to detect a motion vector between frames. And output. In this comparison process, for example, a correlation calculation is performed while shifting the phases of the two images in units of pixels or in units of divided regions of the image, and a process of determining coordinate information of the phase with the highest correlation as a motion vector is performed.

Next, the alignment control unit 11 outputs alignment information for aligning the position of the high-exposure image signal and the position of the low-exposure image signal based on the motion vector information output from the motion vector detection unit 10. To do.
Specifically, the alignment control unit 11 classifies the motion vectors detected by the motion vector detection unit 10 to divide the image area, determines an area representing the motion of the entire screen, and obtains the motion vector from the area. The alignment information is calculated from the obtained motion amount of the entire screen. The area representing the movement of the entire screen is determined from an area where the size of the divided area is large or an area located in the background. In addition, the alignment control unit 11 detects an area with movement that has not been used for alignment as an area with movement other than the area representing the movement of the entire screen.

Next, the alignment processing unit 12 performs high-exposure by performing coordinate conversion for each pixel on the high-exposure image signal or the low-exposure image signal based on the alignment information output from the alignment control unit 11. The image signal and the low exposure image signal are aligned. In this embodiment, alignment processing is performed on the high-exposure image signal, and the position of the high-exposure image signal is aligned with the position of the low-exposure image signal.
Next, the smear component detection unit 6 extracts a smear region and a smear component based on the low exposure image signal output from the frame memory 4 and the high exposure image signal output from the alignment processing unit 12. Output. The smear region and smear component extraction method can be realized by the same method as described in the first embodiment.

Next, the movement region estimation unit 7 estimates a movement region based on the smear region and the smear component detected by the smear component detection unit 6. The moving region estimation method can be realized by the same method as described in the first embodiment.
Here, the region information and the motion amount from the motion amount detection unit 5 in the first embodiment correspond to the alignment information from the alignment control unit 11. When it is detected by the alignment control unit 11 that a region having a motion other than the region representing the motion of the entire screen is present in the smear region, the movement region estimation unit 7 estimates the region as a movement region. Note that, as in the first embodiment, the movement region may be estimated based on the smear component and the movement amount.

Next, the smear correction value generation unit 13 performs smear correction on the smear region based on the smear region and the smear component output from the smear component detection unit 6 and the information on the movement region output from the movement region estimation unit 7. Value and the reliability of the smear correction value are generated.
Since the smear phenomenon changes smoothly, the smear correction value generation unit 13 smoothes the smear component and generates a smear correction value. The smear correction value generation method can be realized by the same method as described in the first embodiment.
The smear correction value generation unit 13 calculates the reliability of the smear correction value according to the ratio of the moving area in the smear area. Specifically, the smear correction value generation unit 13 decreases the reliability of the smear correction value as the ratio of the moving area in the smear area increases. As described in the first embodiment, since the smear correction value is generated by excluding the smear component of the moving region, the larger the moving region, the smaller the number of smear components that can be used.

Next, the smear correction processing unit 9 subtracts the smear correction value output from the smear correction value generation unit 8 from the smear region signal in the high-exposure image signal output from the camera signal processing unit 3 to perform smear correction processing. Make corrections.
Next, the composition control unit 14 outputs the low exposure image signal output from the frame memory 4, the high exposure image signal output from the smear correction processing unit 9, and the smear correction value output from the smear correction value generation unit 13. The composition ratio is determined on the basis of the reliability.
For the area other than the smear area, the same method as the conventional method for generating the synthesis ratio for expanding the dynamic range is adopted. Specifically, the values of the high-exposure image signal and the low-exposure image signal are compared in pixel units, and the ratio of the low-exposure image signal is higher as the difference is larger, and the ratio of the high-exposure image signal is higher as the difference is smaller. Determine the composition ratio.

  On the other hand, in the smear region, in addition to such a composition ratio determination method, the composition ratio of the low-exposure image signal is high when the reliability of the smear correction value is further low, and is high when the reliability is high. The composition ratio is determined so that the composition ratio of the exposure image signal is increased. This is because the higher the reliability of the smear correction value, the more accurately the smear correction process for the high exposure image.

  As described above, in this embodiment, the high exposure image signal and the low exposure image signal are compared to detect a motion vector, and alignment information for aligning the positions of the high exposure image signal and the low exposure image signal is generated. To do. At this time, an area having a motion that is not used for alignment is set as a movement area. Similarly to the first embodiment, the smear region and the smear component are extracted to generate a smear correction value. Further, the reliability of the smear correction value is generated so that the value decreases as the ratio of the moving area in the smear area increases. For the smear area, the composition ratio of the high exposure image signal and the low exposure image signal is determined based on both the pixel unit difference between the pixel values of the high exposure image signal and the low exposure image signal and the reliability of the smear correction value. To do. On the other hand, for a region different from the smear region, the composition ratio is determined based on the pixel unit difference between the pixel values of the high exposure image signal and the low exposure image signal. The high-exposure image signal and the low-exposure image signal are synthesized at the synthesis ratio determined in this way. Therefore, even when synthesizing a high-exposure image signal and a low-exposure image signal to perform dynamic range expansion processing, appropriate smear correction and image composition processing are performed according to the occurrence of smear, resulting in high-performance dynamic Range expansion processing can be performed.

  The above-described embodiments are merely examples of implementation in carrying out the present invention, and the technical scope of the present invention should not be construed in a limited manner. That is, the present invention can be implemented in various forms without departing from the technical idea or the main features thereof.

(Other examples)
The present invention is also realized by executing the following processing. That is, first, software (computer program) for realizing the functions of the above embodiments is supplied to a system or apparatus via a network or various storage media. Then, the computer (or CPU, MPU, etc.) of the system or apparatus reads out and executes the computer program.

  3 camera signal processing unit, 6 smear component detection unit, 7 moving area estimation unit, 8 smear correction value generation unit, 9 smear correction processing unit

Claims (17)

Exposure control means for controlling an exposure amount for the image sensor;
Smear component detection means for detecting a smear region in which smear is generated and a smear component signal based on a plurality of image signals captured by the image sensor with different exposure amounts;
A movement area estimation means for estimating a movement area of a subject in the smear area;
Signal correction means for correcting the smear component signal based on the smear component signal detected by the smear component detection means and the movement region estimated by the movement region estimation means;
Smear correction means for performing smear correction on the image signal in which smear has occurred, using the smear component signal corrected by the signal correction means;
An imaging device comprising:   The signal correction means does not use a smear component signal of the moving region for the moving region in the smear region, but uses a smear component signal in a region different from the moving region in the smear region, The image pickup apparatus according to claim 1, wherein a smear component signal in the moving region is interpolated.   The imaging apparatus according to claim 1, wherein the moving area estimation unit estimates the moving area based on a smear component signal detected by the smear component detection unit. Using a plurality of image signals, motion detection means for detecting movement of a subject in the image signals,
The imaging apparatus according to claim 1, wherein the movement area estimation unit estimates the movement area based on a movement of a subject detected by the movement detection unit. Reliability deriving means for deriving the reliability of the smear component signal corrected by the signal correcting means based on the moving area estimated by the moving area estimating means;
Combining ratio deriving means for deriving a combining ratio of the plurality of image signals;
Synthesizing means for synthesizing the plurality of image signals after smear correction is performed by the smear correction means,
The synthesis ratio deriving unit determines the synthesis ratio for the smear region based on the plurality of image signals and the reliability of the smear component signal corrected by the signal correction unit, and the smear region, 5. The imaging apparatus according to claim 1, wherein for the different areas, the synthesis ratio is determined based on the plurality of image signals.   The reliability deriving unit derives the reliability of the smear component signal corrected by the signal correction unit so that the reliability increases as the ratio of the moving region in the smear region decreases. The imaging device according to claim 5.   The composition ratio deriving unit increases the ratio of the high exposure image signal having a relatively large exposure amount among the plurality of image signals as the reliability of the smear component signal corrected by the signal correction unit increases. The imaging apparatus according to claim 5, wherein the composition ratio is determined.   The plurality of image signals are a high exposure image signal having a relatively large exposure amount and a low exposure image signal having a relatively small exposure amount. Imaging device. An exposure control step for controlling an exposure amount for the image sensor;
A smear component detection step for detecting a smear region, which is a region where smear is generated, and a smear component signal, based on a plurality of image signals imaged by the imaging element in different exposure amounts;
A movement area estimation step of estimating a movement area of a subject in the smear area;
A signal correction step for correcting the smear component signal based on the smear component signal detected by the smear component detection step and the movement region estimated by the movement region estimation step;
A smear correction step of performing smear correction on the image signal in which smear has occurred, using the smear component signal corrected by the signal correction step;
A method for controlling an imaging apparatus, comprising:   The signal correction step uses a smear component signal in a region different from the moving region of the smear region without using a smear component signal of the moving region for the moving region in the smear region, The method of controlling an imaging apparatus according to claim 9, wherein a smear component signal in the moving region is interpolated.   The method of controlling an imaging apparatus according to claim 9 or 10, wherein the moving region estimating step estimates the moving region based on a smear component signal detected by the smear component detecting step. Using a plurality of image signals, a motion detection step of detecting a motion of a subject in the image signals,
The control of the imaging apparatus according to any one of claims 9 to 11, wherein the movement region estimation step estimates the movement region based on the movement of the subject detected by the movement detection step. Method. A reliability derivation step for deriving the reliability of the smear component signal corrected by the signal correction step based on the movement region estimated by the movement region estimation step;
A synthesis ratio deriving step for deriving a synthesis ratio of the plurality of image signals;
And a combining step of combining the plurality of image signals after the smear correction is performed in the smear correction step.
The synthesis ratio derivation step determines the synthesis ratio for the smear region based on the plurality of image signals and the reliability of the smear component signal corrected by the signal correction step, and the smear region, The method for controlling an imaging apparatus according to claim 9, wherein the combination ratio is determined based on the plurality of image signals for different regions.   In the reliability deriving step, the reliability of the smear component signal corrected by the signal correction step is derived so that the reliability increases as the proportion of the moving region in the smear region decreases. The method for controlling an imaging apparatus according to claim 13.   In the synthesis ratio deriving step, the higher the reliability of the smear component signal corrected in the signal correction step, the larger the ratio of the high exposure image signal having a relatively large exposure amount among the plurality of image signals. 15. The method of controlling an image pickup apparatus according to claim 13 or 14, wherein the composition ratio is determined.   The plurality of image signals are a high exposure image signal having a relatively large exposure amount and a low exposure image signal having a relatively small exposure amount, respectively. Method for controlling the imaging apparatus.   A computer program that causes a computer to execute each step of the control method for an imaging apparatus according to any one of claims 9 to 16.

Images