JP5911340B2 - Imaging apparatus and control method thereof - Google Patents

Description

The present invention relates to an imaging apparatus and a control method thereof, in particular, to an imaging device for image processing the image data recorded in a memory or the like.

  In general, an imaging apparatus such as a digital camera is provided with an image processing apparatus for performing image processing on image data recorded in a memory or the like. In a digital camera, image data is obtained by continuously taking images a plurality of times, the image data is stored in a memory, and further, a plurality of image data is synthesized by an image processing device (image processing function). To be done.

  As a photographing function for performing photographing a plurality of times, for example, there is so-called AE (Automatic Exposure) bracket photographing for performing photographing of a plurality of images while changing exposure conditions. Further, as an imaging function for obtaining a single image data by combining a plurality of image data, for example, there is an HDR (High Dynamic Range) imaging function. The HDR imaging function is a function for obtaining a high dynamic range image by combining a plurality of images captured at different exposures (see Patent Document 1).

  On the other hand, in digital cameras, in order to suppress variations in brightness and color of image data recorded in memory or the like in the past, in image data obtained as a result of continuous shooting, in gradation correction processing and white balance processing, etc. There is one that suppresses variations in correction amount. For example, there is a method of determining the white balance of the current frame by weighting according to the similarity between the image data acquired in the past (hereinafter referred to as the past frame) and the latest image data (hereinafter referred to as the current frame) (patent) Reference 2). In this case, the similarity index includes a difference in white balance data, a luminance difference in image data, an elapsed time after shooting, and the like.

JP 2002-247362 A Japanese Patent Laid-Open No. 10-290469

  By the way, in the above-mentioned Patent Document 2, although variations in brightness, color, and the like for each image data (hereinafter also simply referred to as an image) can be suppressed, a plurality of times are continuously performed as in AE bracket shooting or HDR shooting. With respect to image processing when shooting, it is difficult to suppress variations in brightness and color between images.

  For example, in AE bracket shooting, shooting is performed with different exposures of appropriate exposure, underexposure, and overexposure. However, in the image data obtained with different exposures such as proper exposure and underexposure, the above-mentioned similarity is low. Therefore, the correction amount in the past frame is not carried over to the current frame, and variations in color, brightness, and the like occur between the image data.

Accordingly, an object of the present invention is to provide an image pickup apparatus capable of suppressing variations in brightness, color, and the like between image data when image processing is performed on image data obtained by performing a plurality of continuous shootings, and control thereof. It is to provide a method.

To achieve the above object, an imaging apparatus according to the present invention, an imaging apparatus having an imaging mode for acquiring a plurality of images by continuously photographing by changing the exposure, acquired by shooting the shooting mode When a plurality of images are set as an image set, a similarity between a current image satisfying a predetermined condition in the image set and a past image acquired by shooting before shooting the image set is determined. The image processing is performed on a plurality of images included in the image set using a correction amount corresponding to the similarity .

The control method according to the present invention is a method for controlling an image pickup apparatus having a shooting mode in which a plurality of images are acquired by continuously shooting while changing exposure, and a plurality of images acquired by shooting in the shooting mode. Is set as an image set, a similarity between a current image satisfying a predetermined condition in the image set and a past image acquired by shooting before shooting the image set is determined, and according to the similarity Image processing is performed on a plurality of images included in the image set using the correction amount.

  According to the present invention, when image processing is performed on image data (image) obtained by performing a plurality of continuous shootings, variations in brightness, color, and the like among image data can be suppressed.

It is a block diagram showing an example of an imaging device by a 1st embodiment of the present invention. It is a flowchart for demonstrating the similarity determination by the similarity determination part shown in FIG. It is a figure which shows the relationship between the difference of the past frame demonstrated in FIG. 2, the present frame, and similarity. 3 is a flowchart for explaining calculation of a correction amount used when correcting image data in the imaging apparatus shown in FIG. 1. FIGS. 5A and 5B are diagrams schematically illustrating the correction amount calculation process described in FIG. 4, in which FIG. 5A is a diagram illustrating processing when normal shooting is continuously repeated, and FIG. 5B is a diagram illustrating normal shooting and AEB shooting; It is a figure which shows the process at the time of mixing. It is a figure for demonstrating the HDR synthetic | combination performed in the synthetic | combination process part shown in FIG. 1 in the imaging device by the 2nd Embodiment of this invention. FIGS. 2A and 2B are diagrams for explaining HDR synthesis performed by the synthesis processing unit illustrated in FIG. 1, in which FIG. 1A is a diagram illustrating an example of an over image synthesis ratio in HDR synthesis, and FIG. The figure which shows an example of a ratio, (c) is a figure which shows an example of the synthetic | combination ratio of an under image in HDR synthetic | combination. 3 is a flowchart for explaining correction amount calculation at the time of HDR synthesis performed by the imaging apparatus shown in FIG. 1. FIG. 9 is a diagram schematically illustrating correction amount calculation related to image data before composition processing and correction amount calculation related to image data after composition processing in the correction amount calculation processing described in FIG. 8. It is a figure which shows typically calculation of the amount of correction | amendment in the imaging device by the 3rd Embodiment of this invention when normal imaging | photography and AEB imaging | photography are mixed.

  Hereinafter, an example of an imaging apparatus according to an embodiment of the present invention will be described with reference to the drawings.

[First Embodiment]
FIG. 1 is a block diagram illustrating an example of an imaging apparatus according to the first embodiment of the present invention.

  In FIG. 1, an imaging device 10 includes a photographic lens (hereinafter simply referred to as a lens) 100, and when a shutter button (not shown) is depressed, an optical image incident from the photographic lens 100 passes through a diaphragm 101 and a shutter 102. Thus, an image is formed on the image sensor 104. The image sensor 104 outputs an electrical signal (analog signal) corresponding to the optical image.

  This analog signal is converted into a digital signal (hereinafter referred to as RAW data) by the A / D converter 105. Then, the control unit 106 stores the RAW data in the memory 107. The exposure control unit 103 obtains a photometric value based on the RAW data under the control of the control unit 106, and controls the diaphragm 101 and the shutter 102 according to the photometric value to perform exposure control.

  A white balance (WB) processing unit 108 detects a white area in the RAW data. Then, the WB processing unit 108 performs an auto white balance (AWB) process for correcting the color balance of the RAW data (that is, image data) according to the above-described photometric value and the detected white area.

  The signal processing unit 109 performs development processing such as color matrix processing and gamma processing on the image data after the WB processing under the control of the control unit 106 to obtain developed image data. The gradation conversion unit 110 performs gradation conversion processing on the developed image data (hereinafter referred to as YUV data) to generate gradation converted image data.

  The recording unit 111 records the gradation conversion image data on a portable medium (not shown) or the like under the control of the control unit 106. The face detection unit 113 detects the position of the face of the subject from the RAW data or YUV data under the control of the control unit 106, and acquires face information related to the brightness, color, and the like. The histogram detection unit 114 obtains a histogram (frequency distribution table) indicating the brightness distribution state from the RAW data or YUV data under the control of the control unit 106. A composition processing unit 115 (image composition means) composes a plurality of image data into one image data.

  The similarity determination unit 112 determines the similarity between image data (hereinafter referred to as a past frame) obtained as a result of past shooting and the latest image data (hereinafter referred to as a current frame) according to a predetermined similarity determination criterion. Judgment.

  Here, the past frame (past image) and the current frame (current image) are obtained as a result of continuous shooting, and are stored in the memory 107, for example.

  When determining the similarity, the similarity determination unit 112 determines the similarity based on the photometric value for each image data, the color ratio calculated by the WB processing unit 108, the histogram, or the face information. Then, as will be described later, correction amounts in the WB processing unit 108, the signal processing unit 109, and the gradation conversion unit 110 are determined based on the similarity determination result by the similarity determination unit 112.

  Here, the similarity determination process between the past frame and the current frame will be specifically described.

  FIG. 2 is a flowchart for explaining similarity determination by the similarity determination unit 112 shown in FIG. Note that the similarity determination unit 112 performs similarity determination under the control of the control unit 106.

  When the similarity determination is started, the similarity determination unit determines whether or not there is a past frame for similarity determination with the current frame in the memory 107 (step S101). That is, the similarity determination unit 107 determines whether or not a plurality of pieces of image data captured continuously exist in the memory.

  If there is no past frame in the memory 107 (NO in step S101), the similarity determination unit 112 sets the similarity Sim between the current frame and the past frame to “0” (step S102) and ends the similarity determination process. .

  On the other hand, if there is a past frame in the memory 107 (YES in step S101), the similarity determination unit 112 calculates the similarity S1 (from the difference between the photometric value related to the past frame and the photometric value related to the current frame (photometric difference)). First similarity) is calculated (step S103). When calculating the difference between the photometric values, the average value of the entire image may be used. The image is divided into a plurality of areas, the difference is obtained for each area, and the absolute value sum of the differences for each area is calculated. You may make it use.

  Subsequently, the similarity determination unit 112 calculates the similarity S2 (second similarity) according to the difference (color ratio difference) between the color ratio of the previous frame and the current frame (step S2). Then, the similarity determination unit 112 calculates the similarity S3 (third similarity) from the difference (histogram difference) between the histogram of the past frame and the histogram of the current frame (step S105).

  Further, the similarity determination unit 112 calculates the similarity S4 (fourth similarity) from the difference (face difference) between the face information in the previous frame and the face information in the current frame (step S106). Then, the similarity determination unit multiplies the similarity S1 to S4 to obtain a final similarity Sim between the past frame and the current frame (step S107), and ends the similarity determination processing.

  FIG. 3 is a diagram illustrating a relationship between the difference between the past frame and the current frame described in FIG. 2 and the similarity.

  In FIG. 3, when the difference between the past frame and the current frame is equal to or less than a predetermined threshold d1 (first reference threshold) (below the first reference threshold), the similarity determination unit 112 sets the similarity to “1”. And The similarity determination unit 112 sets the similarity to “0” when the difference is greater than or equal to a threshold d2 (second reference threshold) greater than the threshold d1 (greater than or equal to the second reference threshold).

  When the difference exceeds the threshold value d1 and is less than the threshold value d2, the similarity is a value on a straight line connecting the point (d1, 1) and the point (d2, 0). That is, when the difference is between the threshold value d1 and the threshold value d2, the similarity determination unit 112 decreases the similarity as the difference increases.

  The threshold values d1 and d2 are changed every time the similarity is calculated in steps S103 to S106. That is, the thresholds d1 and d2 are different for a difference relating to a photometric value, a difference relating to a color ratio, a difference relating to a histogram, and a difference relating to face information.

  Next, calculation of a correction amount used when correcting image data in the imaging apparatus shown in FIG. 1 will be described.

  FIG. 4 is a flowchart for explaining calculation of a correction amount used when correcting image data in the imaging apparatus shown in FIG. The correction amount refers to a correction value used when various image processing is performed on image data obtained as a result of photographing. Various image processing includes, for example, white balance processing for adjusting the color ratio of image data, gradation conversion processing for changing the gradation of image data, and hue and saturation of a specific color for the image data. Color conversion processing that adjusts. Furthermore, here, the determination result in the similarity determination process described with reference to FIG. 2 is used in order to suppress variations in the correction amount used for each image data.

  When calculation of the correction amount is started, the control unit 106 determines whether or not AE bracket (AEB) shooting (automatic exposure bracket shooting) has been performed (step S201). When ABE shooting is performed (YES in step S201), the control unit 106 determines whether or not shooting of an appropriate image is performed (step S202).

  In the AEB shooting mode (automatic exposure bracket shooting mode), one set of a plurality of images obtained as a result of shooting by changing the exposure is set. The set of images includes an appropriate image (an image defined in advance), an under image, and an over image.

  Here, the appropriate image refers to an image taken at the center exposure (appropriate exposure: predetermined exposure) in AEB photography in which exposure is changed and three pictures are taken. Of the three images, an image taken with the darkest exposure is called an under image, and an image taken with the brightest exposure is called an over image.

  If it is not an appropriate image (NO in step S202), the control unit 106 sets the correction amount used when image processing is performed on the appropriate image as the final correction amount (step S203), and ends the correction amount calculation.

  On the other hand, if it is an appropriate image (YES in step S202), the control unit 106 shifts to the normal shooting process and calculates the correction amount of the current frame using only the current frame (current image) (step S204). ). Then, the control unit 106 controls the similarity determination unit 112 to perform the similarity determination process described with reference to FIG. 2 (step S205) to obtain the similarity Sim.

  Subsequently, the control unit 106 compares the similarity Sim with a predetermined threshold (determination threshold) Th1 and determines whether or not the similarity Sim is less than the determination threshold (Step S206). The threshold value Th1 is 0 <Th1 <1. When the similarity Sim is equal to or greater than the threshold Th1 (determination threshold or greater) (NO in step S206), the control unit 106 determines that the similarity between the past frame (past image) and the current frame is high, and corrects the past frame. The amount is set as a final correction amount (final correction amount) (step S207). Then, the control unit 106 ends the correction amount calculation.

  If similarity Sim is less than threshold Th1 (YES in step S206), control unit 106 assumes that the similarity between the previous frame and the current frame is low, and sets the correction amount related to the current frame as the final correction amount (step S206). S208). Further, the control unit 106 stores the current frame information related to the current frame in the memory 107 as past frame information related to the previous frame, and uses it for similarity determination in the next shooting (step S209: past frame update). Then, the control unit 106 ends the correction amount calculation.

  FIG. 5 is a diagram schematically illustrating the correction amount calculation process described in FIG. FIG. 5A is a diagram showing processing when normal photographing is continuously repeated, and FIG. 5B is a diagram showing processing when normal photographing and AEB photographing are mixed. .

  In FIG. 5A, when the current frame is the first frame, there is no past frame, and in this case, the similarity Sim is set to “0”. Since similarity Sim <Ti, the correction amount of the current frame is set as the final correction amount in step S208 shown in FIG. 4, and the past frame is updated.

  Subsequently, similarity determination between the second frame and the first frame (that is, the past frame) of the current frame is performed. In the illustrated example, since similarity Sim> threshold Th1, the correction amount of the previous frame (first frame) is set as the final correction amount in step S207 shown in FIG. As a result, the third frame of the current frame is determined to be similar to the first frame.

  Since the similarity Sim between the third frame and the first frame is similarity Sim> threshold Th1, the correction amount of the past frame (here, the first frame) is set as the final correction amount. Next, similarity determination between the fourth frame of the current frame and the first frame (past frame) is performed. In the illustrated example, since similarity Sim <threshold Th1 is satisfied, in step S208 shown in FIG. 4, the correction amount for the current frame (here, the fourth frame) is set as the final correction amount.

  Subsequently, similarity determination between the fifth frame and the past frame (here, the fourth frame) is performed. In the illustrated example, since similarity Sim> threshold Th1, the correction amount of the past frame (fourth frame) is set as the final correction amount in step S207 shown in FIG. Further, similarity determination between the sixth frame and the past frame (fourth frame) is performed. Here, since similarity Sim <threshold Th1, the correction amount for the current frame (sixth frame) is set as the final correction amount in step S208 shown in FIG.

  Next, similarity determination between the seventh frame and the past frame (sixth frame) is performed. Here, since similarity Sim> threshold Th1, the correction amount of the past frame (sixth frame) is set as the final correction amount. Similarly, similarity determination between the eighth frame and the past frame (sixth frame) is performed, and the correction amount of the past frame (sixth frame) is set as the final correction amount.

  In FIG. 5B, since the normal shooting is performed up to the second frame, the same processing as the processing described in FIG. 5A is performed. AEB shooting is performed for the third frame to the fifth frame (three shootings are performed).

  Assume that the third frame is a suitable image, the fourth frame is an under image, and the fifth frame is an over image. As shown in the figure, since the similarity Sim between the past frame (first frame) and the appropriate image (third frame) is Sim> Th1, correction in AEB shooting (appropriate image) is performed in step S207 shown in FIG. As the amount, the correction amount of the first frame is taken over and becomes the final correction amount.

  For the fourth frame (under image) and the fifth frame (over image), the appropriate image correction amount is applied in step S203 shown in FIG.

  Subsequently, AEB shooting is performed for the sixth to eighth frames. Here, since the similarity Sim between the past frame (first frame) and the appropriate image (sixth frame) is Sim <Th1, the correction amount obtained from the sixth frame is the final correction amount in AEB shooting. The correction amount applied as the correction amount is stored in the memory 107 as the correction amount of the past frame.

  Thereafter, the correction amount calculation process is performed in the same manner. For the ninth to twelfth frames, the sixth frame correction amount is set as the final correction amount, and the thirteenth and fourteenth frames are corrected for the thirteenth frame. The amount is the final correction amount.

  In this way, in the first embodiment, even when normal shooting and AEB shooting are mixed, when correcting an image obtained by continuous shooting, the correction amount is taken over according to the similarity. Therefore, variations in brightness and color between captured images can be suppressed.

  In the first embodiment, the correction amount of the past frame or the correction amount of the current frame is applied in step S207 and step S208 shown in FIG. 4, respectively. However, the correction of the past frame is performed according to the similarity Sim. The final correction amount may be obtained by performing weighted addition with the correction amount of the current frame using the amount. In this way, variations in the correction amount can be further suppressed.

[Second Embodiment]
Next, an example of an image processing apparatus according to the second embodiment of the present invention will be described.

  Here, the configuration of the imaging apparatus according to the second embodiment is the same as that of the imaging apparatus shown in FIG. In the second embodiment, the function of the composition processing unit 115 is different from that of the first embodiment.

  As described in the first embodiment, the composition processing unit 115 performs a process of compositing a plurality of image data obtained as a result of photographing into one image data (composite image). In the following description, the composition processing unit 115 performs HDR (High Dynamic Range) composition that combines a plurality of images with different exposures to obtain one wide dynamic range image.

  FIG. 6 is a diagram for explaining HDR synthesis performed by the synthesis processing unit 115 shown in FIG. 1 in the imaging apparatus according to the second embodiment of the present invention.

  In the imaging apparatus 100 illustrated in FIG. 1, the control unit 106 controls the exposure control unit 103 to perform exposure while changing the exposure. As a result, three images with different exposures are acquired in the order of the appropriate image (RAW data), the under image (RAW data), and the over image (RAW data). These three pieces of RAW data are subjected to signal processing such as gamma conversion processing by the appropriate image gamma, the under image gamma, and the over image gamma in the signal processing unit 109, respectively.

  Accordingly, the signal processing unit 109 generates appropriate image YUV data (I1), under image YUV data (I2), and over image YUV data (I3). Then, the appropriate image YUV data (I1), the under image YUV data (I2), and the over image YUV data (I3) are given to the synthesis processing unit 115, and the synthesis processing unit 115 performs HDR synthesis of these three pieces of YUV data. .

  FIG. 7 is a diagram for explaining the HDR synthesis performed by the synthesis processing unit 115 shown in FIG. FIG. 7A is a diagram showing an example of the over image synthesis ratio in HDR synthesis, and FIG. 7B is a diagram showing an example of the appropriate image synthesis ratio in HDR synthesis. FIG. 7C is a diagram showing an example of the under image synthesis ratio in HDR synthesis.

  In FIG. 7, the horizontal axis indicates the luminance value (Y), and the vertical axis indicates the composition ratio. In HDR composition, the composition ratio of the three images is determined according to the luminance value Y of the appropriate image YUV data.

  Now, assume that the post-combination YUV data is G, the composition ratio of the appropriate image is M (Y), the composition ratio of the under image is L (Y), and the composition ratio O (Y) of the over image. At this time, the synthesis processing unit 115 synthesizes the pixel values at the respective coordinates of the synthesized YUV data G so that the following expressions (1) and (2) are satisfied.

G = I1 * M (Y) + I2 * L (Y) + I3 * O (Y) (1)
M (Y) + L (Y) + O (Y) = 1 (2)
The face detection unit 113 and the histogram detection unit 114 perform face detection and histogram detection according to the combined YUV data G, respectively, and provide face information and a histogram to the gradation conversion unit 110. The tone conversion unit 110 performs tone conversion processing on the combined YUV data based on the face information and the histogram, and outputs final YUV data. The recording unit 111 records YUV data on a portable medium or the like under the control of the control unit 106.

  FIG. 8 is a flowchart for explaining correction amount calculation at the time of HDR synthesis performed by the imaging apparatus 10 shown in FIG.

  In the first embodiment described above, an example in which the correction amount used for image processing such as white balance processing, gradation conversion processing, and color conversion processing is determined has been described. The processing to be performed is divided from the processing to be performed on the combined image data.

  That is, the white balance process and the color conversion process are performed on the image data before the synthesis, and the correction amount is “A”. On the other hand, the gradation conversion processing is processing performed on the combined image data, and the correction amount is “B”.

  When the correction amount calculation is started, the control unit 106 substitutes 0 for the variable i (step S301). Subsequently, the control unit 106 determines whether or not the image data to be processed is an appropriate image (step S302). If the image data to be processed is an appropriate image (YES in step S302), the control unit 106 calculates a correction amount A related to the current frame (that is, the appropriate image) (step S303).

  Subsequently, the control unit 106 controls the similarity determination unit 112 to obtain the similarity SimA between the past frame and the current frame as described with reference to FIG. 2 (step S304: similarity determination). The control unit 106 determines whether or not the similarity SimA is less than a predetermined threshold Th1A (step S305). Note that 0 <Th1A <1.

  When the similarity SimA is less than the threshold Th1A (YES in step S305), the control unit 106 assumes that the similarity between the past frame and the current frame is low, and sets the final correction amount A as the correction amount A of the current frame. (Step S306). Then, the control unit 106 stores the correction amount A of the current frame information in the memory 107 as the final correction amount A (that is, the correction amount of the past frame), and updates the past frame (step S307).

  Subsequently, the control unit 106 determines whether or not the variable i is 3 or more (step S308). That is, the control unit 106 determines whether signal processing has been completed for the three images before synthesis. If i <3 (NO in step S308), control unit 106 increments variable i (i = i + 1) (step S309), and returns to the process in step S302.

  If the similarity SimA is equal to or greater than the threshold th1A in step S305 (NO in step S305), the control unit 106 assumes that the similarity between the past frame and the current frame is high, and corrects the final correction amount A to the past frame correction. Takes over from the amount A (the final correction amount A is set as the correction amount A of the past frame: step S310). And the control part 106 progresses to the process of step S308.

  In step S302, if the image data to be processed is not an appropriate image (NO in step S302), the control unit 106 applies the correction amount A of the appropriate image as the final correction amount A (step S311), and step S308. Proceed to the process.

  Subsequently, the control unit 106 calculates a correction amount B related to the current frame with respect to the combined image data (step S312). Then, the control unit 106 controls the similarity determination unit 112 to obtain the similarity SimB between the past frame and the current frame (step S313: similarity determination).

  Next, the control unit 106 determines whether or not the similarity SimB is less than a predetermined threshold Th1B (step S314). Note that 0 <TH1B <1. If the similarity SimB is equal to or greater than the threshold Th1B (NO in step S314), the control unit 106 assumes that the similarity between the past frame and the current frame is high, and sets the final correction amount B as the correction amount B of the past frame. (Step S315). Then, the control unit 106 ends the correction amount calculation.

  On the other hand, when the similarity SimB is less than the threshold Th1B (YES in step S314), the control unit 106 determines that the similarity between the past frame and the current frame is low, and sets the final correction amount B as the current frame correction amount B. (Step S316). Then, the control unit 106 stores the correction amount B of the current frame in the memory 107 as the correction amount B of the past frame, updates the past frame (step S317), and ends the correction amount calculation.

  FIG. 9 is a diagram schematically illustrating correction amount calculation related to the image data before the composition processing and correction amount calculation related to the image data after the composition processing in the correction amount calculation processing described with reference to FIG. 8. Note that FIG. 9 illustrates processing when normal shooting and HDR shooting are mixed.

  In the image data before the composition processing, when the current frame is the first frame, there is no past frame, and in this case, the similarity Sim is set to “0”. Since similarity Sim <Ti, the current frame correction amount A is set as the final correction amount A in step S306 shown in FIG. 9, and the past frame is updated.

Subsequently, HDR shooting is performed to obtain the second to fourth frames. Assume that the second frame is a suitable image, the third frame is an under image, and the fifth frame is an over image. As shown in the figure, the similarity SimA between the past frame (first frame) and the appropriate image ( second frame) is SimA> Th1A. Therefore, in step S310 shown in FIG. 8, correction in HDR shooting (appropriate image) is performed. As the amount A, the correction amount A of the first frame is taken over and becomes the final correction amount A.

  For the third frame (under image) and the fourth frame (over image), the appropriate image correction amount A is applied in step S311 shown in FIG. Subsequently, normal shooting is performed to obtain the fifth frame. Since the similarity SimA between the past frame (first frame) and the fifth frame is SimA> Th1A, the correction amount A of the first frame is taken over as the correction amount A in the fifth frame in step S310 shown in FIG. Thus, the final correction amount A is set.

  Subsequently, HDR shooting is performed for the sixth to eighth frames. Here, since the similarity Sim between the past frame (first frame) and the appropriate image (sixth frame) is Sim> Th1, the correction amount A in HDR shooting is obtained from the past frame (first frame). The corrected amount A is applied as the final corrected amount A.

  Thereafter, the correction amount calculation process is performed in the same manner. For the ninth frame to the eleventh frame (HDR shooting), the correction amount A of the first frame is set as the final correction amount A, and in the twelfth frame (normal shooting). Since the similarity SimA <Th1A, the correction amount A at the 12th frame is set as the final correction amount A.

  In the image data after the composition processing, when the current frame is the first frame, there is no past frame. In this case, the similarity SimB is set to “0”. Since similarity SimB <Ti1B, the current frame correction amount B is set as the final correction amount B in step S316 shown in FIG. 9, and the past frame is updated.

  Subsequently, in the second frame obtained by combining the second frame to the fourth frame before image synthesis, the similarity SimB between the past frame (first frame) and the second frame is SimB <Th1B. In step S316, the correction amount B in the second frame after synthesis is set as the final correction amount B.

  Since the similarity SimB between the fifth frame (normal shooting) and the second frame after composition is SimB <Th1B, the correction amount B at the fifth frame, which is the current frame, is finally corrected in step S316 shown in FIG. The amount is B.

  Subsequently, the similarity SimB between the 6th frame and the 5th frame obtained by combining the 6th frame to the 8th frame before synthesis is SimB> Th1B. Therefore, the correction amount B at the 6th frame is set as the correction amount B after the synthesis. The correction amount B obtained from the second frame is applied as the final correction amount B.

  Thereafter, the correction amount calculation process is performed in the same manner, and in the ninth frame obtained by combining the ninth frame to the eleventh frame, the correction amount B of the ninth frame is set as the final correction amount B, and the 12th frame (normal shooting) ), Since the similarity SimB <Th1B, the correction amount B at the 12th frame is set as the final correction amount B.

  In this way, when switching from normal shooting to HDR shooting, the frame SimB and the combined frame in normal shooting have low similarity SimB and become almost zero. Similarly, when the HDR shooting is switched to the R normal shooting, the similarity SimB between the normal shooting frame and the combined frame is almost zero. That is, when switching from normal shooting to HDR shooting and when switching from HDR shooting to normal shooting, the similarity SimB is lower than the threshold value Th1B, and the correction amount B is not inherited.

  In this way, in the second embodiment, even when normal shooting and HDR shooting are mixed, when correcting an image obtained by continuous shooting, the correction amount is taken over according to the degree of similarity. Therefore, variations in brightness and color between captured images can be suppressed.

  In the second embodiment, one final correction amount B is stored in the memory according to the similarity, but a plurality of final correction amounts B may be stored in the memory. For example, if the past frame and final correction amount B for normal shooting and the past frame and final correction amount B for HDR shooting are stored in the memory, when switching between normal shooting and HDR shooting occurs, If either normal shooting or HDR shooting is used for a frame, the similarity SimB can be determined with high accuracy. As a result, even when normal shooting and HDR shooting are mixed, it is possible to more accurately suppress variations in brightness and color.

  In the second embodiment, gradation conversion processing is performed on the combined YUV data, so that inversion of gradation and reduction in contrast due to the combining processing can be easily prevented. The tone conversion process may be performed on each piece of image data before synthesis (appropriate image, under image, and over image). Also in this case, since gradation correction processing different in HDR shooting and normal shooting is performed, the similarity determination described above is used in the similarity determination.

  Also in this embodiment, the correction amount of the past frame or the correction amount of the current frame is applied to the correction amount for each image. However, the correction amount of the past frame is used according to the similarity Sim. The final correction amount may be obtained by performing weighted addition with the correction amount. In this way, variations in the correction amount can be further suppressed.

[Third Embodiment]
Next, an example of an imaging apparatus according to the third embodiment of the present invention will be described.

  Here, the configuration of the imaging apparatus according to the third embodiment is the same as that of the imaging apparatus shown in FIG. In the third embodiment, unlike the first embodiment, the image used for similarity determination in AEB shooting is not an appropriate image but is always used as the first frame image in AEB shooting.

  In AEB shooting, when shooting is performed in the order of an appropriate image, an under image, and an over image, the appropriate image is the first frame. Therefore, if image processing is performed in the order of shooting, correction of the under image and the over image is performed. The amount is the same as the appropriate image correction amount as described in the first embodiment.

  On the other hand, in an imaging apparatus in which the user can change the order of shooting, the appropriate image is not necessarily the first frame in AEB shooting. Therefore, in order to make the correction amount of the under image and the over image the same as the correction amount of the appropriate image, it is necessary to complete the processing of the appropriate image before the processing of the under image and the over image. Therefore, for example, when the appropriate image is captured in the third frame, it is necessary to wait for the image processing of the first frame and the second frame until the processing of the third frame is completed.

  However, depending on the capacity of the memory 107, the first frame and the second frame may not be held in the memory 107. Therefore, in the third embodiment, the first frame image (first image) is always used for similarity determination, not limited to the shooting order.

  FIG. 10 is a diagram schematically illustrating calculation of a correction amount when normal shooting and AEB shooting are mixed in the imaging apparatus according to the third embodiment of the present invention.

  In FIG. 10, since the second frame is normal shooting, the same processing as the processing described in FIG. 5A is performed. AEB shooting is performed for the third frame to the fifth frame (three shootings are performed).

  Assume that shooting is performed in the order of under image, over image, and image in the AEB shooting of the third frame to the fifth frame. In this case, since the under image, which is the third frame of AEB shooting, is significantly different in exposure from the first frame in normal shooting, the similarity Sim is almost zero. For this reason, the correction amount of the first frame in normal shooting is not carried over, and in step S206 shown in FIG. 4, the correction amount of the under image that is the current frame is set as the final correction amount as the correction amount in AEB shooting. The under image (current frame) is stored in the memory 107 as a past frame, and the correction amount is stored in the memory 107.

  In the subsequent second AEB shooting, the similarity Sim between the under image and the second under image in the first AEB shooting is Sim> Th1, and in step S207 shown in FIG. 4, the second AEB shooting is performed. The correction amount obtained from the third frame is applied as the final correction amount.

  Since the 9th frame is normal shooting, the under image, which is the 6th frame of AEB shooting, is significantly different in exposure from the 9th frame of normal shooting, so the similarity Sim is almost zero. For this reason, the correction amount for the third frame at the time of AEB shooting is not carried over, and in step S206 shown in FIG. 4, the correction amount for the ninth frame, which is the current frame, is set as the final correction amount.

  Thereafter, correction amount calculation processing is performed in the same manner, and for the 10th to 12th frames for AEB shooting, the correction amount for the 10th frame is set as the final correction amount, and 13 frames for the 13th frame for normal shooting. The eye correction amount is set as the final correction amount. Then, in the 14 frames of normal shooting following the 13th frame, the correction amount of the 13th frame is taken over as the final correction amount.

  As described above, also in the third embodiment, when normal photography and AEB photography are mixed, when correcting an image obtained by continuous photography, the correction amount is taken over according to the similarity. Therefore, variations in brightness and color between captured images can be suppressed.

  Also in this embodiment, the correction amount of the past frame or the correction amount of the current frame is applied to the correction amount for each image. However, the correction amount of the past frame is used according to the similarity Sim. The final correction amount may be obtained by performing weighted addition with the correction amount. In this way, variations in the correction amount can be further suppressed.

  As mentioned above, although this invention was demonstrated based on embodiment, this invention is not limited to these embodiment, Various forms of the range which does not deviate from the summary of this invention are also contained in this invention. .

  For example, the function of the above embodiment may be used as a control method, and this control method may be executed by the imaging apparatus. Further, a program having the functions of the above-described embodiments may be used as a control program, and the control program may be executed by a computer included in the imaging apparatus. The control program is recorded on a computer-readable recording medium, for example.

  The present invention can also be realized by executing the following processing. That is, software (program) for realizing the functions of the above-described embodiments is supplied to a system or apparatus via a network or various recording media, and a computer (or CPU, MPU, etc.) of the system or apparatus reads the program. It is a process to be executed.

DESCRIPTION OF SYMBOLS 103 Exposure control part 106 Control part 107 Memory 108 WB process part 109 Signal processing part 110 Gradation conversion part 112 Similarity determination part 113 Face detection part 114 Histogram detection part 115 Compositing process part

Claims (8)

In the imaging apparatus having the imaging mode for acquiring a plurality of images by continuously photographing by changing the exposure,
When a plurality of images acquired by shooting in the shooting mode is an image set, a current image satisfying a predetermined condition in the image set and a past image acquired by shooting before shooting the image set And an image processing unit that performs image processing on a plurality of images included in the image set using a correction amount corresponding to the similarity . When the previous shooting is shooting in the shooting mode, the similarity between images satisfying the predetermined condition is determined from the image sets acquired in each shooting, and image processing according to the similarity is performed. the imaging apparatus according to claim 1, characterized in that. The image pickup apparatus according to claim 1 , wherein the image satisfying the predetermined condition is an image obtained by photographing with appropriate exposure among image sets obtained by photographing in the photographing mode . 3. The imaging apparatus according to claim 1 , wherein the image satisfying the predetermined condition is an image acquired by first imaging in an image set acquired by imaging in the imaging mode. 4. . The image processing, white balance processing, color conversion processing, and the imaging apparatus according to any one of claims 1 to 4, characterized in that among the gradation conversion processing is at least one. The shooting mode is an HDR mode for combining a plurality of images included in an image set,
6. The process according to claim 5 , wherein a part of the image processing is performed using a composite image after the composition processing as a current image, and using a correction amount according to the similarity between the current image and the past image . Imaging device. The imaging apparatus according to claim 6 , wherein the partial processing is gradation conversion processing . In a control method of an imaging apparatus having a shooting mode for acquiring a plurality of images by continuously shooting with changing exposure ,
When a plurality of images acquired by shooting in the shooting mode is an image set, a current image satisfying a predetermined condition in the image set and a past image acquired by shooting before shooting the image set A control method, wherein image processing is performed on a plurality of images included in the image set using a correction amount corresponding to the similarity .

Images