JP5483949B2 - Image processing apparatus, image processing method, and program - Google Patents

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program, and more particularly, to a technique suitable for use in correcting an image.

  When shooting a scene using a digital camera outdoors or the like, the brightness range of the shooting scene may be wider than the brightness range that can be shot. At this time, since the gradation information cannot be recorded for a subject outside the photographic brightness range, overexposure / blackout occurs. For example, when a person is photographed outdoors in fine weather, there are cases where the sky and clouds in the background are blown off and the shade of the trees is crushed black when exposure is adjusted to the person. For this reason, the impression when viewing the scene may be different from the impression when viewing the captured image, which is unsatisfactory for the user.

  Therefore, High Dynamic Range Imaging processing (hereinafter referred to as HDR processing) is known as one of techniques for solving such a problem. The HDR process is roughly divided into a dynamic range expansion process and a gradation compression process. The dynamic range expansion process is a process that expands the dynamic range that can be shot and records the gradation information of the brightness range where whiteout or blackout has occurred. As a general method, A group of images shot with different exposures (multi-stage exposure image group) is synthesized. Hereinafter, an image generated by this dynamic range expansion process is referred to as an HDR image. On the other hand, tone compression processing is image processing for preferably reproducing an HDR image with a wide dynamic range on a display / output device with a narrow dynamic range, and compresses the amplitude of the low-frequency component of the luminance of the HDR image. The method of doing is used. By performing the HDR processing in this way, it is possible to display or output a preferable image with reduced whiteout / blackout. Such HDR processing is disclosed in Patent Document 1, for example.

JP 2007-215073 A JP 2005-63406 A

E.H.Land, J.J.McCann, "Lightness and Retinex Theory", Journal of the Optical Society of America, Vol.61, Num1, January, 1971 F.Durand, J.Dorsey, "Fast Bilateral Filtering for the Display of High-Dynamic-Range Images", SIGGRAPH 2002

  Normally, the user takes many images at a time, and when a multi-stage exposure image group is synthesized by the above-described dynamic range expansion processing, the multi-stage exposure image group is selected from the many photographed image groups. There is a need. At this time, since the user selects the multi-stage exposure image group, there is a possibility that an appropriate image may not be selected. As a result, there is a problem that an HDR image different from the user's intention is generated.

  An object of the present invention is to make it possible to appropriately select a multistage exposed image group for performing HDR processing in view of the above-described problems.

The image processing apparatus of the present invention includes a selection means for selecting one of the captured images in response to the user's operation, based on the correlation between one photographing image selected by said selecting means, for generating a HDR image Extraction means for extracting a multi-stage exposure photographed image, display means for grouping and displaying one photographed image selected by the selection means and the multi-stage exposure photographed image extracted by the extraction means on the display unit, and the selection Dynamic range expansion processing means for generating the HDR image by performing dynamic range expansion processing on one captured image selected by the means and the multistage exposure captured image, and the HDR image generated by the dynamic range expansion processing means And a gradation compression processing means for performing gradation compression processing on the image.

  According to the present invention, it is possible to reduce selection mistakes in multistage exposure images. Thereby, dynamic range expansion processing and gradation compression processing can be performed on an appropriate multi-stage exposed image group, and a captured image with reduced whiteout and blackout can be acquired.

1 is a block diagram illustrating a configuration example of an image processing apparatus according to a first embodiment. 6 is a flowchart illustrating an example of a procedure of HDR processing according to the first embodiment. It is a figure which shows an example of the screen for setting an exposure space | interval. It is a figure which shows an example of the additional information of a picked-up image. It is a figure which shows an example of the screen as which the multistage exposure image group and the HRD process UI were displayed. It is a figure which shows an example of the data by which the 8-bit RGB value of all the pixels was shown. It is a flowchart which shows an example of the detailed procedure of step S206. It is a figure which shows the outline | summary of the procedure which produces HDR image data. It is a flowchart which shows an example of the detailed procedure of step S207. It is a figure which shows the flow of the parameter calculation in a gradation compression process. It is a flowchart which shows an example of the detailed procedure of step S905. It is a figure which shows the compression characteristic of a luminance illumination component. It is a flowchart which shows an example of the procedure of the HDR process of 2nd Embodiment. It is a flowchart which shows an example of the detailed procedure of step S1303. It is a figure which shows an example of the histogram of a pixel count. It is a block diagram which shows the structural example of the image processing apparatus which concerns on 3rd Embodiment. 14 is a flowchart illustrating an example of a procedure of HDR processing according to the third embodiment. It is a figure which shows an example of the warning screen to the effect that HDR composition cannot be performed. It is a flowchart which shows an example of the detailed procedure of step S1703.

(First embodiment)
Hereinafter, a first embodiment of the present invention will be described in detail with reference to the drawings.
FIG. 1 is a block diagram illustrating a configuration example of an image processing apparatus 100 according to the present embodiment.
In FIG. 1, an imaging unit 101 is for detecting light of a subject. The imaging unit 101 includes, for example, a zoom lens, a focus lens, a shake correction lens, a diaphragm, a shutter, an optical low-pass filter, an iR cut filter, a color filter, and a sensor such as a CMOS or a CCD. The A / D conversion unit 102 is for converting the detected light amount of the subject into a digital signal. The signal processing unit 103 performs signal processing on the digital signal converted by the A / D conversion unit 102 to generate digital image data, and performs, for example, gamma processing. The encoder unit 104 is for compressing data with respect to digital image data. For example, the encoder unit 104 performs compression according to the JPEG method.

  The media interface unit 105 is an interface for connecting to media such as a hard disk, a memory card, an SD card, and a USB memory. The CPU 106 is for controlling processing of each component of the entire image processing apparatus 100. The ROM 107 and the RAM 108 are provided to provide the CPU 106 with programs, data, work areas, and the like necessary for the processing of each configuration. Further, when a control program necessary for processing to be described later is stored in the ROM 107, it is executed after being temporarily expanded in the RAM 108.

  The operation unit 109 is for inputting an instruction from the user, and corresponds to, for example, a button or a mode dial. The character generation unit 110 is for generating characters and graphics. The D / A converter 111 is for converting digital image data into an analog signal. The display unit 112 is a device for displaying a captured image, an image such as a GUI, and is generally a CRT, a liquid crystal display, or the like. In addition, a known touch screen may be used, and in this case, input via the touch screen can be handled as input from the operation unit 109. The imaging unit controller 113 is for controlling the imaging system instructed by the CPU 106, and controls, for example, focusing, shutter opening / closing, aperture adjustment, and the like.

FIG. 2 is a flowchart illustrating a procedure of HDR processing executed by the image processing apparatus 100 according to the present embodiment. 2 is performed under the control of the CPU 106.
When the user presses a shooting start button (not shown), the process starts. In step S201, the imaging unit 101 performs auto bracket shooting at an exposure interval selected by the user according to an instruction from the CPU. Here, as a procedure for setting the exposure interval, first, a shooting menu UI is displayed on the display unit 112 as shown in FIG. When the “auto bracket shooting” menu is selected by the user's selection, a UI for setting the exposure interval (exposure width) of bracket shooting shown in FIG. 3 is displayed on the right side of the screen. The exposure interval set from these exposure intervals is selected by the user.

  In step S201, the fact that the CPU 106 has performed auto bracket shooting is described in the additional information of the shot image. Here, as shown in FIG. 4, the additional information is data including information related to shooting such as shooting date / time and exposure time, for example, data such as Exif format which is a known image data format. At the time of auto bracket shooting, the auto bracket flag is “ON”, and information on the number of auto bracket shooting is held as a bracket group number. Also, the same bracket number is assigned to these captured images.

  Next, in step S202, from the shooting menu UI displayed on the left side of the screen of the display unit 112 as shown in FIG. 5A, the “display shot image” menu is selected via the operation unit 109 according to a user instruction. It is determined whether or not it has been done. If it is not selected, it waits as it is. When the “display photographed image” menu is selected, in step S203, the CPU 106 functions as a detection unit, and reads information about the auto bracket flag and the group number from the additional information described above. Then, the CPU 106 extracts the images having the same bracket group number having the correlation with the auto bracket flag being “ON”, and displays the images as a grouped multi-exposure image group on the display unit 112.

  Next, in step S204, it is determined whether or not an HDR processing parameter to be set through the operation unit 109 is selected from the parameters illustrated in FIG. The HDR processing UI shown in FIG. 5B is “HDR processing” for the grouped multi-stage exposure image group described above on the screen shown in FIG. Is displayed when is selected. If the HDR processing parameter is not selected, the process waits as it is. If the HDR processing parameter is selected, the selected HDR processing parameter is stored in the RAM 108 in step S205. Next, in step S206, the HDR image data is generated by synthesizing the images of the multistage exposed image group. The generated HDR image data is data in which 8-bit RGB values of all the pixels are shown as in the image data shown in FIG. Details of step S206 will be described later.

  Next, in step S207, tone compression processing is performed on the generated HDR image data based on the HDR processing parameters stored in the RAM 108. Details of step S207 will be described later. In step S208, the HDR image subjected to the gradation compression process is displayed on the display unit 112, and the HDR image data is recorded on a medium or the like via the media interface unit 105, and the process ends.

Next, details of the process executed in step S206 of FIG. 2 will be described.
FIG. 7 is a flowchart illustrating an example of a processing procedure for generating HDR image data executed in step S206. Each process shown in FIG. 7 is performed by the CPU 106. In step S206, as shown in FIG. 8, the image with much exposure is multiplied by the synthesis gain, and the images with low exposure are sequentially synthesized.
First, in step S701, information on the exposure time ET is read from the additional information of each image in the multistage exposed image group. Next, in step S702, the maximum value ETmax in these exposure times ET is detected.

  Next, in step S703, the RGB value of pixel number 1 is read for the image with the longest exposure time ET. In step S704, it is determined whether or not the read RGB value is a pixel value that does not have whiteout or blackout (a pixel value within an effective range). As a result of the determination, if the pixel value is whiteout / blackout, the process proceeds to step S705. If the pixel value is not whiteout / blackout, the process proceeds to step S709. Here, the pixel values that are not overexposure / blackout are pixel values that satisfy 0 <RGB <255.

  Next, in step S705, the RGB value of the same pixel number is read out for an image whose exposure time ET is one step smaller. In step S706, it is determined whether or not the read RGB value is a pixel value without whiteout or blackout. As a result of the determination, if the pixel value is whiteout / blackout, the process returns to step S705. If the pixel value is not whiteout / blackout, the process proceeds to step S707. Then, in step S707, the exposure time ET and the exposure time maximum value ETmax of the image (hereinafter referred to as a selected image) determined as a pixel value without whiteout / blackout in step S706 according to the following equation (1). Calculate the exposure time ratio ETR.

Next, in step S708, using the RGB value of the selected image and the calculated exposure time ratio ETR, the RGB value (RGB _HDR ) after combining the HDR image is calculated according to the following equation (2).

In step S709, the calculated RGB value (RGB _HDR ) of the HDR image is stored in the RAM 108. In step S710, it is determined whether or not the RGB values (RGB _HDR ) after combining HDR images are stored in the RAM 108 for all pixels. As a result of this determination, when the RGB values (RGB _HDR ) after synthesizing the HDR image are stored for all the pixels, the processing is terminated. On the other hand, if the result of determination in step S710 is that there are pixels not yet stored in the RAM 108, processing proceeds to step S711, and the RGB value of the next pixel number is read for the image with the longest exposure time, and processing returns to step S704.

Next, details of the processing executed in step S207 will be described.
FIG. 9 is a flowchart showing an example of the gradation compression processing procedure executed in step S207. Note that each process shown in FIG. 8 is performed by the CPU 106. FIG. 10 is a diagram showing a flow of parameter calculation in the gradation compression process. In step S207, tone compression is performed using the Retinex model proposed by Land et al. Described in Non-Patent Document 1, for example. Tone compression processing using the Retinex model separates the luminance component of a real scene into an illumination component and a reflectance component, and assumes that the illumination component is insensitive to human vision and compresses only the illumination component That's it. Note that the luminance illumination component is acquired by performing low-pass filter processing on the HDR image.

First, in step S901, the luminance component Y _HDR is calculated from the RGB value (RGB _HDR ) of the HDR image according to the following equation (3).

  Next, in step S902, a Gaussian filter Filter (a, b) is calculated according to the following equation (4). In the present embodiment, the variance S of the Gaussian filter Filter (a, b) is set to 1/5 the number of pixels of the image width W, and the filter processing range includes approximately 95% of the integral value of the Gaussian function− S to S.

Next, in step S903, according to the equation shown in Expression 5 below, it performs a discrete convolution operation between the luminance component Y _HDR and Gaussian filter Filter (a, b), calculates the luminance illumination component Y _Illum _ _HDR.

Next, in step S904, the luminance reflectance component Y _Refrectance is calculated according to the following equation (6).

Next, in step S905, the calculated brightness illumination component Y _Illum _ _HDR compressed in step S903, it calculates the luminance illumination component Y _Illum _ _LDR after compression. Details of step S905 will be described later. Next, in step S906, according to the equation shown in the following Equation 7, it combines the luminance illumination component Y _Illum _ _LDR and the luminance reflectance component Y _Refrectance after compression, to calculate the luminance component Y _LDR.

Next, in step S907, the RGB values (RGB _HDR ) of the HDR image are adjusted using the luminance components Y _HDR and Y _LDR before and after the gradation compression process according to the following equation (8), and the gradation compression process is performed. The later RGB value (RGB _LDR ) is calculated.

Then, calculated in step S908, according to the equation shown in the following numbers 9, RGB values after gradation compression processing to (RGB _LDR) performs a camera gamma processing, the camera gamma processed RGB values (RGB _LDR _ _g) And stored in the RAM 108. Then, the process ends. As shown in FIG. 10, it compresses only luminance illumination component, calculates the RGB values of the HDR image from (RGB _HDR) camera gamma processing after the RGB values (RGB _LDR _ _g).

Next, details of the processing executed in step S905 will be described.
Figure 11 is a flowchart illustrating an example of the compression processing procedure of the luminance illumination component Y _Illum _ _HDR executed in step S905. Each process illustrated in FIG. 11 is performed by the CPU 106. FIG. 12A is a diagram showing the compression characteristics of the luminance illumination component in each of the cases “correct blackout”, “correct whiteout”, and “correct blackout and whiteout”.
First, in step S1101, the HDR processing parameters (reference brightness Y _{Ref of} selected image, correction method, correction strength) are read from the RAM 108. Then, in step S1102, detects a maximum value Y _Illum _ _MAX from the calculated at the step S903 the luminance illumination component Y _Illum _ _HDR.

Next, in step S1103, it determines whether reference luminance Y _Ref is greater than the luminance illumination component Y _Illum _ _HDR read out in step S1101. If the result of this determination is that it is smaller than the reference brightness Y _Ref , the process proceeds to step S1104, and if it is greater than or equal to the reference brightness Y _Ref , the process proceeds to step S1105.

In step S1104, compressing the luminance illumination component Y _Illum _ _HDR according having 10 and Table 1 below, calculates the luminance illumination component Y _Illum _ _LDR after compression. Note that the SCD in the equation shown in Equation 10 is a shadow correction degree (Shadow Correction Degree), and is corrected with a compression characteristic as shown in FIG. 12B according to the set black shadow correction strength. I do.

On the other hand, in step S1105, compressing the luminance illumination component Y _Illum _ _HDR according to the formula shown in the following Equation 11 to calculate the luminance illumination component Y _Illum _ _LDR after compression. Here, the HCD in the equation shown in Expression 11 is the highlight correction degree (Highlight Correction Degree), and has a compression characteristic as shown in FIG. 12 (c) according to the set whiteout correction strength. Make corrections.

  As described above, according to the present embodiment, the auto bracket flag is “ON”, images with the same bracket group number are set as multi-stage exposure images, and multi-stage exposure image groups are grouped and displayed on the display unit 112. did. Thereby, the user can confirm an appropriate multistage exposure image group, and can reduce selection mistakes of the multistage exposure image. Therefore, dynamic range expansion processing and gradation compression processing can be performed on an appropriate multistage exposure image group, and a captured image with reduced whiteout and blackout can be acquired.

  In the present embodiment, as shown in FIG. 5A, the example in which the multistage exposed image group is grouped and a plurality of groups is displayed has been described. On the other hand, before grouping, the user may select an image captured by one auto bracket, and only the multi-stage exposed image group of the selected image may be grouped and displayed.

  In the present embodiment, the 8-bit RGB values of all the pixels are recorded as image data in the image file. However, the data type and format are not limited to this. For example, the recorded RGB values may be 16 bits. Further, in the present embodiment, the image having a large amount of exposure is multiplied by the composite gain, and the image is sequentially combined with the image having a small amount of exposure. However, a method of combining images having different exposures using the composite gain. For example, it is not limited to this. For example, an image with little exposure may be multiplied by a synthesis gain and sequentially synthesized with an image with much exposure.

Furthermore, the gradation compression processing in the present embodiment compresses the low frequency component of the image extracted by the Gaussian filter. However, the present invention is not limited to this if it compresses the low frequency component of the image based on the Retinex model. is not. For example, as described in Non-Patent Document 2, a frequency component extracted using a bilateral filter may be compressed. Further, although the reference luminance Y _Ref is maintained before and after the gradation compression process, the present invention is not limited to this. For example, a main subject such as a face may be detected from the image, and the detected luminance Y _{face of the} main subject may be corrected so as to be maintained before and after the gradation compression process. In this case, the reference luminance Y _Ref in the equations shown in Equations 10 and 11 is replaced with the detected luminance Y _face of the main subject. A method for detecting a main subject is described in Patent Document 2, for example.

(Second Embodiment)
Hereinafter, a second embodiment of the present invention will be described in detail with reference to the drawings. In the first embodiment, an example in which auto bracket shooting is performed has been described. However, in the present embodiment, an example in which a captured image generated by normal shooting is used will be described. In the present embodiment, a correlated captured image is used for HDR processing, as determined from the capturing conditions of each captured image without a bracket group number. Note that the configuration of the image processing apparatus in the present embodiment is the same as that in the first embodiment, and a description thereof will be omitted.

FIG. 13 is a flowchart illustrating an example of a procedure of HDR processing executed by the image processing apparatus 100 according to the present embodiment. Each process shown in FIG. 13 is performed under the control of the CPU 106.
When the user presses a shooting start button (not shown), the process starts. In step S1301, the imaging unit 101 performs shooting while changing the exposure time for each shooting in accordance with a user instruction. Note that step S1302 is the same as step S202 of FIG. 2 in the first embodiment.

  If the “display photographed image” menu is selected as a result of the determination in step S1302, the picked-up multistage exposure image group is grouped and displayed on the display unit 112 in step S1303. Details of step S1303 will be described later. Steps S1304 to S1308 are the same as steps S204 to S208 in FIG. 2 of the first embodiment, and thus description thereof is omitted.

Next, details of the processing executed in step S1303 of FIG. 13 will be described. FIG. 14 is a flowchart illustrating an example of a processing procedure for displaying the multi-stage exposure image group executed in step S1303. Each process illustrated in FIG. 14 is performed by the CPU 106.
First, in step S1401, arbitrary two pieces of image data are read from the RAM 108 from the captured image group held. In step S1402, it is determined whether additional information at the time of shooting as shown in FIG. 4 exists in the two read image data. As a result of this determination, if additional information exists, the process proceeds to step S1403. If additional information does not exist, the process proceeds to step S1405.

  Next, in step S1403, the additional information is referred to, the shooting time is approximately the same time (within 1 minute difference), and the image size such as the shooting location, the image width, and the image height, the lens aperture value, and the ISO sensitivity are set. It is determined whether or not the exposure time is different. As a result of this determination, if the condition is satisfied, the process proceeds to step S1404. If the condition is not satisfied, the process proceeds to step S1407. In step S1404, the same bracket group number is added to the additional information of both image data. If a bracket group number has already been added to any one of the image data, the same bracket group number is added to the image data for which no bracket group number is described.

On the other hand, in step S1405, a histogram of pixel distribution as shown in FIG. 15 is calculated from the two image data, and their correlation coefficient C is calculated as a comparison target according to the following equation (12). Note that Hist1 and Hist2 in the equation shown in Expression 12 indicate histograms of two read image data, and Hist1 _ave and Hist1 _ave indicate average values of histograms of both image data. Then, it is determined whether or not the calculated correlation coefficient C is 0.9 or more. As a result of this determination, if the correlation coefficient C is 0.9 or more, the process proceeds to step S1406, and if the correlation coefficient is less than 0.9, the process proceeds to step S1407.

  Next, in step S1406, additional information of both image data is newly created, the bracket shooting flag is turned ON, the same bracket group number is added, and other additional information is not added. When a bracket group number is already added to any one of the image data, the same bracket group number is added to the image data not including the bracket group number. In step S1407, it is determined whether or not the processing has been completed for all the two combinations in the captured image group. As a result of this determination, if the processing has been completed for all combinations, the process proceeds to step S1408. If the process has not been completed, the process returns to step S1401. Next, in step S1408, as shown in FIG. 5A, image groups having the same bracket group number are grouped and displayed, and the process ends.

  As described above, according to the present embodiment, the correlation is detected from the additional information of the captured image or the pixel distribution of the captured image, and the correlated captured images are grouped and displayed on the display unit 112 as a multistage exposed image group. I tried to do it. Thereby, the user can confirm an appropriate multistage exposure image group, and can reduce selection mistakes of the multistage exposure image. Therefore, dynamic range expansion processing and gradation compression processing can be performed on an appropriate multistage exposure image group, and a captured image with reduced whiteout and blackout can be acquired.

  In the present embodiment, as shown in FIG. 5A, the example in which the multistage exposed image group is grouped and a plurality of groups is displayed has been described. On the other hand, similarly to the first embodiment, before grouping, the user may select one photographed image and display only a group of multi-stage exposed images of the selected image.

(Third embodiment)
Hereinafter, a third embodiment according to the present invention will be described in detail with reference to the drawings. In the second embodiment, a correlated captured image is determined and assigned the same bracket group number, and the apparatus side selects an image used for HDR processing. On the other hand, in the present embodiment, it is determined whether or not the captured image selected by the user is suitable for HDR processing, and a warning is displayed when a captured image that is not suitable is included. An example will be described.

FIG. 16 is a block diagram illustrating a configuration example of the image processing apparatus 300 according to the present embodiment.
In FIG. 16, an input unit 301 is used to input an instruction from the user and data, and includes, for example, a keyboard and a pointing device. Note that examples of the pointing device include a mouse, a trackball, a trackpad, and a tablet.

  The data storage unit 302 is a recording medium for holding image data, such as a hard disk, a flexible disk, a CD-ROM, a CD-R, a DVD, a memory card, a smart media SD card, a USB memory, or the like. The data storage unit 302 can store programs and other data in addition to image data. Further, a part of the RAM 306 described later may be used as the data storage unit 302. Furthermore, the form which utilizes the data storage part of the apparatus of the other side connected by the communication part 307 mentioned later via the communication part 307 may be sufficient.

  The display unit 303 is a device that displays an image before or after image processing, or displays an image such as a GUI, and generally uses a CRT, a liquid crystal display, or the like. The display unit 303 may be a display device external to the device connected by a cable or the like, or a known touch screen. In this case, the input via the touch screen can be handled as the input of the input unit 301.

  The CPU 304 is for controlling processing of each component of the entire image processing apparatus 300. The ROM 305 and the RAM 306 are provided for providing the CPU 304 with programs, data, work areas, and the like necessary for the processing. Further, when a control program necessary for processing to be described later is stored in the data storage unit 302 or the ROM 305, the control program is temporarily expanded in the RAM 306 and executed. When the apparatus receives a program via the communication unit 307, the program is recorded in the data storage unit 302 and then temporarily expanded in the RAM 306, or directly read from the communication unit 307 to the RAM 306 and executed. .

  The communication unit 307 is an I / F for performing communication between devices. For example, the communication unit 307 is a wired communication method such as a well-known Ethernet (registered trademark), USB, IEEE1284, IEEE1394, or telephone line. Further, a wireless communication method such as infrared (IrDA), IEEE802.11a, IEEE802.11b, IEEE802.11g, Bluetooth, UWB (Ultra Wide Band) may be used. Note that FIG. 16 shows a configuration in which the input unit 301, the data storage unit 302, and the display unit 303 are all included in one device, but these configurations are connected via a communication path using a known communication method. It may be configured as such as a whole. In addition to the above, there are various components of the system configuration, but since these are not the main points of the present invention, description thereof will be omitted.

FIG. 17 is a flowchart illustrating an example of a procedure of HDR processing executed by the image processing apparatus 300 according to the present embodiment. Each process shown in FIG. 17 is performed under the control of the CPU 304.
First, when the user receives an instruction to display a captured image from the input unit 301, the process is started. In step S1701, a group of captured images (for example, thumbnail images) is displayed on the display unit 303. In step S1702, a multi-stage exposure image group is selected from the captured image group displayed by the user via the input unit 301, and whether or not the selected multi-stage exposure image group is suitable for HDR synthesis. judge. As a result of this determination, if it is suitable for HDR synthesis, the process proceeds to step S1704, and if it is not suitable for HDR synthesis, the process proceeds to step S1703. Details of step S1702 will be described later.

  In step S1703, for example, a warning message as shown in FIG. 18 is displayed on the display unit 303, and the process returns to step S1701. On the other hand, in step S1704, the multistage exposed image group is grouped and displayed on the display unit 303. Steps S1705 to S1708 are the same as steps S203 to S206 in FIG. 2 described in the first embodiment, and thus the description thereof is omitted.

FIG. 19 is a flowchart illustrating an example of a processing procedure for generating a determination criterion for determination performed in step S1702. Each process shown in FIG. 19 is performed under the control of the CPU 304.
First, in step S1901, arbitrary two pieces of image data are read from the data storage unit 302 from the multistage exposed image group selected by the user. Note that steps S1902 to S1907 are the same as steps S1402 to S1407 described in the second embodiment, and a description thereof will be omitted. Further, in the present embodiment, unlike the second embodiment, only the image selected by the user is targeted.

  In step S1908, the bracket group numbers of all the multistage exposure images selected by the user are confirmed, and the process ends. As described above, the bracket group number is confirmed, and if at least one bracket group number is different or does not exist in the multistage exposed image group, the process proceeds to step S1703. If all the multistage exposure images have the same bracket group number, the process advances to step S1704.

  As described above, according to the present embodiment, the correlation is detected from the additional information of the captured image selected by the user as the multistage exposed image or the pixel distribution of the captured image. If all selected photographed images have a correlation, the photographed images are grouped as a multi-stage exposed image group and displayed on the display unit 112. Otherwise, a warning screen is displayed. Thereby, the user can confirm whether it is an appropriate multistage exposure image group, and can reduce selection mistakes of a multistage exposure image. Therefore, dynamic range expansion processing and gradation compression processing can be performed on an appropriate multistage exposure image group, and a captured image with reduced whiteout and blackout can be acquired.

(Other embodiments)
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

106 CPU, 112 Display unit

Claims (8)

Selection means for selecting one captured image in response to a user operation;
Extracting means for extracting a multi-exposure shooting image for based on the correlation between one photographing image selected by said selecting means, for generating a HDR image,
Display means for grouping and displaying one photographed image selected by the selecting means and the multistage exposure photographed image extracted by the extracting means on the display unit;
Dynamic range expansion processing means for performing dynamic range expansion processing on the one captured image selected by the selection means and the multistage exposure captured image to generate the HDR image;
An image processing apparatus comprising: gradation compression processing means for performing gradation compression processing on the HDR image generated by the dynamic range expansion processing means. The image processing apparatus according to claim 1, wherein the extraction unit determines the correlation based on additional information of the captured image. The image processing apparatus according to claim 2, wherein the extracting unit determines the correlation based on bracket shooting information included in the additional information of the captured image. 3. The correlation according to claim 2, wherein the extracting unit determines the correlation based on a shooting time, ISO sensitivity, lens aperture, exposure time, image size, and shooting location included in the additional information of the shot image. The image processing apparatus described. 5. The image processing according to claim 2, wherein the extraction unit determines the correlation based on the comparison of the captured images when no additional information is present in the captured image. 6. apparatus. The image processing apparatus according to claim 5, wherein the extraction unit determines the correlation based on a pixel distribution of the photographed image. A selection step of selecting one captured image in response to a user operation;
Based on the correlation between one photographing image selected by said selecting step, an extraction step of extracting a multistage exposure shooting image for generating a HDR image,
A display step of grouping and displaying one captured image selected in the selection step and the multistage exposure captured image extracted in the extraction step on a display unit;
A dynamic range expansion process step of generating a HDR image by performing a dynamic range expansion process on the one captured image selected in the selection step and the multi-stage exposure captured image;
A gradation compression processing step of performing gradation compression processing on the HDR image generated by the dynamic range expansion processing step. A selection step of selecting one captured image in response to a user operation;
Based on the correlation between one photographing image selected by said selecting step, an extraction step of extracting a multistage exposure shooting image for generating a HDR image,
A display step of grouping and displaying one captured image selected in the selection step and the multistage exposure captured image extracted in the extraction step on a display unit;
A dynamic range expansion process step of generating a HDR image by performing a dynamic range expansion process on the one captured image selected in the selection step and the multi-stage exposure captured image;
A program causing a computer to execute a gradation compression processing step of performing gradation compression processing on an HDR image generated by the dynamic range expansion processing step.

Images