JP2018198456A - Image processing program, image processing apparatus, and imaging apparatus - Google Patents

Abstract

To prevent unnecessary correction of flare to a saturated region where the flare does not occur.SOLUTION: An image processing program causes a computer to perform steps of: acquiring an input image generated by capturing an image; generating a processed image by threshold processing of the input image in a range between a threshold upper limit and a threshold lower limit; and detecting flare on the basis of the processed image.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image processing program, an image processing apparatus, and an imaging apparatus.

  When shooting with an electronic camera, if an extremely bright object (bright spot) is present in the shooting range, flare may occur around the bright spot image on the shot image. For example, a technique for estimating a flare component based on pre-stored imaging characteristic data of a photographing optical system and the position of a bright spot on the photographed image and removing the estimated flare component from the photographed image has been conventionally known. It has been.

Japanese Patent No. 4250513

  In the conventional technique, correction is performed by regarding a saturated region on a photographed image as a light source in which flare occurs. For this reason, in the related art, unnecessary correction is performed even in a saturated region where flare does not actually occur, and thus there is a possibility that a sense of incongruity may occur in the corrected image.

  An image processing program according to one aspect includes: a procedure for acquiring an input image generated by imaging; a procedure for generating a processed image by performing threshold processing on the input image within a range of a threshold upper limit and a threshold lower limit; and And causing the computer to execute a procedure for detecting the flare based on the information.

  An image processing apparatus according to one aspect includes an acquisition unit that acquires an input image generated by imaging, a generation unit that generates a processed image by performing threshold processing on the input image in a range of a threshold upper limit and a threshold lower limit, and the processing A detection unit that detects flare based on the image.

  An imaging apparatus according to one aspect includes an imaging unit that captures an image of an imaging optical system, and the image processing apparatus described above.

  According to the image processing program, the image processing apparatus, and the imaging apparatus of the present disclosure, it is possible to prevent unnecessary flare correction for a saturated region where no flare occurs.

The figure which shows the structural example of the computer of 1st Embodiment. A flow chart showing an example of operation of a computer in a 1st embodiment Diagram showing an example of an input image The figure which shows the state which extracted the saturation area | region from FIG. (A)-(f): The figure which shows the example of the binarized image corresponding to FIG. (A)-(f): The figure which shows the example of the 6th binarized image from the 1st binarized image corresponding to FIG. The figure which shows an example of the frequency distribution corresponding to the light source which the flare has generate | occur | produced The figure which shows an example of the frequency distribution corresponding to the light source which flare does not generate | occur | produce Diagram showing how to determine the sharpness of frequency distribution The figure which shows the structural example of the electronic camera of 2nd Embodiment. Flow chart showing an operation example of the electronic camera in the second embodiment

<Description of First Embodiment>
FIG. 1 is a diagram illustrating a configuration example of a computer according to the first embodiment. The computer according to the first embodiment executes flare correction processing on a captured image obtained by capturing an image of the imaging optical system by executing an image processing program.

  A computer 101 illustrated in FIG. 1 includes a data reading unit 102, a storage device 103, a CPU (Central Processing Unit) 104, a memory 105, an input / output interface (I / F) 106, and a bus 107. The data reading unit 102, the storage device 103, the CPU 104, the memory 105, and the input / output I / F 106 are connected to each other via a bus 107. Furthermore, an input device 108 (keyboard, pointing device, etc.) and a monitor 109 are connected to the computer 101 via an input / output I / F 106. The input / output I / F 106 receives various inputs from the input device 108 and outputs display data to the monitor 109.

  The data reading unit 102 is an example of an acquisition unit, and acquires captured image data and an image processing program from the outside. The data reading unit 102 communicates with, for example, a reading device that acquires data from a removable storage medium (such as an optical disk, a magnetic disk, or a magneto-optical disk reading device) or an external device in accordance with a known communication standard. A communication device to perform (USB interface, LAN module, wireless LAN module, etc.).

  The storage device 103 is a storage medium such as a hard disk or a nonvolatile semiconductor memory. The storage device 103 stores the above-described image processing program and various types of information necessary for executing the image processing program. Note that the storage device 103 can store data of a captured image read from the data reading unit 102.

  The CPU 104 is a processor that comprehensively controls each unit of the computer 101. The CPU 104 functions as the image processing unit 200 that is an example of the image processing apparatus disclosed herein by executing the image processing program. The image processing unit 200 includes an image generation unit 201, an annular zone detection unit 202, a light source estimation unit 203, and a flare correction unit 204. The operations of the image processing unit 200, the image generation unit 201, the annular zone detection unit 202, the light source estimation unit 203, and the flare correction unit 204 will be described later.

  Note that each functional block (201 to 204) of the image processing unit 200 shown in FIG. 1 is realized by a program loaded in a memory in terms of software. However, some or all of the functional blocks of the image processing unit 200 may be realized in hardware by a combination of a processor, a memory, and an electronic circuit.

  The memory 105 temporarily stores calculation results and the like in the image processing program. The memory 105 is a volatile semiconductor memory, for example.

  Hereinafter, an exemplary operation of the computer according to the first embodiment will be described with reference to the flowchart of FIG. The processing in FIG. 2 is started when the CPU 104 executes the image processing program in response to a program execution instruction from the user.

  In step S <b> 11, the image processing unit 200 acquires data of a captured image (hereinafter also referred to as an input image) to be processed from the outside via the data reading unit 102. The input image data acquired in step S <b> 11 is recorded in the storage device 103 or the memory 105 under the control of the image processing unit 200. Note that the image processing unit 200 may omit the process of step S11 when the data of the input image is stored in the storage device 13 in advance. FIG. 3 shows an example of the input image.

  As an example, the input image of the first embodiment is a color image in an RGB color space obtained by photographing (capturing) a subject image of the photographing optical system with an electronic camera. Further, the input image of the first embodiment is a photographed image by a photographing optical system having a diffractive optical element (DOE: Diffractive Optical Element). When photographing with a photographing optical system having a diffractive optical element, if an extremely bright object (bright spot) exists in the photographing range, a flare (color flare) with color blur around the bright spot image on the photographed image Will occur. The color flare derived from the diffractive optical element is remarkably unnatural as compared with the colorless flare, and thus it is desired to be corrected by the flare correction process.

  In step S12, the image processing unit 200 determines whether or not it is a flare correction mode in which a flare correction process is performed on the input image. As an example, the image processing unit 200 determines that the flare correction mode is set when the user turns on the setting of the flare correction mode. Further, the image processing unit 200 refers to the shooting information added to the header area of the input image, and determines that the flare correction mode is set when the input image is shot with a shooting optical system including a diffractive optical element. To do.

  If it is the flare correction mode (affirmative determination in step S12), the process proceeds to step S13. On the other hand, when the mode is not the flare correction mode (negative determination in step S12), the process proceeds to step S24.

  In step S13, the image processing unit 200 generates a luminance image indicating the luminance component of the input image. For example, the image processing unit 200 may convert the input image into a YCbCr color space and use the Y channel of the input image as a luminance image. Alternatively, the image processing unit 200 may generate a luminance image by obtaining an average ((R + G + B) / 3) of RGB values for each pixel of the input image.

  In step S <b> 14, the image processing unit 200 detects a saturated pixel whose pixel value has reached a saturation level in the luminance image. Then, the image processing unit 200 performs a labeling process for obtaining a group of detected saturated pixels. Hereinafter, a group of saturated pixels after the labeling process may be referred to as a saturated region. In the saturation region, there is a possibility that not only the light source in which the flare is generated but also the light source unrelated to the flare is mixed. FIG. 4 shows a state where the saturation region is extracted from FIG.

  In step S15, the image processing unit 200 numbers each saturated region existing in the luminance image in order from the zero value. Hereinafter, the variable indicating the saturation region number is i (where i is a positive integer including 0). By the above numbering, the image processing unit 200 acquires the total number imax of saturated regions in the luminance image. Further, the image processing unit 200 acquires coordinates (Xi, Yi) indicating the position in the luminance image for each saturated region i. For example, the image processing unit 200 acquires the coordinates of the center of gravity of the saturated region i as the above coordinates.

  In step S16, the image generation unit 201 binarizes the luminance image with different threshold values, and generates a plurality of binarized images. At this time, the image generation unit 201 generates a plurality of binarized images by extracting a plurality of types of pixel value ranges by changing the combination of the threshold upper limit and the threshold lower limit.

  As an example, when the gradation of the luminance image is 8 bits (0 to 255), the image generation unit 201 generates six binarized images with the following settings.

  First, the image generation unit 201 binarizes the luminance image depending on whether or not the pixel value falls within the range of pixel values 210 to 250, and generates a first binarized image. In addition, the image generation unit 201 binarizes the luminance image depending on whether or not the pixel is within the range of the pixel values 170 to 210, and generates a second binarized image. In addition, the image generation unit 201 binarizes the luminance image depending on whether or not the pixel is within the range of the pixel values 130 to 170, and generates a third binarized image. In addition, the image generation unit 201 binarizes the luminance image depending on whether or not the pixel is in the range of the pixel values 90 to 130, and generates a fourth binarized image. Further, the image generation unit 201 binarizes the luminance image depending on whether or not it is a pixel included in the range of the pixel values 50 to 90, and generates a fifth binarized image. Further, the image generation unit 201 binarizes the luminance image depending on whether or not the pixel is in the range of the pixel values 30 to 50, and generates a sixth binarized image. Note that the number of binarized images generated and the above-described threshold range may be changed as appropriate.

  FIGS. 5A to 5F show examples of binarized images corresponding to FIG. FIG. 5A is a binarized image with the pixel value 250 as a threshold, and FIG. 5B is a binarized image with the pixel value 210 as a threshold. FIG. 5C shows a binarized image with the pixel value 170 as a threshold, and FIG. 5D shows a binarized image with the pixel value 130 as a threshold. FIG. 5E shows a binarized image with a pixel value 90 as a threshold, and FIG. 5F shows a binarized image with a pixel value 50 as a threshold. In FIGS. 5A to 5F, portions where the pixel value is greater than or equal to the threshold value are indicated by white pixels, and portions where the pixel value is less than the threshold value are indicated by black pixels.

  5A to 5F, it can be seen that a circular pattern is generated in the binarized image around the light source where the flare is generated. The circular pattern has an image structure that spreads outward while gradually decreasing the luminance value around the light source. Here, when a difference between two different binarized images is taken, the image structure by the flare is expressed as a concentric image structure (annular zone) centered on the light source. In other words, if a binarized image is generated by extracting a certain band of pixel values defined by the threshold upper limit and the threshold lower limit, the occurrence of flare can be determined using the above-described annular zone. Therefore, when a binarized image obtained by extracting a certain band of pixel values is used, it is easy to estimate a light source in which flare occurs.

  FIGS. 6A to 6F show examples of the first to sixth binarized images corresponding to FIG. 3, respectively. In FIGS. 6A to 6F, a portion where the pixel value is within the threshold value range is indicated by a white pixel, and a portion where the pixel value is outside the threshold value range is indicated by a black pixel. The first binarized image shown in FIG. 6 (a) corresponds to the difference between FIG. 5 (a) and FIG. 5 (b). The second binarized image shown in FIG. 6 (b) corresponds to the difference between FIG. 5 (b) and FIG. 5 (c). The third binarized image shown in FIG. 6 (c) corresponds to the difference between FIG. 5 (c) and FIG. 5 (d). The fourth binarized image shown in FIG. 6 (d) corresponds to the difference between FIG. 5 (d) and FIG. 5 (e). The fifth binarized image shown in FIG. 6 (e) corresponds to the difference between FIG. 5 (e) and FIG. 5 (f). The sixth binarized image shown in FIG. 6 (f) corresponds to the difference between FIG. 5 (f) and the binarized image with the pixel value 30 as a threshold value. According to FIGS. 6A to 6F, it can be seen that an annular zone is formed around the light source where the flare is generated.

  In step S <b> 17, the image processing unit 200 initializes a counter k indicating the number of the light source for which the occurrence of flare is determined to a zero value (k = 0). For example, when k = i, the image processing unit 200 determines the occurrence of flare for the saturated region to which the number i is assigned in step S15.

  In step S <b> 18, the annular zone detection unit 202 detects an annular zone from a plurality of binarized images with the saturated region k corresponding to the value of the counter k as a determination target. As an example, the annular zone detection unit 202 detects an annular zone in each of the first binarized image to the sixth binarized image by the following procedures (A) and (B).

  (A) First, the annular zone detection unit 202 calculates a frequency distribution indicating the correlation between the distance from the saturation region k to be determined and the number of extracted pixels extracted from the binarized image in each binarized image. Ask.

  For example, the annular zone detection unit 202 sets the coordinates of the saturation region acquired in step S15 as the reference point (Xk, Yk) for the saturation region k to be determined. Then, the annular zone detection unit 202 counts the white pixels of the binarized image in association with the distance from the reference point. Thereby, the annular zone detection unit 202 can obtain a frequency distribution indicating a correlation between the distance from the reference point and the number of white pixels. Note that the annular zone detection unit 202 obtains the above-described frequency distribution for each of the first binarized image to the sixth binarized image for one saturation region.

  FIG. 7 is an example of a frequency distribution corresponding to a light source in which flare occurs. FIG. 8 is an example of a frequency distribution corresponding to a light source in which no flare occurs. The vertical axis in FIGS. 7 and 8 indicates the number of white pixels, and the horizontal axis in FIGS. 7 and 8 indicates the distance from the reference point in the saturation region. 7 and 8 collectively show six frequency distributions obtained from different binarized images for the same light source.

  In the case of a light source in which flare occurs, an annular zone appears around the light source in each binarized image as described above. Thereby, in the frequency distribution diagram (FIG. 7) of the light source in which the flare is generated, the peak of the white pixel appears clearly on the graph according to the position of the annular zone.

  As described above, the flare image structure spreads outward while gradually decreasing the luminance value around the light source. Therefore, in a binarized image in which a range having a relatively high pixel value is extracted, the position of the annular zone approaches the light source, and thus the distance at which the white pixel peak appears on the frequency distribution graph is close to the reference point. On the contrary, in the binarized image in which the range having a relatively low pixel value is extracted, the position of the annular zone is far from the light source, so compared to the binarized image in which the range having a relatively high pixel value is extracted, The distance at which the white pixel peak appears on the frequency distribution graph is separated from the reference point.

  As a comparative example of FIG. 7, in the frequency distribution diagram of the light source in which no flare occurs (FIG. 8), no clear peak appears in the number of white pixels in any of the plurality of binarized images, and each frequency distribution It can be seen that this graph is flat.

  (B) Next, the annular zone detection unit 202 detects the annular zone based on the peak of the number of extracted pixels in the frequency distribution obtained in the procedure (A) and the width of the peak section of the extracted pixel.

For example, the annular zone detection unit 202 obtains an evaluation value EV indicating the degree of sharpness of the graph for each frequency distribution by the following equation (1). The annular zone detection unit 202 determines that the annular zone has been detected from the binarized image when the height of the evaluation value EV exceeds a predetermined threshold value.
EV = peak / width% (1)
The above “peak” is a value indicating the peak intensity in the frequency distribution (for example, the maximum value of white pixels in the frequency distribution or the number of white pixels at the position where the center of gravity is near the maximum value) (FIG. 9). The value of peak increases as the number of white pixels increases at the peak position.

  Further, “width%” is a ratio (β / α) occupied by the peak section of white pixels (indicated by “β” in FIG. 9) to the entire frequency distribution section (indicated by “α” in FIG. 9). Is a value indicating The above-described peak interval is a width connecting two points where the number of white pixels takes a predetermined value across the peak of white pixels (see FIG. 9). For example, the half width of the peak may be used as the peak section. Under the same peak height, if the slope of the graph in the peak section is steep, the peak section becomes narrow, so the width% value is low. If the slope of the graph in the peak section is gentle, the peak section Since the width becomes wider, the width% value becomes higher.

  In the above equation (1), the evaluation value EV increases as the peak value increases, and the evaluation value EV increases as the width% value decreases.

  By the way, the annular zone detection unit 202, when the annular zone detection positions (white pixel peaks) in each binarized image are not arranged in the order of the pixel values extracted as white pixels, It may be determined that the annular zone has not been detected. For example, the annular zone detection unit 202, when the white pixel peak in the first binarized image is far from the reference point compared to the white pixel peak in the second binarized image, It is determined that the ring zone is not detected. This is because in the above case, there is a high possibility that white pixels are unrelated to flare.

  In the above example, the case where the coordinates of the saturation region acquired in step S15 are set as the reference point (Xk, Yk) has been described. However, in reality, the center of gravity of the saturation region and the center of the annular zone do not always coincide. Therefore, the annular zone detection unit 202 may shift the reference point in the plane direction from the coordinates of the saturation region acquired in step S15. Then, the annular zone detection unit 202 may obtain an evaluation value EV at each of a plurality of reference points, and finally determine a position where the evaluation value EV has a maximum value as a reference point.

  In step S19, the annular zone detection unit 202 determines whether or not the annular zone has been detected in step S18. If an annular zone has been detected (Yes determination in step S19), the process proceeds to step S20. On the other hand, when the annular zone cannot be detected (No in step S19), the process proceeds to step S21.

  In step S20, the light source estimation unit 203 estimates that the saturation region k to be determined is a light source that causes flare. And the light source estimation part 203 preserve | saves the reference point of the saturation area | region k located in the center of an annular zone as a light source position. At this time, the light source estimation unit 203 also stores information indicating the size of the saturation region k and information on the size of the annular zone (flare size) corresponding to the saturation region k. For example, the light source estimation unit 203 may calculate the size of the annular zone from the distance indicating the peak of the white pixel in step S18.

  When the reference point is determined by searching for the position where the evaluation value EV has the maximum value in step S18, the annular zone detection unit 202 in step S20 stores the reference point determined by the search as a light source position. To do.

  In step S <b> 21, the image processing unit 200 determines whether or not all saturated regions are to be determined (k = imax). If k = imax (Yes determination in step S21), the process proceeds to step S23. On the other hand, if k = imax is not satisfied (No in step S21), the process proceeds to step S22.

  In step S22, the image processing unit 200 increments the value of the counter k (k = k + 1), and shifts the processing to step S18. By updating the value of k in step S22, in the processing after S18 after the loop, the above operation is repeated with different saturated regions as determination targets.

  In the example of steps S14 to S22 described above, the case where the annular zone is extracted from the luminance image has been described. However, the image processing unit 200 may further perform the same processing as in steps S14 to S22 for each of the RGB channels.

  In step S <b> 23, the flare correction unit 204 performs correction processing (flare correction processing) for suppressing flare around the light source that is estimated to have flare in the input image. For example, the flare correction unit 204 may perform flare correction processing by applying the method disclosed in Japanese Patent Application Laid-Open No. 2012-134630.

  The outline of the flare correction process will be described. The flare correction unit 204 performs the flare correction process on the basis of each light source in which flare is estimated to occur by the following procedures (C) to (E). Can be applied.

  (C) First, the flare correction unit 204 sets a flare correction range for performing flare correction processing in the input image. For example, the flare correction unit 204 determines the center of the flare correction range using the information on the light source position stored in step S20. And the flare correction | amendment part 204 sets the position of the flare correction range centering | focusing on a light source position using the information on the size of the annular zone preserve | saved at step S20. The shape of the flare correction range is not limited to a circle, and may be, for example, a rectangle or a similar shape of an annular zone. The flare correction range is set for each light source that is estimated to have flare.

  (D) Secondly, the flare correction unit 204 determines whether or not color flare (color flare) occurs in the flare correction range. For example, the flare correction unit 204 calculates the average value of the input image in the flare correction range for each color component (RGB). When the variation between the color components of the average value exceeds the predetermined range, the flare correction unit 204 determines that a color flare has occurred in the flare correction range. In the flare correction range where the color flare occurs, the flare correction unit 204 performs the process (E) described later.

  On the other hand, when the variation between the color components of the average value does not exceed the predetermined range, the flare correction unit 204 determines that no color flare is generated in the flare correction range (a colorless flare). Note that the flare correction unit 204 does not perform the process (E) described later in a flare correction range in which no color flare occurs. Note that the flare correction unit 204 may perform a general correction process for a colorless luminance flare in a flare correction range in which no color flare occurs.

  (E) Third, the flare correction unit 204 divides the color component (for example, R component or B component) to be corrected by another reference color component (for example, G component) for each pixel in the flare correction range. Thereby, a correlated color distribution within the flare correction range is generated. Then, the flare correction unit 204 multiplies the reference color component by the value of the correlated color distribution in each pixel in the flare correction range. Thereby, the flare correction unit 204 can acquire the correction pixel value of the color component to be corrected in the flare correction range. The correction of the color components in the flare correction range described above may target only one color component or two color components. Through the above processing, color flare that has occurred around the saturation region on the input image is suppressed.

  The flare correction unit 204 generates a grayscale mask image by applying a Gaussian filter to the mask image corresponding to the flare correction range, and smoothes the color component obtained by correcting the color flare with the grayscale mask image. The treatment may be performed. By applying the above gray scale mask, it is possible to make the change in gradation at the boundary of the flare correction range less noticeable.

  In step S24, the image processing unit 200 reproduces and displays the input image on the monitor 109. When correction by the image processing unit 200 is performed, the corrected input image is displayed on the monitor 109. In step S24, when a save instruction is input from the user, the image processing unit 200 records the corrected image in a save destination (for example, the storage device 103) designated by the user.

  In step S <b> 25, the image processing unit 200 determines whether or not an instruction for retouching processing including image processing such as gradation conversion, noise removal, edge enhancement, brightness correction, and saturation correction has been received from the user. If an instruction for retouching processing has been received (affirmative determination in step S25), the process proceeds to step S26. On the other hand, when the instruction for the retouching process is not received (No at Step S25), the process proceeds to Step S27.

  In step S <b> 26, the image processing unit 200 performs image processing (retouch processing) of the content instructed by the user on the input image, and shifts the processing to step S <b> 24.

  In step S27, the image processing unit 200 determines whether an end instruction has been received. If an end instruction has been received (affirmative determination in step S27), the image processing unit 200 ends the process. On the other hand, when the termination instruction has not been received (No at Step S27), the process proceeds to Step S24. Above, description of the flowchart of FIG. 2 is complete | finished.

  Hereinafter, the operational effects of the first embodiment will be described.

  The image processing unit 200 of the first embodiment detects an annular zone included in the input image using a plurality of binarized images (S16 to S18). Then, the image processing unit 200 estimates a light source position that causes flare from the detected position of the annular zone (S20), and performs a flare correction process on the flare correction range set with the estimated light source position as a reference ( S23). Thereby, the image processing unit 200 can prevent unnecessary flare correction from being performed on a saturated region where there is no ring zone and flare is not generated. In addition, since the image processing unit 200 can identify the flare occurrence location from the pixel value information of the input image, the flare can be corrected accurately even in an environment where information indicating the imaging characteristics of the imaging optical system is not available. It becomes possible.

  Further, the image processing unit 200 uses a plurality of binarized images (FIG. 6) in which the combination of the threshold upper limit and the threshold lower limit is changed when the binarized image is generated, and the range of white pixels to be extracted is different. The ring zone is detected (S16 to S18). Thereby, the image processing unit 200 can clearly extract the shape of the annular zone included in the input image, and thus can easily estimate the light source in which the flare is generated.

  In addition, the image processing unit 200 obtains a frequency distribution indicating a correlation between the distance from the saturated region in the binarized image and the number of white pixels in the binarized image, and calculates the peak height of the white pixel in the frequency distribution. The ring zone is detected based on the peak width (S18). Thereby, the image processing unit 200 can clearly extract the position of the annular zone from each binarized image.

  Further, the image processing unit 200 corrects color flare generated around the light source of the input image by flare correction processing (S23). By the above processing, for example, in an image photographed by a photographing optical system including a diffractive optical element, unnatural color flare due to the diffractive optical element can be removed, so that the appearance of the image can be improved.

<Description of Second Embodiment>
FIG. 10 is a diagram illustrating a configuration example of the electronic camera according to the second embodiment which is an example of the imaging device disclosed herein. Note that in the second embodiment, the same or similar components as those in the first embodiment are denoted by the same reference numerals and description thereof is omitted.

  The electronic camera 300 includes a lens unit 301, an imaging unit 302, a camera microcomputer 303, a first memory 304, a second memory 305, a display unit 306 including a display device that displays various images, a media I / O F307 and an operation unit 308 that receives an operation from the user. Here, the imaging unit 302, the first memory 304, the second memory 305, the display unit 306, the media I / F 307, and the operation unit 308 are each connected to the camera microcomputer 303.

  The lens unit 301 can be attached to and detached from the electronic camera 300, and includes, for example, a photographing optical system 309 including a plurality of lenses such as a diffractive optical element. Lens information indicating the lens configuration and the like of the photographing optical system 309 is input to the camera microcomputer 303 via a mount (not shown) between the electronic camera 300 and the lens unit 301. In FIG. 10, the photographing optical system 309 is shown as a single lens.

  The imaging unit 302 is a module that captures (captures) an image of a subject by a light beam incident via the imaging optical system 309. For example, the imaging unit 302 includes an imaging device that performs photoelectric conversion, and a signal processing circuit that performs analog signal processing, A / D conversion processing, and the like on the output of the imaging device. Note that, for example, RGB color filters are arranged in the pixels of the image sensor according to a known Bayer array, and image signals corresponding to the respective colors are output by color separation in the color filters. Thereby, the imaging unit 302 can acquire, for example, an RGB color image at the time of shooting.

  The camera microcomputer 303 is a processor that comprehensively controls the operation of the electronic camera 300. For example, the camera microcomputer 303 performs shooting control in the imaging unit 302, automatic exposure control, image display control in the display unit 306, and image recording control in the media I / F 307. In addition, the camera microcomputer 303 in the second embodiment includes an image processing unit 200 that is an example of an image processing apparatus disclosed herein. Also in the second embodiment, the image processing unit 200 includes an image generation unit 201, an annular zone detection unit 202, a light source estimation unit 203, and a flare correction unit 204. Note that the image processing unit 200 illustrated in FIG. 10 may be realized by an image processing program loaded in a memory, or may be realized in hardware by an arbitrary processor, memory, or electronic circuit.

  The first memory 304 is a buffer memory that temporarily stores image data captured in the pre-process and post-process of image processing. For example, the first memory 304 is a volatile semiconductor memory. The second memory 305 is a memory that stores programs processed by the camera microcomputer 303 and various data used in the programs. For example, the second memory 305 is a nonvolatile memory such as a flash memory.

  The media I / F 307 has a connector for connecting a nonvolatile storage medium 310. The media I / F 307 executes writing / reading of data (recording image) with respect to the storage medium 310 connected to the connector. The storage medium 310 is, for example, a hard disk or a memory card incorporating a semiconductor memory.

  FIG. 11 is a flowchart illustrating an operation example of the electronic camera according to the second embodiment. In the example of FIG. 11, a case where the electronic camera 300 performs flare correction processing on an image immediately after shooting will be described. In the example of FIG. 11, steps S11 and S12 shown in FIG. 2 are replaced with steps S11 'and S12', respectively. Further, in the example of FIG. 11, the processing after step S24 shown in FIG. 2 is replaced with step S31. Note that the processing in steps S13 to S23 in FIG. 11 is the same as that in FIG.

  In step S <b> 11 ′, the camera microcomputer 303 drives the imaging unit 302 to capture an image of the subject. Data of the captured image captured by the imaging unit 302 is input to the camera microcomputer 303. As a result, the camera microcomputer 303 acquires an input image to be subjected to flare correction processing from the imaging unit 302.

  In step S12 ', the image processing unit 200 determines whether or not it is a flare correction mode in which flare correction processing is performed on the input image. As an example, when the user turns on the setting of the flare correction mode, or when the lens unit 301 being mounted has a diffractive optical element with reference to the lens information of the lens unit 301, the image processing unit 200 is determined to be the flare correction mode.

  If it is the flare correction mode (Yes determination in step S12 '), the process proceeds to step S13 in FIG. On the other hand, when the mode is not the flare correction mode (negative determination in step S12 '), the process proceeds to step S31 in FIG.

  In step S <b> 31, the camera microcomputer 303 records captured image data in the storage medium 310 via the media I / F 307. Above, description of the flowchart of FIG. 11 is complete | finished.

  Note that in the second embodiment, the image processing unit 200 can also perform flare correction processing in a post-processing step on the captured image data acquired via the media I / F 307. Since the process in the above case is the same as the process of FIG. 2 described in the first embodiment, the description thereof is omitted.

  As described above, also in the electronic camera of the second embodiment, the same effect as the configuration of the first embodiment can be obtained.

<Supplementary items of the embodiment>
(1) In the above embodiment, an example in which a binarized image is generated from an input image has been described. However, in the configuration of the present disclosure, a reduced image obtained by thinning or reducing the resolution of the input image may be used instead of the input image. In the case of using a reduced image, the number of pixels to be subjected to calculation processing is reduced, so that the calculation load can be reduced.

  In the configuration of the present disclosure, a smoothed image obtained by smoothing the input image may be used instead of the input image. In the case of using a smoothed image, for example, fine bright spots and white spots due to noise are removed, so that it is possible to more accurately estimate the light source that causes flare.

  In the configuration of the present disclosure, a reduced image subjected to smoothing processing may be used instead of the input image. In the above case, the order of the smoothing process and the process of generating the reduced image can be changed as appropriate.

  (2) In the above embodiment, the example in which the annular zone is detected using the luminance component of the color image has been described. However, in the configuration disclosed herein, the G component of the input image may be regarded as the luminance component, and the annular zone may be detected from the binary image of the G component.

  (3) The flare correction processing described in the above embodiment is merely an example, and the configuration disclosed herein may correct flare by other methods. For example, when correcting the color flare, a process of reducing the saturation of the flare correction range may be performed. Note that the estimation of the light source position of the flare according to the configuration of the present disclosure is not limited to the color flare by the diffractive optical element, but can also be applied to the estimation of the light source position of another flare.

  (4) In the second embodiment, the configuration example of the interchangeable lens type electronic camera has been described. However, the imaging device disclosed herein may be a lens-integrated electronic camera. In addition, the imaging device disclosed herein may be a camera module incorporated in a portable electronic device (smart phone, mobile phone, tablet computer, or the like), for example.

  From the above detailed description, features and advantages of the embodiments will become apparent. It is intended that the scope of the claims extend to the features and advantages of the embodiments as described above without departing from the spirit and scope of the right. Also, any improvement and modification should be readily conceivable by those having ordinary knowledge in the art. Therefore, there is no intention to limit the scope of the inventive embodiments to those described above, and appropriate modifications and equivalents included in the scope disclosed in the embodiments can be used.

DESCRIPTION OF SYMBOLS 101 ... Computer, 102 ... Data reading part, 103 ... Memory | storage device, 104 ... CPU, 105 ... Memory, 106 ... Input / output I / F, 107 ... Bus, 108 ... Input device, 109 ... Monitor, 200 ... Image processing part, DESCRIPTION OF SYMBOLS 201 ... Image generation part, 202 ... Ring zone detection part, 203 ... Light source estimation part, 204 ... Flare correction part, 300 ... Electronic camera, 301 ... Lens unit, 302 ... Imaging part, 303 ... Camera microcomputer, 304 ... 1st memory 305 ... second memory, 306 ... display unit, 307 ... media I / F, 308 ... operation unit, 309 ... shooting optical system, 310 ... storage medium

Claims (9)

A procedure for acquiring an input image generated by imaging;
A procedure for generating a processed image by performing threshold processing on the input image in a range of a threshold upper limit and a threshold lower limit;
Detecting a flare based on the processed image;
An image processing program for causing a computer to execute. In the image processing program according to claim 1,
In the procedure for generating the processed image,
An image processing program for generating a plurality of processed images by performing threshold processing with a plurality of types of thresholds having different combinations of the upper threshold and lower threshold. In the image processing program according to claim 1 or 2,
An image processing program for causing a computer to execute a procedure for correcting the flare detected by the procedure for detecting the flare based on the processed image. The image processing program according to claim 3.
A procedure for obtaining a predetermined distribution using a plurality of the processed images is executed,
An image processing program that suppresses the flare based on the distribution in the correction procedure. In the image processing program according to claim 3 or 4,
In the procedure of acquiring the input image, an input image obtained by capturing an image formed by a photographing optical system including a diffractive optical element is acquired,
An image processing program characterized in that, in the correcting step, color blur in the flare derived from the diffractive optical element is corrected. In the image processing program according to any one of claims 1 to 5,
In the procedure for generating the processed image, an image processing program for generating the processed image for a luminance component of the input image. An acquisition unit for acquiring an input image generated by imaging;
A generating unit that generates a processed image by performing threshold processing on the input image in a range of a threshold upper limit and a threshold lower limit;
A detection unit for detecting flare based on the processed image;
An image processing apparatus comprising: An imaging unit for imaging the imaging optical system;
An image processing apparatus according to claim 7;
An imaging apparatus comprising: The imaging device according to claim 8,
The imaging optical system is an imaging apparatus including a diffractive optical element.