JP4415188B2 - Image shooting device - Google Patents

Description

  The present invention relates to an image photographing device.

With a digital camera that captures still images and moving images and records them on a recording medium such as a memory card, it is possible to record captured images without worrying about film costs or development costs. Recently, the price of recording media has decreased, the recording capacity has increased significantly, and a large number of images can be recorded (see, for example, Patent Document 1). When the capacity of the recording medium is full, the image data recorded on the recording medium is transferred to a PC (Personal Computer) or the like, and then the recording medium is emptied, or the user can select from recorded images. An empty area of the recording medium is secured by determining whether the image is good or bad and deleting unnecessary images.
Japanese Patent No. 2787781

  However, conventionally, when an unnecessary image is deleted from images recorded on a recording medium of a digital camera, the user visually evaluates the image quality of the recorded image and determines an unnecessary image based on the evaluation result. In fact, there is a possibility that even an image with good image quality may be deleted.

An object of the present invention is to automatically determine a captured image as a deletion candidate in an image capturing device such as a digital camera.

To solve the above problems, an image capturing apparatus according to claim 1 includes an imaging means for obtaining captured image by capturing an object to be subject, and recording means for recording the obtained captured image to the recording medium, wherein A display means for displaying a recorded photographed image, a first evaluation means for evaluating the image quality of the acquired photographed image, and a photographed image recorded on the recording medium, obtained by the first evaluation means. A first deciding unit that decides a photographed image whose evaluation result satisfies a predetermined condition as a candidate for correction; and a display unit that displays the photographed image decided as a candidate to be corrected by the first deciding unit on the display unit. 1 display control means, a first instruction means for instructing whether or not to correct the captured image displayed by the first display control means, and an instruction for correction by the first instruction means If it, made a correction means for correcting the image quality of the captured image, a second evaluation means for image quality evaluating an image quality of the corrected captured image by the correction means, the correction by the first instructing means A second deciding unit that decides, as a candidate to be deleted from the recording medium , a photographic image instructed to be deleted and a photographic image whose evaluation result by the second evaluation unit satisfies a predetermined condition. It is said.

According to a second aspect of the present invention, in the image photographing device according to the first aspect of the present invention, the image photographing apparatus includes a determination unit that determines whether or not a remaining capacity of the recording medium is equal to or less than a preset capacity . The determining means, when the determining means determines that the remaining capacity of the recording medium is equal to or less than a preset capacity, the captured image instructed not to be corrected by the first instructing means, A photographed image whose evaluation result by the second evaluation means satisfies a predetermined condition is determined as a candidate to be deleted from the recording medium.
According to a third aspect of the present invention, in the image photographing device according to the first or second aspect, the display means displays a photographed image determined as a candidate to be deleted by the second determining means. The display control means, the second instruction means for instructing whether or not to delete the captured image displayed by the second display control means from the recording medium, and the second instruction means are instructed to delete. And a deletion means for deleting the photographed image from the recording medium.

According to a fourth aspect of the present invention, in the image photographing device according to any one of the first to third aspects, the first evaluation means and the second evaluation means are histogram distributions of luminance or density of the photographed image. The image quality of the captured image is evaluated based on the above.

According to a fifth aspect of the present invention, in the image photographing device according to any one of the first to third aspects, the first evaluation unit and the second evaluation unit are based on frequency characteristics of a captured image. It is characterized by evaluating the image quality of the captured image.

According to a sixth aspect of the present invention, in the image photographing device according to any one of the first to third aspects, the first evaluation unit and the second evaluation unit are degrees of blurring or defocusing of a captured image. The image quality of the captured image is evaluated based on the above.

According to a seventh aspect of the present invention, in the image photographing device according to any one of the first to sixth aspects, the correction unit performs gamma correction on the captured image.

According to an eighth aspect of the present invention, in the image photographing device according to any one of the first to sixth aspects, the correction unit converts a histogram distribution of luminance or density of the photographed image by a predetermined conversion formula. Thus, the captured image is corrected.

According to a ninth aspect of the present invention, in the image photographing device according to any one of the first to sixth aspects, the correction unit performs the flattening of the histogram distribution of luminance or density of the photographed image, thereby It is characterized by correcting captured images.

According to a tenth aspect of the present invention, in the image photographing device according to any one of the first to sixth aspects, the correction unit corrects the captured image using a spatial filter for smoothing the image. It is characterized by.

The invention of claim 1 1, in the image capturing apparatus according to any one of claim 1 to 6, wherein the correction means performing the process of sharpening the photographed image by using a predetermined differential filter It is characterized by.

Invention according to claim 1 2, in the image capturing apparatus according to any one of claim 1 to 6, wherein the correcting means, the photographic image frequency conversion for the frequency-converted photographic image, It is characterized by performing a sharpening process or a blurring process.

The invention according to claim 1 3, in the image capturing apparatus according to any one of claim 1 to 6, wherein the correcting means uses the point spread function or line spread function, linear photographed image It is characterized by correcting blurring or defocusing.

The invention according to claim 1 4, in the image capturing apparatus according to any one of claims 1 to 1 3, a degraded image pickup means for photographing the sample image in a manner that the image deteriorates, the deteriorated image capturing A calculation unit that calculates a degradation component of the image obtained by the unit, and a storage unit that stores information on the degradation component of the image calculated by the calculation unit according to the degree of image degradation, and is acquired by the imaging unit. When the evaluation result of the captured image satisfies a predetermined condition, the correction unit reads out the degradation component corresponding to the captured image from the storage unit, and corrects the captured image based on the readout degradation component. It is characterized by doing.

According to the present invention, a photographed image is automatically evaluated, an image whose evaluation result satisfies a predetermined condition (for example, an image whose evaluation value is a predetermined value or less) is determined as a correction candidate, and a correction instruction is included among the correction candidates. The correction process is performed on the image that has occurred , the image is evaluated again after the correction process, and only the image whose evaluation result satisfies the predetermined condition (for example, the image whose evaluation value is equal to or less than the predetermined value) is selected as the deletion candidate. As a result, an image is not deleted more than necessary, and a free space in the recording medium can be ensured.

  Further, when a sample image is photographed in advance by a method in which image degradation occurs, a degradation component is calculated and stored for each degree of image degradation, and the evaluation result of the actually photographed image satisfies a predetermined condition (for example, When the evaluation value is equal to or less than a predetermined value), by performing the image correction based on the deterioration component stored in the storage unit, it is possible to easily perform the image correction according to the performance of each image capturing apparatus. Further, since it is not necessary to calculate a deterioration component at the time of image correction, the image correction process can be performed at high speed.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
First, the configuration in the present embodiment will be described.

  FIG. 1 shows a circuit configuration of a digital camera 1 according to an embodiment to which the image photographing apparatus of the present invention is applied. As shown in FIG. 1, the digital camera 1 includes a motor (M) 21, a lens optical system 22, a CCD 23, a TG (timing generator) 24, a vertical driver 25, an S / H (sample hold circuit) 26, and an A / D. Converter 27, color process circuit 28, DMA (Direct Memory Access) controller 29, DRAM (Dynamic RAM) I / F (DRAM interface) 30, DRAM 31, control unit 32, VRAM (Video RAM) controller 33, VRAM 34, digital video The encoder 35, key input unit 36, JPEG (Joint Photograph coding Experts Group) circuit 37, flash memory 38, communication unit 39, camera shake detection sensor 40, and display unit 13 are configured.

  The motor (M) 21 is driven according to a control signal input from the control unit 32 to move the aperture position of the lens optical system 22.

  The CCD 23 has a structure in which imaging elements are arranged in a planar shape (two-dimensional), converts a light input into an electrical signal and stores it, a scanning unit that reads out the stored charge, and outputs it as an electrical signal. It is configured by an output unit, and is driven by a timing generator (TG) 24 and a vertical driver 25 in a monitoring state in the photographing mode.

  The electrical signal output from the output unit of the CCD 23 is appropriately gain-adjusted for each primary color component of RGB in an analog value signal state, and the S / H (sample hold circuit) 26 samples the gain-adjusted signal. Hold. The A / D converter 27 converts the analog signal output from the S / H (sample hold circuit) 26 into a digital signal and outputs the digital signal to the color process circuit 28.

  The color process circuit 28 performs color process processing including pixel interpolation processing and γ correction processing on the digital signal output from the A / D converter 27 to generate a digital luminance signal Y and color difference signals Cb and Cr. And output to the DMA controller 29.

  The DMA controller 29 uses the luminance signal Y and the color difference signals Cb and Cr output from the color process circuit 28 and the DMA controller once using the composite synchronization signal, the memory write enable signal and the clock signal output from the color process circuit 28. The data is written in the internal buffer 29 and DMA transfer is performed to the DRAM 31 used as a buffer memory via the DRAM I / F 30.

  The control unit 32 includes a CPU (Central Processing Unit), a ROM (Read Only Memory) in which an operation program executed by the CPU is fixedly stored, a RAM (Random Access Memory) used as a work memory, and the like. The operation of each part of the camera 1 is controlled. Hereinafter, the control operation by the control unit 32 will be described.

  The control unit 32 executes an image evaluation process for evaluating the image quality of the captured image. When the shooting mode is designated, the control unit 32 determines whether or not the remaining capacity of the flash memory 38 (recording medium) is equal to or less than a preset capacity, and the remaining capacity of the recording medium is preset. When it is determined that the capacity is equal to or less than the capacity, a correction process is executed to correct the image quality of a captured image whose evaluation value calculated by the image evaluation process is equal to or less than a predetermined value among images recorded on the recording medium. Then, the control unit 32 performs the evaluation process of the image subjected to the image quality correction again, and determines an image whose evaluation value calculated by the image evaluation process is a predetermined value or less as a candidate to be deleted from the recording medium. Then, the control unit 32 performs processing for deleting the image designated by the operation of the key input unit 36 from the deletion candidate images from the recording medium. In the following embodiments, not only when erasing recorded data on a recording medium but also when overwriting recorded data with new data, it is assumed that the recorded data is “deleted”. To do.

  As an image evaluation method, a method of evaluating from a luminance or density histogram distribution of a captured image (image evaluation method 1: see FIGS. 7 and 8), a method of evaluating based on a frequency characteristic of the captured image, and (image evaluation) Method 2: refer to FIGS. 9 to 11), and a method of evaluating based on the degree of blurring or defocusing of a captured image (image evaluation method 3: FIGS. 12 to 16). In the image evaluation process of the present embodiment, a case where all of the plurality of image evaluation methods are applied is shown, but any one or two or more of these image evaluation methods 1 to 3 may be combined.

  As a method for correcting the image quality of the photographed image, a method of performing gamma correction on the photographed image (image correction method 1: refer to FIG. 18), a method of converting the luminance or density histogram distribution of the photographed image by a predetermined conversion formula, (Image correction method 2: refer to FIG. 19 and FIG. 20), a method of flattening the histogram distribution of luminance or density of the captured image (image correction method 3: refer to FIG. 21 and FIG. 22), and a method of smoothing the captured image (Image correction method 4: refer to FIG. 23), a method of sharpening a captured image using a differential filter (image correction method 5: refer to FIG. 24 and FIG. 25), and a sharpening process to the frequency-converted captured image. And an image correction method (image correction method 6), a PSF (Point Spread Function), a LPF (Line Spread Function), or the like is used to correct linear blurring or defocusing of a captured image (image correction method 7). In the present embodiment, an image correction method combining any one or two or more of the image correction methods 1 to 7 can be applied.

  The VRAM controller 33 reads the image data (luminance signal Y and color difference signals Cb, Cr) temporarily stored in the DRAM 31 through the DRAM I / F 30 and writes them in the VRAM 34 under the control of the control unit 32. In the reproduction mode, the image data expanded by the control unit 32 is expanded and stored in the VRAM 34.

  The digital video encoder 35 periodically reads image data stored in the VRAM 34 from the VRAM 34 via the VRAM controller 33, generates a video signal based on these data, and outputs the video signal to the display unit 13.

  The key input unit 36 includes various keys such as a power key, a shutter key, a mode switch, a menu key, and a cross key, and outputs a signal corresponding to the key operation to the control unit 32. The shutter key of the key input unit 36 operates with a two-stage stroke, and performs an AE (automatic exposure) process and an AF (autofocus) process in a first-stage operation state generally expressed as “half-press”. Preparation for the first shooting is performed, and the shooting is performed in the second-stage operation state in which the pressing operation is more strongly expressed, which is generally expressed as “full press”.

  The JPEG circuit 37 converts the image signal taken from the CCD 23 and temporarily stored in the DRAM 31 into data by processing such as ADCT (Adaptive Discrete Cosine Transform), Huffman coding which is an entropy coding method. Compress.

  The flash memory 38 is a non-volatile memory enclosed in a memory card that is detachably mounted as a recording medium of the digital camera 1, and an image signal encoded by the JEPG circuit 37 is recorded on the flash memory 38. Note that the flash memory 38 may be built in the digital camera 1.

  The communication unit 39 includes an interface for performing data communication with an external device via a cable. For example, the communication unit 39 includes an interface for performing communication using IEEE1394, USB (Universal Serial Bus), or the like. Note that communication with an external device may be possible by wireless communication means.

  The camera shake detection sensor 40 is an acceleration sensor or the like, and detects an amount of camera shake at the time of shooting (such as a shake direction (θ) and a shake distance (L), which will be described later). The display unit 13 functions as a monitor display unit (electronic finder) in the shooting mode, and performs display based on the video signal from the digital video encoder 35, thereby displaying image information captured from the VRAM controller 33 at that time. Display the through image based on real time.

Next, the operation in this embodiment will be described.
First, the image capturing / deleting process executed in the digital camera 1 will be described with reference to the flowchart of FIG.

  First, a menu screen as shown in FIG. 4A is displayed on the display unit 13 (step S1). On the menu screen shown in FIG. 4A, a “shooting mode” button for specifying a shooting mode, a “playback mode” button for specifying playback of a shot image, and a “playback mode” button for specifying deletion of a shot image. A “delete image” button, a “various settings” button for specifying various other settings, and the like are displayed.

  When the “delete image” button is designated on the menu screen (step S3; YES), a delete mode screen as shown in FIGS. 4B and 5A is displayed, and the key input unit 36 is operated. Thus, various setting processes such as automatic image deletion and image quality correction are performed (step S4). After the setting process, the process returns to step S1. When a button other than the “shooting mode” button and the “delete image” button is designated on the menu screen (step S3; NO), processing corresponding to the designated mode is performed (step S5), and step S1. Return to.

  When an “automatic deletion setting” button is designated on the deletion mode screen, an automatic deletion setting screen as shown in FIG. 5B is displayed. In the automatic deletion setting screen, as shown in FIG. 5B, “automatic deletion” for setting whether the automatic deletion function of the captured image is enabled (ON) or disabled (OFF) is set as a setting item. "Function" field, "Deletion method" field for setting the image deletion method, "Evaluation method" field for setting the image evaluation method, and each evaluation function is enabled (ON) or disabled (OFF) "Evaluation item" field (for example, "exposure condition evaluation" item, "contrast evaluation" item, "blur / blur evaluation" item) is displayed. In each evaluation item column, a “reference value” column for setting a reference value of the evaluation item is displayed. Here, the reference value of the evaluation item is a numerical value serving as a reference for correction and deletion, and the evaluation value of the image is determined based on this reference value.

  When the “image quality correction” button is designated on the deletion mode screen shown in FIG. 5A, a correction item screen as shown in FIG. 5C is displayed. On the correction item screen, as shown in FIG. 5C, as the image quality correction items, an “exposure (density level) correction” button, a “tone curve (contrast) correction” button, and “enhancement of contour edge (sharpness) ) "Button," Blur / blur correction "button," Noise removal "button, and" Color tone (brightness / hue / saturation) correction "button.

  When the “image quality automatic correction setting” button is designated on the deletion mode screen shown in FIG. 5A, an image quality automatic correction setting screen as shown in FIG. 5E is displayed. In the automatic correction setting screen shown in FIG. 5E, the automatic image correction function, exposure condition correction function, outline edge enhancement function, blur / blur correction function, and noise removal function are enabled. “Automatic correction function” field, “Exposure condition correction” field, “Outline edge enhancement” field, “Blur / blur correction” field, The “Noise removal” column is displayed.

  When the “exposure (density level) correction” button is designated on the correction item screen shown in FIG. 5C, an exposure (density level) correction screen as shown in FIG. By operating the input unit 36, the density level of the captured image is set for each of the RGB colors. When the “tone curve (contrast) correction” button is designated on the correction item screen shown in FIG. 5C, a tone curve correction screen as shown in FIG. By this operation, the contrast of the captured image is set.

  When the “shooting mode” button is designated on the menu screen shown in FIG. 4A (step S2; YES), initial settings such as the lens position of the digital camera 1 are made (step S6), and the recording medium (flash It is determined whether the remaining capacity of the memory 38) is less than or equal to the set capacity (step S7).

  If it is determined in step S7 that the remaining capacity of the recording medium is equal to or less than the set capacity (step S7; YES), an image deletion process is performed to delete image data from the recording medium based on the evaluation value of each captured image. (Step S8). Details of the image deletion processing will be described later with reference to FIG.

  If it is determined in step S7 that the remaining capacity of the recording medium is larger than the set capacity (step S7; NO), or after the image deletion process in step S8 is completed, photometric processing, setting of exposure conditions, and the like are performed (step S9). ). Next, zoom processing is performed (step S10), and autofocus processing is performed (step S11).

  A shooting through image as shown in FIG. 6A is displayed on the display unit 13 (step S12). When shooting is instructed by pressing the shutter key of the key input unit 36 (step S13; YES), shooting processing is performed. Is performed (step S14), and photographed image data is acquired. After the shooting process, an image evaluation process is performed on the shot image data acquired by the shooting process (step S15). Details of the image evaluation process will be described later with reference to FIGS. If the shutter key has not been pressed (step S13; NO), other (other than shooting mode) key input processing, display processing, and the like are performed (step S25), and the image shooting / deletion processing ends.

  When the image evaluation process for the photographed image data is completed, the photographed image data and its evaluation value are associated and recorded on the recording medium (step S16), and the photographed image data and its evaluation value are displayed on the display unit 13 (step S17). ). In step S17, as shown in FIG. 6B, detailed image data such as the size of image data, shooting conditions, and image quality evaluation values may be displayed. In addition, each item of the image quality evaluation value and the comprehensive evaluation value may be displayed after being converted into a 5-point method, a 10-point method, or%.

  Next, it is determined whether or not the evaluation value of the captured image is equal to or less than a predetermined value (step S18). Here, the predetermined value is determined based on the reference value set on the automatic deletion setting screen in FIG. 5B, and is, for example, the total value of the reference values set in each evaluation item. If it is determined in step S18 that the evaluation value of the photographed image is greater than the predetermined value (step S18; NO), other (other than photographing mode) key input processing, display processing, and the like are performed (step S25). The shooting / deleting process ends.

  When it is determined in step S18 that the evaluation value of the captured image is equal to or less than the predetermined value (step S18; YES), an automatic correction confirmation prompt screen as shown in FIG. 6C is displayed on the display unit 13. The On the automatic correction confirmation prompt screen, as shown in FIG. 6C, the corresponding photographed image, a message “Are you sure you want to automatically correct this image? (Y / N)” and an evaluation value are displayed. Is done.

  If automatic image correction is not specified on the automatic correction confirmation prompt screen shown in FIG. 6C, the process proceeds to step S25. If automatic image correction is designated on the automatic correction confirmation prompt screen (step S19; YES), automatic image quality correction processing of the corresponding captured image is performed (step S20). Details of the automatic image quality correction processing will be described later with reference to FIGS.

  When the automatic image quality correction process in step S20 is completed, an image evaluation process is performed on the corrected captured image (step S21). Details of the image evaluation process will be described later with reference to FIGS. Next, an update confirmation prompt screen for the corrected image as shown in FIG. 6D is displayed on the display unit 13. As shown in FIG. 6D, the corrected image update confirmation prompt screen is displayed with the corresponding photographed image and “Image quality is automatically corrected. Do you want to update to this image? (Y / N)”. Messages and evaluation values are displayed.

  If the update of the corrected image is not designated on the corrected image update confirmation prompt screen (step S23; NO), the process proceeds to step S25. When update of the corrected image is designated on the corrected image update confirmation prompt screen (step S23; YES), the corrected image is encoded and recorded on the recording medium together with the evaluation value (step S24). Then, other key input processing (except for the shooting mode), display processing, and the like are performed (step S25), and the main image shooting / deleting processing ends.

  Note that the image evaluation processing in steps S15 and S21 may be performed every time one image is captured, or processing for all images may be performed collectively after a series of imaging processing is completed. In particular, in the continuous shooting and the moving image shooting, it is preferable to use the latter method in order to maintain the shooting performance.

  Next, details of the image deletion processing shown in FIG. 2 will be described with reference to the flowchart of FIG.

  First, it is determined whether or not the image data automatic deletion setting is ON (whether or not the image data automatic deletion function is enabled) (step S30). If it is determined in step S30 that the automatic deletion setting is OFF (step S30; NO), the display unit 13 displays that the remaining capacity of the recording medium is exhausted (step S31) and manual operation. A confirmation prompt screen for confirming image deletion at is displayed (step S32).

  When image deletion by manual operation is designated on the confirmation prompt screen displayed on the display unit 13 (step S33; YES), the mode shifts to an image deletion mode by manual operation (step S34), and this image deletion processing is performed. Ends. In step S34, the user deletes the image while checking the capacity of the recording medium. When image deletion by manual operation is not designated (step S33; NO), the image deletion process ends.

  If it is determined in step S30 that the automatic deletion setting is ON (step S30; YES), “There is no remaining memory! Search for candidates to be deleted” as shown in FIG. Is displayed, and it is determined whether the image recorded on the recording medium has been subjected to the image evaluation process (step S35).

  If it is determined in step S35 that the recorded image has not been subjected to the image evaluation process (step S35; NO), the image evaluation process for the recorded image is performed (step S36), and the process proceeds to step S37 described later. Details of the image evaluation processing in step S36 will be described later with reference to FIGS.

  In step S35, when it is determined that the recorded image has been subjected to the image evaluation process (step S35; YES) or when the image evaluation process in step S36 is completed, an evaluation value is set to a predetermined value among the images recorded on the recording medium. Images below the value are searched (step S37), and correction target images are selected in the order of lowest evaluation value (step S38).

  When a photographic image to be corrected is selected, an automatic correction confirmation prompt screen as shown in FIG. 6C is displayed on the display unit 13 (step S39). On the automatic correction confirmation prompt screen, as shown in FIG. 6C, the selected photographed image, the message “Are you sure you want to automatically correct this image? (Y / N)” and the evaluation value are displayed. Is displayed.

  When automatic image correction is not designated on the automatic correction confirmation prompt screen shown in FIG. 6C (step S40; NO), the process proceeds to step S45 described later. If automatic image correction is designated on the automatic correction confirmation prompt screen (step S40; YES), automatic correction processing of the image quality of the corresponding captured image is performed (step S41). Details of the automatic image quality correction processing will be described later with reference to FIGS.

  When the automatic image quality correction process is completed, an image evaluation process for evaluating the corrected image again is performed (step S42). Details of the image evaluation processing in step S42 will be described later with reference to FIGS. Next, the corrected image and its evaluation value are displayed on the display unit 13 (step S43), and it is determined whether or not the evaluation value of the corrected image is equal to or less than a predetermined value (step S44). Here, the predetermined value is determined based on the reference value set on the automatic deletion setting screen in FIG. 5B, and is, for example, the total value of the reference values set in each evaluation item.

  If it is determined in step S44 that the evaluation value of the corrected image is greater than the predetermined value (step S44; NO), the next candidate for correction is searched (step S49), and the process returns to step S38. In step S44, when it is determined that the evaluation value of the corrected image is equal to or smaller than the predetermined value (step S44; YES), the corrected image is determined as a candidate to be deleted from the recording medium, and the display unit 13 displays FIG. As shown in (f), a deletion confirmation prompt screen is displayed (step S45). The deletion confirmation prompt screen displays a corresponding image, a message “Are you sure you want to delete this image? (Y / N)”, and an evaluation value.

  If deletion of the corresponding image is not designated on the deletion confirmation prompt screen (step S46; NO), a candidate for the next correction target is searched from images already recorded on the recording medium (step S49), and step Return to S38. When deletion of the corresponding image is designated on the deletion confirmation prompt screen (step S46; YES), the corresponding image is deleted from the recording medium (step S47), and the remaining capacity of the current recording medium is obtained by new shooting. It is determined whether the data is sufficient to save (step S48).

  If it is determined in step S48 that the remaining capacity of the recording medium is sufficient to store the newly captured image data (step S48; YES), the image deletion process ends. If it is determined in step S48 that the remaining capacity of the recording medium is not sufficient to store newly captured image data (step S48; NO), the next correction target is selected from the images already recorded on the recording medium. Candidates are searched (step S49), and the process returns to step S38.

  In the flowchart of FIG. 3, a deletion confirmation prompt screen as shown in FIG. 6F is displayed on the display unit 13 so that the user confirms the image to be deleted, and the image specified by the user is deleted. However, an image determined as a deletion candidate may be automatically deleted.

  Next, details of the image evaluation method applied in the image evaluation processing of FIGS. 2 and 3 will be described.

<Image evaluation method 1: Evaluation based on luminance histogram distribution>
First, a method for evaluating image quality from a histogram distribution of luminance or density of a captured image will be described with reference to FIGS.

  A luminance distribution diagram (brightness histogram) as shown in FIG. 7 is obtained by setting the luminance value of each point (i, j) of the two-dimensional image data to f (i, j) = x and counting the number of pixels for each luminance value. Distribution) P (x) is created. In general, if the luminance histogram distribution deviates from the center and is biased to the right end side (large luminance value: left end side in the density histogram distribution), the image is overexposed and the high luminance area is saturated or a so-called “out-of-white” image. Become. Conversely, if the luminance histogram distribution is biased to the left end side (right end side in the density histogram distribution), the image is underexposed and the low luminance area is saturated or “blackout”. Even if the distribution is concentrated in the center, if the distribution width is narrow and not uniformly distributed, the dynamic range is narrow, and the image lacks intermediate gradation and fine gradation. Both of these are typical images that have poor exposure conditions at the time of shooting and are not well captured. Whether the exposure condition is good or bad can be evaluated from the index value or statistic of the luminance or density distribution characteristic of the photographed image. Many of the recent digital cameras can calculate the luminance values and display the histogram (frequency distribution diagram) as a graph, so if this function is used, image evaluation can be performed relatively easily. Can do.

  In this embodiment, from the histogram distribution of the luminance or density of the photographed image, the mode of luminance or density (the luminance value or density value having the highest frequency), the average value, and the median (the order of the frequency numbers of the distribution is the center) The representative value of the distribution (such as the luminance value or density value), the maximum value, the minimum value, the variance, the standard deviation, the distortion degree, the kurtosis degree, etc. of the distribution are calculated, and the evaluation value for image evaluation is calculated from the calculation result. It is determined.

The average value x _M of the histogram distribution of luminance or density, variance σ ² , standard deviation σ, skewness Skew, kurtosis Kurt is the luminance value of pixel i x _i. Assuming that the total number of pixels of the evaluation target image is N, these are expressed as equations (1) to (5), respectively.

  Here, the degree of distortion Skew of the distribution indicates asymmetry whether the distribution is symmetrical. If the distribution is bilaterally symmetrical, it is 0, if there is a large outlier value on the right (when overexposed), and positive if there is a large outlier value on the left (underexposed). The kurtosis Kurt of the distribution is 0 if the distribution is a normal distribution, positive if the spread is larger than the normal distribution (when the dynamic range is wide), and negative if the spread is smaller than the normal distribution (if the dynamic range is narrow). .

  In this way, based on the statistics of the histogram distribution of luminance or density, it is possible to determine the width and spread of the distribution, its bias, the shape and characteristics of the distribution curve, and to evaluate the image quality. The value is determined.

When the captured image is a color image, as shown in FIG. 8, histogram distributions P _R (x), P _G (x), P _B (x) of luminance or density are created for each color of RGB, and the color Each time, statistics such as distribution average, variance, standard deviation, degree of distortion, kurtosis, coefficient of variation CV, etc. are calculated, and the evaluation value of image quality is determined by the difference and comparison of the calculated statistics for each color. You may make it do. Here, the variation coefficient CV is defined as in Expression (6).

<Image evaluation method 2: Evaluation based on frequency characteristics>
Next, a method for evaluating the image quality from the frequency characteristics of the captured image will be described with reference to FIGS.

  F (u, v) obtained by frequency-converting the luminance value or density value f (x, y) of each point of the captured image (two-dimensional image data) is calculated, and the frequency characteristic of the image is obtained from the amplitude value and coefficient for each frequency. From the ratio of high-frequency components and low-frequency components in the entire image, the degree of image blur, sharpness, contrast, and the like can be evaluated. As the frequency transform, Fourier transform, discrete Fourier transform (DFT), discrete cosine transform (DCT), fast Fourier transform (FFT) algorithm for calculating them at high speed, or the like is used. Can do. Many recent digital cameras have a built-in DCT DSP (Digital Signal Processor) and software for JPEG and other compression coding, so if this function is used, image evaluation is relatively easy. It can be carried out.

A discrete Fourier transform expression of the luminance value or density value f (x, y) of the two-dimensional image data is expressed as Expression (7).

Also, the discrete cosine transform equation for the luminance value or density value f (x, y) of the two-dimensional image data is expressed as equation (8).

  As described above, the two-dimensional image f (x, y) is converted into the frequency space, the frequency component F (u, v) is calculated, the ratio of the high frequency component occupying all the frequency components, and the low frequency occupying all the frequency components. The evaluation value of the image can be determined based on the ratio of the components. Also, as shown in FIG. 9, the captured image may be scanned in advance and rearranged into a one-dimensional signal and then subjected to frequency conversion to calculate the evaluation value, or as shown in FIG. The evaluation value may be calculated after the amplitude value of F (u, v) obtained by frequency-converting the image is rearranged in one dimension by zigzag scanning or the like.

In the present embodiment, the ratio of high frequency components to all frequency components is set as an image quality evaluation value, and the ratio of low frequency components to all frequency components is set as a blur evaluation value. That is, the image quality evaluation value and the blur evaluation value can be expressed as Expression (9) and Expression (10), respectively.
Image quality evaluation value = (total amplitude value of high frequency components) / (total amplitude value of all frequency components) (9)
Blur evaluation value = (total amplitude value of all frequency components−total amplitude value of high frequency components) / (total amplitude value of all frequency components) (10)

Further, as shown in FIG. 11, the frequency distribution (amplitude (frequency) -frequency relationship) of the luminance value or density value f (x, y) of the photographed image is obtained, and this frequency distribution is obtained in the same manner as the image evaluation method 1. Statistical values such as average value, variance, standard deviation, and the like may be calculated, and an image quality evaluation value may be determined based on the calculated statistical values. _Assuming that the amplitude value of the frequency component i is F [i] and the total of the frequency components is N, the average value f _M , variance σ ² , and standard deviation σ of this distribution are represented by the following equations (11) to (13), respectively. ).

<Image Evaluation Method 3: Evaluation Based on Image Blur or Degree of Blur>
Next, with reference to FIGS. 12 to 16, a method for evaluating the image quality based on the degree of blurring or the degree of blur of the captured image will be described.

FIG. 12 shows a method for discriminating and evaluating blur and blur using a PSF (Point Spread Function) method used for image conversion and restoration. In general, in image restoration by the PSF method, a degraded image g (x, y) deteriorated due to an original image i (x, y) without blurring or blurring and a component p (x, y) that causes blurring or blurring. Represents. That is, the deteriorated image g (x, y) is expressed as in Expression (14).
g (x, y) = p (x, y) * i (x, y) (14)
Here, * represents a convolution (convolution integration) operation.

  If p (x, y) deteriorated by calculating p (x, y) as PSF and LSF (Line Spread Function), the original image i (x, y) It can be obtained by deconvolution (deconvolution integration) calculation. Hereinafter, a method for calculating the original image i (x, y) will be described.

First, the degraded image g (x, y) shown in Expression (14) is converted into the frequency axis by Fourier transform or the like. Assuming that the degraded image subjected to frequency conversion is G (u, v), G (u, v) is expressed as in Expression (15-1).

Also, if p (x, y) and i (x, y) are frequency-converted as P (u, v) and I (u, v), respectively, G (u, v) is expressed by equation (15). -2).
G (u, v) = P (u, v) · I (u, v) (15-2)
Here, the convolution is a multiplication of Fourier transforms on the frequency axis. If P (u, v) in equation (15-2) can be calculated in the frequency domain, 1 / P (u, v) can be calculated as an inverse filter to restore the original image i (x, y). it can. That is, I (u, v) = G (u, v) / P (u, v) (division between Fourier transforms) is calculated, and i (x, y) is obtained by inverse Fourier transform. . Expression (16) shows an inverse Fourier transform expression of I (u, v).

  In this embodiment, the PSF or P (u, v) calculation method in the PSF method is a general amateur photographed image, along with failure of exposure setting, which is often a “focus blur” image or a “linear blur” image. Applies to discrimination and evaluation. In general, the distribution of amplitude values of G (u, v) obtained by performing a Fourier transform on an image that has undergone linear blurring has a linear blur pattern as shown in FIG. 12B, and high-frequency components near the center are several in accordance with the blur angle. A distorted pattern appears in the form of elongated stripes or slanted ellipses. Further, G (u, v) obtained by performing Fourier transform on the defocused image becomes a defocused image pattern as shown in FIG. 12 (d), and becomes a distorted pattern in which the high-frequency component in the center portion is lost concentrically. . When both straight blurring and out-of-focus blur occur, a pattern having both features as shown in FIG. 12C is obtained.

  In the present embodiment, the evaluation value of linear blur is the blur direction angle (θ) in the direction in which the amplitude value of G (u, v) zero-crosses with a period 1 / L, and the blur distance or the number of pixels (L). It shall be determined based on Further, the evaluation value of the focal blur is determined based on the concentric focal spread radius where the amplitude value of G (u, v) zero-crosses with a period of 1.01π / r or the number of pixels (r). And In order to calculate the blur direction angle (θ), the blur distance (L), and the focal spread radius (r), the PSF method, the multi-tone Hough transform method, the inverse filter method, and the like can be applied. The calculation method using the method and multi-gradation Hough transform will be described.

式（１７）のＨsum(θ)は、劣化画像Ｇ（ｕ，ｖ）のフーリエ・スペクトルが示す円内の画素値の合計である。式（１７）に示すエントロピーＥ（θ）が最小となる角度θを、直線ブレ方向（θ）とみなすことができる。また、図１３（ｂ）に示すように、直線ブレ方向（θ）のＨθ（ρ）波形の極小点の周期（ＴＬ）を求めると、ブレ距離（Ｌ）＝1／ＴＬが求まる。 First, as shown in FIG. 13 (a), within a circle at the center of the Fourier spectrum of G (u, v) obtained by Fourier transform of a deteriorated image such as linear blur (estimation and detection of straight lines passing through a plurality of points). Hough (ρ, which is widely used for the above), is converted into a multi-gradation two-dimensional image H (θ, ρ) in the θ-ρ space and cut out for each angle θ of the H (θ, ρ) value. ) Waveforms are compared to determine entropy E (θ) at each angle θ (0 ° ≦ θ <180 °). Entropy E (θ) is expressed as shown in Equation (17).

Hsum (θ) in Expression (17) is the sum of the pixel values in the circle indicated by the Fourier spectrum of the degraded image G (u, v). The angle θ at which the entropy E (θ) shown in the equation (17) is minimized can be regarded as the linear blur direction (θ). Further, as shown in FIG. 13B, when the period (TL) of the minimum point of the Hθ (ρ) waveform in the linear blur direction (θ) is obtained, the blur distance (L) = 1 / TL is obtained.

Ｃ（ρ）波形の極小点の周期（Ｔｒ）から、焦点ボケの広がり半径（ｒ）は、半径（ｒ）＝１．０１×π／Ｔｒより求まる。 To calculate the defocusing radius (r), first, as shown in FIG. 14, a C (ρ) waveform is created by averaging H (θ, ρ) from 0 ° to 179 ° in the θ-axis direction. To do. C (ρ) is expressed as in Expression (18).

From the period (Tr) of the local minimum point of the C (ρ) waveform, the spread radius (r) of the focal blur is obtained from the radius (r) = 1.01 × π / Tr.

15 and 16 show the principle of Hough transform (Hough transform) used in the above method. In the Hough transform, a general two-dimensional coordinate (a point on the xy plane) is converted into a curve on the θ-ρ plane by a conversion formula of ρ = x cos θ + y sin θ, and a straight line on the xy plane is θ-ρ Can be converted back to a point on the plane. For example, when a straight line L passing through points A, B, and C on the xy plane is obtained, a curve A on the θ-ρ plane obtained by performing Hough transform on the points A, B, and C as shown in FIG. Find the intersection P (θ _L , ρ _L ) where ', B' and C 'intersect. Then, the intersection P (θ _L , ρ _L ) is inversely transformed into Hough on the xy plane, and the length of the perpendicular drawn from the zero point is ρ _L , and the straight line L having the perpendicular angle θ _L is The straight line to be obtained.

  The brightness (brightness or density having multiple gradations) of the Hough transform array H (θ, ρ) is defined as H (θ, ρ) = f (x, y) as a two-dimensional image having multiple gradations. As shown in FIG. 16, the lightness (gradation value) is added and the point where the gradation value is the largest (the shaded area in FIG. 16) is used as the intersection. Can be quickly obtained by approximate calculation.

  As described above, G (u, v) obtained by Fourier-transforming the deteriorated image g (x, y) on the frequency axis by the method of calculating the PSF function that deteriorates the image, the zero cross method, the multi-gradation Hough transform, or the like. From this, the direction of the straight blur image (angle θ), the straight blur distance (number of pixels L), and the focal spread radius (number of pixels r) of the defocus image are calculated, and the image is evaluated based on these θ, L, and r. The value is determined. The evaluation value decreases as the values of θ, L, and r increase.

FIG. 17 shows a conceptual diagram of image quality correction processing applied in the present embodiment.
The image quality correction processing is performed by referring to a predetermined conversion table or the like for the luminance value (or density value) f (i, j) of each pixel of the captured image input to the digital camera 1 or the surrounding pixel array. A correction calculation is performed, and a corrected output image g (i, j) or a predetermined feature amount is output.

  As the conversion table, for example, conversion expressions of input / output characteristics such as luminance conversion, LUT (Look Up Table) (conversion table) for conversion, weighting matrix (matrix), operators (operators) such as spatial filter coefficients, etc. As the image quality correction calculation, a point processing calculation, a local processing calculation targeting the pixel of interest and eight surrounding points, and a global processing calculation for calculating the entire image can be applied.

  The point processing calculation includes density histogram conversion processing, binarization processing, coordinate conversion processing, and inter-image calculation. Examples of the local processing calculation include noise removal calculation, differentiation calculation, product-sum calculation, filter calculation, edge extraction processing, expansion / contraction processing, thinning processing, and the like. As the global processing computation, orthogonal transformation computation such as Fourier transformation, texture analysis processing, or the like can be applied.

  Next, image correction methods 1 to 7 applied in the automatic image quality correction processing of FIGS. 2 and 3 will be described.

図１８の変換テーブルαは、式（１９）においてγ＝０．５、（定数）＝（２５５）^-2とした場合の入力Ｌと出力Ｌ'の関係を示すテーブルである。 <Image correction method 1: Gamma correction>
First, gamma correction processing will be described with reference to FIG.
The gamma (γ) correction is to correct an input / output characteristic curve of an image sensor such as a CCD (Charge Coupled Device) using a conversion table. The luminance or density of an input image is f (i, j) = L, Assuming that the luminance or density of the output image after γ correction is g (i, j) = L ′, the conversion equation for γ correction is defined as in equation (19).

The conversion table α in FIG. 18 is a table showing the relationship between the input L and the output L ′ when γ = 0.5 and (constant) = (255) ⁻² in equation (19).

<Image correction method 2: Correction by conversion of luminance or density histogram distribution>
Next, correction by converting a histogram distribution of luminance or density of a captured image will be described with reference to FIGS.

ｖ−ｕ＞ｂ−ａである場合、ダイナミックレンジが狭くコントラストの悪い入力画像の輝度ヒストグラム分布Ｐ（ｘ）を輝度変換式（２０）により変換することにより、ダイナミックレンジが広がり、コントラストや中間諧調の濃淡表現を改善することができる。なお、式（２０）における定数ｖ、ｕ、ａ、ｂの値を変えれば、ダイナミックレンジの収縮や、輝度ヒストグラム分布の左右シフト（全体の輝度を上げる、下げる）等にも応用できる。 Assuming that the luminance of the input photographed image is f (i, j) = x and the luminance of the converted image is g (i, j) = y, as shown in FIG. When (x) is converted by a linear or non-linear luminance conversion equation y = Q (x), a luminance histogram distribution P (y) having characteristics different from P (x) is obtained. If the minimum value of luminance x in the luminance histogram distribution P (x) is a, the maximum value is b, the minimum value of luminance y in the converted luminance histogram distribution P (y) is u, and the maximum value is v, FIG. The shown luminance conversion equation y = Q (x) is expressed as equation (20).

When v−u> b−a, the dynamic range is expanded by converting the luminance histogram distribution P (x) of the input image with a narrow dynamic range and poor contrast by the luminance conversion equation (20), and the contrast and intermediate gradation are increased. It is possible to improve the shading expression. Note that if the values of constants v, u, a, and b in Equation (20) are changed, the present invention can also be applied to dynamic range contraction, left / right shift of the luminance histogram distribution (increasing or decreasing the overall luminance), and the like.

FIG. 20 shows examples of other luminance conversion expressions.
FIG. 20A is an example of a non-linear luminance conversion equation in the case of performing a correction process in which the luminance of an image is darkened as a whole (lower luminance), and is expressed as equation (21). .
y = v (x / b) ² (21)

FIGS. 20B and 20C are examples of non-linear luminance conversion formulas in the case of performing a correction process in which the luminance of an image is brightened as a whole (the higher the luminance), respectively, the equations (22) , Expressed as equation (23).
y = −v {(x−b) / b} ² + v (22)
y / v = log (1 + μ x / b) / log (1 + μ) (23)

図２０（ｅ）に示す輝度変換式は、式（２５）のように表される。

図２０（ｄ）又は（ｅ）に示すような輝度変換式を用いると、階調表現のトーンカーブ（色調曲線）やコントラストを改善することができる。 20D and 20E are examples of luminance conversion formulas in the case where the luminance of a region with high luminance is increased and the luminance of a region with low luminance is decreased. The S-shaped luminance conversion equation shown in FIG. 20D is expressed as equation (24).

The luminance conversion equation shown in FIG. 20 (e) is expressed as equation (25).

By using a luminance conversion formula as shown in FIG. 20D or 20E, the tone curve (tone curve) and contrast of gradation expression can be improved.

  In this way, by converting the histogram distribution of density and brightness using a linear or non-linear brightness or density conversion formula, images with “exposure” or “blackout” due to poor exposure condition settings, contrast and gradation Image quality correction and processing can be performed, such as widening the dynamic range of images with poor tonal expression, improving contrast, light and dark on the screen, halftone gradation expression, and tone curve improvement.

  Further, the above correction processing may be performed for each luminance histogram distribution for each RGB color to improve the histogram distribution or correct the weighted balance of the luminance of each RGB color. For example, as is often the case with underwater photography, the B (blue) color component is overexposed and the brightness is high, but the R (red) color component is reduced (absorbed by water), resulting in underexposure. In the image, the B (blue) color component histogram is shifted slightly to the left to weaken, and the R (red) color component histogram is shifted slightly to the right to strengthen the histogram. Thus, the overall exposure balance can be corrected.

<Image correction method 3: Correction by flattening histogram distribution of luminance or density>
Next, a correction method by flattening the histogram distribution of luminance or density of a captured image will be described with reference to FIGS.

  As shown in FIG. 21, by using a distribution (FIG. 21 (a)) in which the luminance or density histogram distribution P (x) of the input image is biased and using the integral value of P (x) as a gradation conversion equation, A uniformly flat histogram distribution P (y) (FIG. 21B) is obtained.

  The histogram distribution flattening process shown in FIG. 21 will be described with reference to the flowchart of FIG.

  When an image to be corrected is input (step S60), the number of horizontal pixels × the number of vertical pixels of the input image is set as the total number of pixels N (step S61). Next, the number of gradations m (0 to m-1) of the output image after correction is determined (step S62), and the average number of appearance frequencies (number of pixels per gradation) is set to N / m (step S62). S63).

  Next, a histogram of the input image is calculated (step S64). Next, the counter value k (0 to m-1) for counting the gradation luminance (or density) is set to 0 (step S65), and the counter value Hist (k for counting the total number of gradations k. ) Is set to 0 (step S66).

  Next, pixels are extracted in order from the lowest gradation luminance (or density) of the input image, and the gradation of the corresponding pixel of the output image is set to k (step S67). Then, the total number Hist (k) of the gradation k is incremented (step S68), and it is determined whether or not the current total number Hist (k) is larger than the average appearance frequency number N / m (step S69).

  If it is determined in step S69 that Hist (k) is N / m or less (step S69; NO), the process returns to step S67, and the processes of steps S67 and S68 are repeated. If it is determined in step S69 that Hist (k) is greater than N / m (step S69; YES), the gradation k is incremented (step S70), and the current gradation k is the maximum gradation value m. It is determined whether it is larger than −1 (step S71).

  If it is determined in step S71 that the gradation k is equal to or less than m−1 (step S71; NO), the process returns to step S66, and the processes of steps S66 to S70 are repeated. If it is determined in step S71 that the gradation k is greater than m−1 (step S71; YES), an image after gradation conversion is output (step S72), and the histogram flattening process ends.

<Image Correction Method 4: Correction by Smoothing Image>
Next, with reference to FIG. 23, a correction method by smoothing a captured image will be described.

このようなフィルタ演算によって、入力画像のごま塩ノイズが除去され、明瞭度が落ち、ぼかす効果がある。明瞭度をあまり落とさずに、ノイズを除去するためには、メディアン（中央値）フィルタを用いた処理が有効である。メディアンフィルタは、着目画素とその周囲の９画素のうち、輝度が大きい方から５番目の中央値を、着目画素の新しい輝度値とするフィルタである。 By performing a spatial filter operation with an average value (MEAN) filter A (K, L) as shown in FIG. 23A or 9B on the pixel of interest of the input image and its surrounding pixels, the image The whole is smoothed. Assuming that the input image is f (i, j) and the output image after the filter operation is g (i, j), the spatial filter operation using the average value filter A (K, L) is as shown in Expression (26). expressed.

By such a filter operation, the sesame salt noise of the input image is removed, the clarity is lowered, and there is an effect of blurring. A process using a median (median) filter is effective in removing noise without significantly reducing the intelligibility. The median filter is a filter that uses, as the new luminance value of the pixel of interest, the fifth median value from the highest luminance among the pixel of interest and the surrounding nine pixels.

<Image correction method 5: Correction by sharpening using a differential filter>
Next, with reference to FIG. 24 and FIG. 25, correction processing for enhancing the outline and edge components of a captured image, reducing the blur of the image, and sharpening will be described.

As shown in FIG. 24A, when the gradient of the image f (i, j) is obtained by first-order differentiation of the image f (i, j) in which the edge of the portion where the gradation changes is blurred, As shown in FIG. 24B, a gradient portion where the gradation changes is extracted, and unevenness is obtained. The first derivative of the image f (i, j) is expressed as in Expression (27).
Δxf (i, j) = f (i + 1, j) −f (i−1, j), Δyf (i, j) = f (i, j + 1) −f (i, j−1) ( 27)
Here, Δx is the first derivative of the x direction (lateral direction), and Δy represents the first derivative of the y direction (vertical direction). In addition, the gradient having no direction is represented by Expression (28), Expression (29), and Expression (30).

In the second derivative for further differentiating the first derivative of the image f (i, j), as shown in FIG. 24C, an N-shaped waveform in the opposite direction is obtained, and an image in which high frequency components are emphasized is obtained. . The second derivative (Laplaciant) of the image f (i, j) is expressed as in Expression (31) or Expression (32).

  For the first derivative shown in the equation (27), a Prewitt filter as shown in FIG. 25A, a Sobel filter, a Kirsch filter, a Roberts filter as shown in FIG. In FIG. 25, when the Prewitt filter is applied to the input image, the input image on the line AA ′ becomes an output image as shown in FIG. 25D, and when the Sobel filter is applied to the input image, AA 'The input image on the line is an output image as shown in FIG. FIG. 25C shows an example of a filter that performs an operation of subtracting an image subjected to second order differentiation from the original image (input image). When a filter as shown in FIG. 25C is used, an image in which the high-frequency component at the edge portion is emphasized is obtained as shown in FIG. That is, this filter can be used for correction that enhances the sharpness of an image by enhancing the blurred edge or outline of the image, and sharpening the image.

例えば、図２５（ｃ）に示したフィルタＷ（Ｋ,Ｌ）による演算は、式（３３−１）のように表される。

Assuming that the differential filter as shown in FIGS. 25A to 25C is W (K, L), the filter operation using the differential filter W (K, L) is expressed as in Expression (33). .

For example, the calculation by the filter W (K, L) illustrated in FIG. 25C is expressed as Expression (33-1).

<Image Correction Method 6: Correction for sharpening the frequency-converted image>
Next, correction processing by performing sharpening processing on a frequency-converted captured image will be described.

  The image correction method 6 is a sharpening process for low-frequency components or high-frequency components of image data converted into a frequency space by frequency conversion such as two-dimensional discrete Fourier transform (DFT) or discrete cosine transform (DCT). Filter processing such as blurring processing and contour enhancement processing is performed. The conversion formula of the discrete Fourier transform and the conversion formula of the discrete cosine transform of the two-dimensional image f (x, y) are the same as the expressions (7) and (8) shown in the image evaluation method 2, respectively.

  If the DC component and low-frequency component of F (u, v) subjected to frequency conversion are left (or enhanced) and the high-frequency component is set to 0, the processing becomes the same as that of the low-frequency pass filter (LPF), and edges and contours are removed. It is a weakening correction that blurs the image. Further, if the DC component and the high-frequency component of F (u, v) subjected to frequency conversion are left (or enhanced) and the low-frequency component is set to 0, the processing becomes the same as that of the high-frequency pass filter (HPF), and edges and contours are removed. This is a correction to emphasize. Further, when the value of the high frequency component of F (u, v) subjected to frequency conversion is increased or when a primary differential filter or a secondary differential filter (Laplacian filter) is applied to F (u, v), This is a correction that emphasizes the contour. When an image subjected to the above-described processing in the frequency space is subjected to frequency inverse transform (discrete Fourier inverse transform, discrete cosine inverse transform, etc.), an image subjected to sharpening processing, blurring processing, and the like can be obtained.

<Image Correction Method 7: Correction Using PSF Method>
Next, correction processing of a linearly blurred image due to camera shake or the like and a defocused image such as a blurred image using the PSF method will be described.

  G (u, v) obtained by Fourier transforming an image g (x, y) in which an original image i (x, y) having no linear blurring or defocusing is deteriorated by a degradation component p (x, y) such as blurring or blurring. ), And from the pattern on the frequency space, as shown in the image evaluation method 3, the straight blur direction (angle θ), the straight blur distance (number of pixels L), the focal blur spread radius (pixel number r) ) Is calculated.

In the case where a straight line blur that crosses zero with a period of 1 / L occurs in the θ direction, P (u, v) in Expression (15-2) is expressed as Expression (34) below.
P (u, v) = sin (πfL) / πfL (34)
Here, f = u cos θ + v sin θ. Then, from G (u, v) and P (u, v) in equation (34), the original Fourier-transformed image I (u, v) = G (u, v) / P (u , V), and the original image i (x, y) can be obtained by inverse Fourier transforming this I (u, v) as shown in equation (16).

In the case where a focal blur having a concentric zero crossing occurs at a cycle of 1.01π / r, P (u, v) in Expression (15-2) is expressed as Expression (35) below.
P (u, v) = 2 · J1 (r · R) / r · R (35)
Here, J1 in Equation (35) is a first-order first-type Bessel function. Then, similarly to the above, from the G (u, v) obtained by Fourier transform of the received image data and P (u, v) in the equation (35), the Fourier transform image I ( u, v) = G (u, v) / P (u, v) is obtained, and the original image i (x, y) is obtained by performing inverse Fourier transform on this I (u, v) as shown in Expression (16). ).

  Note that p (x, y) or P (u, v) may be calculated by using a general PSF, LSF, inverse filter method, least square method, or other methods such as a Wiener filter. An inverse filter (inverse filter) may be obtained by a method, or a deconvolution operation may be performed to restore an image or correct blur and blur.

  As described above, according to the digital camera 1 of the present embodiment, when the remaining capacity of the recording medium is equal to or less than a preset capacity, the captured image is automatically evaluated, and an image whose evaluation value is equal to or less than a predetermined value. Is automatically corrected, and even after correction processing, only images whose image evaluation values are below a predetermined value are deleted, so that images are not deleted more than necessary and recording media is free. It is possible to ensure the area.

(Modification)
Next, a modification of the above-described embodiment will be described with reference to FIGS. In this modification, in the production process of the digital camera 1, degradation components such as blurring and blurring are measured and set in the digital camera 1, and a captured image is corrected based on the set degradation components. An example will be described. In addition, although this modification shows the case where the degradation component of the image set in the production process of the digital camera 1 is linear blurring or defocusing, this modification can also be applied to other degradation components. .

  First, with reference to the flowchart of FIG. 26, the blur deterioration component setting process executed in the production process of the digital camera 1 will be described.

  First, a sample image i (x, y) to be photographed is set (step S80), the linear blur direction angle θ is set to 0 (step S81), and the linear blur distance L is set to 0. (Step S82). Next, a sample image is captured by causing a linear blur having a blur direction angle θ and a blur distance L, and a captured image g (x, y) is obtained (step S83). In step S83, the image is shot so as to cause blurring using a robot hand, a dedicated experimental device, or the like.

  Next, the captured image g (x, y) is Fourier-transformed to obtain a captured image G (u, v) in the frequency space (step S84). Next, the degradation component p (x, y) is subjected to Fourier transform P (I (u, v) obtained by Fourier transform of the sample image i (x, y) and G (u, v) obtained by Fourier transform of the photographed image. u, v) = G (u, v) / I (u, v) is calculated, and the calculated P (u, v) is a degradation component P1 (θ with a blur direction angle θ and a blur distance L. , L) is stored in the memory (ROM) of the control unit 32 (step S85).

  Next, the blur distance L is incremented (step S86), and it is determined whether or not the current blur distance L is equal to or greater than a preset maximum value Lmax (step S87). If it is determined in step S87 that the blur distance L is less than Lmax (step S87; NO), the process returns to step S83, and the processes in steps S83 to S86 are repeated.

  If it is determined in step S87 that the blur distance L is greater than or equal to Lmax (step S87; YES), the blur direction angle θ is incremented (step S88), and the current blur direction angle θ is set to a preset maximum value θmax. It is determined whether or not this is the case (step S89).

  If it is determined in step S89 that the blur direction angle θ is less than θmax (step S89; NO), the process returns to step S83, and the processes of steps S83 to S88 are repeated. In step S89, when it is determined that the blur direction angle θ is equal to or greater than θmax (step S89; YES), the blur deterioration component setting process ends.

  Next, with reference to the flowchart of FIG. 27, the defocusing deterioration component setting process executed in the production process of the digital camera 1 will be described.

  First, a sample image i (x, y) to be photographed is set (step S100), and the focal blur spread radius r is set to 0 (step S101). Next, a sample image is shot with a focal blur having a spread radius r, and a shot image g (x, y) is acquired (step S102). In step S102, an image is taken using a robot hand, a dedicated experimental device, or the like so as to cause out of focus.

  Next, the photographed image g (x, y) is Fourier transformed to obtain a photographed image G (u, v) in the frequency space (step S103). Next, the degradation component p (x, y) is subjected to Fourier transform P (I (u, v) obtained by Fourier transform of the sample image i (x, y) and G (u, v) obtained by Fourier transform of the photographed image. u, v) = G (u, v) / I (u, v) is calculated, and the calculated P (u, v) is used as the deterioration component P2 (r) of the focal blur spread radius r, and the control unit 32 is stored in the memory (ROM) 32 (step S104).

  Next, the spread radius r is incremented (step S105), and it is determined whether or not the current spread radius r is greater than or equal to a preset maximum value rmax (step S106). If it is determined in step S106 that the spread radius r is less than rmax (step S106; NO), the process returns to step S102, and the processes of steps S102 to S105 are repeated. If it is determined in step S106 that the spread radius r is equal to or greater than rmax (step S106; YES), the blur deterioration component setting process ends.

  Next, image shooting / correction processing executed in the digital camera 1 after shipment will be described with reference to the flowchart of FIG.

  When the photographing mode is designated by operating the key input unit 36, photometric processing, exposure condition setting, and the like are performed (step S110). After the light metering process, when shooting is instructed by pressing the shutter key of the key input unit 36, the shooting process is performed and a shot image g (x, y) is acquired (step S111). The acquired captured image g (x, y) is recorded on a recording medium (flash memory 38).

  In the photographing process of step S111, when the camera shake detection sensor 40 detects a camera shake, the detected camera shake direction θ and the camera shake distance L are stored in the RAM of the control unit 32 (step S112). In addition, when a focus blur is detected from the captured image g (x, y), the spread radius r of the focus blur is stored in the memory (RAM) of the control unit 32 (step S112).

  Next, the captured image g (x, y) is Fourier-transformed to obtain a captured image G (u, v) in the frequency space (step S113). Next, it is determined whether or not the blur distance L stored in step S112 is greater than or equal to a preset value (set value) (that is, whether or not the evaluation value is less than or equal to a predetermined value) (step S114). .

  If it is determined in step S114 that the blur distance L is less than the set value (that is, the evaluation value is greater than the predetermined value) (step S114; NO), the process proceeds to step S119 described later. If it is determined in step S114 that the blur distance L is greater than or equal to the set value (that is, the evaluation value is less than or equal to a predetermined value) (step S114; YES), the blur direction θ and the blur distance L stored in step S112 are stored. Is read from the memory (ROM) of the control unit 32 (step S115), and from G (u, v) and P1 (θ, L), the original blur-free component P1 (θ, L) is read. I (u, v) = G (u, v) / P1 (θ, L) obtained by Fourier transform of the image i (x, y) is calculated (step S116).

  Next, the restored image i (x, y) is calculated by inverse Fourier transform of I (u, v) calculated in step S116 (step S117), and g (x, y) recorded on the recording medium. ) Is updated to the restored image i (x, y) (step S118).

  Next, it is determined whether or not the focal blur spread radius r stored in step S112 is greater than or equal to a preset value (set value) (that is, whether or not the evaluation value is less than or equal to a predetermined value) ( Step S119). When it is determined in step S119 that the defocusing radius r of the defocus is less than the set value (that is, the evaluation value is greater than the predetermined value) (step S119; NO), the image photographing / correction process ends.

  When it is determined in step S119 that the focal blur spread radius r is greater than or equal to the set value (that is, the evaluation value is smaller than or equal to a predetermined value) (step S119; YES), the focal blur spread stored in step S112 is determined. The degradation component P2 (r) corresponding to the radius r is read from the memory (ROM) of the control unit 32 (step S120), and an original image without defocusing is obtained from G (u, v) and P2 (r). I (u, v) = G (u, v) / P2 (r) obtained by Fourier transform of i (x, y) is calculated (step S121).

  Next, I (u, v) calculated in step S121 is inversely Fourier transformed to calculate a restored image i (x, y) (step S122), and g (x, y) recorded on the recording medium. ) Is updated to the restored image i (x, y) (step S123), and the main image photographing / correcting process is completed.

  As described above, in the production process of the digital camera 1, a sample image is taken in advance so as to cause blurring and blurring, and the deterioration component P (u, v) is calculated for each degree of blurring and blurring and stored in the memory. By setting and correcting the image taken with the digital camera 1 after shipment based on the degradation component P (u, v) set in the memory, image correction that matches the performance of the lens of each camera is performed. It can be done easily. Further, since it is not necessary to calculate the degradation component P (u, v) after shipment, the image correction process can be performed at high speed.

  Recently, digital cameras equipped with an autofocus function have become widespread, but the accuracy and characteristics of autofocus differ for each camera model. Also, depending on the exposure conditions, ambient brightness, subject placement, etc., it is not always possible to shoot an image focused on the desired subject, and the depth of field may vary depending on the degree of aperture and focal length. Or change. Accordingly, as shown in FIGS. 26 to 28, before production shipment, degradation components are calculated for each degree of image degradation and set in the memory, and images taken after shipment are converted into degradation components set in the memory. By performing correction based on the above, it is possible to perform correction corresponding to differences in lens characteristics and focus characteristics among individual models.

  In addition, the description content in this embodiment and a modification can be suitably changed in the range which does not deviate from the meaning of this invention.

1 is a block diagram showing a circuit configuration of a digital camera to which an image photographing apparatus of the present invention is applied. 5 is a flowchart showing image shooting / deleting processing executed in the digital camera of the present embodiment. The flowchart which shows the detail of the image deletion process in FIG. The figure which shows the example of a screen display in the display part of a digital camera. The figure which shows the example of a screen display in the display part of a digital camera. The figure which shows the example of a screen display in the display part of a digital camera. The figure which shows an example of the method of evaluating an image from the brightness | luminance histogram distribution of an image. The figure which shows an example of the method of evaluating an image from the brightness | luminance histogram distribution of an image. The figure which shows an example of the method of evaluating an image with the frequency characteristic of an image. The figure which shows an example of the method of evaluating an image with the frequency characteristic of an image. The figure which shows an example of the method of evaluating an image with the frequency characteristic of an image. The figure which shows an example of the image evaluation method by the measurement of the linear blurring of an image and the degree of a focus blur. The figure which shows the calculation method of the angle ((theta)) of the direction of a linear blurring, and the calculation method of the distance (L) of a linear blurring. The figure which shows the calculation method of the expansion radius (r) of a focus blur. The figure for demonstrating Hough conversion. The figure for demonstrating multi-gradation Hough conversion. The conceptual diagram of an image quality correction process. The figure which shows the image correction method by a gamma correction. The figure which shows the image correction method by conversion of the histogram distribution of a brightness | luminance or a density | concentration. The figure which shows the image correction method by conversion of the histogram distribution of a brightness | luminance or a density | concentration. The figure which shows the image correction method by flattening the histogram of a brightness | luminance or a density | concentration. The flowchart which shows a histogram flattening process. The figure which shows the image correction method by the smoothing of an image. The figure which shows the image correction method by sharpening using a differential filter. The figure which shows the image correction method by sharpening using a differential filter. The flowchart which shows the blurring deterioration component setting process performed in the production process of a digital camera. The flowchart which shows the focus blurring degradation component setting process performed in the production process of a digital camera. 7 is a flowchart showing image photographing / correction processing.

Explanation of symbols

1 Digital camera (image capture device)
2 External device 3 Image processing device 4 Display device 5 Printing device 13 Display unit 21 Motor (M)
22 Lens optical system 23 CCD
24 TG (timing generator)
25 Vertical driver 26 S / H (sample hold circuit)
27 A / D converter 28 Color process circuit 29 DMA controller 30 DRAM I / F
31 DRAM
32 Control unit 33 VRAM controller 34 VRAM
35 Digital video encoder 36 Key input unit 37 JPEG circuit 38 Flash memory 39 Communication unit 40 Camera shake detection sensor

Claims (14)

Imaging means for photographing a subject and obtaining a captured image;
Recording means for recording the acquired photographed image on a recording medium;
Display means for displaying the recorded photographed image;
First evaluation means for evaluating the image quality of the acquired captured image;
First deciding means for deciding, as a candidate for correcting a photographic image whose evaluation result obtained by the first evaluation means satisfies a predetermined condition among the photographic images recorded on the recording medium ;
First display control means for causing the display means to display a captured image determined as a candidate to be corrected by the first determination means;
First instruction means for instructing whether or not to correct the captured image displayed by the first display control means;
A correction unit that corrects the image quality of the captured image when it is instructed to perform correction by the first instruction unit ;
Second evaluation means for evaluating the image quality of the captured image whose image quality has been corrected by the correction means;
Said first instruction photographed image is instructed not to perform correction by means, and second the result of evaluation by the second evaluation means for determining a predetermined condition is satisfied captured image, as a candidate to be deleted from the recording medium And means for determining
An image photographing apparatus comprising: Determination means for determining whether or not the remaining capacity of the recording medium is equal to or less than a preset capacity;
The second determining means is instructed not to perform correction by the first instructing means when the determining means determines that the remaining capacity of the recording medium is equal to or less than a preset capacity . photographic images, and the image capturing apparatus according to claim 1, evaluation results of the second evaluation means and determines as a candidate to delete predetermined condition is satisfied captured image from the recording medium. Second display control means for causing the display means to display a captured image determined as a candidate for deletion by the second determination means;
Second instruction means for instructing whether or not to delete the captured image displayed by the second display control means from the recording medium;
The image photographing apparatus according to claim 1, further comprising a deleting unit that deletes the captured image from the recording medium when the second instruction unit instructs to delete the image. The said 1st evaluation means and 2nd evaluation means evaluate the image quality of the said picked-up image based on the histogram distribution of the brightness | luminance or density | concentration of a picked-up image, The any one of Claims 1-3 characterized by the above-mentioned. The image photographing device described in 1. The image according to any one of claims 1 to 3, wherein the first evaluation unit and the second evaluation unit evaluate the image quality of the captured image based on the frequency characteristics of the captured image. Shooting device. The said 1st evaluation means and 2nd evaluation means evaluate the image quality of the said picked-up image based on the degree of the blurring or a focus blur of a picked-up image, The any one of Claims 1-3 characterized by the above-mentioned. The image photographing device described. It said correction means, the image capturing apparatus according to any one of claim 1 to 6, characterized in that performs gamma correction on the captured image. The image according to any one of claims 1 to 6 , wherein the correction unit corrects the captured image by converting a histogram distribution of luminance or density of the captured image by a predetermined conversion formula. Shooting device. Said correcting means by performing flattening the histogram distribution of brightness or density of the captured image, the image capturing apparatus according to any one of claim 1 to 6, characterized in that to correct the captured image. It said correction means, the image capturing apparatus according to any one of claim 1 to 6, wherein the correcting the captured image using a spatial filter for smoothing the image. It said correction means, the image capturing apparatus according to any one of claim 1 to 6, characterized in that to perform the process of sharpening the photographed image by using a predetermined differential filter. Wherein the correction means, the captured image frequency conversion for the frequency-converted photographic image, the image capturing according to any one of claim 1 to 6, characterized in that a sharpening process or blurring apparatus. Wherein the correction means uses the point spread function or line spread function, an image capturing apparatus according to any one of claim 1 to 6, characterized in that to correct the linear blur or defocus of the captured image . A degraded image capturing means for capturing a sample image in a manner that causes image degradation;
Calculating means for calculating a deterioration component of the image obtained by the deteriorated image capturing means;
Storage means for storing information on degradation components of the image calculated by the calculation means according to the degree of image degradation,
When the evaluation result of the captured image acquired by the imaging unit satisfies a predetermined condition, the correction unit reads out the degradation component corresponding to the captured image from the storage unit, and based on the read degradation component image capturing apparatus according to any one of claim 1 to 13, characterized in that to correct the captured image.

Images