JP2006180078A - Electronic camera, image processor and image processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an electronic camera which can easily judge the picture quality of a face region and has a small calculation load on the camera side. <P>SOLUTION: The electronic camera includes: an imaging device which photoelectrically converts a subject image to generate an image signal; an image processing section which applies image processing to the image signal to generate a photographic image data; a mode switching section which switches the mode between a 1st image processing mode where the picture quality of the face region in the photographic image data is judged and a 2nd image processing mode where no picture quality judgment is made according to parameters of photographic conditions; a face recognition section which extracts the face region from the photographic image data in the 1st image processing mode; a spatial frequency analysis section which analyzes a spatial frequency component in the face region; and a face picture quality judgment section which judges the picture quality of the face region based on the spatial frequency component having high correlation with a feeling of granularity of an image and detects special image data whose picture quality is equal to or below a threshold. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an electronic camera that photoelectrically converts a subject image to generate image data, and more particularly to an electronic camera that can determine the image quality of a face area of a captured image by a face recognition function.

  In recent years, electronic cameras that electronically record subject images with an image sensor have been rapidly spreading. In such an electronic camera, depending on the shooting conditions, a sense of graininess may appear in the shot image and the appearance may deteriorate. In particular, when the subject is a person, the face portion of the photographed image is required to have high image quality, and if the face portion of the photographed image has a graininess, it is often regarded as a photographing failure.

On the other hand, Patent Document 1 discloses an electronic camera that obtains a good captured image by recognizing a face from image data and associating color balance with the presence of the face.
JP, 2001-309225, A [0034]

The captured image of the electronic camera is recorded on a recording medium such as a semiconductor memory. When a photographed image having a grainy appearance on the face portion as described above is deleted or corrected on the PC, the user has to check a large number of photographed images one by one, and a great deal of labor is required.
By the way, the graininess of the captured image is caused by the appearance of the high frequency component of the spatial frequency component included in the captured image as noise. Therefore, if the spatial frequency of the photographed image is analyzed, the image quality determination of the face area can be fully automated. On the other hand, since the image quality determination of the face area has a large calculation load on a computer or the like, it is preferable to exclude, for example, a camera shake image from the image quality determination target.

Note that the technique of Patent Document 1 merely suggests that the result of face recognition can be applied to skin tone correction, and does not solve the above problem.
The present invention is to solve the above-described problems of the prior art, and an object of the present invention is to provide an electronic camera that can easily determine the image quality of a face area and has a low calculation load on the camera side. is there.

  An electronic camera according to a first aspect of the present invention includes an image sensor that photoelectrically converts a subject image to generate an image signal, an image processing unit that performs image processing on the image signal to generate captured image data, and the captured image data A mode switching unit that switches between a first image processing mode for determining the image quality of the face area and a second image processing mode for which the image quality determination is not performed according to a parameter of the shooting condition, and a face area in the captured image data. A face recognition unit for extracting; a face region analysis unit for analyzing the face region; and determining an image quality of the face region based on an analysis result of the face region analysis unit in the first image processing mode. And a face image quality determination unit that detects special image data equal to or less than a threshold value.

  An electronic camera according to a second aspect of the present invention includes an image sensor that photoelectrically converts a subject image to generate an image signal, an image processing unit that performs image processing on the image signal to generate captured image data, and the captured image data A face recognition unit that extracts a face region in the image, a spatial frequency analysis unit that analyzes a spatial frequency component of the face region and any partial region other than the face region, and first image processing that performs image quality determination of the face region A mode switching unit that switches between a mode and a second image processing mode that does not perform the image quality determination based on the analysis result; and a spatial frequency component that has a high correlation with the graininess of the image in the first image processing mode. And a face image quality determination unit for determining the image quality of the face area and detecting special image data having the image quality equal to or less than a threshold value.

According to a third invention, in the first or second invention, there is further provided notification means for notifying a camera user that the photographed image data is special image data during photographing or reproduction.
According to a fourth invention, in the first or second invention, further comprising: a connector for connecting a recording medium; and a control unit for recording the photographed image data on the recording medium. The special image data is excluded from the recording target of the medium.

According to a fifth invention, in the first or second invention, the image processing unit performs a correction process for reducing a graininess of an image on a face area of the special image data.
In a sixth aspect based on the fifth aspect, the face recognition unit extracts a predetermined face part in the face area of the special image data, and the image processing unit extracts the face part area from the correction processing target. It is characterized by excluding.

According to a seventh invention, in the fifth or sixth invention, the correction processing of the image processing unit has a high graininess in the vicinity of the boundary portion of the face region, and is higher than the boundary portion inside the face region. Is also weighted to reduce the graininess of the image.
An electronic camera according to an eighth aspect of the present invention is an image sensor that photoelectrically converts a subject image to generate an image signal, an image processing unit that performs image processing on the image signal to generate captured image data, and the captured image data A face recognition unit that extracts a face region in the image, a control unit that changes the gain setting of the image signal and at least one of shutter times, and performs a plurality of shootings, and a spatial frequency component of the face region in the captured image data A face image quality determination unit that determines the image quality of the face region based on a spatial frequency component having a high correlation with the granularity of the image, and detects captured image data having the highest image quality in multiple shootings; It is characterized by having.

  A ninth aspect of the invention is an image processing apparatus that reads an image file including image data generated by an electronic camera and additional data indicating a shooting condition parameter of the electronic camera, and processes the image data. Switching between the reading unit for reading the image file, the first image processing mode for determining the image quality of the face area in the captured image data, and the second image processing mode for not performing the image quality determination, according to the parameters of the imaging condition Based on the analysis result of the face area analysis unit in the first image processing mode, a face recognition unit that extracts a face area in the image data, a face area analysis unit that analyzes the face area And a face image quality determination unit that determines image quality of the face area and detects special image data having the image quality equal to or less than a threshold value.

  A tenth aspect of the invention is an image processing program for reading an image file including image data generated by an electronic camera and additional data indicating a parameter of photographing conditions of the electronic camera, and processing the image data, A procedure for reading the image file, and a procedure for switching between a first image processing mode for determining the image quality of the face area in the captured image data and a second image processing mode for not performing the image quality determination according to the parameters of the imaging condition. Determining the image quality of the face area based on the analysis result of the face area in the first image processing mode, a procedure for extracting the face area in the image data, a procedure for analyzing the face area, And causing the computer to execute a procedure for detecting special image data whose image quality is equal to or less than a threshold value.

  An eleventh aspect of the invention is an image processing apparatus that reads an image file including image data generated by an electronic camera and processes the image data, a reading unit that reads the image file, and a face region from the image data A face recognition unit that extracts a spatial frequency component of the face region and an arbitrary partial region other than the face region, a first image processing mode that performs image quality determination of the face region, and A mode switching unit that switches a second image processing mode that does not perform image quality determination based on the analysis result; and the face based on a spatial frequency component that has a high correlation with the graininess of the image in the first image processing mode. A face image quality determining unit that determines the image quality of the area and detects special image data having the image quality equal to or less than a threshold value.

  A twelfth aspect of the invention is an image processing program for reading an image file including image data generated by an electronic camera and processing the image data, a procedure for reading the image file, and a face area from the image data. A procedure for extracting, a procedure for analyzing a spatial frequency component of the face region and an arbitrary partial region other than the face region, a first image processing mode for determining the image quality of the face region, and a step for not performing the image quality determination. Determining the image quality of the face region based on a procedure for switching between the two image processing modes based on the analysis result, and a spatial frequency component highly correlated with the graininess of the image in the first image processing mode, And causing the computer to execute a procedure for detecting special image data whose image quality is equal to or less than a threshold value.

In the electronic camera according to the present invention, the first image processing mode for determining the image quality of the face area and the second image processing mode for not performing the image quality determination are switched according to the parameters of the shooting condition, so that appropriate image processing is executed. can do.
In the electronic camera, image processing apparatus, and image processing program of the present invention, a face area is extracted from captured image data, and special image data having a rough face area image quality is obtained based on a spatial frequency component having a high correlation with the graininess of the image. Since it is detected, it is possible to easily determine the image quality of the face area of the captured image data.

  In the electronic camera, the image processing apparatus, and the image processing program of the present invention, it is difficult to determine the image quality of the face area by determining the presence or absence of camera shake according to the parameters of the shooting conditions and the analysis results of the spatial frequency components at a plurality of locations. Since the image quality of the face area is determined after the captured image data is excluded from the processing target in advance, the calculation load on the camera or image processing apparatus is reduced.

(Description of the first embodiment)
FIG. 1 is a block diagram of the electronic camera of the first embodiment (corresponding to the electronic camera of claims 1 to 3). The electronic camera 10 includes an imaging lens 11, an image sensor 12, an analog signal processing unit 13, an A / D conversion unit 14, an image processing unit 15, a compression / decompression processing unit 16, a memory 17, and a capacitor storage. A flash light emitting unit 18 that causes the xenon arc tube to emit light with electric charge, a timing generator (TG) 19, a card interface (card I / F) 20, a recording medium 21, a monitor 22, an operation unit 23, a CPU 24, and each unit And a data bus 25 for connecting the two.

The image sensor 12 is disposed on the image space side of the photographing lens 11. On the light receiving surface of the image sensor 12 (the surface opposite to the photographing lens 11), light receiving pixels for photoelectrically converting a subject image to generate an analog image signal are two-dimensionally arranged. The output of the image sensor 12 is connected to the analog signal processing unit 13.
The analog signal processing unit 13 includes a CDS circuit that performs correlated double sampling, a gain circuit that amplifies the output of the analog image signal, a clamp circuit that clamps the waveform of the input signal to a certain voltage level, and the like. The A / D conversion unit 14 converts the analog image signal output from the analog signal processing unit 13 into a digital image signal.

The image processing unit 15 performs image processing (defective pixel correction, gamma correction, interpolation, color conversion, edge enhancement, etc.) on the digital image signal to generate captured image data. In particular, the image processing unit 15 of the first embodiment has the following functions.
(A) First, the image processing unit 15 performs a known face recognition process on the captured image data to detect the face area of the subject. In addition, as an example of face recognition processing, Japanese Patent Laid-Open No. 8-63595 discloses a method for extracting a contour of a skin color region based on color and detecting a face with a matching degree with a face contour template prepared in advance. Prepared in advance including eye detection method using face matching with eye template by finding eye candidate area, 2D Fourier transform result of face candidate area obtained from face outline template, eyes, nose, mouth, hair, etc. A method is disclosed in which a feature amount defined from the two-dimensional Fourier transform result of the face template image obtained is obtained, and the feature amount is subjected to threshold processing to perform face detection.

  (B) Secondly, the image processing unit 15 analyzes a spatial frequency component included in the face area of the captured image data by fast Fourier transform (FFT) or the like. Then, the image processing unit 15 detects special image data in which the image quality of the face area is rough (the face area has a strong grain feeling) in the captured image data based on a spatial frequency component having a high correlation with the graininess of the image. . Note that the “spatial frequency component having a high correlation with the granularity of the image” is set based on the characteristics of the electronic camera statistically obtained in advance (for example, noise is likely to occur if a specific spatial frequency component is strong). .

(C) Thirdly, the image processing unit 15 performs a correction process for reducing the granularity of the image on the face area of the special image data by a blur filter or the like according to an instruction from the CPU 24.
The compression / decompression processing unit 16 executes processing for compressing image data after image processing in JPEG format, and processing for decompressing and restoring JPEG format compressed image data. The memory 17 is composed of SDRAM or the like, and temporarily stores photographed image data before or after image processing by the image processing unit 15.

The TG 19 generates a drive signal at a predetermined timing based on an instruction from the CPU. The drive signal for the TG 19 is supplied to the image sensor 12 and is used for discharging unnecessary charges from the light receiving pixels, accumulating charges and outputting accumulated charges.
The card I / F 20 is formed with a connector for detachably connecting the recording medium 21. The recording medium 21 is composed of a known semiconductor memory or the like, and the above-described photographic image data is finally recorded on the recording medium 21.

The monitor 22 is mainly disposed on the back portion of the electronic camera 10. The monitor 22 displays a captured image data reproduction screen, a setting screen for changing various settings of the electronic camera 10, a notification screen described later, and the like. The operation unit 23 includes a cross-shaped cursor key. The operation unit 23 is used for various inputs on the setting screen.
The CPU 24 executes, for example, arithmetic processing such as AE calculation and AF calculation, a series of image processing control at the time of shooting, light emission timing control of the flash light emitting unit 18, data writing / reading control to the recording medium 21, and the like. Each part of the camera 10 is controlled.

In particular, the CPU 24 according to the first embodiment switches ON / OFF of the image quality determination of the face area by the image processing unit 15 according to the parameter of the shooting condition. Here, the above switching is based on “whether or not the scene mode of the electronic camera is person-oriented” or “whether camera shake is likely to occur during shooting”.
For example, when the scene mode is the landscape mode, the CPU 24 turns off the image quality determination of the face area, and when the scene mode is the portrait mode, the person is the main subject and turns on the image quality determination of the face area. In addition, when the flash light emitting unit is not illuminated during long-time shooting, there is a high possibility that it will be difficult to determine the image quality of the face area due to camera shake during shooting, so the CPU 24 turns off the image quality determination of the face area in advance. Furthermore, when a known camera shake detection mechanism that detects camera shake by the output of the acceleration sensor is mounted on the electronic camera 10, the CPU turns off the image quality determination of the face area when the camera shake detection mechanism detects camera shake. It may be. When a known camera shake correction mechanism that shifts the photographing lens 11 to cancel camera shake is mounted on the electronic camera 10, the CPU 24 turns on the image quality determination of the face area when camera shake correction is ON. (The illustration of the camera shake detection mechanism and the camera shake correction mechanism is omitted).

When the image quality determination of the face area is ON, the CPU 24 performs processing at the time of detecting the special image data based on the input on the setting screen, (1) automatically corrects the image quality of the face area of the special image data. 2) The special image data is automatically deleted by prohibiting writing to the recording medium, and (3) the notification screen is displayed on the monitor.
Hereinafter, the photographing operation of the electronic camera of the first embodiment will be described with reference to the flowchart of FIG.

Step S101: The camera user performs processing when special image data is detected on the setting screen as "(1) Automatic correction of image quality of face area", "(2) Automatic deletion of special image data", "(3) Notification screen" Select and input from "Display".
Step S102: The camera user instructs the electronic camera 10 to shoot a subject by fully pressing a release button (not shown). The CPU 24 of the electronic camera 10 drives the image sensor 12 via the TG 19 and the image sensor 12 outputs an analog image signal of the subject image. The analog image signal passes through the analog signal processing unit 13 and is then converted into a digital image signal by the A / D conversion unit 14. Then, the image processing unit 15 performs predetermined image processing on the digital image signal to generate captured image data.

Step S103: The CPU 24 determines whether or not to determine the image quality of the face area based on the shooting condition parameters in the shooting of S102. When the image quality determination of the face area is performed (YES side), the process proceeds to S104. On the other hand, when the image quality determination of the face area is not performed (NO side), the process proceeds to S110.
Step S104: The image processing unit 15 performs face recognition processing on the captured image data, and outputs the presence or absence of a face in the captured image data to the CPU 24. The CPU 24 determines whether a face is detected from the captured image data based on the output of the image processing unit 15. If a face is detected from the captured image data (YES side), the process proceeds to S105. On the other hand, if no face is detected from the captured image data (NO side), the process proceeds to S110.

Step S105: The image processing unit 15 analyzes the spatial frequency component included in the face area of the captured image data.
Step S106: The CPU 24 determines whether the image quality of the face area is special image data having a threshold value or less (specifically, whether the amount of the spatial frequency component having a high correlation with the graininess of the image in the face area is equal to or more than the threshold value). ). If the captured image data is special image data (YES side), the process proceeds to S107. On the other hand, if the captured image data is not special image data (NO side), the process proceeds to S110.

  Step S107: The CPU 24 determines the processing at the time of detecting special image data based on the setting content in S101. If “(1) Automatic correction of image quality of face area” is selected and input in S101, the process proceeds to S112. If “(2) Automatic deletion of special image data” is selected and input in S101, the process proceeds to S111. When “(3) Display of notification screen” is selected and input in S101, the process proceeds to S108.

  Step S108: The CPU 24 displays a notification screen on the monitor 22. In this notification screen, a message indicating that the captured image is special image data (rough face image) and the face is not photographed beautifully, and “Save (special image to recording medium) as selection options for special image data processing” "Data write permission", "Deletion (prohibition of writing special image data to recording medium)", and "Correction (face area image quality correction processing)" are displayed (see FIG. 3). Then, the camera user operates the icon displayed on the monitor 22 with the operation unit 23 to select and input the special image data processing.

Step S109: The CPU 24 determines which process is selected and input by the camera user in S108. When “Save” is selected, the process proceeds to S110. When “Delete” is selected, the process proceeds to S111. If “Correction” is selected, the process proceeds to S112.
Step S110: In this case, the CPU 24 records the captured image data generated in S102 on the recording medium 21 as it is, and thereafter ends the photographing operation.

Step S111: In this case, the CPU 24 deletes the photographed image data (special image data) generated in S102 and ends the photographing operation without recording the special image data on the recording medium 21.
Step S112: In this case, the image processing unit 15 performs a filter operation with a blur filter on the face area of the special image data, and performs a correction process to reduce the graininess of the face area (see FIG. 4). In the filter calculation using the blur filter described above, the image processing unit 15 excludes the face part regions such as the eyes, nose, mouth, and eyebrows extracted during face recognition from the correction processing target, and the contour and fineness of the face parts are excluded. It is preferable to avoid losing unnecessary information.

  Further, as a result of the correction process using the blur filter, there is a significant difference in the graininess of the image between the face portion in the image and the outside of the face portion, and the face portion may float from the surroundings, resulting in a feeling of strangeness. Therefore, the image processing unit 15 may perform the correction process by weighting so that the intensity of the blurring effect changes between the vicinity of the boundary portion and the inside of the face region. For example, the image processing unit 15 applies a filter with a weak blur effect near the boundary portion in the face area, and applies a filter with a strong blur effect to the other parts in the face area. Alternatively, the image processing unit 15 may apply a filter with a weak blur effect near the boundary portion outside the face area, and apply a filter with a strong blur effect in the face area. In these cases, the difference in the graininess of the image is smoothed from the vicinity of the boundary portion of the face region toward the inside of the face region, so that the possibility that a sense of incongruity between the face portion and its surroundings is reduced is also reduced.

Then, the CPU 24 records the special image data after the correction process on the recording medium 21, and thereafter ends the photographing operation. Note that the CPU 24 can of course perform a correction process for reducing the graininess of the face area on the special image data recorded on the recording medium 21 after the fact.
The electronic camera of the first embodiment described above can detect special image data with poor image quality of the face area (high image granularity in the face area) based on the result of the spatial frequency analysis of the face area. Particularly in the first embodiment, since the image quality determination ON / OFF of the face area is switched according to the parameters of the shooting conditions, the shot image data that does not require the image quality determination of the face area or the shot image data that is shaken is excluded from the determination target. The calculation load of the electronic camera is reduced.

In the electronic camera according to the first embodiment, as special image data detection processing, any one of automatic deletion of special image data, automatic correction processing, and notification to the camera user can be selected depending on the situation. User convenience is greatly improved.
(Description of Second Embodiment)
FIG. 5 is a flowchart showing the operation of the electronic camera of the second embodiment (corresponding to the electronic camera of claims 2 to 7). The second embodiment is a modification of the first embodiment, and the face area and the spatial frequency components other than the face area are compared, and the image quality of the face area is not determined when the possibility of camera shake or blur is high. It is a configuration. Note that, in the electronic cameras of the second embodiment and the third embodiment, the same reference numerals are given to the same components as those in the first embodiment, and description thereof will be omitted. Also, S201, S202, and S206 to S212 in FIG. 5 correspond to S101, S102, and S106 to S112 in FIG.

  Step S203: The image processing unit 15 performs face recognition processing on the captured image data, and outputs the presence / absence of a face in the captured image data to the CPU 24. The CPU 24 determines whether a face is detected from the captured image data based on the output of the image processing unit 15. If a face is detected from the captured image data (YES side), the process proceeds to S204. On the other hand, if no face is detected from the captured image data (NO side), the process proceeds to S210.

Step S204: The image processing unit 15 analyzes the spatial frequency component of the face area of the captured image data and the spatial frequency component of an arbitrary partial area other than the face area. The spatial frequency component of this partial area is used for determination of camera shake and defocusing together with the spatial frequency component of the face area.
Step S205: The CPU 24 determines whether or not the spatial frequency component of at least one of the face area and the partial area includes a high frequency component equal to or greater than a predetermined value. If a high frequency component is included (YES side), the contour of the image is clear and the possibility of camera shake and blurring is low, so the process proceeds to face image quality determination in S206. On the other hand, when the high frequency component is not included (NO side), the outline of the image is unclear and there is a high possibility that the image is blurred or out of focus. Therefore, the CPU 24 proceeds to S210 and saves the captured image data without performing the image quality determination of the face area. In this case, the CPU 24 may automatically delete the captured image data to secure the capacity of the recording medium 21.

The electronic camera of the second embodiment can obtain substantially the same effect as that of the first embodiment. Particularly in the second embodiment, the image quality of the face area is determined after individually removing images of camera shake and blur based on the high-frequency components of the face area and partial area of each captured image data. The detection accuracy of the image quality determination of the face area can be improved as compared with the case.
(Description of the third embodiment)
FIG. 6 is a flowchart showing the operation of the electronic camera of the third embodiment (corresponding to the electronic camera of claim 8). The third embodiment is an example in which the combination of image signal gain setting and shutter time is changed and bracketing shooting is executed a plurality of times, and the shot image data with the highest image quality in the face area is recorded.

  Step S301: The camera user sets the number of frames to be shot (number of times of shooting) and the amount of parameter fluctuation for each shooting (in shutter speed, gain setting of image signal) on the setting screen. In the third embodiment, the charge accumulation time of the image sensor 12 and the shutter travel interval of the shutter mechanism (not shown) are adjusted to change the shutter time, and the gain setting of the analog signal processing unit 13 is adjusted. Change the gain setting of the image signal.

  Step S302: The camera user instructs the electronic camera 10 to perform bracketing photographing of the subject by fully pressing a release button (not shown). In this bracketing shooting, the CPU 24 generates shot image data of the first frame, and then changes the parameters according to the setting in S301 to generate shot image data of the next frame. Then, the CPU 24 repeats the above steps until the end of shooting for the number of shooting frames set in S301.

  Step S303: The image processing unit 15 performs face recognition processing on the captured image data, and outputs the presence / absence of a face in the captured image data to the CPU 24. The CPU 24 determines whether a face is detected from each captured image data based on the output of the image processing unit 15. If a face is detected from the captured image data (YES side), the process proceeds to S304. On the other hand, when a face is not detected from the captured image data (NO side), the process proceeds to S307, and the CPU stores captured image data in which a face area cannot be detected. Note that the determination in S303 is executed for each captured image data of bracketing shooting.

Step S304: The image processing unit 15 analyzes the spatial frequency component included in the face area for the captured image data in which the face is detected in S303. Then, the image processing unit 15 generates an evaluation value indicating the strength of the granularity of the image in the face area for each of the above captured image data based on the amount of the spatial frequency component having a high correlation with the granularity of the image. .
Step S305: The CPU 24 selects the photographed image data having the best image quality of the face (that is, the granularity of the image in the face region is weak and the evaluation value is small) among the photographed image data in which the face is detected in S303. Determine based on the evaluation value.

Step S306: The CPU 24 records the photographed image data determined in S305 on the recording medium 21, and ends a series of photographing operations.
In the electronic camera of the third embodiment, bracketing shooting is performed by changing the combination of the gain setting of the image signal and the shutter speed, and the shot image data with the best image quality of the face is recorded. Therefore, the possibility that a captured image with a low image granularity in the face region and a good face image quality can be acquired is improved.

(Explanation of Fourth Embodiment)
FIG. 7 is a functional block diagram of the image processing apparatus of the fourth embodiment (corresponding to the image processing apparatus of claim 9 and the image processing program of claim 10). The fourth embodiment is an example in which the image processing apparatus 30 is configured by causing a personal computer or the like to execute an image processing program. Here, the image processing program according to the fourth embodiment targets an image file including “captured image data” generated by the electronic camera and “additional data” indicating the parameters of the imaging conditions of the electronic camera. Examples of this image file include an Exif standard-compliant image file, a RAW data file having information on parameters of imaging conditions, and the like.

The image processing apparatus 30 includes a microprocessor 31, a memory 32, a bus interface unit (bus I / F unit) 33, a display driver unit 34, a disk drive unit 35, and a system bus 36 that connects the units. is doing. The image processing apparatus 30 is connected to an input unit 37 such as a keyboard and a pointing device, and a monitor 38.
The microprocessor 31 controls each part of the image processing apparatus 30 and executes an image processing program described later. The memory 32 stores programs and data, sends data to the microprocessor 31, and stores processing results returned from the microprocessor 31. The bus I / F unit 33 transmits / receives data to / from peripheral devices (for example, an electronic camera or a recording medium) connected to the image processing device 30 in conformity with a serial communication standard such as USB (Universal Serial Bus) or IEEE1394. Control. The display driver unit 34 outputs an image to the monitor 38. The disk drive unit 35 reads / writes captured image data to / from a recording medium such as an optical disk or a magneto-optical disk.

The image processing apparatus of the fourth embodiment is configured as described above, and the operation of the image processing program of the fourth embodiment will be described below with reference to the flowchart of FIG.
Step S401: The microprocessor 31 reads an image file from the bus I / F unit 33 or the disk drive unit 35, and displays a list of the read image files on the monitor 38. Then, the user is caused to select and input an image file for determining the image quality of the face area.

  Step S402: The microprocessor 31 reads the image file to be processed among the image files selected in S401 into the memory 32, and acquires the shooting condition parameters of the shot image data from the additional data. Then, the microprocessor 31 determines whether or not to determine the image quality of the face area for the captured image data currently being processed based on the parameters of the imaging condition. In the above determination, as in the first embodiment, the parameter of the shooting condition is based on “whether or not the scene mode of the electronic camera is person-oriented” or “whether camera shake is likely to occur during shooting”. When the image quality determination of the face area is performed (YES side), the process proceeds to S403. On the other hand, when the image quality determination of the face area is not performed (NO side), the process proceeds to S407.

Step S403: The microprocessor 31 performs face recognition processing on the captured image data, and determines whether or not a face has been detected from the captured image data currently being processed. If a face is detected from the captured image data (YES side), the process proceeds to S404. On the other hand, if no face is detected from the captured image data (NO side), the process proceeds to S407.
Step S404: The microprocessor 31 analyzes the spatial frequency component included in the face area of the captured image data currently being processed.

  Step S405: The microprocessor 31 determines whether or not the image quality of the face area in the captured image data currently being processed is less than or equal to the threshold value (whether the amount of the spatial frequency component having a high correlation with the granularity of the image in the face area is equal to or greater than the threshold value). Or not). If the image quality of the face area is less than or equal to the threshold (YES side), the process proceeds to S406. On the other hand, when the image quality of the face area exceeds the threshold value (NO side), the process proceeds to S407.

Step S406: The microprocessor 31 generates marker information indicating that the image quality of the face area is equal to or less than the threshold value, and associates the marker information with the specific information (file name, directory path, etc.) of the image file in the memory 32. Remember.
Step S407: The microprocessor 31 determines whether there is an image file to be processed next. If the next image file exists (YES side), the process returns to S402 and the microprocessor repeats the above operation. On the other hand, if the next image file does not exist (NO side), the process proceeds to S408.

  Step S408: The microprocessor 31 displays, on the monitor 38, a list of image files in which the image quality of the face area of the photographed image data is equal to or less than the threshold value based on the marker information and the image file specifying information. The user designates all or individual image files on the list screen to instruct image file processing, and the microprocessor 31 executes processing such as image file deletion and image file correction. The image file is corrected as in S112 of the first embodiment. This completes the series of operations of the image processing program.

  In the fourth embodiment, in the post-processing step by the computer, it is possible to detect photographed image data with a poor image quality of the face area based on the result of the spatial frequency analysis of the face area. In the fourth embodiment, as in the first embodiment, the face area image quality determination is turned on / off according to the parameters of the image capturing condition. It is excluded from the determination target, and the computation load of the computer is reduced.

(Description of Fifth Embodiment)
FIG. 9 is a flowchart showing the operation of the image processing program of the fifth embodiment (corresponding to the image processing apparatus of claim 11 and the image processing program of claim 12). The fifth embodiment is a modification of the fourth embodiment, and compares the face frequency and the spatial frequency components other than the face area, and does not determine the image quality of the face area when the possibility of camera shake or blur is high. It is a configuration. Note that in the image processing device of the fourth embodiment, the same reference numerals are given to the same components as in the fourth embodiment, and description thereof will be omitted. Also, S501 and S505 to S508 in FIG. 9 correspond to S401 and S405 to S408 in FIG.

  Step S502: The microprocessor 31 performs face recognition processing on the captured image data of the image file to be processed among the image files selected in S501, and determines whether a face is detected from the captured image data. If a face is detected from the captured image data (YES side), the process proceeds to S503. On the other hand, if a face is not detected from the captured image data (NO side), the process proceeds to S507.

Step S503: The microprocessor 31 analyzes the spatial frequency component of the face area of the captured image data and the spatial frequency component of an arbitrary partial area other than the face area. The spatial frequency component of this partial area is used for determination of camera shake and defocusing together with the spatial frequency component of the face area.
Step S504: The microprocessor 31 determines whether or not the spatial frequency component of at least one of the face region and the partial region includes a high frequency component equal to or greater than a predetermined value. When a high frequency component is included (YES side), the contour of the image is clear and the possibility of camera shake and blurring is low, so the process proceeds to face image quality determination in S505. On the other hand, when the high frequency component is not included (NO side), the outline of the image is unclear and there is a high possibility that the image is blurred or out of focus. Therefore, the microprocessor proceeds to S507.

In the fifth embodiment, substantially the same effect as that of the fourth embodiment can be obtained. In particular, in the fifth embodiment, image quality determination of the face area is performed after individually removing images of camera shake and blur based on the high-frequency components of the face area and partial area of each captured image data. The image quality of the face area can also be determined for the file.
(Correspondence between Claims and Embodiments)
Here, the correspondence relationship between the configuration of the claims and the embodiment will be described. Note that the correspondence shown below is an interpretation given for reference only, and does not limit the technical scope of the present invention.

  In the electronic camera according to claim 1, the image pickup device 12 corresponds to the “image pickup device”. The image processing unit 15 corresponds to the “image processing unit”, “face recognition unit”, “face region analysis unit”, and “spatial frequency analysis unit”. The CPU 24 corresponds to the “mode switching unit”, “face image quality determination unit”, and “control unit”. The “notification means” corresponds to the monitor 22 and the CPU 24. The card I / F 20 corresponds to the “connector”.

  In the image processing apparatus according to claims 9 and 11, the “reading unit” corresponds to the bus I / F unit 33 and the disk drive unit 35. The microprocessor 31 corresponds to the “mode switching unit”, “face recognition unit”, “face region analysis unit”, “spatial frequency analysis unit”, and “face image quality determination unit”. The image processing programs according to claims 10 and 12 are executed by the microprocessor 31.

(Supplementary items of the embodiment)
As mentioned above, although this invention has been demonstrated by said embodiment, the technical scope of this invention is not limited to the said embodiment.
(1) In the electronic camera of the above-described embodiment, an example in which correction processing is performed by applying a blur filter to the face area of special image data has been described. However, in the present invention, the graininess of an image in the face area is reduced by applying a median filter. Correction processing may be performed so that

  (2) In the electronic camera of the above embodiment, marker information may be added to a tag of image data when detecting special image data. In this case, since the CPU can extract the special image data based on the marker information after the photographing is completed, it becomes easy for the user to identify the special image data during reproduction.

Block diagram of the electronic camera of the first embodiment Flowchart of photographing operation of electronic camera of first embodiment Figure showing an example of the notification screen The figure which shows an example of a face recognition process and a correction process Flowchart of photographing operation of electronic camera of second embodiment Flowchart of photographing operation of electronic camera of third embodiment Block diagram of an image processing apparatus according to the fourth embodiment Flow chart of operation of image processing program of fourth embodiment Flow chart of operation of image processing program of fifth embodiment

Explanation of symbols

10 Electronic Camera 12 Image Sensor 13 Analog Signal Processing Unit 15 Image Processing Unit 20 Card I / F
21 Recording medium 22 Monitor 24 CPU
30 Image processing device 31 Microprocessor 32 Memory 33 Bus I / F unit 35 Disk drive unit 38 Monitor

Claims (12)

An image sensor that photoelectrically converts a subject image to generate an image signal;
An image processing unit that performs image processing on the image signal to generate captured image data;
A mode switching unit that switches between a first image processing mode for determining the image quality of the face area in the captured image data and a second image processing mode for not performing the image quality determination according to a parameter of the imaging condition;
A face recognition unit for extracting a face region in the captured image data;
A face area analysis unit for analyzing the face area;
A face image quality determination unit that determines image quality of the face region based on an analysis result of the face region analysis unit in the first image processing mode, and detects special image data having the image quality equal to or less than a threshold;
An electronic camera comprising: An image sensor that photoelectrically converts a subject image to generate an image signal;
An image processing unit that performs image processing on the image signal to generate captured image data;
A face recognition unit for extracting a face region in the captured image data;
A spatial frequency analysis unit that analyzes a spatial frequency component of the face region and an arbitrary partial region other than the face region;
A mode switching unit that switches between a first image processing mode that performs image quality determination of the face area and a second image processing mode that does not perform the image quality determination based on the analysis result;
A face image quality determination unit for determining image quality of the face region based on a spatial frequency component having a high correlation with the graininess of the image in the first image processing mode, and detecting special image data having the image quality equal to or less than a threshold;
An electronic camera comprising:   3. The electronic camera according to claim 1, further comprising notification means for notifying a camera user that the photographed image data is special image data at the time of photographing or reproduction. A connector for connecting a recording medium; and a control unit for recording the captured image data on the recording medium,
The electronic camera according to claim 1, wherein the control unit excludes the special image data from a recording target of the recording medium.   The electronic camera according to claim 1, wherein the image processing unit performs a correction process for reducing a graininess of an image on a face area of the special image data. The face recognition unit extracts a predetermined face part in the face area of the special image data,
The electronic camera according to claim 5, wherein the image processing unit excludes the area of the face part from the correction processing target.   The correction processing of the image processing unit is weighted so that the granularity of the image is high near the boundary portion of the face area, and the granularity of the image is lower than the boundary portion inside the face area. The electronic camera according to claim 5 or 6, wherein the electronic camera is characterized in that: An image sensor that photoelectrically converts a subject image to generate an image signal;
An image processing unit that performs image processing on the image signal to generate captured image data;
A face recognition unit for extracting a face region in the captured image data;
A control unit that performs shooting a plurality of times by changing at least one of the gain setting of the image signal and the shutter speed;
The spatial frequency component of the face area in the captured image data is analyzed, the image quality of the face area is determined based on the spatial frequency component highly correlated with the granularity of the image, and the image quality is the highest in multiple shootings A face image quality determination unit for detecting captured image data;
An electronic camera comprising: An image processing apparatus that reads an image file including image data generated by an electronic camera and additional data indicating parameters of photographing conditions of the electronic camera and processes the image data,
A reading unit for reading the image file;
A mode switching unit that switches between a first image processing mode for determining the image quality of the face area in the captured image data and a second image processing mode for not performing the image quality determination according to a parameter of the imaging condition;
A face recognition unit for extracting a face region in the image data;
A face area analysis unit for analyzing the face area;
A face image quality determination unit that determines image quality of the face region based on an analysis result of the face region analysis unit in the first image processing mode, and detects special image data having the image quality equal to or less than a threshold;
An image processing apparatus comprising: An image processing program that reads an image file including image data generated by an electronic camera and additional data indicating parameters of photographing conditions of the electronic camera, and processes the image data,
A procedure for reading the image file;
A procedure for switching between a first image processing mode for determining the image quality of a face area in the captured image data and a second image processing mode for not performing the image quality determination according to a parameter of the imaging condition;
A procedure for extracting a face region in the image data;
Analyzing the face region;
Determining the image quality of the face area based on the analysis result of the face area in the first image processing mode, and detecting special image data having the image quality of a threshold value or less;
An image processing program for causing a computer to execute. An image processing apparatus that reads an image file including image data generated by an electronic camera and processes the image data,
A reading unit for reading the image file;
A face recognition unit for extracting a face region from the image data;
A spatial frequency analysis unit that analyzes a spatial frequency component of the face region and an arbitrary partial region other than the face region;
A mode switching unit that switches between a first image processing mode that performs image quality determination of the face area and a second image processing mode that does not perform the image quality determination based on the analysis result;
A face image quality determination unit for determining image quality of the face region based on a spatial frequency component having a high correlation with the graininess of the image in the first image processing mode, and detecting special image data having the image quality equal to or less than a threshold;
An image processing apparatus comprising: An image processing program that reads an image file including image data generated by an electronic camera and processes the image data,
A procedure for reading the image file;
A procedure for extracting a face region from the image data;
Analyzing the spatial frequency components of the face region and any partial region other than the face region;
A procedure for switching between a first image processing mode for determining the image quality of the face area and a second image processing mode for not performing the image quality determination based on the analysis result;
Determining the image quality of the face area based on a spatial frequency component having a high correlation with the graininess of the image in the first image processing mode, and detecting special image data having the image quality equal to or lower than a threshold;
An image processing program for causing a computer to execute.

Images