JP2013141104A - Imaging apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging apparatus capable of simply correcting an image of a face with closed eyelids into an image of a face with opened eyelids without much time and labor.SOLUTION: An imaging apparatus includes an imaging element, a face detection section, a face recognition section, a determination section, a collation section, an extraction section, and a correction section. The imaging element picks up an image of a subject and generates an image. The face detection section detects the area of a face from the image. The face recognition section recognizes features for identifying the face from the area of the face. The determination section determines the opening/closing state of eyelids within the area of the face on the basis of the recognition result of the face recognition section. The collation section searches for another image recorded in a memory when the determination section determines that the eyelids are closed, and collates whether the face with the opened eyelids in the same subject exists or not. The extraction section extracts at least the area of eyes of the face with the opened eyelids when the face with the opened eyelids in the same subject is collated by the collation section. The correction section performs correction processing by allowing an extraction area extracted by the extraction section to be applied to the image of the face with the closed eyelids.

Description

  The present invention relates to an imaging apparatus.

  Conventionally, in photographing with an electronic camera which is a kind of imaging apparatus, a person who is a subject closes his eyelid at the moment of photographing, so that a photographed image of the person who closed the eyelid may be generated.

  Therefore, a method has been proposed in which a database that stores a large number of eye images is used, eye images are selected from the database, and the closed eyelid region is corrected to a desired eye image (for example, Patent Document 1). reference).

JP 2001-197296 A

  However, in the technique of Patent Document 1, for example, an eye to be corrected is selectively input by an operator to correct the eyes of the same person. Therefore, it may take time for the operator to determine matching eyes.

  SUMMARY OF THE INVENTION In view of the above circumstances, an object of the present invention is to provide an imaging apparatus that can easily and easily correct an image of a face with a closed eyelid to an image of a face with an open eyelid without taking time and effort. .

  An imaging apparatus according to a first invention includes an imaging element, a face detection unit, a face recognition unit, a determination unit, a collation unit, an extraction unit, and a correction unit. The imaging element captures an image of a subject and generates an image. The face detection unit analyzes the image and detects a face region from the image. The face recognition unit recognizes a feature for identifying the face from the face area. The determination unit determines the open / close state of the eyelid in the face region based on the recognition result of the face recognition unit. When the determination unit determines that the eyelid is closed, the verification unit searches other images recorded in the memory to verify whether there is a face that is the same subject and has the eyelid open. . The extraction unit extracts at least an eye area of the face with the eyelid open when the matching unit collates faces with the same subject and with the eyelid open. The correction unit performs correction processing by adapting the extraction region extracted by the extraction unit to the face image with the eyelid closed.

  In a second aspect based on the first aspect, the correction unit performs the correction process by adapting to the face image with the eyelid closed by enlarging or reducing the extraction region.

  In a third aspect based on the first aspect, the apparatus further comprises an inclination detection unit. The tilt detection unit detects the tilt of the face. The inclination detection unit calculates an inclination correction angle for correcting the inclination, and the correction unit performs correction processing by adapting the extraction region to the face image with the eyelid closed based on the inclination correction angle.

  In a fourth aspect based on the first aspect, the apparatus further comprises a luminance analysis unit. The luminance analysis unit analyzes the luminance distribution of the image. The correction unit performs correction processing by adapting the brightness of the extraction region to the brightness of the face with the eyelid closed based on the analysis result of the luminance analysis unit.

  A fifth invention further comprises a database according to any one of the first to fourth inventions. The database registers the image corrected by the correction unit and the feature information indicating the feature in association with each other. When the determination unit determines that the eyelid is closed, the verification unit searches the database and verifies whether there is a face that is the same subject and has the eyelid open based on the feature information.

  In a sixth aspect based on any one of the first to fifth aspects, when the face detection unit detects a plurality of faces, a face recognition unit, a determination unit, a collation unit, an extraction unit, a correction unit, Performs processing for each face.

  In a seventh aspect based on any one of the first to fifth aspects, the face is a human face.

  INDUSTRIAL APPLICABILITY The present invention can provide an imaging apparatus that can easily and easily correct an image of a face with a closed eyelid to an image of a face with an open eyelid.

FIG. 2 is a block diagram illustrating a configuration example of the electronic camera 1 of the present embodiment. Flow chart showing an example of operation in automatic correction mode Diagram explaining the flow of processing in automatic correction mode A diagram for schematically explaining the flow of tilt correction processing The figure which illustrates brightness correction processing typically The figure explaining the 1st modification The figure explaining the 2nd modification

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Here, an electronic camera which is an embodiment of the imaging apparatus of the present invention will be described as an example.

  FIG. 1 is a block diagram illustrating a configuration example of an electronic camera 1 according to the present embodiment. Here, the electronic camera 1 of the present embodiment uses a face image (hereinafter, “image with the eyelids opened”) from a face image with the eyelids closed (hereinafter referred to as “image with the eyelids closed”). Mode) (hereinafter referred to as “automatic correction mode”). Specific contents of the automatic correction mode will be described later. An image with the eyelid closed may be expressed as an image with the eyes closed conventionally, and an image with the eyelid open may be expressed with an image with the eyes open conventionally.

  As shown in FIG. 1, the electronic camera 1 includes a photographing optical system 10, an image sensor 11, a signal processing unit 12, a RAM (Random Access Memory) 13, an image processing unit 14, a flash memory 15, and a display monitor. 16, a recording interface unit (hereinafter referred to as “recording I / F unit”) 17, an operation unit 18, a release button 19, a CPU 20, and a bus 28.

  Among these, the signal processing unit 12, the RAM 13, the image processing unit 14, the flash memory 15, the recording I / F unit 17, the display monitor 16, and the CPU 20 are connected to each other via a bus 21.

  The photographing optical system 10 includes a plurality of lens groups including a zoom lens and a focus lens. For the sake of simplicity, FIG. 1 shows the photographing optical system 10 as a single lens.

  The image sensor 11 captures an image of a subject and generates an analog image signal (image). The analog image signal output from the image sensor 11 is input to the signal processing unit 12. Note that three types of color filters of R (red), G (green), and B (blue) are arranged on the imaging surface of the imaging device 11 in, for example, a Bayer array. Further, the charge accumulation time of the image sensor 11 and the reading of the image signal are controlled by a timing generator (not shown).

  Here, the imaging device 11 captures a recording image for recording in response to a full pressing operation of the release button 19 in the imaging mode of the electronic camera. In addition, the image sensor 11 continuously captures images for composition confirmation (through images) at predetermined intervals even during standby for shooting.

  The imaging device 11 is a CCD (Charge Coupled Device) type or CMOS (Complementary Metal-Oxide Semiconductor) type image sensor.

  The signal processing unit 12 performs an analog front end circuit (AFE) that performs analog signal processing on an image signal output from the image sensor 11, and performs digital signal processing on the image signal that has been subjected to analog signal processing by the AFE. A digital front-end circuit (DFE). The AFE of the signal processing unit 12 performs correlated double sampling and gain adjustment on the analog image signal, and then performs a process of converting the analog image signal into a digital image signal (A / D conversion). Here, the AFE of the signal processing unit 12 can adjust the imaging sensitivity corresponding to the ISO sensitivity by adjusting the gain of the image signal.

  Further, the DFE of the signal processing unit 12 performs a correction process on the digital image signal after A / D conversion. The image signal output from the signal processing unit 12 is temporarily recorded in the RAM 13 as image data of RGB signals. The RAM 13 is a buffer memory that temporarily records image data.

  The image processing unit 14 reads out the image data recorded in the RAM 13 and performs various image processing (for example, white balance processing, color interpolation processing, gradation conversion processing, contour enhancement processing, etc.) as necessary.

  The flash memory 15 is a non-volatile memory that stores a program for controlling the electronic camera 1 in advance. In the flash memory 15, an automatic correction mode program for executing the automatic correction mode is recorded in advance. The flash memory 15 has a database 15a. The database 15a has an information table used in the automatic correction mode. Details of the contents of this information table will be described later with reference to FIG.

  The display monitor 16 is composed of, for example, a liquid crystal display medium. The display monitor 16 displays various images, an operation menu of the electronic camera 1 and the like according to instructions from the CPU 20.

  The recording I / F unit 17 is formed with a connector (not shown) for connecting a detachable recording medium M1. The recording I / F unit 17 accesses the recording medium M1 connected to the connector and performs image recording processing and the like. The recording medium M1 is, for example, a nonvolatile memory card. FIG. 1 shows the recording medium M1 after being connected to the connector.

  The operation unit 18 has a plurality of buttons (not shown) that receive user operations. The release button 19 receives an instruction input for a half-press operation (start of imaging preparation operation) and an instruction input for a full-press operation (start of imaging operation).

  The CPU 20 is a processor that performs various calculations and control of the electronic camera 1. The CPU 20 controls each part of the electronic camera 1 by executing a sequence program stored in advance in the flash memory 15.

  In the present embodiment, the automatic correction mode program in the flash memory 15 is read into the CPU 20. Then, the CPU 20 executes the automatic correction mode program, thereby performing the face detection unit 20a, the face recognition unit 20b, the determination unit 20c, the collation unit 20d, the extraction unit 20e, the inclination detection unit 20f, the luminance It also functions as an analysis unit 20g and a correction unit 20h.

  The face detection unit 20a analyzes the image and detects a face region from the image. Specifically, the face detection unit 20a acquires color components and luminance values based on the recording image data output from the image sensor 11. In this case, the face detection unit 20a detects the face region by specifying the skin color component region. Then, the face detection unit 20a identifies position information (position coordinates and the like) in the face area image.

  The face recognition unit 20b recognizes a feature for identifying a face from the face area. Specifically, the face recognition unit 20b extracts facial features (feature parts) by a feature extraction process described in, for example, Japanese Patent Application Laid-Open No. 2001-16573. Examples of the characteristic part include eyebrow, eye, nose, lip end points, face contour points, head apex, chin lower end point, and the like. Then, the face recognition unit 20b specifies the position information of the face area.

  The determination unit 20c determines the open / closed state of the eyelids in the face region based on the recognition result of the face recognition unit 20b. Specifically, the determination unit 20c detects the degree of blinking for the eye area recognized by the face recognition unit 20b, for example, by blink detection processing described in Japanese Patent Application Laid-Open No. 2000-235648. Then, the determination unit 20c determines the open / close state of the kite from the degree of blinking.

  When the determination unit 20c determines that the eyelid is closed, the collation unit 20d searches for another image recorded in the flash memory 15 or the recording medium M1, and a face with the same subject and the eyelid opened. Check whether it exists. The collation unit 20d performs collation by, for example, a well-known pattern matching process.

  However, since the open / closed state of the eyelid is different between the face with the eyelid open and the face with the eyelid closed, the pattern matching near the eyes does not match. Therefore, the collation unit 20d performs a collation process based on pattern matching of other feature parts excluding the eye region.

  The extraction unit 20e extracts at least the eye region of the face with the eyelid open when the collation unit 20d collates the same subject and the face with the eyelid open. Specifically, the extraction unit 20e extracts the eye area as an extraction area based on the coordinates of the captured image. Or the extraction part 20e is good also considering the area | region containing eyes as an extraction area | region. Here, extracting means, for example, extracting the extraction region. In other words, in the present embodiment, not only the eyes but also other characteristic parts such as eyebrows may be corrected as the extraction region in combination with the eye region.

  The tilt detection unit 20f analyzes the image and detects the tilt of the face. The luminance analysis unit 20g analyzes the luminance distribution of the image. The correction unit 20h performs correction processing by adapting the extraction region extracted by the extraction unit 20e to an image with the eyelids closed. Details of the inclination detection unit 20f, the luminance analysis unit 20g, and the correction unit 20h will be described later when the operation is described.

  Next, an example of the operation in the automatic correction mode in the electronic camera 1 of the present embodiment will be described.

  FIG. 2 is a flowchart showing an example of the operation in the automatic correction mode. FIG. 3 is a diagram schematically illustrating the processing flow in the automatic correction mode.

  Here, it is assumed that the photographer has fully pressed the release button 19 and the electronic camera 1 has acquired an image. In this embodiment, in order to make the explanation easy to understand, an image that has been captured this time and is subjected to eye correction processing is referred to as a captured image, and is a captured image that has already been recorded in the recording medium M1 or the flash memory 15. Is sometimes simply called an image. FIG. 3A shows an example of a captured image. In FIG. 3A, the subject person P1 with the eyelid closed is shown.

  When the operation unit 18 illustrated in FIG. 1 first receives an instruction input in the automatic correction mode, the CPU 20 starts the processing of the flow illustrated in FIG. For convenience of explanation, it is assumed that a plurality of images in which the face of the subject is reflected are already recorded on the recording medium M1.

  Step S101: The face detection unit 20a of the CPU 20 performs face detection processing. Further, the face recognition unit 20b of the CPU 20 performs recognition processing. Specifically, the face detection unit 20a detects the face area by analyzing the image data of the photographed image taken this time and specifying the skin color component area. Subsequently, the face recognizing unit 20b extracts a feature portion of the face from the face region through feature extraction processing. And CPU20 transfers to the process of step S102.

  Step S102: The determination unit 20c of the CPU 20 first performs blink detection processing. Specifically, the determination unit 20c detects the degree of blinking for the eye area recognized by the face recognition unit 20b by the blink detection process described above. And CPU20 transfers to the process of step S103.

  Step S103: The determination unit 20c determines the open / close state of the kite from the degree of blinking. That is, the determination unit 20c determines whether or not the image is a closed eyelid. In the case of an image with the eyelid closed (step S103: Yes), the CPU 20 proceeds to the process of step S104. On the other hand, in the case of an image with the eyelid opened (step S103: No), the CPU 20 does not need to correct the image, and thus ends the processing of the flow shown in FIG.

  Step S104: The CPU 20 refers to the memory such as the recording medium M1, and whenever there is a transition to Step S104, whether there is an image photographed one time before in time series with respect to the image selected in the previous Step S104. Determine whether or not. However, when the process proceeds to step S104 at the beginning, it is determined whether or not there is an image captured one image before the correction target captured image captured this time. At this time, for example, when the image is an Exif (Exchangeable Image File Format) format image file, the CPU 20 searches for the image based on the shooting date / time information of the header file.

  FIG. 3B schematically shows the processing from step S104 to step S108. Here, when there is an image photographed immediately before (step S104: Yes), the CPU 20 proceeds to the process of step S105. On the other hand, when there is no image taken immediately before (step S104: No), the CPU 20 ends the processing of the flow shown in FIG.

  Step S105: The collation unit 20d of the CPU 20 performs collation processing. Specifically, the collation unit 20d collates the face recognized by the face recognition unit 20b from the captured image and an image including the face recorded on the recording medium M1. And the collation part 20d performs a collation process based on the pattern matching of the other characteristic site | parts except the area | region of an eye. And CPU20 transfers to the process of step S106.

  Step S106: If the image of the same person who has opened the eyelids can be collated (step S106: Yes), the CPU 20 proceeds to the process of step S107. On the other hand, when the images of the same person with open eyes could not be collated (step S106: No), the process returns to step S104.

  Step S107: The extraction unit 20e of the CPU 20 performs an extraction process. Specifically, the extraction unit 20e extracts an extraction region including the eye region of the face with the eyelid open based on the image collated by the collation unit 20d.

  Step S108: The correction unit 20h of the CPU 20 performs a correction process. Specifically, the correction unit 20h first performs a correction process for replacing a portion estimated as a face eye region with an extraction region on a captured image. Here, the correction unit 20h performs correction processing by selecting at least one of enlargement / reduction processing, tilt correction processing, and brightness correction processing as necessary.

(1) Enlarging / Reducing Process The correcting unit 20h performs a correcting process by adapting the image with the eyelid closed by enlarging or reducing the extraction area. Specifically, as illustrated in FIG. 3, for example, the correction unit 20 h matches the facial contour (first contour) of the captured image with the facial contour (second contour) of the image collated by the collating unit 20 d. In this way, the second contour is enlarged or reduced. As a result, the extraction region (eye region) matches the eye region of the captured image within an allowable range of errors. Then, the correction unit 20h replaces the portion estimated as the face eye region with the extracted region in the captured image. As a result, the correction unit 20h increases the size of the eye region by enlarging or reducing the extraction region even if the face size of the captured image is different from the face size of the image collated by the collation unit 20d. After correction, it is possible to correct the image with the eyelid closed from the image with the eyelid closed.

(2) Inclination Correction Process FIG. 4 is a diagram schematically illustrating the flow of the inclination correction process. FIG. 4A illustrates the inclination of the face. In the present embodiment, for example, the face in the image collated by the collation unit 20d may be inclined in the shooting screen. Therefore, the inclination detection unit 20f of the CPU 20 detects the inclination of the face. FIG. 4B is a diagram illustrating a face inclination detection method. As shown in FIG. 4B, the inclination detection unit 20f regards the contour of the image collated by the collation unit 20d as an ellipse, and obtains the major axis of the ellipse as the center line (L1). The inclination detection unit 20f calculates an angle θ formed by the perpendicular line (L2) in the Y-axis direction and the center line (L1) in the XY coordinates in the image. If the angle θ is 0 degree, the inclination detection unit 20f determines that no face inclination has occurred. On the other hand, the inclination detection unit 20f determines that the face is inclined if the angle θ is 45 degrees clockwise, for example. Here, in the present embodiment, the angle θ is an inclination correction angle.

  FIG. 4C is a diagram exemplifying inclination correction. The correction unit 20h corrects the inclination based on the inclination correction angle, and performs the correction process by adapting the extraction region to the image with the eyelids closed. Thereby, the correction unit 20h can correct the inclination of the eye region and correct the image with the eyelid closed to the image with the eyelid closed.

(3) Brightness Correction Process FIG. 5 is a diagram schematically illustrating the brightness correction process. FIG. 5A schematically shows that the person P1 is photographed in a clear sky environment by the sun S1. FIG. 5B schematically shows that the person P1 is photographed under room lighting by the lighting S2.

  In the present embodiment, as shown in FIGS. 5A and 5B, for example, a captured image may be captured this time in a clear sky environment, and an image verified by the verification unit 20d may be captured in the past in an indoor environment. Can happen. In this case, the brightness of the face is also different. Therefore, the luminance analysis unit 20g analyzes the luminance of both images. The extraction unit 20e extracts a region including the eyes from the image collated by the collation unit 20d (see FIG. 5C). Then, the correction unit 20h performs correction processing by adapting the brightness of the peripheral portion of the eye of the extraction region to the brightness of the face of the captured image (see FIG. 5D). Thereby, the correction unit 20h can perform the correction process in consideration of the brightness of the face. When the correction process by the correction unit 20h ends, the CPU 20 proceeds to the process of step S109.

  Step S109: The CPU 20 determines whether the corrected captured image is acceptable. For example, when the resolution of the face image collated by the collation unit 20d is significantly lower than the captured image (below a predetermined threshold value), the CPU 20 determines that the captured image is no and searches for another image. And For this reason, if the determination result of the corrected captured image is negative (step S109: No), the CPU 20 returns to the process of step S104. On the other hand, if the determination result of the corrected captured image is good (OK) (step S109: Yes), the CPU 20 proceeds to the process of step S111.

  Step S110: The CPU 20 displays the corrected captured image on the display monitor 16. This is to allow the photographer to determine the quality of the captured image that has been corrected.

  Step S111: The CPU 20 accepts a user input (selection input) as to whether or not the corrected captured image is determined to be good for the photographer. When receiving a user input indicating that the corrected captured image is determined to be good for the photographer (step S111: Yes), the CPU 20 proceeds to the process of step S112. On the other hand, when receiving a user input indicating that the corrected photographed image is not determined to be good for the photographer (step S111: No), the CPU 20 proceeds to the process of step S113.

  Step S112: The CPU 20 performs a saving process for the corrected photographed image. Specifically, the CPU 20 records the corrected captured image on the recording medium M1 via the recording I / F unit 17. Then, the CPU 20 ends the process of the flow shown in FIG.

  Step S113: The CPU 20 accepts a user input (selection input) as to whether or not to search for another image from the photographer. When receiving a user input for searching for another image (step S113: Yes), the CPU 20 returns to the process of step S104. On the other hand, when the user input indicating that no other image is searched is received (step S113: No), the CPU 20 ends the processing of the flow shown in FIG.

  As described above, the electronic camera 1 according to the present embodiment does not need to correct the eyes of the same person by causing the operator to select and input the eyes to be corrected, as in Patent Document 1, for example.

  Therefore, in the electronic camera 1, it is possible to easily correct an image with the eyelids closed to an image with the eyelids open without taking time and effort.

That is, the electronic camera 1 searches for a photograph of the same person from the past photographs (captured images) taken and stored in the recording medium M1 and the flash memory 15 by face identification processing (face authentication). The electronic camera 1 extracts a region including the eyes from the searched photograph, adjusts the color and size (size, angle), and replaces the region including the eyes, thereby obtaining a photograph close to the photographer's image. Can be realized.
(First modification)
Next, a first modification of the above embodiment will be described. In the above embodiment, the collation unit 20d sequentially searches the images recorded in the recording medium M1 and the flash memory 15 so as to go back to the shooting date and time. Here, in the first modification, the collation unit 20d may first collate by searching the database 15a.

  FIG. 6 is a diagram illustrating a first modification. FIG. 6A is a diagram illustrating an example of a captured image. For convenience of explanation, it is assumed that the person of the subject in FIG. 6A is P1, and is taken after the photographed image of the same person P1 shown in FIG. FIG. 6B is a diagram showing a configuration example (information table) of the database 15a. FIG. 6C is a diagram illustrating an example of a captured image after the correction process.

  In the first modified example, in the storage process in step S112 of the flowchart shown in FIG. 2, the CPU 20 extracts feature information (for example, eyebrows, etc.) indicating the features extracted by the feature extraction process of the face recognition unit 20b with respect to the corrected captured image. (Each eye, nose, lip end point, face outline point, head apex, lower end point of chin, etc.) are registered for each person. Furthermore, the CPU 20 associates the storage destination address of the corrected captured image with the database. FIG. 6B illustrates that feature information A, B, and C are registered for each person (P1, P2, and P3).

  Thereby, when a new photographed image (see FIG. 6A) is photographed, the collation unit 20d refers to the feature information in the database 15a and collates the faces of the same subject (see FIG. 6B). ). Then, when the collation unit 20d has been able to collate, the extraction unit 20e refers to the recording destination address and extracts an extraction region from the person specified image. Furthermore, the correction unit 20h performs a correction process on the new captured image (see FIG. 6A) to correct the captured image with an open eyelid (see FIG. 6C).

Therefore, in the first modified example, it is necessary to go to search for an image with the eyelids opened in advance and to perform the feature extraction process of the face recognition unit 20b again on the image for matching registered in the database 15a. As a result, the search processing speed is improved.
(Second modification)
Next, a second modification of the above embodiment will be described. In the above embodiment, the case where one person is present in the photographed image is illustrated, but a plurality of persons may be present in the photographed image.

FIG. 7 is a diagram illustrating a second modification. FIG. 7A shows an example of a captured image. FIG. 7B shows a photographed image after correction processing for the image shown in FIG.
In FIG. 7A, two persons P1 and P2 are shown in the photographed image. Here, when the face detection unit 20a detects a plurality of faces, the face recognition unit 20b, the determination unit 20c, the collation unit 20d, the extraction unit 20e, and the correction unit 20h each perform processing for each face. good. Moreover, the inclination detection unit 20f and the luminance analysis unit 20g may perform processing for each face as necessary.

Thereby, in the 2nd modification, as shown in Drawing 7 (b), it can amend about a plurality of persons from the face which closed the eyelid individually to the face which opened the eyelid.
(Supplementary explanation)
(1) In the above embodiment, a lens-integrated compact electronic camera is exemplified as an imaging apparatus to which the present invention can be applied. However, the present invention is not limited to this type of electronic camera. The imaging apparatus to which the present invention can be applied includes, for example, a single-lens reflex type electronic camera and an interchangeable lens type electronic camera without a quick return mirror. The imaging apparatus to which the present invention can be applied also includes a mobile phone having a camera function and a personal digital assistant (PDA).

  (2) In the above embodiment, the face of a person is exemplified as the face of the subject. However, for example, the face of an animal such as a pet may be targeted. In this case, the face detection unit 20a and the face recognition unit 20b may perform face detection and feature extraction processing using a known pet face recognition process. Thereby, this invention is not restricted to a person's face, It can apply also to animals, such as a pet.

  (3) In the above embodiment, as the image with the heel closed, for example, a state in which the person P1 completely closed the heel is illustrated. In this case, the image may be a closed image.

  (4) In the above embodiment, in the first modification, the CPU 20 registers the feature information extracted by the feature extraction processing of the face recognition unit 20b in the database 15a for each person for the corrected captured image. Here, the CPU 20 may further register the feature information and the recording destination address of the image collated by the collation unit 20d (image that is the basis of the correction process) in the database 15a.

  (5) In the above embodiment, the tilt detection unit 20f detects the tilt of the face for the image collated by the collation unit 20d as shown in FIG. 4B, but the same applies to the face of the captured image. The tilt may be detected. Therefore, when the face of the captured image is inclined, the correction unit 20h may correct the extraction region so as to match the inclination. Thereby, even when the face of the photographed image is tilted, it is possible to easily correct an image with a closed eyelid to an image with an open eyelid without trouble.

  (6) In the above-described embodiment, the recording medium M1 may be used as a computer-readable recording medium on which an instruction for causing a computer to execute the automatic correction mode program is recorded. Here, as described above, the automatic correction mode program is characterized by causing the computer to execute face detection processing, face recognition processing, determination processing, collation processing, extraction processing, and correction processing.

  (7) In the above embodiment, the automatic correction mode program is incorporated into the electronic camera 1 and used. However, the automatic correction mode program may be installed in a personal computer, for example.

DESCRIPTION OF SYMBOLS 1 ... Electronic camera, 11 ... Imaging device, 20a ... Face detection part, 20b ... Face recognition part, 20c ... Judgment part, 20d ... Collation part, 20e ... Extraction part , 20h ... correction part

Claims (7)

An image sensor that captures an image of a subject and generates an image;
A face detection unit that analyzes the image and detects a face region from the image;
A face recognition unit for recognizing a feature for identifying the face from the face region;
A determination unit that determines an open / close state of a wrinkle in the face region based on a recognition result of the face recognition unit;
When the determination unit determines that the eyelid is closed, another image recorded in the memory is searched to verify whether there is a face that is the same subject and has the eyelid open. And
An extraction unit that extracts at least an eye area of the face with the eyelid open when the matching unit matches the face with the same subject and the eyelid open;
A correction unit that performs correction processing by adapting the extraction region extracted by the extraction unit to an image of a face closing the eyelid;
An imaging apparatus comprising: The imaging device according to claim 1,
The imaging device according to claim 1, wherein the correction unit performs a correction process by adapting to an image of a face closing the eyelid by enlarging or reducing the extraction area. The imaging device according to claim 1,
An inclination detector for detecting the inclination of the face;
The inclination detection unit calculates an inclination correction angle for correcting the inclination,
The correction device performs correction processing by adapting the extraction region to an image of a face with the eyelid closed based on the tilt correction angle. The imaging device according to claim 1,
A luminance analysis unit for analyzing the luminance distribution of the image;
The imaging device according to claim 1, wherein the correction unit performs correction processing by adapting the brightness of the extraction region to the brightness of the face closing the eyelid based on the analysis result of the luminance analysis unit. In the imaging device according to any one of claims 1 to 4,
A database that associates and registers the image corrected by the correction unit and the feature information indicating the feature;
When the determination unit determines that the eyelid is closed by the determination unit, the collation unit searches the database and determines whether there is a face that is the same subject and has the eyelid open based on the feature information. An image pickup apparatus characterized by collating them. In the imaging device according to any one of claims 1 to 5,
When the face detection unit detects a plurality of faces, the face recognition unit, the determination unit, the collation unit, the extraction unit, and the correction unit perform processing for each face, respectively. An imaging device. The imaging apparatus according to any one of claims 1 to 6,
The imaging apparatus according to claim 1, wherein the face is a human face.

Images