JP2012200023A - Electronic camera - Google Patents

Abstract

PROBLEM TO BE SOLVED: To facilitate photographing of a face in accordance with contents of a photographing mode.SOLUTION: An electronic camera is characterized by comprising: an imaging device which captures a subject image and generates an image signal; a face detection unit for detecting a face region on the basis of the image signal from the imaging device; setting means for setting a photographing mode; and setting means for setting detection accuracy of the face region in accordance with the photographing mode set by the above-mentioned setting means.

Description

  The present invention relates to an electronic camera.

  Conventionally, a technique has been disclosed in which feature points of a person's face are extracted from a photographed image of the person's face, and face recognition is performed based on the degree of coincidence between face recognition data related to the person to be recognized and the above feature points. (See Patent Document 1). Such face recognition technology requires that face recognition data be registered in advance in an electronic camera or the like. In particular, in order to improve the accuracy of face recognition, more face recognition data is required, so it is necessary to generate face recognition data from a plurality of frames of captured images.

JP-A-6-259534

  By the way, in registered photographing for generating face recognition data, it is necessary to photograph the face of the subject so that facial feature points can be easily extracted.

  An electronic camera that is an example of the present invention captures a subject image and generates an image signal, a face detection unit that detects a face area based on the image signal from the image sensor, and a setting for setting a shooting mode And setting means for setting the detection accuracy of the face area according to the shooting mode set by the setting means.

  In the electronic camera of the present invention, the detection accuracy of the face area is set according to the set shooting mode.

The block diagram which shows the structure of the electronic camera of 1st Embodiment. Flow chart showing the operation in the “face registration mode” of the first embodiment Figure showing an example of the mode selection screen The figure which shows an example of the selection screen of a registered person The figure which shows an example of the detailed information screen of a registered person The figure which shows the example of a display of the preview image at the time of face detection A flowchart showing the operation of the “face recognition photographing mode” of the first embodiment Flow chart showing the operation in the “face registration mode” of the second embodiment Flow chart showing the operation in the “face registration mode” of the third embodiment Flow chart showing the operation in the “face registration mode” of the fourth embodiment The figure which shows the example of a display of the preview image in 4th Embodiment

<Description of First Embodiment>
FIG. 1 is a block diagram showing the configuration of the electronic camera of the first embodiment.

  The electronic camera includes a photographing optical system 11, a first lens driving unit 12, a swing sensor unit 13, an image blur correction lens 14, a second lens driving unit 15, an image sensor 16, and an analog signal processing unit 17. The A / D converter 18, the built-in memory 19, the image processor 20, the card I / F 21, the input / output I / F 22, the operation member 23 and the liquid crystal monitor 24, the flash light emitter 25, and the CPU 26. And a face registration memory 27 and a data bus 28. The internal memory 19, the image processing unit 20, the card I / F 21, the input / output I / F 22, and the CPU 26 are connected via a data bus 28.

  The photographing optical system 11 includes a plurality of lens groups including a zoom lens 11a and a focusing lens 11b for adjusting a focus position. Each lens constituting the photographing optical system 11 is driven and adjusted in the optical axis direction by the first lens driving unit 12.

  The swing sensor unit 13 includes a vertical angular velocity sensor that detects the pitch of the electronic camera and a horizontal angular velocity sensor that detects the roll of the electronic camera. The swing sensor unit 13 outputs camera swing data indicating the swing of the electronic camera to the CPU 26. This camera swing data is used to calculate the amount of movement of the image blur correction lens 14.

  The image blur correction lens 14 is configured to be swingable in the direction perpendicular to the optical axis. The second lens driving unit 15 includes a vertical swing unit that swings the image blur correction lens 14 in the vertical direction and a horizontal swing unit that swings the image blur correction lens 14 in the horizontal direction. The second lens driving unit 15 swings the image blur correction lens 14 based on the movement amount of the image blur correction lens 14 calculated by the CPU 26 and executes blur correction.

  The image sensor 16 photoelectrically converts the light beam that has passed through the photographing optical system 11 to generate an analog image signal of the subject image. The image sensor 16 also exposes the subject at predetermined intervals even when the shutter is not released (during shooting standby), and outputs an analog image signal (through image signal) by thinning readout. The through image signal is used for AF calculation, AE calculation, face detection operation, and finder image generation, which will be described later.

  The analog signal processing unit 17 includes a CDS circuit that performs correlated double sampling, a gain circuit that amplifies the output of the analog image signal, a clamp circuit that clamps the waveform of the input signal to a certain voltage level, and the like. The A / D converter 18 converts the analog image signal of the image sensor 16 into a digital image signal. The built-in memory 19 temporarily stores data in the pre-process and post-process of image processing in the image processing unit 20. The built-in memory 19 can be provided with a recording area in which captured image data described later can be recorded.

  The image processing unit 20 performs predetermined image processing on the digital image signal. The image processing unit 20 generates captured image data based on the image signal at the time of release. Further, the image processing unit 20 sequentially generates through image data based on the through image signal. The resolution of the through image data is set lower than the resolution of the captured image data. The through image data is used for face recognition processing by the CPU 26 and also used as a display image (preview image) on the liquid crystal monitor 24. Furthermore, the image processing unit 20 also performs a process of compressing captured image data. Furthermore, the image processing unit 20 can also perform an electronic zoom process that cuts out a part of the captured image and converts the resolution.

  In addition, the image processing unit 20 generates video data for various displays (such as a frame display during face recognition) according to instructions from the CPU 26, superimposes the video data on the through image data, and outputs the video data to the liquid crystal monitor 24. As a result, the various displays described above are combined with the preview image.

  The card I / F 21 is formed with a connector for connecting the recording medium 29. The recording medium 29 is composed of a known semiconductor memory or the like. The card I / F 21 controls writing / reading of photographed image data with respect to the recording medium 29.

  An operation member 23 and a liquid crystal monitor 24 are connected to the input / output I / F 22. The operation member 23 includes an input button, a release button, and the like. The input button of the operation member 23 accepts, for example, a switching input of a photographing mode (normal photographing mode, reproduction mode, face registration mode, face recognition photographing mode, etc.) of the electronic camera. The release button of the operation member 23 accepts an AF operation instruction input by half pressing and a release input by full pressing.

  On the other hand, the liquid crystal monitor 24 is mainly disposed on the back surface portion of the electronic camera. The liquid crystal monitor 24 displays a playback image of the captured image data, a setting screen for changing various settings of the electronic camera, and the like. The setting screen includes an editing screen for editing face recognition data described later. In addition, the preview image can be displayed as a moving image on the liquid crystal monitor 24 in the shooting mode. Therefore, the user can perform framing of the subject while confirming the preview image on the liquid crystal monitor 24.

  Furthermore, an external connection terminal is formed in the input / output I / F 22. The input / output I / F 22 controls data transmission / reception with a computer or the like connected via an external connection terminal in accordance with a serial communication standard such as USB (Universal Serial Bus) or IEEE1394. Further, it is possible to connect a communication unit 30 which is an expansion device for performing wireless communication with the outside to the external connection terminal of the input / output I / F 22.

  The flash light emitting unit 25 includes a xenon arc tube, a main capacitor for storing light emission energy, a reflector and a lens member for efficiently irradiating the subject with flash light, a light emission control circuit for adjusting light emission timing and light quantity, and the like. .

  The CPU 26 controls the operation of each part of the electronic camera according to a sequence program stored in a ROM (not shown). For example, the CPU 26 executes a known contrast AF (autofocus) calculation or a known AE (automatic exposure) calculation based on the through image signal. Further, the CPU 26 calculates the movement amount of the image blur correction lens 14 based on the camera swing data.

  Further, the CPU 26 functions as a face detection unit 31 and a face recognition unit 32. The face detection unit 31 extracts feature points from through image data or captured image data, and detects a face area, a face size, and the like of the subject. For example, the face detection unit 31 extracts a face region by a feature point extraction process described in Japanese Patent Laid-Open No. 2001-16573. Examples of the feature points include eyebrow, eye, nose, and lip end points, face contour points, head vertices, and chin lower end points.

  In the “face registration mode”, the face recognition unit 32 generates face recognition data based on the detected feature points of the face. For example, the face recognizing unit 32 generates face recognition data of a registered person from the position of the detected feature point of the face, the size of the face part obtained from the feature point, the relative distance of each feature point, and the like.

  Further, in the “face recognition shooting mode”, the face recognition unit 32 performs face recognition processing for determining whether or not a person's face in the shooting screen is a face of a registered person in the face recognition data. Specifically, the face recognition unit 32 first calculates the position of the feature point of the photographed person's face, the size of each face part, the relative distance of each feature point, and the like based on the detected feature points of the face. Next, the face recognition unit 32 compares the above calculation result with the face recognition data, and obtains the similarity between the face of the predetermined registered person and the face of the photographed person. Then, the face recognizing unit 32 determines that the photographed person matches a predetermined registered person when the similarity is greater than the threshold value.

  Note that the CPU 26 can execute a change in shooting conditions or image processing settings, a release instruction to the image sensor 16, and the like based on the result of the face recognition process.

  In the face registration memory 27, face recognition data generated by the face recognition unit 32 of the CPU 26 is recorded. In the face registration memory 27, a group folder is generated for each registered person. In the group folder, face recognition data of each registered person is recorded. That is, the face recognition data in the face registration memory 27 is grouped for each registered person by the group folder. For example, in the group folder, a plurality of face recognition data with different face orientations, presence / absence of glasses, photographing conditions, and the like can be recorded for the same registered person. Then, by designating the registered person, the CPU 26 can collectively read the face recognition data in the group folder from the face registration memory 27.

  The face registration memory 27 stores data of “name of registered person” and “processing setting for face recognition”. The “registered person name” and “processing setting at the time of face recognition” are associated with the group folder of each registered person.

  Here, in the “processing setting at the time of face recognition”, settings of various controls of the electronic camera that are applied when a registered person is recognized are recorded. For example, “processing setting for face recognition” includes (1) setting of shooting conditions at the time of shooting, (2) setting of image processing of shot image data, (3) setting of a recording destination of shot image data, (4 And (5) setting for transmission of photographed image data, and the like.

  The setting of the shooting condition in (1) includes an exposure correction value when shooting each registered person, setting of the depth of field (shallow, standard selection), and the like. It is also possible to set the CPU 26 to automatically take a picture when a registered person is detected. The image processing settings in (2) above include settings for presence / absence of soft focus processing of captured image data, settings for suppressing edge enhancement, and the like.

  In the setting of the recording destination in (3) above, the directory path of the recording folder in the recording medium 29 that is the recording destination of the photographed image data can be designated for each registered person. That is, the CPU 26 can record the photographed image data in a different recording folder for each registered person by referring to the data in the face registration memory 27.

  In the print designation setting in (4) above, for example, it is possible to designate, for each registered person, whether or not to print photographed image data according to a standard such as DPOF (Digital Print Order Format). In the transmission setting in (5) above, it is possible to designate whether or not the captured image data needs to be transmitted when the registered person is photographed, and address information (e-mail address) of the transmission destination.

  The face registration memory 27 stores data of “face registration image”, “face recognition data registration time”, and “shooting conditions at the time of face recognition data acquisition”. The above-mentioned “face registration image”, “face recognition data registration time”, and “imaging conditions when acquiring face recognition data” are associated with individual face recognition data in the group folder.

  The above “face registration image” is photographed image data at the time of face recognition data acquisition, and is resized to a predetermined pixel size. Then, the “face registration image” is displayed on the liquid crystal monitor 24 in the above editing screen. That is, the “face registration image” functions as an index image that enhances the identifiability and searchability of the registered person or face recognition data. For example, the “face registration image” allows the user to check on the editing screen which face recognition data has been generated from what kind of image (face front, landscape orientation, etc.).

  “Registration time of face recognition data” is data of photographing date and time of face registration image data that is the basis of the face recognition data. Further, “shooting conditions when acquiring face recognition data” is data of shooting conditions of face registration image data. The above photographing conditions include, for example, focal length data of the photographing optical system 11, white balance and exposure conditions at the time of photographing, presence / absence of flash emission at the time of photographing, and the like. Note that the data of “registration time of face recognition data” and “shooting conditions when acquiring face recognition data” are also used for determination when the CPU 26 narrows down the face recognition data used in the face recognition processing.

  Hereinafter, the operation of the electronic camera of the first embodiment will be described by dividing it into “face registration mode” and “face recognition photographing mode”.

(Operation in face registration mode)
FIG. 2 is a flowchart showing the operation of the “face registration mode” of the first embodiment. This “face registration mode” is a kind of shooting mode, in which the user captures the face of a registered person and generates face recognition data.

  Step 101: First, the user sets the photographing mode of the electronic camera to the “face registration mode” using the operation member 23.

  For example, in a state where the electronic camera is set to “normal shooting mode” or “portrait shooting mode”, the user performs an operation for calling a mode selection screen. As a result, the CPU 26 displays a mode selection screen (see FIG. 3A) on the liquid crystal monitor 24. Then, the user operates the selection frame on the mode selection screen with the operation member 23 and instructs the CPU 26 to activate the “face registration mode”. Thereby, the CPU 26 starts a series of operations in the “face registration mode”.

  Step 102: The CPU 26 displays a menu of “face registration mode” on the liquid crystal monitor 24 (see FIG. 3B). In this menu, the user operates the selection frame with the operation member 23 to select and input “new” or “add” to the CPU 26. When registering a new person's face, the user selects and inputs “new”. On the other hand, when further photographing a face of a person who has already been registered, the user selects and inputs “Add”.

  Step 103: The CPU 26 determines whether or not “new” is selected in the menu (S102). In the case of “new” (YES side), the process proceeds to S104. On the other hand, in the case of “addition” (NO side), the process proceeds to S105.

  Step 104: When “new” is selected in S102, the CPU 26 generates a group folder of newly registered persons (registered persons to be photographed this time) in the face registration memory 27. Then, the CPU 26 displays a “registered person detailed information screen” (see FIG. 5) on the liquid crystal monitor 24 and prompts the user to input information of “registered person name” and “processing setting for face recognition”. Information input on the “registered person detailed information screen” in FIG. 5 is recorded in the face registration memory 27 in a state of being associated with the group folder. Thereafter, the process proceeds to S106.

  Note that the user can omit the input of “name of registered person” and “processing setting for face recognition”. In this case, the user can start up the “registered person detailed information screen” of FIG. 5 from the editing item of FIG. 3B and later input the information to the face registration memory 27 again. it can.

  Step 105: On the other hand, if “add” is selected in S102, the CPU 26 displays a registered person selection screen (see FIG. 4) on the liquid crystal monitor 24, and allows the user to select and input the registered person to be photographed this time.

  Step 106: The CPU 26 changes the setting at the time of shooting with the electronic camera to a setting suitable for face registration shooting. Specifically, the CPU 26 performs the following settings (1) to (7).

  (1) The CPU 26 sets the AF method to the face detection mode in the “face registration mode”. In this case, the CPU 26 automatically repeats focusing according to the face area detected by the through image data regardless of the operation of the release button. This makes it possible to focus on the detected face. Note that the CPU 26 may perform AF lock when the user half-presses the release button.

  (2) The CPU 26 sets the AE method to the spot AE mode based on the face detection area in the “face registration mode”. In this case, the CPU 26 executes an AE calculation in accordance with the face area detected by the through image data. Thereby, the exposure of the detected face is optimized.

  (3) The CPU 26 disables the electronic zoom process of the image processing unit 20 in the “face registration mode”. Alternatively, the CPU 26 restricts the maximum magnification of the electronic zoom so as to be smaller than normal by means such as limiting the magnification of the electronic zoom to, for example, about 2. This is because when the electronic zoom is performed, the information amount of the captured face registration image is reduced, and the extraction accuracy of the facial feature points is lowered.

  (4) In the “face registration mode”, the CPU 26 reduces the maximum magnification by narrowing the magnification adjustment range of the zoom lens 11a to the wide-angle side compared to normal shooting. This is because, when the face is detected, the possibility that the face can be detected increases if the zoom lens is set to the wide angle side to increase the depth of field. Also, if the zoom lens is set to the wide angle side, the possibility that the captured image will be shaken is reduced. In this case, it is particularly preferable to fix the position of the zoom lens 11a at the wide angle end because the depth of field becomes deeper.

  By the way, although the depth of field is deepened even if the aperture (not shown) of the imaging optical system is reduced, in this case, the imaging time corresponding to the shutter speed becomes longer and the possibility that the face registration image is shaken increases. . Therefore, from the viewpoint of avoiding camera shake, it is more preferable for the CPU 26 to set the aperture as open as possible. Note that the CPU 26 may perform control to gradually increase the depth of field when face detection cannot be performed.

  (5) In the “face registration mode”, the CPU 26 performs optical blur correction by moving the image blur correction lens 14 based on the output of the swing sensor unit 13. This is to avoid a failure in photographing the face registration image due to camera shake.

  (6) In the “face registration mode”, the CPU 26 increases the gain setting (imaging sensitivity setting) of the image signal in the analog signal processing unit 17 higher than the setting during normal shooting. This is because when the imaging sensitivity is increased, the above-described shooting time is shortened, and thus the possibility that the face registration image is shaken is also reduced.

  (7) The CPU 26 disables the flash emission of the flash emission unit 25 in the “face registration mode”. This is because if the flash light is emitted at the time of photographing, a shadow is formed on the face of the photographed person and the accuracy of face detection is lowered. Note that the CPU 26 uses the flash light emitting unit 25 as an exception when forcibly shooting in a state where the scene is dark and sufficient exposure cannot be secured, or when it is intended to acquire face recognition data during flash light emission. You may make it light-emit.

  Step 107: The CPU 26 drives the image sensor 16 to acquire a through image signal. The image sensor 16 outputs a through image signal by thinning readout at predetermined intervals. The image processing unit 20 generates through image data based on the through image signal. A preview image is displayed as a moving image on the liquid crystal monitor 24 based on the through image data.

  Step 108: On the other hand, the face detection unit 31 of the CPU 26 performs a known face detection process on the through image data to detect a face area in the shooting screen.

  In S <b> 108, it is more preferable that the face detection unit 31 lowers the threshold value that defines the detection accuracy of the face area (face detection accuracy rate) to increase the detection rate of the face candidates higher than usual. This is because a face registration image may be taken in a face orientation (for example, landscape orientation) in which face detection is difficult.

  When the face detection unit 31 detects a person's face in the shooting screen, the image processing unit 20 displays a rectangular frame at the position of the face area of the preview image (see FIG. 6). Thereby, the user can confirm the presence / absence of face detection in the preview image.

  Step 109: The CPU 26 determines whether or not a face is detected from the shooting screen in S108. If a face is detected (YES side), the process proceeds to S110. On the other hand, if no face is detected (NO side), the process returns to S107.

  Step 110: The CPU 26 estimates the face orientation based on the detection state of the facial feature points in the face area. Then, the CPU 26 determines whether or not the estimated face orientation matches a face orientation (front orientation, landscape orientation, etc.) designated in advance. If the face orientation is the predetermined direction (YES side), the process proceeds to S111. On the other hand, if the face orientation is other than the predetermined direction (NO side), the process returns to S107. At this time, the CPU 26 may cause the liquid crystal monitor 24 or the like to display a warning display such as “Please change the face direction of the subject and take a picture again”.

  Here, the detection of the face orientation in S110 is performed by a known means. For example, as in JP-A-9-163212, the CPU 26 may detect the face outline and nose coordinates from the face image to calculate the face orientation. Alternatively, as in JP-A-2005-209137, the CPU 26 may identify the face orientation based on an image database in which face images are classified and registered for each face direction.

  In the case of NO in S110, the CPU 26 does not release, and no shooting is performed until the face direction of the subject changes. Therefore, face recognition data can be generated from a face registration image having a predetermined face orientation.

  Step 111: The CPU 26 estimates the facial expression based on the detection state of the facial feature points in the facial area. Then, the CPU 26 determines whether or not the estimated facial expression is in a neutral state (a standard state without blinking). When the facial expression is neutral (YES side), the process proceeds to S112. On the other hand, if the facial expression is not neutral (NO side), the process returns to S107. At this time, the CPU 26 may display a warning display such as “Please change the facial expression of the subject and take a picture again” on the liquid crystal monitor 24 or the like.

  Here, the detection of the facial expression in S111 is performed by a known means. For example, as disclosed in Japanese Patent Application Laid-Open No. 2004-272933, learning face images classified into classes such as a normal expression, a face with closed eyes, a face with open mouth, and a smile are prepared in advance, and the CPU 26 canonically. After performing discrimination by discriminant analysis, facial expressions may be discriminated based on evaluation values set for each class. Alternatively, standard patterns such as open eyes, closed eyes, open mouths, closed mouths, etc., may be prepared in advance, and the CPU 26 may match each pattern with the face area and estimate the facial expression from the combination of patterns. .

  In the case of NO at S111, the CPU 26 does not release, and no image is taken until the facial expression of the subject changes. Therefore, the face registration image in which the facial expression is neutral can be acquired more reliably. If face recognition data is generated from an image with a smile or closed eyes, the accuracy of face recognition may be greatly reduced, so a face registration image with a neutral expression is considered to be the most suitable image for face recognition. Because.

  Note that the CPU 26 may omit the processes of S110 and S111 according to user settings.

  Step 112: The CPU 26 performs an AF calculation and an AE calculation based on the detected position of the face area.

  Step 113: The CPU 26 determines whether or not the release button has been fully pressed. When the release button is fully pressed (YES side), the process proceeds to S114. On the other hand, when there is no input to the release button (NO side), the CPU 26 waits for a full pressing operation of the release button.

  Step 114: The CPU 26 drives the image sensor 16 to take a subject image. Thereafter, the image processing unit 20 generates photographed image data (face registration image data) based on the image signal at the time of release.

  Step 115: The face recognition unit 32 of the CPU 26 generates registered person face recognition data from the face registration image data (S114). Further, the image processing unit 20 trims the face area of the face registration image data and resizes the face area to a predetermined size by resolution conversion to generate index image data.

  Step 116: The CPU 26 records the photographing conditions and photographing date / time of the face recognition data, the index image data, and the face registration image data in the face registration memory 27. Each of the above data is recorded in the group folder of the registered person specified in S104 or S105 in a state of being associated with each other. Among the above data, the index image data corresponds to “face registration image”. The shooting condition corresponds to “registration time of face recognition data”, and the shooting condition corresponds to “shooting condition when acquiring face recognition data”.

  Thus, a series of operations in the “face registration mode” is completed. Note that the CPU 26 may return to S107 after S116 to further continue registration of the registered person's face recognition data.

  Hereinafter, the effect of the “face registration mode” of the first embodiment will be described. In the face registration shooting described above, settings suitable for face registration are changed as shooting settings, and AF calculation and AE calculation are performed based on the detection position of the face area (S106, S112). Then, face recognition data is generated based on face registration image data obtained by photographing a person's face (S114, S115). Therefore, the user can easily acquire face recognition data based on a properly registered face registration image by simply pointing the electronic camera at the person to be registered and releasing the image.

(Operation in face recognition shooting mode)
FIG. 7 is a flowchart showing the operation of the “face recognition photographing mode” of the first embodiment. This “face recognition shooting mode” is a shooting mode in which, when a designated registered person exists on the shooting screen, the electronic camera performs predetermined control to acquire shot image data.

  Step 201: First, the user sets the photographing mode of the electronic camera to the “face recognition photographing mode” using the operation member 23.

  For example, in a state where the electronic camera is set to “normal shooting mode” or “portrait shooting mode”, the user performs an operation for calling a mode selection screen. As a result, the CPU 26 displays a mode selection screen (see FIG. 3A) on the liquid crystal monitor 24. Then, the user operates the selection frame on the mode selection screen with the operation member 23 and instructs the CPU 26 to activate the “face recognition photographing mode”. Thereby, the CPU 26 starts a series of operations in the “face recognition photographing mode”.

  Step 202: The CPU 26 displays a registered person selection screen (see FIG. 4) on the liquid crystal monitor 24. Then, the CPU 26 receives a designation input of a registered person to be face recognized (a registered person who performs face recognition at the time of shooting). It is also possible for the user to designate and input a registered person for face recognition into the CPU 26 in advance before photographing.

  Here, in the “registered person selection screen”, the CPU 26 can accept designation of one or more registered persons. For example, the CPU 26 can accept all designations of registered persons in the face registration memory 27. In addition, the group folder may be linked in advance with a predetermined category (for example, family, circle, etc.), and the user may input a batch designation of registered persons linked to the CPU 26 by designating and inputting the category on the selection screen. it can.

  Step 203: The CPU 26 drives the image sensor 16 to acquire a through image signal. The image sensor 16 outputs a through image signal by thinning readout at predetermined intervals. The image processing unit 20 generates through image data based on the through image signal. A preview image is displayed as a moving image on the liquid crystal monitor 24 based on the through image data.

  Step 204: On the other hand, the face detection unit 31 of the CPU 26 performs a known face detection process on the through image data to detect a face area in the shooting screen.

  Step 205: The CPU 26 determines whether or not a face is detected from the shooting screen in S204. If a face is detected (YES side), the process proceeds to S206. On the other hand, if the face is not detected (NO side), the process proceeds to S209.

  Step 206: The CPU 26 performs AF calculation with reference to the detected position of the face area. If a plurality of face areas are detected, the CPU 26 prioritizes the face area located at the center of the shooting screen or the face area located closest to the shooting screen, and executes the AF calculation.

  Step 207: The face recognition unit 32 of the CPU 26 executes face recognition processing on the detected face area (S204). Then, the face recognition unit 32 determines whether or not the recognition target face (S202) is included. If the face to be recognized is included (YES side), the process proceeds to S208. On the other hand, if the face to be recognized is not included (NO side), the process proceeds to S210.

  Step 208: The CPU 26 changes the setting of the photographing condition of the electronic camera and the setting of the image processing based on the data of “processing setting at the time of face recognition” corresponding to the detected registered person. Thereafter, the process proceeds to S210.

  Here, when a plurality of faces to be recognized are detected in the shooting screen, the CPU 26 determines the priority order of the registered person using a predetermined algorithm. Then, the CPU 26 changes various settings of the electronic camera based on the “processing setting for face recognition” corresponding to the registered person with the highest priority. Note that if the automatic shooting of the registered person is set in the “processing setting for face recognition”, the CPU 26 proceeds to S211 and automatically images the registered person.

  Step 209: On the other hand, in this case, the CPU 26 selects the AF area by the algorithm at the time of normal shooting and executes the AF calculation in response to the half-press operation of the release button by the user.

  Step 210: The CPU 26 determines whether or not the release button has been fully pressed. When the release button is fully pressed (YES side), the process proceeds to S211. On the other hand, when there is no input to the release button (NO side), the CPU 26 waits for a full pressing operation of the release button.

  Step 211: The CPU 26 drives the image sensor 16 to capture a subject image. Thereafter, the image processing unit 20 generates captured image data based on the image signal at the time of release. Note that when the face to be recognized is photographed, the image processing unit 20 performs the image processing set in S208 on the photographed image data.

  Step 212: The CPU 26 records the photographed image data on the recording medium 29. When the face to be recognized is photographed, the CPU 26 can also classify and record the photographed image data in a predetermined recording folder based on the directory path specified in the face registration memory 27. Thus, a series of operations in the “face recognition photographing mode” is completed.

  Hereinafter, the effect of the “face recognition photographing mode” of the present embodiment will be described. In the face recognition shooting mode, face recognition of the photographed person is performed based on the face recognition data (S207). When a face to be recognized is detected, various processes at the time of shooting are performed with settings corresponding to the registered person (S208, S211 and S212). Therefore, the process reflecting the user's intention can be automatically performed when the registered person is photographed, and the convenience of the user who operates the electronic camera is remarkably improved.

<Description of Second Embodiment>
FIG. 8 is a flowchart showing the operation in the “face registration mode” of the second embodiment. Note that S301 to S312 in FIG. 8 correspond to S101 to S112 in FIG. 2, respectively, and S315 in FIG. 8 corresponds to S116 in FIG.

  Step 313: The CPU 26 automatically releases after the end of AF and drives the image sensor 16 to shoot a subject image. Thereafter, the image processing unit 20 generates photographed image data (face registration image data) based on the image signal at the time of release.

  Step 314: The face recognition unit 32 of the CPU 26 generates registered person face recognition data from the face registration image data (S313). Further, the image processing unit 20 generates index image data from the face registration image data.

  The effects of the “face registration mode” of the second embodiment will be described below. In the second embodiment, when a face is detected, a human face is automatically photographed to generate face recognition data (S313, S314). Therefore, the user can easily acquire face recognition data based on an appropriate face registration image simply by pointing the electronic camera toward the person to be registered. In particular, in the second embodiment, since focusing and release are automated, it is possible to perform face registration photographing of the user with only one person with the photographing optical system facing the user himself / herself.

<Description of Third Embodiment>
FIG. 9 is a flowchart showing the operation in the “face registration mode” of the third embodiment. Note that S401 to S409 in FIG. 9 correspond to S101 to S109 in FIG. 2, respectively, and S414 in FIG. 9 corresponds to S116 in FIG.

  Step 410: The CPU 26 executes AF calculation and AE calculation with reference to the detected position of the face area.

  Step 411: The CPU 26 automatically performs a predetermined number of releases in succession to obtain a plurality of photographed image data (face registration image data). Here, in S411, the CPU 26 shoots a plurality of face registration image data under the same shooting conditions. For example, when the subject is not stationary or when camera shake occurs, different images are captured even under the same shooting conditions. Note that the CPU 26 may perform bracketing shooting by changing parameters such as exposure conditions and white balance in stages.

  Step 412: The face detection unit 31 of the CPU 26 performs a face detection process on each of the plurality of face registration image data acquired in S411, and extracts feature points. Then, the CPU 26 sets the priority order for each face registration image data in the order in which the detection state of the facial feature points is favorable (for example, in the descending order of the number of extracted feature points). Note that, as shown in S110 and S111 of the first embodiment, the CPU 26 may set the above-mentioned priority order in consideration of the face orientation and facial expression of the face registration image data.

  Step 413: The CPU 26 selects face registration image data with the highest priority among the plurality of face registration image data acquired in S411. Then, the face recognition unit 32 of the CPU 26 generates face recognition data from the selected face registration image data. Further, the image processing unit 20 generates index image data from the selected face registration image data.

  The effects of the “face registration mode” of the third embodiment will be described below. In the third embodiment, face recognition data is generated by selecting an optimum image from a plurality of face registration image data. Therefore, according to the third embodiment, it becomes easier to improve the accuracy of face recognition during face recognition shooting.

<Description of Fourth Embodiment>
FIG. 10 is a flowchart showing the operation in the “face registration mode” of the fourth embodiment. Note that S501 to S505 in FIG. 10 correspond to S101 to S105 in FIG. 2, respectively, and S513 and S514 in FIG. 10 correspond to S115 and S116 in FIG.

  Step 506: The CPU 26 changes the setting at the time of shooting with the electronic camera to a setting suitable for face registration shooting. Specifically, the CPU 26 performs the same settings as (3) to (7) in S106, and performs the following settings (a) and (b).

  (A) In the “face registration mode”, the CPU 26 selects the AF area at the center of the shooting screen and performs AF calculation. In this case, the CPU 26 performs focus control with so-called center-weighted AF in response to a half-press operation of the release button, and executes control to lock AF when focus is achieved. This is because by acquiring a face registration image at the center of the shooting screen, the influence of the aberration of the shooting optical system is reduced, and highly accurate face recognition data is generated.

  (B) In the “face registration mode”, the CPU 26 sets the AE method to the spot AE mode based on the center of the shooting screen. Thereby, the exposure of the face photographed at the center of the photographing screen is optimized.

  Step 507: The CPU 26 drives the image sensor 16 to acquire a through image signal. The image sensor 16 outputs a through image signal by thinning readout at predetermined intervals. The image processing unit 20 generates through image data based on the through image signal. A preview image is displayed as a moving image on the liquid crystal monitor 24 based on the through image data.

  Here, the image processing unit 20 in S507 synthesizes a composition auxiliary display at the center of the preview image shooting screen (see FIG. 11). The composition auxiliary display in FIG. 11 shows the position where the face is stored in the shooting screen by a rectangular frame. This composition assist display allows the user to easily position the face during framing. Therefore, the user can easily shoot a face registration image in good condition. As another example of composition assistance display, only the four corners of the rectangular frame may be combined with the preview image. Alternatively, as another example of the composition assistance display, for example, the display color of the image may be changed inside and outside the face-covering range by means such as monochrome display other than the face-filling range (both shown) Is omitted).

  In step S507, the image processing unit 20 synthesizes the photographic sample display in the peripheral area other than the center of the preview image (see FIG. 11). The photographic sample display of FIG. 11 displays a photograph or illustration of a person's face facing a predetermined direction. For example, when a front-facing face registration image is shot, a front-facing face image is displayed on the shooting sample display. By this shooting sample display, the user can visually understand the face state (face orientation, etc.) suitable for the current shooting. Therefore, the user can easily shoot a face registration image in good condition. Note that the image processing unit 20 may synthesize a message display such as “Please photograph the front face” with the preview image.

  Step 508: The CPU 26 determines whether or not the release button is half-pressed. If the release button is half-pressed (YES side), the flow shifts to S509. On the other hand, if there is no input on the release button (NO side), the process returns to S507.

  Step 509: The CPU 26 executes AF calculation and AE calculation based on the center of the shooting screen in response to a half-press operation of the release button by the user.

  Step 510: The CPU 26 determines whether or not the release button has been fully pressed. If the release button is fully pressed (YES side), the flow shifts to S511. On the other hand, when there is no input to the release button (NO side), the CPU 26 waits for a full pressing operation of the release button.

  Step 511: The CPU 26 acquires photographed image data (face registration image data) in response to a release button full-press operation by the user.

  Step 512: The face detection unit 31 of the CPU 26 performs a known face detection process on the center of the shooting screen of the face registration image data (S511) to detect a face area in the shooting screen.

  The effects of the “face registration mode” of the fourth embodiment will be described below. In the fourth embodiment, face recognition data is generated based on face registration image data manually shot by a user. Therefore, it is possible to create face recognition data according to the user's intention. In addition, since the face registration image is acquired at the center of the shooting screen that is less affected by the aberration of the shooting optical system, it is possible to generate more accurate face recognition data.

(Modification of face registration mode in the first to fourth embodiments)
In the above embodiment, the example in which the face recognition data is generated from the face registration image at the time of shooting has been described. However, instead of generating face recognition data from the face registration image, the CPU generates face image data by cutting out the face area portion from the face registration image, and records this face image data in a memory or a recording medium. Also good. Then, face recognition data can be generated from face image data by post-processing by a computer.

  Here, the face image data is generated by cutting out the face area of the face registration image with a size of about 80 × 80 pixels, for example. It is preferable not to perform a scaling process by resolution conversion on the face image data image. As in the case of face registration image data, the CPU preferably records the face image data in association with the index image data, the shooting conditions of the face registration image data, and the shooting date and time. Further, with respect to the face image data, the CPU preferably records a plurality of face image data generated from the same photographed person in the same folder or the like.

(Supplementary items of the embodiment)
As mentioned above, although this invention has been demonstrated by said embodiment, the technical scope of this invention is not limited to the said embodiment, For example, the following forms may be sufficient.

  (1) Even in the “face registration mode” from the first embodiment to the third embodiment, composition auxiliary display and photographing sample display may be combined and displayed on the preview image as in the fourth embodiment. Also in the “face registration mode” of the fourth embodiment, face detection processing may be performed with through image data before release.

  (2) In the embodiment described above, the CPU 26 may perform AF calculation based on face detection with a half-press of the release button as a trigger.

  (3) In the above embodiment, “Face registration mode” and “Face detection shooting mode” are directly designated by a mode dial or the like in the same way as “Normal shooting mode”, “Portrait shooting mode”, “Playback mode”, etc. May be launched.

  (4) In the “face registration mode” of the third embodiment, the CPU 26 may generate two or more face recognition data in descending order of priority.

  (5) In the “face registration mode” of the fourth embodiment, the CPU 26 captures the face registration image data on the recording medium 29 or the like, batch-processes the plurality of face registration image data, and generates face recognition data collectively. It may be. At this time, as in the third embodiment, priority may be set for the face registration image data, and face recognition data may be generated in descending order of priority.

  (6) The “registration time of face recognition data” data in the face registration memory may be sufficient to indicate when the original face registration image data was captured. Accordingly, the “registration time of the face recognition data” may be terminated by recording at any of the shooting year, shooting month, and shooting date, and recording of detailed time and the like may be omitted.

  (7) In the face recognition process (S207) of the “face recognition shooting mode”, the face recognition unit 32 may narrow down the face recognition data used for the determination from the grouped face recognition data under a predetermined condition. . For example, the face recognizing unit 32 may exclude face recognition data that is older than a shooting date and time by a certain period or more based on the “face recognition data registration time”. Similarly, the face recognizing unit 32 may exclude face recognition data generated under shooting conditions that are significantly different from the current shooting conditions based on the “shooting conditions when acquiring face recognition data”.

  (8) In the face recognition process (S207) in the “face recognition shooting mode”, the face recognition unit 32 corrects the face recognition data according to the degree of distortion calculated from the focal length data of the shooting optical system 11. May be.

  (9) In the “face registration mode”, the CPU 26 may generate identification information relating to the face direction of the face recognition data and record the identification information in the face registration memory 27. The identification information regarding the face orientation may be generated by the CPU 26 based on the result of the process shown in S110 of the first embodiment. Alternatively, the CPU 26 may generate the identification information based on a user input during shooting or editing.

  (10) The electronic camera of the present invention does not necessarily match the configuration of the electronic camera of the embodiment. For example, the built-in memory 19 and the face registration memory 27 may be configured by a common memory. The face registration memory 27 may be connected to the CPU 26 via the data bus 28. Further, the optical blur correction mechanism including the swing sensor unit 13, the image blur correction lens 14, and the second lens driving unit 15 may be omitted. Furthermore, the communication unit 30 may be built in the electronic camera.

  (11) The CPU 26 may record the data in the face registration memory 27 on the recording medium 29 so that the face registration memory 27 can be backed up. Further, the CPU 26 may acquire face recognition data or the like generated in advance by another electronic camera from the recording medium 29 so that the data in the face registration memory 27 can be shared between the electronic cameras.

DESCRIPTION OF SYMBOLS 11 ... Imaging optical system, 11a ... Zoom lens, 11b ... Focusing lens, 12 ... 1st lens drive part, 13 ... Swing sensor part, 14 ... Image blur correction lens, 15 ... 2nd lens drive part, 16 ... Imaging element , 17 ... Analog signal processing unit, 20 ... Image processing unit, 24 ... Liquid crystal monitor, 25 ... Flash emission unit, 26 ... CPU, 27 ... Face registration memory, 31 ... Face detection unit, 32 ... Face recognition unit

Claims (6)

An image sensor that captures a subject image and generates an image signal;
A face detection unit that detects a face region based on the image signal from the image sensor;
Setting means for setting the shooting mode;
Setting means for setting the detection accuracy of the face area according to the shooting mode set by the setting means;
An electronic camera characterized by comprising: The electronic camera according to claim 1,
An electronic camera, further comprising: a face recognition data generation unit that extracts a feature point of a photographed person's face from the face area portion and generates face recognition data based on the feature point. The electronic camera according to claim 2,
An electronic camera, further comprising a control unit that acquires a plurality of pieces of captured image data by a plurality of releases when generating the face recognition data. The electronic camera according to claim 3.
The control unit is configured to acquire a plurality of data of the photographed images by changing photographing conditions in stages. The electronic camera according to claim 4,
The electronic camera according to claim 1, wherein the control unit obtains a plurality of data of the photographed images by changing exposure parameters or white balance parameters stepwise. The electronic camera according to any one of claims 3 to 5,
The face recognition data generation unit extracts the feature points from a plurality of the captured image data, and prioritizes the face recognition data generated from the captured image data according to the number of extracted feature points. An electronic camera characterized by giving a ranking.

Images