JP2022027841A - Electronic apparatus, program, and storage medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To control display of an AF result according to an operation mode set by a user to perform display of the AF result along an intention of the user.

SOLUTION: An electronic apparatus detects the face of a subject and an organ on the face from an image, when an area in the image is designated according to a user instruction, controls to perform AF for an AF target corresponding to the designated area, and displays, on a display, display of the AF target superimposed on the image. When a first operation mode is set in which either one of the face and the organ of the subject is designated as the AF target according to an area designated according to the user instruction in a partial area corresponding to the face of the subject in the image or in a specific area including the organ on the face, even if the partial area is designated, when the organ included in the face of the subject is detected, the electronic apparatus uses the detected organ as the AF target and performs display of the AF target for the detected organ.

SELECTED DRAWING: Figure 2

COPYRIGHT: (C)2022,JPO&INPIT

Description

The present invention relates to an image pickup device, and relates to an image pickup device capable of detecting a face and facial organs (pupil, etc.).

Many conventional digital cameras are capable of autofocus (AF) on a subject automatically detected by the camera or a subject arbitrarily selected by the user. Particularly in recent years, cameras that not only detect the face as a subject but also detect facial organs (for example, pupils) and realize AF to a more precise position of the subject (hereinafter referred to as pupil AF) are also known. There is.

In such a camera, it is important to make the specifications so that the user's intention can be reflected more by making it possible to select the AF target position from the face and the pupil. For example, Patent Document 1 describes a technique in which a user touches an LCD screen to determine whether the position is a face or a pupil and selects it as an AF target. This allows the user to easily specify the AF target position according to the shooting scene, thus realizing a camera with high usability.

Japanese Unexamined Patent Publication No. 2013-07164

However, as in the technique disclosed in the above patent document, when the pupil is selected, the AF target position is always the pupil, and when the face is selected, the AF target position is always the face. Inconvenience occurs with cameras that can be switched on / off. For example, a user who has set the pupil AF on (an operation mode in which an operation targeting the pupil is performed) should expect that AF is appropriately performed with the pupil of the selected subject. However, if the AF frame is always displayed near the face when the face is selected, the user may feel that AF has not been performed with the eyes, and the AF operation may be repeated and shooting may not be possible indefinitely.

The present invention has been made in view of the above problems, and by controlling the AF result display according to the operation mode set by the user, it is possible to display the AF result according to the user's intention. With the goal.

The electronic device according to one aspect of the present invention has the following configurations. That is,
A detection means that detects the subject's face and facial organs from within the image,
A designated means for designating an area in an image according to user instructions,
AF control means that controls AF target corresponding to the area designated by the designated means so as to perform AF, and
The region corresponding to the face of the subject in the detected image is defined as a partial region, the region including the detected facial organs is defined as a specific region, and the partial region or the specific region included in the partial region is included. A setting means for setting one of a plurality of operation modes including a first operation mode in which either the face of the subject or the organ can be designated as an AF target according to the area designated by the designating means. ,
A display control means for superimposing the display of the AF target on the image and displaying it on the display unit is provided.
When the first operation mode is set, in the AF control means, even if the partial area is designated by the designated means, the organ included in the face corresponding to the partial area is detected by the detection means. When it is detected, the detected organ is used as an AF target, and the display control means displays the AF target on the detected organ.

According to the present invention, by controlling the AF result display according to the operation mode set by the user, the AF result display is performed according to the user's intention. Therefore, for example, in each of the settings of a plurality of operation modes having different AF targets that can be specified, it is possible to notify an appropriate AF result that the user does not feel a sense of discomfort.

The external view of the camera in embodiment. The functional block diagram which shows the configuration example of the camera in Embodiment. The flowchart which shows the shooting mode processing in embodiment. The flowchart which shows the shooting mode processing in embodiment. The flowchart which shows the frame display process in an embodiment. The flowchart which shows the frame display process in an embodiment. The flowchart which shows the frame display process in an embodiment. The flowchart which shows the AF process in an embodiment. The flowchart which shows the AF process in an embodiment. The schematic diagram which shows the screen example in an embodiment. The schematic diagram which shows the screen example in an embodiment. The schematic diagram which shows the screen example in an embodiment. The schematic diagram which shows the touch reaction area in an embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the embodiment described below, the case where the electronic device according to the present invention and the control method thereof are applied to a digital camera capable of shooting still images / moving images is taken as an example.

FIG. 1 shows a rear view of the digital camera 100 as an example of the image pickup apparatus according to the present embodiment. The display unit 28 is a display unit that displays images and various information. The display unit 28 includes a rear display panel 28a and an electronic viewfinder 28b, which is a display unit in the finder, as described with reference to FIG. The shutter button 61 is an operation unit for giving a shooting instruction. The mode changeover switch 60 is an operation unit for switching various modes. The connector 112 is a connector for connecting a connection cable 111 for connecting to an external device such as a personal computer or a printer and a digital camera 100. The operation unit 70 includes operation members such as various switches, buttons, and a touch panel that receive various operations from the user. The controller wheel 73 is a rotation-operable operating member included in the operating unit 70. The power switch 72 is a push button for switching between power on and power off.

The recording medium 200 includes, for example, a memory card, a hard disk, or the like, and stores an image or the like taken by the digital camera 100. The recording medium slot 201 is a slot for detachably storing the recording medium 200. The recording medium 200 stored in the recording medium slot 201 can communicate with the digital camera 100, and can record and play back. The lid 202 is the lid of the recording medium slot 201. FIG. 1 shows a state in which a lid 202 is opened and a part of the recording medium 200 is taken out from the recording medium slot 201 to be exposed.

FIG. 2 is a block diagram showing a configuration example of the digital camera 100 according to the present embodiment. In FIG. 2, the photographing lens 103 is a lens group including a zoom lens and a focus lens. The shutter 101 is a shutter having an aperture function. The image pickup unit 22 is an image pickup device composed of a CCD, a CMOS element, or the like that converts an optical image into an electric signal. The A / D converter 23 converts an analog signal into a digital signal. The A / D converter 23 is used to convert an analog signal output from the image pickup unit 22 into a digital signal. The barrier 102 covers the imaging system including the photographing lens 103 of the digital camera 100 to prevent the imaging system including the photographing lens 103, the shutter 101, and the imaging unit 22 from being soiled or damaged.

The image processing unit 24 performs resizing processing such as predetermined pixel interpolation and reduction and color conversion processing on the data from the A / D converter 23 or the data from the memory control unit 15. In addition, the image processing unit 24 performs predetermined arithmetic processing using the captured image data. The system control unit 50 performs exposure control and distance measurement control based on the calculation result obtained by the image processing unit 24. As a result, TTL (through-the-lens) AF (autofocus) processing, AE (automatic exposure) processing, and EF (flash pre-flash) processing are performed. The image processing unit 24 further performs a predetermined calculation process using the captured image data, and performs a TTL method AWB (auto white balance) process based on the obtained calculation result.

The output data from the A / D converter 23 is written directly to the memory 32 via the image processing unit 24 and the memory control unit 15 or via the memory control unit 15. The memory 32 stores, for example, image data obtained by the image pickup unit 22 and converted into digital data by the A / D converter 23, and image data to be displayed on the display unit 28. The memory 32 has a storage capacity sufficient to store a predetermined number of still images, moving images for a predetermined time, and audio.

Further, the memory 32 also serves as a memory (video memory) for displaying an image. The D / A converter 13 converts the image display data stored in the memory 32 into an analog signal and supplies it to the display unit 28. In this way, the image data for display written in the memory 32 is displayed by the display unit 28 via the D / A converter 13. The display unit 28 displays on a display such as an LCD according to the analog signal from the D / A converter 13. The digital signal once A / D converted by the A / D converter 23 and stored in the memory 32 is analog-converted by the D / A converter 13 and sequentially transferred to the display unit 28 for display, thereby displaying a through image (through image display). Live view display (LV display)) can be performed. Hereinafter, the image displayed in the live view is referred to as an LV image.

The non-volatile memory 56 is a memory as a recording medium that can be electrically erased and recorded, and for example, EEPROM or the like is used. The non-volatile memory 56 stores constants, programs, and the like for the operation of the system control unit 50. The program referred to here is a computer program for executing various flowcharts described later in this embodiment.

The system control unit 50 is at least one processor or circuit that controls the entire digital camera 100. By executing the program recorded in the non-volatile memory 56 described above, each process of the present embodiment described later is realized. For example, RAM is used as the system memory 52. In the system memory 52, constants and variables for the operation of the system control unit 50, a program read from the non-volatile memory 56, and the like are expanded. The system control unit 50 also controls the display by controlling the memory 32, the D / A converter 13, the display unit 28, and the like.

The system timer 53 is a time measuring unit that measures the time used for various controls and the time of the built-in clock. The mode changeover switch 60, the shutter button 61, and the operation unit 70 are operation means for inputting various operation instructions to the system control unit 50.

The mode changeover switch 60 switches the operation mode of the system control unit 50 to any one of a still image recording mode, a moving image shooting mode, a reproduction mode, and the like. The modes included in the still image recording mode include an auto shooting mode, an auto scene discrimination mode, a manual mode, various scene modes for shooting settings for each shooting scene, a program AE mode, a custom mode, and the like. The mode selector switch 60 allows the user to switch directly to any of these modes. Alternatively, after switching to the menu button once with the mode changeover switch 60, the mode may be switched to any of these modes included in the menu button by using another operating member. Similarly, the moving image shooting mode may include a plurality of modes.

The shutter button 61 has a first shutter switch 62 and a second shutter switch 64. The first shutter switch 62 is turned on by a so-called half-pressing (shooting preparation instruction) during the operation of the shutter button 61 provided on the digital camera 100, and the first shutter switch signal SW1 is generated. The first shutter switch signal SW1 starts operations such as AF (autofocus) processing, AE (automatic exposure) processing, AWB (auto white balance) processing, and EF (flash pre-flash) processing. The second shutter switch 64 is turned on when the operation of the shutter button 61 is completed, that is, when the shutter button 61 is fully pressed (shooting instruction), and the second shutter switch signal SW2 is generated. The system control unit 50 starts a series of shooting processes from reading the signal from the image pickup unit 22 to writing the image data to the recording medium 200 by the second shutter switch signal SW2.

Each operation member of the operation unit 70 is assigned a function as appropriate for each scene by selecting and operating various function icons displayed on the display unit 28, and acts as various function buttons. Examples of the function button include an end button, a back button, an image feed button, a jump button, a narrowing down button, an attribute change button, and the like. For example, when the menu button is pressed, various configurable menu screens are displayed on the display unit 28. The user can intuitively make various settings by using the menu screen displayed on the display unit 28 and the up / down / left / right four-way buttons and the SET button.

The controller wheel 73 is a rotation-operable operation member included in the operation unit 70, and is used when instructing a selection item together with a direction button. When the controller wheel 73 is rotated, an electrical pulse signal is generated according to the amount of operation, and the system control unit 50 controls each unit of the digital camera 100 based on this pulse signal. From this pulse signal, it is possible to determine the angle at which the controller wheel 73 is rotated, the number of rotations, and the like. The controller wheel 73 may be any operation member as long as it can detect the rotation operation. For example, the controller wheel 73 itself may be a dial operation member that rotates in response to a user's rotation operation to generate a pulse signal. Further, the operation member including the touch sensor may be one that does not rotate the controller wheel 73 itself and detects the rotational movement of the user's finger on the controller wheel 73 (so-called touch wheel).

The power supply control unit 80 includes a battery detection circuit, a DC-DC converter, a switch circuit for switching a block to be energized, and the like, and detects whether or not a battery is installed, the type of battery, and the remaining battery level. Further, the power supply control unit 80 controls the DC-DC converter based on the detection result and the instruction of the system control unit 50, and supplies a necessary voltage to each unit including the recording medium 200 for a necessary period. The power supply unit 30 includes a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, a NiMH battery, or a Li battery, an AC adapter, or the like.

The recording medium I / F 18 is an interface with a recording medium 200 such as a memory card or a hard disk. The recording medium 200 is a recording medium such as a memory card for recording a captured image, and is composed of a semiconductor memory, an optical disk, a magnetic disk, or the like. The communication unit 54 is connected by a wireless or wired cable to transmit and receive video signals, audio signals, and the like. The communication unit 54 can also be connected to a wireless LAN (Local Area Network) or the Internet. The communication unit 54 can transmit an image (including a through image) captured by the image pickup unit 22 and an image recorded on the recording medium 200, and can also receive image data and other various information from an external device. can.

The posture detection unit 55 detects the posture of the digital camera 100 with respect to the direction of gravity. Whether the image taken by the image pickup unit 22 based on the posture detected by the posture detection unit 55 is an image taken by holding the digital camera 100 horizontally or an image taken by holding the digital camera 100 vertically. Can be determined. The system control unit 50 can add orientation information according to the posture detected by the posture detection unit 55 to the image file of the image captured by the image pickup unit 22, or rotate and record the image. be. As the attitude detection unit 55, an acceleration sensor, a gyro sensor, or the like can be used.

The eyepiece detection unit 57 detects the approach (eyepiece) of the eye (object). The system control unit switches between display / non-display of the rear display panel 28a and the electronic viewfinder 28b according to the state detected by the eyepiece detection unit 57. That is, when the eyepiece detection unit 57 detects the eyepiece, the display destination is the electronic viewfinder 28b, and when the eyepiece detection unit 57 does not detect the eyepiece, the display destination is the rear display panel 28a.

As one of the operation units 70, there is a touch panel 70a capable of detecting contact (position) with respect to the rear display panel 28a. The touch panel 70a and the rear display panel 28a can be integrally configured. For example, the touch panel 70a can be a built-in type (in-cell type) that is configured so that the light transmittance does not interfere with the display of the rear display panel 28a and is incorporated inside the display screen of the rear display panel 28a. By associating the input coordinates on the touch panel 70a with the display coordinates on the display screen of the rear display panel 28a, it is as if the user can directly operate the display screen of the rear display panel 28a GUI (graphical user). Interface) can be provided. The system control unit 50 can detect the following user operations or states on the touch panel 70a.

-Detection that a finger or pen that has not touched the touch panel 70a newly touches the touch panel 70a, that is, the start of touch (hereinafter referred to as "touch-down").
-Detection that the touch panel 70a is in a state of being touched with a finger or a pen (hereinafter referred to as "Touch-On").
-Detection that the touch panel 70a is moving while being touched with a finger or a pen (hereinafter referred to as "Touch-Move").
-Detection that the finger or pen that was touching the touch panel 70a is released, that is, the end of the touch (hereinafter referred to as "Touch-Up") is detected.
-Detection of a state in which nothing is touched on the touch panel 70a (hereinafter referred to as "touch-off").

When a touchdown is detected, it is also detected that it is touch-on. After a touchdown, touch-on usually continues to be detected unless touch-up is detected. The touch move is also detected while the touch-on is detected. Even if touch-on is detected, touch move is not detected unless the touch position is moved. After it is detected that all the fingers and pens that have been touched are touched up, the touch is turned off.

These operations / states and the position coordinates of the finger or pen touching the touch panel 70a are notified to the system control unit 50 through the internal bus, and the system control unit 50 determines which on the touch panel 70a based on the notified information. Determine if such an operation has been performed. For example, regarding the touch move, the system control unit 50 can determine the moving direction of the finger or the pen moving on the touch panel 70a for each vertical component / horizontal component on the touch panel 70a based on the change in the position coordinates. Further, it is assumed that a stroke is drawn when the touch panel 70a is touched up from the touch down through a certain touch move.

The operation of drawing a stroke quickly is called flicking. Flicking is an operation in which a finger is quickly moved on the touch panel 70a by a certain distance while being touched, and then released as it is, in other words, an operation in which the touch panel 70a is swiftly traced with a finger. It can be determined that the flick has been performed when it is detected that the touch move is performed at a predetermined speed or more over a predetermined distance and the touch-up is detected as it is. In addition, if it is detected that the touch move is performed at a speed equal to or longer than a predetermined distance, it is determined that the drag has been performed. Further, a touch operation in which a plurality of points (for example, two points) are touched at the same time to bring the touch positions closer to each other is called a pinch-in, and a touch operation to move the touch positions away from each other is called a pinch-out. Pinch out and pinch in are collectively called pinch operation (or simply pinch).

The touch panel 70a may use any of various touch panels such as a resistance film method, a capacitance method, a surface acoustic wave method, an infrared ray method, an electromagnetic induction method, an image recognition method, and an optical sensor method. good. Depending on the method, there are a method of detecting that there is a touch due to contact with the touch panel and a method of detecting that there is a touch due to the approach of a finger or a pen to the touch panel, but either method may be used. ..

The digital camera 100 can be used by switching between a playback mode for reproducing at least an image and a shooting mode for shooting, and the shooting modes include an auto mode, a manual mode, and shooting for each of a plurality of scenes. It has a mode. The auto mode is a mode in which various parameters of the camera are automatically determined by a program incorporated in the digital camera 100 based on the measured exposure value. The manual mode is a mode in which the user can freely change various parameters of the camera. The shooting mode for each scene is a shooting mode in which various parameters are determined by combining the shutter speed, aperture value, strobe light emission state, sensitivity setting, white balance (WB) setting, etc. suitable for the shooting scene. .. The digital camera 100 has, for example, the scene-specific shooting modes (1) to (3) described below. However, it is not limited to these scene-specific shooting modes. The photographer can set the digital camera 100 to a desired shooting mode from the shooting mode selection menu and shoot.

(1) Portrait shooting mode: A mode specialized for portrait shooting by blurring the background to make the person stand out.
(2) Flower shooting mode: A mode in which the macro mode is set and the saturation is set high.
(3) Sports shooting mode: A shooting mode with settings specialized for shooting fast-moving subjects.

With the above-mentioned digital camera 100, it is possible to take a picture using the central one-point AF, the face AF, and the pupil AF. The central 1-point AF is to perform AF on the central position 1 point in the shooting screen. Face AF is to perform AF on the face in the shooting screen detected by the face detection function. Eye AF is to perform AF on the pupil included in the face in the photographing screen detected by the organ detection function, which is also a kind of face detection function.

The face detection function will be described. The system control unit 50 functions as a detection unit capable of detecting the face and the facial organs (eyes, nose, mouth, ears, etc.) from the image. The system control unit 50 sends the image data of the face detection target to the image processing unit 24. Under the control of the system control unit 50, the image processing unit 24 causes a horizontal bandpass filter to act on the image data. Further, under the control of the system control unit 50, the image processing unit 24 applies a vertical bandpass filter to the image data. By applying these horizontal and vertical bandpass filters, edge components are detected from the image data.

After that, the system control unit 50 performs pattern matching on the detected edge components, and extracts candidate groups of facial organs such as eyes, nose, mouth, and ears. Then, the system control unit 50 determines from the extracted eye candidate groups that the eyes satisfy preset conditions (for example, the distance between two eyes, the inclination, etc.), and narrows down the eye candidate groups. I do. Then, the system control unit 50 associates the narrowed-down eye candidate group with other parts (organs such as the nose, mouth, and ears) that form the corresponding face, and sets a preset non-face condition filter. By passing it through, the face is detected. The system control unit 50 determines the number of detected faces, the position, size, orientation of each face, the position and size of the internal organs (eyes, nose, mouth, ears), etc., according to the face detection result. Outputs the face information of and ends the process. At this time, the system control unit 50 stores a feature amount such as the number of faces in the system memory 52. The area once detected as a face continues to be detected as a face for a predetermined period (about 1 second) if conditions such as contrast, color, and size are met. As a result, even if the facial organs are not detected due to the subject temporarily facing backward, closing the eyes, etc., it is possible to continue to detect the face as a face.

The system control unit 50 determines the eyes that have been extracted for the face detected by the above-mentioned face detection function as the detected eyes (pupils), and outputs the pupil information as the pupil detection result. The pupil information includes, for example, the position of the eyes in the image, the position of the eyes on the face, the size, and the like, and the pupil region based on the position and size of the eyes. Eye detection is a type of organ detection that detects facial elements (parts).

As described above, it is possible to perform image analysis on the image data displayed in the live view or reproduced, extract the feature amount of the image data, and detect specific subject information such as a face and eyes.

Face AE, face FE, and face WB can be performed at the same time as face AF. Face AE is to optimize the exposure of the entire screen according to the detected brightness of the face. Face FE is to adjust the flash around the detected face. The face WB is to optimize the WB of the entire screen according to the detected face color.

Further, the system control unit 50 can also use the image processing unit 24 to detect an object (an object other than a person's face and organs) that is assumed to be the main subject depending on conditions such as color, contrast, and motion vector. Is.

Further, in the digital camera 100, it is possible to set one AF mode from a plurality of AF modes according to the user operation as the operation mode of the autofocus (AF) at the time of shooting. The AF mode setting is performed based on the user operation on the AF mode setting screen in which the menu item for setting the AF mode is selected and displayed on the setting menu screen. A plurality of AF modes are prepared for each AF position determination method. In the present embodiment, either the one-point AF mode or the tracking priority mode can be set as the AF mode.

The one-point AF mode is an AF mode in which an AF frame representing the focus adjustment position is set at the center of the shooting range or at one point designated by the user. In the one-point AF mode, the AF frame does not move even if the subject changes, and the information obtained from the position of the AF frame (contrast value and phase difference AF) regardless of whether or not a subject such as a face is detected. AF is performed based on the defocus amount for.

In the tracking priority mode, when there is no tracking designation from the user (tracking standby state, tracking release state), the subject automatically determined by the digital camera 100 as the main subject is the AF target (focus adjustment position). When a person's face is detected, the eyes or face of the detected person is given priority and the AF target is set as the main subject. When the face of a person is not detected, the digital camera 100 automatically determines the main subject according to predetermined conditions such as a moving object, a subject having a high contrast value, and a subject close to the center, and sets it as an AF target. In addition, after the tracking is specified by the user, the subject specified in the LV image is continuously tracked, and even if the position of the tracked subject within the shooting range changes, the tracked subject is AF. set to target. For example, when the user specifies to track the pupil or face of the person A (during tracking), even if the person A moves on the LV image, the person A continues to follow the pupil or face of the person A and makes it an AF target.

It is also possible to track other than a person (mono tracking), and even if the same subject moves in the LV image, it will continue to be tracked, subject to the color, contrast, shape, etc. of the position specified for tracking. Target for AF. That is, the tracking priority mode is an AF mode in which the AF position can be determined by performing tracking. The AF mode is not limited to the one-point AF mode and the tracking mode. For example, there may be an AF mode (“zone AF”) in which tracking is performed within a limited area specified by the user. The set AF mode is stored in the non-volatile memory 56, and is read out in the system memory 52 in the shooting mode processing.

3A and 3B are flowcharts relating to the shooting mode processing of the digital camera 100 according to the present embodiment. In the processing of FIGS. 3A to 3B and the processing of FIGS. 4A to 4C and 5A to 5B described later, the system control unit 50 executes the program recorded in the non-volatile memory 56 with the system memory 52 as the work memory. Is realized by. When the digital camera 100 is started in the live view shooting mode, the processes of FIGS. 3A to 3B are started. The processes of FIGS. 3A to 3B are processes when the AF mode is set to the tracking priority mode. The description of the processing when the one-point AF mode is set will be omitted.

In the tracking priority mode, the face and facial organs are detected from the image, and the detected face or organ is a candidate designated as an AF target (tracking target) by user operation (touch operation in this embodiment). .. Further, a plurality of operation modes having different AF targets that can be specified by such a touch operation are prepared, and the user can set any of them from the MENU screen. In the following, the facial organs detected from the image are the eyes (pupils), and the face and pupil can be designated as AF targets as multiple operation modes with different AF targets that can be specified. Eye AF on and face are designated as AF targets. An image pickup device that can set the pupil AF off (the pupil cannot be specified) will be described.

In S301 (FIG. 3A), the system control unit 50 displays the image acquired through the image pickup unit 22 as a live view (LV display) on the display unit 28 after performing the shooting mode initialization process. The display destination at this time is the rear display panel 28a when the display destination switching is set to "automatic" and the eyepiece is not detected, and the electronic viewfinder 28b when the eyepiece is detected. If the display destination is switched to "manual" and the display destination is set to the rear display panel 28a, the rear display panel 28a is used. If the display destination is set to the electronic viewfinder 28b, the electronic viewfinder 28b is used. Is. Here, the shooting mode initialization process is a process of reading parameters, setting values, and setting modes including flags and control variables from the non-volatile memory 56. Further, the state of the recording medium 200 is confirmed, and if there is an abnormality or the like, a warning or the like is displayed by superimposing it on the live view.

In S302, the system control unit 50 performs a frame display process for displaying a frame indicating that the pupil, face, or object is being detected, or a frame indicating that tracking is in progress. The frame display process will be described later with reference to FIGS. 4A to 4C.

In S303, the system control unit 50 determines whether or not the MENU button included in the operation unit 70 has been pressed. If it is determined that the MENU button has been pressed, the process proceeds to S304, otherwise the process proceeds to S305. In S304, the system control unit 50 displays the MENU screen. The MENU screen includes the pupil AF setting items, and when the user selects the pupil AF setting items, the pupil AF setting screen is displayed on the display unit 28. On the pupil AF setting screen, the choices of "on" and "off" are displayed as setting candidates, and the user can select either one to set either on or off of pupil AF. Is possible. When the ON / OFF setting of the pupil AF is changed by the user operation, the changed setting value is stored in the non-volatile memory 56 and set.

In S305, the system control unit 50 determines whether or not a touch operation (position designation operation) for the touch panel 70a is detected. If a touch operation is detected, the process proceeds to S306 ((a) in FIG. 3B), otherwise the process proceeds to S316 (FIG. 3A). In S306 to S315, processing for setting the tracking target is performed based on the on / off of the pupil AF and the designated position on the display screen by the touch operation. The position designation operation by the touch operation detected here is touch-down when the display destination is the rear display panel 28a, and touch-up when the display destination is the electronic viewfinder 28b. When displaying and outputting to the electronic viewfinder 28b, the user generally cannot visually check the touch position on the touch panel. Therefore, by performing the confirmation operation as a touch-up after touchdown, it becomes easier to confirm the target position. Because.

In S306 (in FIG. 3B (a)), the system control unit 50 determines whether or not the pupil AF setting is on (whether the pupil AF is on or the pupil AF is off). If the pupil AF is on, the process proceeds to S307, and if not, the process proceeds to S308. In S307, the system control unit 50 detects the pupil from the live view image, and the designated position when the touch operation is detected in S305 (hereinafter, also referred to as the touch operation position) is detected. Judge whether it is in the position of (whether it is in the pupil area) or not. If the position of the touch operation is within the pupil region, the process proceeds to S308, and if not, the process proceeds to S311. In S307, it is possible to distinguish between the right pupil and the left pupil of the subject in the pupil AF on operation mode.

When the display destination is the rear display panel 28a, the position of the touch operation when the touchdown is performed within the area corresponding to the detected pupil in the live view displayed on the rear display panel 28a. Is determined to be within the detected pupil area. If the display destination is the electronic view finder 28b, the cursor indicating the specified position is moved in the live view displayed on the electronic view finder 28b by the touch move, and the position of the cursor when touched up is the position of the touch operation. And. Therefore, when the cursor is moved to the area corresponding to the detected pupil of the live view displayed on the electronic viewfinder 28b and touched up, the position of the touch operation is detected in the pupil area. Judge that.

In S308, the system control unit 50 determines that the designated position in S307 is within the selection area of the pupil (left eye of the subject) on the right side of the subject on the live view (in the upper right area 703 described later). Judge whether or not. If it is the pupil on the right side, proceed to S309. In other cases, that is, when the designated position determined in S307 is within the selection area (inside the upper left area 702 described later) of the left pupil (right eye of the subject) when viewed from the user with respect to the subject on the live view. The process proceeds to S310.

In S309, the system control unit 50 tracks the right pupil of the face detected at the designated position (the left eye of the detected face) as a tracking target. In S310, the system control unit 50 tracks the left pupil of the face detected at the designated position (the right eye of the detected face) as a tracking target.

On the other hand, when the designated position is not the pupil region in S307, in S311 the system control unit 50 determines whether or not the designated position by the touch operation is the detected face position (inside the face region). If it is the position of the face (in the case of the face areas 701, 701a, and 701b described later), the process proceeds to S312, and if not, the process proceeds to S313. In S312, the system control unit 50 tracks the face detected at the designated position of the touch operation as the tracking target.

The processing in S307, S308, and S311 will be described in detail with reference to FIG. 7. 7 (a) to 7 (e) are schematic views showing a touch reaction region for selecting a face or pupil in the digital camera 100 when the display destination is the rear display panel 28a, and are detected from the live view. It is a figure which extracted and explained only one face part. Actually, the live view image includes an area outside the face, other faces, and the like, but these are omitted in the description of FIG. 7.

FIG. 7A is a diagram showing a touch reaction region in a state where only the face is detected. It is assumed that the pupil is not detected because the eyes are closed. At this time, since only the face can be selected by the user, the face area 701, which is the touch reaction area, is set as a square area (both height and width L) that covers the entire face. As described above, when the face is detected from the image, the system control unit 50 sets the face area corresponding to the face area on the display screen, and associates the detected face with the face area. Therefore, when the user's touch operation is detected on the face area 701 in this state, the face is selected.

When a predetermined organ (pupil in the present embodiment) is detected from the detected face, the system control unit 50 divides the set face region into a plurality of regions and associates each region with the face and the predetermined organ. FIG. 7B shows a state in which the face and both the left and right eyes are detected. At this time, the user can select three, the face, the pupil on the right side of the subject, and the pupil on the left side. Therefore, as shown in FIG. 7B, the face area 701 is divided into three parts, the upper left area 702, which is the upper left touch reaction area, is the left pupil, and the upper right area 703, which is the upper right touch reaction area, is the right pupil. The lower region 704, which is the lower half touch reaction region, is set as the face selection region. In this embodiment, the side where the pupil is located is upward from the mouth and nose with respect to the face.

It should be noted that the division of the face region is performed according to a predetermined rule regardless of the size at the time of detection of the organ. For example, even if the size of the face area or organ area changes depending on the size of the detected face or organ, the face area may maintain the ratio and positional relationship between the areas of the face area and the multiple areas. The division is done. More specifically, in the present embodiment, the height and width of the upper left region 702, the upper right region 703, and the lower region 704 are the face region 701 regardless of the relative size (ratio) of the pupil to the detected face. It is a fixed ratio to the size of, and is set as follows. That is, the heights of the upper left region 702, the upper right region 703, and the lower region 704 are the same as each other (length L / 2, that is, half the height of the face region 701). Further, the widths of the upper left region 702 and the upper right region 703 are the same as each other (length L / 2, that is, half the width of the face region 701). The width of the lower region 704 is twice the width of the upper left region 702 and the upper right region 703 (length L). This divides the face area 701 into four equal parts vertically and horizontally, and the upper left part, the upper right part, and the lower two parts are the left pupil, the right pupil, and the touch reaction region corresponding to the face, respectively. It is a thing. By aligning the height of the reaction area between the face and the pupil, the user can select the face and the pupil with the same operation feeling. In addition, by aligning the widths of the reaction areas of the left and right pupils, even if the face faces diagonally and one pupil is detected small, both pupils can be selected with the same operation feeling.

FIG. 7C shows a state in which only the face and one pupil (here, the left pupil) are detected. At this time, the upper right region 703, which is the upper right touch reaction region in FIG. 7B, is set as the face touch reaction region 705 in combination with the lower region 704, which is the lower half touch reaction region. That is, when the user touches an undetected pupil area, the face is selected.

In pupil AF, there is a use case where the photographer or the subject standing position and orientation are adjusted so that the pupil is detected when the pupil to be focused is not detected. It is annoying because other subjects may be detected as the main subject during the process. Therefore, in the present embodiment, the face is tracked when the non-detected pupil is touched. Such control makes it possible to prevent transfer from the main subject to another subject. This makes it easier for the photographer to move himself or the subject to focus on the desired pupil even if the targeted pupil is not detected.

Although the face area 701 is a square in FIGS. 7 (a) to 7 (c), it may have a different shape such as a rectangle, a rhombus, a circle, or an ellipse depending on the subject to be detected. For example, since the human face is actually close to a rhombus or a circle, the diamond-shaped face region 701a as shown in FIG. 7D and the circular face region 701b as shown in FIG. 7E may be used as the touch reaction region. .. In the case of FIGS. 7 (d) and 7 (e), the upper left touch reaction area 706 and 709 are the left pupil, and the upper right touch reaction area 707 and 710 are the right pupil and the lower half touch reaction region. Certain areas 708,711 may be set as face selection areas. Further, in FIGS. 7 (a) to 7 (c), the width of the upper left region 702 and the upper right region 703, and the heights of the upper left region 702, the upper right region 703, and the lower region 704 are the same length, but either of them has the same length. It may be carried out only on one side. For example, as shown in FIG. 7 (f), the vertical size (height) of the upper left region 702 and the upper right region 703 and the lower region 704 is divided by La and Lb (La ≠ Lb, La + Lb = L). You may try to do it.

When the display destination is the electronic viewfinder 28b, the cursor indicating the designated position when touched up is described with reference to FIG. 7 among the faces in the LV displayed on the electronic viewfinder 28b. Determine in which area the position is. Further, in S307, S308, and S311, when the rear display panel 28a is the display destination, the area is determined based on the touchdown position (touch position when touched down), but the area is not based on the touchdown position. The judgment may be based on the touch-up position.

In S313 ((a) of FIG. 3B), the system control unit 50 determines whether or not the position where the touch operation is performed is the position of the release touch button described later. If it is not the position of the release touch button, the process proceeds to S314, and if it is the position of the release touch button, the process proceeds to S315. In S314, the system control unit 50 executes mono-tracking with the subject at the designated position as the tracking target. That is, even if the same subject moves in the LV image, it continues to follow and is targeted for AF, subject to the color, contrast, shape, and the like of the subject at the designated position in the LV image. In S315, the system control unit 50 releases the tracking state and enters the tracking standby state.

In S316 (FIG. 3A), the system control unit 50 determines whether or not the shutter button 61 is half-pressed and the first shutter switch 62 is turned on (whether or not a shooting preparation instruction is given). If the shutter button 61 is pressed halfway, the process proceeds to S317 ((b) in FIG. 3B), otherwise the process proceeds to S324.

In S317 to S323 ((b) in FIG. 3B), the shooting preparation operation is executed by half-pressing the shutter button 61, and the shooting process is executed according to the detection of the full-pressing of the shutter button 61. First, in S317, the system control unit 50 performs AF processing described later. The AF process will be described later with reference to FIGS. 5A to 5B. In S318, the system control unit 50 performs photometric processing. In S319, the system control unit 50 determines whether or not the shutter button 61 is fully pressed and the second shutter switch 64 is turned on. If the shutter button 61 is fully pressed, the process proceeds to S321, otherwise the process proceeds to S320. In S320, the system control unit 50 determines whether or not the first shutter switch 62 is kept on (half-pressing the shutter button 61). If it is maintained, the process proceeds to S319, otherwise it proceeds to S302.

When it is determined in S319 that the second shutter switch 64 is on (the shutter button 61 is fully pressed), the system control unit 50 performs a shooting process in S321. This is not a shooting for live view shooting, but a main shooting for recording an image as an image file on a recording medium. Exposure is performed under the set exposure conditions (shutter speed, aperture value), a signal is read out from the image pickup unit 22, image processing is performed, and an image for recording on the recording medium 200 is generated. In S322, the system control unit 50 records the image taken in S321 on the recording medium 200. At the same time, the system control unit 50 may perform a quick review (REC review) display for confirming and displaying the captured image before restarting the LV display in S323. In S323, the system control unit 50 restarts the imaging of the LV image and displays the live view image on the display unit 28.

If the on (half-pressed state of the shutter button) of the first shutter switch 62 is not detected in S316 (FIG. 3A), in S324, the system control unit 50 determines whether or not another operation has been performed on the operation unit 70. Judge. If it is determined that another operation has been performed, the process proceeds to S325, and if not, the process proceeds to S326. In S325, the system control unit 50 performs other processing. For example, in response to the operation of the operation unit 70, processing such as changing a shooting parameter such as a shutter speed is performed. On the other hand, in S326, the system control unit 50 determines whether or not an end operation (power off operation, change operation to another operation mode such as a reproduction mode, etc.) has been performed on the operation unit 70. If it is determined that the end operation has been performed, this process ends. If it is determined that the end operation has not been performed, the process returns to S302, and the above operation is repeated.

4A to 4C are flowcharts showing the details of the frame display process of S302 described with reference to FIGS. 3A to 3B. 6A to 6C are schematic views showing screen examples in this embodiment. In the present embodiment, an item (frame) indicating that the subject is detected and an item (frame) indicating that the subject is being tracked are superimposed and displayed on the LV image.

In S401 (FIG. 4A), the system control unit 50 refers to the setting information recorded in the non-volatile memory 56 and determines whether or not the pupil AF setting is on (whether the pupil AF is on or the pupil AF is off). do. If the pupil AF is on, the process proceeds to S408, otherwise (if the pupil AF is off), the process proceeds to S402. In S402, the system control unit 50 determines whether or not the subject is being tracked. If the subject is being tracked, the process proceeds to S403, and if the subject is in the tracking standby state, the process proceeds to S404.

In S404, the system control unit 50 determines whether or not a face or an object has been detected. If a face or an object is detected, the process proceeds to S405, and if not, the process proceeds to S431. FIG. 6A (a) is an example of displaying the LV image 601 in the tracking standby state when the face is not detected. This is a case where the condition for proceeding to S431 is satisfied, and corresponds to the display state before the shutter button 61 is half-pressed. The shooting information display 602 is superimposed and displayed on the LV image 601. In the LV image 601 as an example, the subject 603 (the subject 603a of a person and the subject 603b of a thing (automobile)) is shown, but (a) of FIG. 6A is a display example when these are not detected. .. Actually, this display state occurs when a subject such as a wall or the sky that cannot detect a person or an object is photographed. When it is determined in S404 that a face or an object is detected, in S405, the system control unit 50 determines whether or not the face of a person is detected. If a person's face is detected, the process proceeds to S427 of FIG. 4C described later (a face detection frame 604 as shown in FIG. 6A (b) is displayed), otherwise S429 of FIG. 4C described later. (The object detection frame 610 as shown in FIG. 6B (h) is displayed).

When it is determined in S402 (FIG. 4A) that the subject is not being tracked, in S403, the system control unit 50 determines whether or not the face of the subject is being tracked. If the face is being tracked, the process proceeds to S406, otherwise it proceeds to S407. In S406, the system control unit 50 displays the face tracking frame on the display unit 28, and proceeds to process in S418 of FIG. 4B (continuous AF is performed on the face being tracked). FIG. 6A (c) is a display example when the face of the subject 603a is being tracked. The system control unit 50 displays the face tracking frame 605 so as to surround the face of a person, and displays the tracking release button 606, which is a touch button for canceling the tracking state, at the edge of the screen. On the other hand, in S407 of FIG. 4A, the system control unit 50 displays the object tracking frame on the display unit 28, and advances the processing to S420 of FIG. 4B (continuous AF is performed by the object being tracked). FIG. 6A (d) is a display example when tracking an object. A mono tracking frame 607 is displayed so as to surround the car.

When it is determined in S401 (FIG. 4A) that the pupil AF is not on (pupil AF off), in S408, the system control unit 50 determines whether or not the subject is being tracked. If the subject is being tracked, the process proceeds to S409 (FIG. 4B), otherwise it proceeds to S421 (FIG. 4A). In S409 (FIG. 4B), the system control unit 50 determines whether or not the pupil of the subject is being tracked. If the pupil is being tracked, the process proceeds to S410, otherwise it proceeds to S413.

In S410, the system control unit 50 displays the pupil tracking frame on the display unit 28. If the pupil is detected, the face is also detected. Therefore, in S411, the system control unit 50 superimposes and displays the face detection frame on the position of the detected face on the LV image 601 on the display unit 28. FIG. 6B (e) is a display example of a state in which the pupil is being tracked. In the LV image 601 the pupil tracking frame 608 is displayed so as to surround the pupil. Further, when the pupil is detected, the face is also detected, so that the face detection frame 604 is also displayed on the LV image 601.

In S412, the system control unit 50 sets the position of the pupil during tracking to the target position (AF position) of continuous AF, and performs continuous AF. Here, of the detected right and left eyes of the face, the pupil on the side designated as the tracking target by the user is the target of continuous AF. That is, regardless of whether the eye is closer to the digital camera 100 or the eye is larger in size, the eye designated by the user as the tracking target is the target of AF. By targeting the pupil in this way, the pupil as intended by the user can be targeted for AF. The continuous AF here refers to a function that continuously and automatically focuses on the subject according to the position of the subject during standby when no shooting operation is performed.

When it is determined in S409 that the pupil is not being tracked, in S413, the system control unit 50 determines whether or not the face of the subject is being tracked. If the subject's face is being tracked, the process proceeds to S414, otherwise it proceeds to S419. In S414, the system control unit 50 displays the face tracking frame on the display unit 28. As shown in FIG. 6A (c), the face tracking frame 605 is displayed so as to surround the tracking face on the LV image 601. In S415, the system control unit 50 determines whether or not the pupil of the subject is detected. If the pupil is detected, the process proceeds to S416, otherwise it proceeds to S418.

In S416, the system control unit 50 displays the pupil detection frame on the display unit 28 in addition to the face tracking frame. FIG. 6B (f) is a display example of a state in which the pupil is detected and the face is tracked. In the LV image 601 the pupil detection frame 609 is displayed so as to surround the pupil. Further, since the face is tracked, the face tracking frame 605 displayed in S414 is still displayed. After that, in S417, the system control unit 50 sets the position of the detected pupil to the target position of the continuous AF, and performs the continuous AF. In this way, even if the pupil is not being tracked, if the face is being tracked and the pupil is also detected, AF is performed on the detected pupil. However, the pupil to be AF here is the pupil selected by the system control unit 50 from the right eye and the left eye of the tracking face, which are automatically determined to be the main subject. Basically, the system control unit 50 selects the eye closer to the digital camera 100 (closest eye) or the eye with the larger size as the AF target.

When it is determined in S415 that the pupil is not detected, in S418, the system control unit 50 sets the position of the face being tracked to the target position of continuous AF, and performs continuous AF. Here, since the pupil is not detected, the AF target is the entire face or one point in the center of the face, which is not related to the position of the pupil.

When it is determined that neither the pupil nor the face is being tracked (NO in S409 and S413), in S419, the system control unit 50 displays the mono tracking frame on the display unit 28. A display example of the mono tracking frame is as shown in FIG. 6A (d) described above. In S420, the system control unit 50 sets the position of the object being tracked to the target position of continuous AF, and performs continuous AF.

When it is determined in S408 (FIG. 4A) that the subject being tracked does not exist, in S421, the system control unit 50 determines whether or not any of the human face, pupil, and object is detected. .. If it is determined that such a subject has been detected, the process proceeds to S422 (FIG. 4C), otherwise (that is, if none of the face, person, or object is detected). Proceed to S431 (FIG. 4A).

In S422 (FIG. 4C), the system control unit 50 determines whether or not the pupil of the subject is detected. If the pupil of the subject is detected, the process proceeds to S423, otherwise the process proceeds to S426. In S423, the system control unit 50 superimposes on the live view displayed on the display unit 28, and displays the pupil detection frame at the position of the pupil of the detected face. If the pupil is detected, the face is also detected. Therefore, in S424, the system control unit 50 superimposes on the live view displayed on the display unit 28 and detects the face at the detected face position. Display the frame. FIG. 6B (g) is a display example in a state where the face and the pupil are detected in S424. A face detection frame 604 and a pupil detection frame 609 are displayed so as to surround the face and the pupil, respectively. In S425, the system control unit 50 sets the detected pupil position to the continuous AF position and performs continuous AF, as in S417. Here, the pupil to be AF is the pupil selected by the system control unit 50 from the right eye and the left eye of the tracking face, which are automatically determined to be the main subject.

When it is determined in S422 that the pupil is not detected, in S426, the system control unit 50 determines whether or not the face of the person is detected. If it is determined that the face is detected, the process proceeds to S427, and if not, the process proceeds to S429. In S427, the system control unit 50 superimposes on the live view displayed on the display unit 28 and displays the face detection frame at the position of the detected face. FIG. 6A (b) shows a state in which the face detection frame 604 is displayed. In S428, the system control unit 50 sets the detected face position to the continuous AF position and performs continuous AF.

When the subject being detected is neither a pupil nor a face (NO in S422 and S426), in S429, the system control unit 50 superimposes on the live view displayed on the display unit 28 to the position of the detected object. Display the object detection frame. FIG. 6B (h) is a diagram showing a state in which an object is detected. The object detection frame 610 is displayed so as to surround the car (subject 603b). In S430, the system control unit 50 sets the position of the detected object to the continuous AF position and performs continuous AF.

When the subject is not being tracked or detected (NO in S404 or NO in S421) regardless of the on / off state of the pupil AF, the system control unit 50 sets other positions to the continuous AF position in S431. Then, perform continuous AF.

The face detection frame 604, the pupil detection frame 609, and the mono detection frame 610, which are items indicating the detected subject (items indicating that the subject is in the detection state), have a single frame indicating the detected range. It is used. In this way, the same expression is used for the item indicating that the item is in the detected state, regardless of the subject. Further, in the face tracking frame 605, the mono tracking frame 607, and the pupil tracking frame 608, which are items indicating that the state is designated as the tracking target, a double frame indicating the range of the subject to be tracked is used. There is. As described above, the same expression is used for the item indicating the state designated as the tracking target regardless of the subject, but the expression is different from the item indicating the detected state. The display form of the item indicating the detection state and the tracking state is not limited to the above example (single frame and double frame).

5A to 5B are flowcharts relating to AF processing of the digital camera 100 according to the present embodiment.

In S501 (FIG. 5A), the system control unit 50 refers to the setting information recorded in the non-volatile memory 56 and determines whether or not the pupil AF setting is on. If the pupil AF is on, the process proceeds to S513 (FIG. 5B), otherwise the process proceeds to S502 (FIG. 5A). The processing of S502 to S512 (FIG. 5A) is the AF processing in the pupil AF off. Further, the processing of S513 to S526 (FIG. 5B) is the AF processing in the pupil AF input.

In S502 (FIG. 5A), the system control unit 50 determines whether or not the face of the subject is being tracked. If the face is being tracked, proceed to S503, otherwise proceed to S507. In S503, the system control unit 50 determines whether or not the pupil of the subject is detected. If the pupil is detected, the process proceeds to S504, otherwise the process proceeds to S505. In S504, the system control unit 50 sets the detected pupil position to the AF position and performs AF. Here, the pupil to be AF is the pupil selected by the system control unit 50 from the right eye and the left eye of the tracking face, which are automatically determined to be the main subject. On the other hand, in S505, the system control unit 50 sets the position of the face being tracked to the AF position and performs AF.

In S506, the system control unit 50 displays the AF result frame on the display unit 28 at the position of the pupil subjected to AF in S504 or the position of the face subjected to AF in S505. When AF is executed in response to the half-pressing of the shutter button 61, an item indicating that the shutter button 61 is in the detected state (face, pupil, object detection frame) and an item indicating that the shutter button 61 is in the specified state (face, pupil). , Mono tracking frame) is hidden and the AF result frame is displayed. For example, when AF is completed from the state of (f) in FIG. 6B, the tracking frame and the detection frame are hidden, and the AF result frame is displayed as shown in (j) of FIG. 6C. FIG. 6C (j) is a display example when the face AF result frame 612 is displayed at the position of the face in the LV image 601. The face AF result frame 612 is displayed so as to surround the face. Even when the S506 is reached via S504, the face AF result frame 612 is displayed instead of the pupil AF result frame. This is because it is assumed that the user expects the entire face of the subject to perform AF because the user has turned off the pupil AF setting.

If the AF result frame is displayed for the pupil instead of the face here, the user says that the pupil AF was performed because the setting was not reflected even though the pupil AF setting was turned off. There is a possibility of misunderstanding. Alternatively, the user may feel uncomfortable because the AF result frame is displayed on the pupil even though the user expected the AF result frame to be displayed on the face. When such a misunderstanding or discomfort occurs, the user releases the half-pressed state of the shutter button 61 and presses the shutter button 61 halfway again until the AF result frame is displayed on the face as expected (the user performs AF). (Re-AF) may be repeated. In that case, the user may miss a photo opportunity. In order to prevent such a situation and to meet the user's assumption, in the present embodiment, even when the S506 is reached via S504, the AF result frame is displayed so as to surround the face instead of the pupil. As a result, it is possible to realize the notification of the AF result that is not uncomfortable for the user, and it is possible to smoothly perform the subsequent shooting operation.

When it is determined in S502 that the face is not being tracked, in S507, the system control unit 50 determines whether or not the object is being tracked. If it is determined that the object is being tracked, the process proceeds to S511, otherwise it proceeds to S508. In S508, the system control unit 50 determines whether or not the pupil of the subject is detected. If it is determined that the pupil has been detected, the process proceeds to S504 (performing AF with the automatically selected pupil) described above, and if not, the process proceeds to S509. In S509, the system control unit 50 determines whether or not the face of the subject is detected. If the face of the subject is detected, the process proceeds to S505 (AF for the face being tracked) described above, and if not, the process proceeds to S510.

In S510, the system control unit 50 automatically determines the other positions (because none of the face, pupil, or object is detected, the system control unit 50, which is neither face, pupil, or object) as the main subject. Position) is set to the AF position, and AF is performed. When the object is being tracked (YES in S507), in S511, the system control unit 50 sets the position of the object being tracked to the AF position and performs AF. In S512, the system control unit 50 displays the AF result frame on the display unit 28 for the object or other position set as the AF position. FIG. 6C (l) is a display example when the AF result frame at another position is displayed when the process reaches S512 via S510. The multi-point AF result frame 614 is displayed for the position automatically detected by the camera on the entire surface of the LV image 601. Further, FIG. 6C (k) is a display example when the AF result frame is displayed for the mono position being tracked when the process reaches S512 via S511. The mono AF result frame 613 is displayed so as to surround the object (subject 603b).

Next, the AF operation when the pupil AF is on (YES in S501) will be described. First, in S513 (FIG. 5B), the system control unit 50 determines whether or not the pupil of the subject is being tracked. If it is determined that the pupil is being tracked, the process proceeds to S514, otherwise it proceeds to S516. In S514, the system control unit 50 sets the position of the pupil during tracking to the AF position and performs AF. Then, in S515, the system control unit 50 displays the AF result frame on the display unit 28 with respect to the pupil (the pupil being tracked) of the subject set as the AF position. FIG. 6C (i) is a diagram showing a state in which the AF result frame is displayed in the pupil. The pupil AF result frame 611 is displayed so as to surround the pupil.

When it is determined in S513 that the pupil is not being tracked, in S516, the system control unit 50 determines whether or not the face of the person is being tracked. If it is determined that the face is being tracked, the process proceeds to S517, and if not, the process proceeds to S521. In S517, the system control unit 50 determines whether or not the pupil of the subject is detected. If it is determined that the pupil has been detected, the process proceeds to S518, and if not, the process proceeds to S519. In S518, the system control unit 50 sets the detected pupil position to the AF position and performs AF. Subsequently, the process proceeds to S515, and the pupil AF result frame is displayed at the position where AF is performed. Even when reaching S515 via S518 (when not in pupil tracking), the pupil AF result frame is displayed because the user has set the pupil AF setting, so the user can see the subject. This is because it is expected that AF will be performed with the eyes. By displaying the AF result frame so as to surround the eyes instead of the face, it is possible to realize the notification of the AF result that is not uncomfortable for the user, and it is possible to smoothly perform the subsequent shooting operation.

When the face of the subject is being tracked but the pupil is not detected (NO in S517), in S519, the system control unit 50 sets the position of the face being tracked to the AF position and performs AF. Then, in S520, the system control unit 50 displays the AF result frame on the display unit 28 with respect to the face of the subject (the face being tracked) set as the AF position.

When it is determined by S513 and S516 that neither the pupil nor the face is being tracked, in S521, the system control unit 50 determines whether or not the object is being tracked. If the object is being tracked, proceed to S525, otherwise proceed to S522. In S522, the system control unit 50 determines whether or not the pupil of the subject is detected. If it is determined that the pupil has been detected, the process proceeds to S518 described above, and if not, the process proceeds to S523. In S523, the system control unit 50 determines whether or not the face of the subject is detected. If it is determined that the face has been detected, the process proceeds to S519 described above, and if not, the process proceeds to S524.

In S524, the system control unit 50 sets other positions to the AF position as in S510. On the other hand, when it is determined in S521 that the object is being tracked, in S525, the system control unit 50 sets the position of the object being tracked to the AF position in the same manner as in S511 described above. Then, in S526, the system control unit 50 displays the AF result frame on the display unit 28 for the object or other position set as the AF position, similarly to the above-mentioned S512.

It should be noted that the various AF result frames displayed in S506, S512, S515, S520, and S526 described above are displayed in different display formats depending on whether the focus is achieved or not as a result of AF. .. For example, the AF result frame is displayed in green when it is in focus, and the AF result frame is displayed in gray when it is out of focus because it cannot be focused. Further, the AF result notification sound may be sounded together with the display of various AF result frames. In this case as well, the notification shall be made with different sounds depending on whether the focus is achieved or not as a result of AF. For example, when it is in focus, it is pronounced as "pip", and when it cannot be focused and is out of focus, it is pronounced as "pip".

According to the above-described embodiment, in the pupil AF, the face region is divided into upper and lower equal parts, and the upper half is the pupil / lower half as the face selection region, so that both the face and the pupil can be easily selected. A highly convenient image pickup device is provided. In addition, it is possible to display an appropriate AF result frame according to the on / off setting of the pupil AF, so that the user does not feel uncomfortable and it is possible to concentrate on shooting.

In the above embodiment, the face, pupil, and object detection frame / tracking frame have the same shape, but the shape is not limited to this, and frames having different shapes are displayed depending on the type of subject. May be good. When a frame with the same shape is used, the UI selects all of the face, eyes, and objects as equivalent objects, and the user can easily switch the subject and enjoy shooting. Further, when frames having different shapes depending on the type of subject are used, it becomes easy to clearly understand which target is detected / tracked. Of course, frames having different shapes may be used on the left and right sides of the pupil.

Although the electronic viewfinder has been described as an example so far, an optical viewfinder may also be used. Further, the frame representation described so far may be variable in size depending on the situation, and may not be composed of one frame. For example, when a plurality of AF points are displayed on the display unit in advance, such as in an optical viewfinder, when indicating the touch position, the colors of the plurality of AF points are changed according to the position and area of the touched finger. The touch position may be indicated by such means.

It should be noted that the above-mentioned various controls described as those performed by the system control unit 50 may be performed by one hardware, or a plurality of hardware (for example, a plurality of processors and circuits) share the processing to be a device. The whole control may be performed.

Further, although the present invention has been described in detail based on the preferred embodiment thereof, the present invention is not limited to these specific embodiments, and various embodiments within the range not deviating from the gist of the present invention are also included in the present invention. included. Further, each of the above-described embodiments is merely an embodiment of the present invention, and each embodiment can be appropriately combined.

Further, in the above-described embodiment, the case where the present invention is applied to the image pickup apparatus has been described as an example, but the present invention is not limited to this example. For example, the display of the tracking frame and the detection frame can be applied to any electronic device having a function of displaying a captured image and a function of designating a position on the image. For example, a touch operation when playing back a captured image to detect the face, eyes, mouth, and nose and selecting the eyes for red-eye correction, or for another reason (for example, entering a person's name). Applicable to. Other processes for selecting a face or organ include the following. Select the entire face or eyes to apply a mosaic or mask, select the eyes to enhance the eyes (such as enlarging), select the nose to enhance the image, select the entire face or cheeks, etc. Select and apply image processing for skin-beautifying effect. Select the mouth and apply image processing such as emphasizing the redness of the lips and adding gloss. As a method for selecting facial organs in such cases, the above-mentioned processing can be applied to a recorded image instead of the LV image. Further, the display of the AF frame can also be applied to various electronic devices having an image pickup function. That is, the present invention can be applied to a personal computer, a PDA, a mobile phone terminal, a portable image viewer, a printer device equipped with a display, a digital photo frame, a music player, a game machine, an electronic book reader, a sticker printing system, and the like. ..

Further, in the above embodiment, the area on the display screen is specified by the touch operation on the touch panel, but the present invention is not limited to this. For example, the area on the screen (face area or pupil area) is specified by operating the mouse pointer with the mouse. You may specify it. Further, in the above embodiment, the pupil is exemplified as the organ to be tracked, but the present invention is not limited to this, and for example, other organs such as the nose and mouth may be designated as the tracking target (AF target). For example, when the mouth is the tracking target (AF target), the face region may be divided in half in the vertical direction, the lower region may be associated with the mouth, and the upper region may be associated with the face.

Further, the present invention can be applied not only to the image pickup device main body but also to a control device that communicates with an image pickup device (including a network camera) via wired or wireless communication and remotely controls the image pickup device. As a device for remotely controlling the image pickup device, for example, there is a device such as a smartphone, a tablet PC, or a desktop PC. The image pickup device can be controlled remotely by notifying the image pickup device of commands to perform various operations and settings from the control device side based on the operations performed on the control device side and the processing performed on the control device side. be. Further, the live view image taken by the image pickup device may be received via wired or wireless communication and displayed on the control device side. Further, the live view image taken by the image pickup device may be received via wired or wireless communication and displayed on the control device side. In that case, such a control device will execute AF control for controlling the AF mechanism of the image pickup device. That is, the AF process described above with reference to FIGS. 5A and 5B may be AF control for controlling the AF mechanism of an external device such as an image pickup device.

(Other embodiments)
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

28: Display panel, 28a rear display panel, 28b: Electronic viewfinder, 50: System control unit, 70: Operation unit, 70a: Touch panel, 100: Digital camera, 200: Recording medium

Claims (15)

A detection means that detects the subject's face and facial organs from within the image,
A designated means for designating an area in an image according to user instructions,
AF control means that controls AF target corresponding to the area designated by the designated means so as to perform AF, and
The region corresponding to the face of the subject in the detected image is defined as a partial region, the region including the detected facial organs is defined as a specific region, and the partial region or the specific region included in the partial region is included. A setting means for setting one of a plurality of operation modes including a first operation mode in which either the face of the subject or the organ can be designated as an AF target according to the area designated by the designating means. ,
A display control means for superimposing the display of the AF target on the image and displaying it on the display unit is provided.
When the first operation mode is set, in the AF control means, even if the partial area is designated by the designated means, the organ included in the face corresponding to the partial area is detected by the detection means. An electronic device characterized in that, when detected, the detected organ is used as an AF target, and the display control means displays the AF target on the detected organ. When the second operation mode for designating the face of the subject as the AF target is set, the AF control means is the AF control means regardless of whether the designated means designates the partial area or the specific area. When an organ about the face of the subject designated by the designated means is detected by the detecting means, the detected organ is used as an AF target, and the display control means is designated to display the AF target. The electronic device according to claim 1, wherein the operation is performed on the face of a subject. The electronic device according to claim 1 or 2, wherein the detection means detects the pupil as the organ. The detection means can detect the right pupil and the left pupil of the subject separately, and the designation means has the first identification corresponding to the right pupil of the subject in the first operation mode. When an instruction is given to an area, the right pupil of the subject is specified, and when an instruction is given to a second specific area corresponding to the left pupil of the subject, the left pupil of the subject is specified separately. The electronic device according to claim 3, characterized in that it is possible. Claims 1 to 4, wherein the display control means causes the display unit to display an item indicating that the state is detected by the detection means on the detected face or organ of the image. The electronic device according to any one of the above. The electronic device according to claim 5, wherein the item indicating the detected state has the same expression for a face or an organ. 5. Electronic devices listed in. The item indicating the specified state is the same expression for the face or organ, and is
The electronic device according to claim 7, wherein the expression of the item indicating that the detected state and the item indicating the specified state are different. The electronic according to claim 7 or 8, wherein the expression of the AF target is a different expression from the item indicating the detected state and the item indicating the specified state. machine. When AF is executed by the control of the AF control means, the display control means hides the item indicating the detected state and the item indicating the specified state. The electronic device according to any one of claims 7 to 9, wherein the electronic device is characterized. One of claims 1 to 10, wherein the designating means includes a touch panel and designates the partial area or the specific area based on the designated position indicated by the touch operation detected by the touch panel. The electronic device described in the section. The display unit has a first display unit and a second display unit different from the first display unit.
While the image is displayed on the first display unit, the designating means determines a designated position on the display screen of the first display unit by a first touch operation.
While the image is displayed on the second display unit, the designated position on the display screen of the second display unit is determined by a second touch operation different from the first touch operation. The electronic device according to claim 11. A detection step that detects the subject's face and facial organs from within the image,
A specification step that specifies an area in the image according to user instructions,
An AF control step that controls AF targets corresponding to the area designated by the designated step, and an AF control step that controls AF.
The region corresponding to the face of the subject in the detected image is defined as a partial region, the region including the detected facial organs is defined as a specific region, and the partial region or the specific region included in the partial region is included. A setting step for setting one of a plurality of operation modes including a first operation mode in which either the face of the subject or the organ can be designated as an AF target according to the area designated by the designation step. ,
A display control step for superimposing the display of the AF target on the image and displaying it on the display unit is provided.
When the first operation mode is set, in the AF control step, even if the partial area is designated by the designated step, the organ included in the face corresponding to the partial area is detected by the detection step. An electronic device characterized in that, when detected, the detected organ is used as an AF target, and the display control step displays the AF target on the detected organ. A program for making a computer function as each means of the electronic device according to any one of claims 1 to 12. A computer-readable storage medium containing a program for causing the computer to function as each means of the electronic device according to any one of claims 1 to 12.

Images