JP7342883B2 - Imaging control device, imaging device, imaging control method - Google Patents

Description

The present technology relates to an imaging control device, an imaging device, and an imaging control method, and particularly relates to a technique suitable for adjusting and controlling imaging control parameters such as autofocus control.

Some imaging devices perform adjustment control on imaging control parameters, such as autofocus control and automatic exposure control (AE). In particular, with regard to autofocus control, in recent years there are some systems that use captured image analysis and subject recognition techniques to track a target subject and perform autofocus on that subject. Regarding autofocus control, there is also known a technique of detecting the face or eyes of a subject and performing autofocus on the detected face or eyes.
Note that Patent Document 1 listed below describes techniques related to autofocus operation and tracking operation.

Japanese Patent Application Publication No. 2015-111746

Here, when performing autofocus based on the subject's face or eyes as described above, the user of the imaging device is asked to select the target subject's face or eyes from the image displayed on the display unit by touch operation. Alternatively, you can select the face or eyes by moving the cursor using the cross key, or change the direction of the imaging device to move the AF (autofocus) frame displayed on the display to the face or eye. It is conceivable to have the user perform an operation such as positioning it in the pupil area.

However, when the above-mentioned touch operation is adopted, the user cannot select the face or eyes while looking through the viewfinder. Furthermore, no matter which method is used, including the above-mentioned cursor and AF area movement operations, it is difficult to accurately select the position of the face and eyes of a moving subject, such as in a sports scene. be.

The present technology has been developed in view of the above circumstances, and aims to facilitate operations for realizing adjustment of imaging control parameters based on the position of a subject's face and eyes.

An imaging control device according to the present technology includes a face/pupil determination unit that determines the face or eyes of a selected subject in response to selection of a subject in a captured image by an operation;
and an adjustment control section that performs adjustment control of imaging control parameters based on the position information of the face or eyes determined by the face/eye determination section.

The imaging control parameters refer to various control parameters related to imaging using an imaging device, such as focus, exposure, and white balance. According to the above configuration, as long as an arbitrary part of the subject, such as the torso of the subject, is selected, the face or eyes of the subject are determined, and the imaging control is performed based on the determined position of the face or eyes. Parameter adjustment control is performed. That is, when adjusting imaging control parameters such as autofocus based on the subject's face or eyes, there is no need for the user to directly select the subject's face or eyes.

In the imaging control device according to the present technology described above, the face/eye determination unit detects a face or eyes present in the captured image, and determines whether the detected face or eyes correspond to the face or eyes of the subject. It is conceivable to determine whether or not.

As a method of determining the face or eyes of the selected subject, a method of performing image analysis to determine the human body part of the selected subject may be considered, but this type of image analysis processing has a relatively high calculation cost. According to the above configuration, it is possible to determine the face or eyes of the subject without performing such image analysis processing that requires high computational cost.

In the imaging control device according to the present technology described above, the face/eye determination unit may determine the face or eyes of the subject based on the positional relationship between the detected face or eyes and the subject.

This makes it possible to prevent faces or eyes detected in a captured image that are located far apart from each other from being determined as faces or eyes of the subject.

In the above-described imaging control device according to the present technology, the face/eye determination unit detects the subject on the condition that the subject is located within a geometrically set search range from the position of the detected face or pupil. It is conceivable to determine that the detected face or eyes are the face or eyes of the subject.

Thereby, in determining the face or eyes of a subject, it is sufficient to perform at least a range setting process and a process of determining whether the selected subject is included within the set range for each detected face or eye.

In the imaging control device according to the present technology described above, the face/eye determination unit may determine the face or eyes of the subject based on the positional relationship in the depth direction between the detected face or eyes and the subject. Conceivable.

This makes it possible to determine the face or eyes of the subject based not only on the positional relationship in the image plane direction but also on the positional relationship in the depth direction.

In the imaging control device according to the present technology described above, it is possible that the face/eye determination unit determines the face or eyes of the subject based on consistency of movement between the detected face or eyes and the subject. It will be done.

As a result, in response to cases where the face or eyes of the subject cannot be appropriately determined based on positional relationships alone, the face or eyes of the subject can be appropriately determined from among the detected faces or eyes based on consistency of movement. is possible.

In the imaging control device according to the present technology described above, the face/eye determination unit determines the face or eyes of the subject based on the consistency of movement in the depth direction between the detected face or eyes and the subject. It is possible that

This makes it possible to determine the face or eyes of the subject based not only on the consistency of movement in the plane direction of the image but also on the consistency of movement in the depth direction.

In the imaging control device according to the present technology described above, the adjustment control unit performs autofocus control as adjustment control of the imaging control parameter, and the face/pupil determination unit is configured to It is conceivable to use distance measurement information for autofocus as the depth information.

This eliminates the need to provide a separate distance measurement sensor other than the distance measurement sensor for autofocus control in the imaging device when determining the face or eyes of the subject based on the positional relationship in the depth direction and consistency of movement.

The imaging control device according to the present technology described above includes a tracking unit that tracks a target subject, and when the face/eye determination unit determines the face or eyes of the subject, the tracking unit The tracking target subject, which is the subject to be photographed, is updated to the determined face or eyes, and in subsequent determinations, based on the position tracked by the tracking unit, the detected face or eyes are updated to the face or eyes of the subject. Alternatively, it is conceivable to determine whether the object is a pupil or not.

This makes it possible to track the subject's head even if the subject's face or eyes are no longer detected because the subject's face or eyes are turned backwards or the like after the subject's face or eyes have been determined.

In the imaging control device according to the present technology described above, after determining the face or eyes of the subject from among the detected face or eyes, the face/eye determination unit determines that the face or eyes are the face or eyes of the subject. It is conceivable to exclude all faces or eyes that do not appear from the determination target.

This helps prevent a face or eyes other than the subject's face or eyes from being misjudged when the face or eyes of the subject are lost after the face or eyes of the subject have been determined even once. It will be done.

In the imaging control device according to the present technology described above, after determining the face or eyes of the subject from among the detected faces or eyes, the face/eye determination unit changes the search range from the first search range. It is conceivable to make a determination by switching to a second search range different from the first search range.

This makes it possible to set the search range to a second search range, which is narrower than the first search range, in the judgment performed when the face or eyes of the subject are lost after the face or eyes of the subject have been determined at least once. Thus, it is possible to prevent faces or eyes other than those of the subject from being erroneously determined.

The imaging control device according to the present technology described above includes a tracking unit that tracks a target subject, and the tracking unit determines whether or not the face/eye determination unit has determined the face or eyes of the subject. Regardless, the subject selected by the operation is continued to be tracked, and the face/eye determination unit determines whether the detected face or pupil is the face or pupil of the subject based on the position tracked by the tracking unit. It is conceivable to determine whether or not.

As a result, even if a subject whose face or eyes are not detected because the face is turned backwards etc. is selected and the face or eyes of a subject other than the selected subject is incorrectly determined, subsequent judgments will be made. Since the judgment is performed based on the selected subject position, when the face or eyes of the selected subject are detected, such as when the face of the selected subject is facing forward, the face or eyes of the selected subject are detected. It is possible to judge correctly.

In the imaging control device according to the present technology described above, in response to the selection of a subject by the operation, first visual information is displayed at the position of the subject on the display unit that displays the captured image, and the first visual information is displayed at the position of the subject. In response to the face or eyes of the subject being determined by the eye determination unit, displaying second visual information at the position of the determined face or eyes on the display unit, and hiding the first visual information. It is conceivable to include a display control unit that allows

This makes it possible to intuitively and easily notify the user that tracking of the selected subject has been switched to tracking of the face or eyes.

In the imaging control device according to the present technology described above, the display control unit may display information having a different shape from the first visual information as the second visual information.

This makes it possible to intuitively and more easily notify the user that tracking of the selected subject has been switched to tracking of the face or eyes.

The imaging control device according to the present technology described above includes a tracking unit that tracks a target subject, and when the face/eye determination unit determines the face or eyes of the subject, the tracking unit The tracking target subject, which is the target subject, is updated to the determined face or eyes, and if the determined face or eyes are no longer detected in the captured image, the tracking target subject is updated to the previously determined The adjustment control unit performs adjustment control of the imaging control parameter based on information on the position tracked by the tracking unit, and the display control unit controls the image capture control parameter based on the information of the position tracked by the tracking unit, In response to the fact that the face or eyes cannot be detected, control may be performed to display visual information different from the second visual information at the position tracked by the tracking unit.

According to the above configuration, after the face or eyes of the selected subject is determined, until the face or eyes of the selected subject becomes undetected, for example, when the face of the subject turns backward, the face or eyes of the selected subject are determined. The imaging control parameters are adjusted based on the position of . According to the above configuration, when the face or eyes of the selected subject is not detected, the face (head) position of the selected subject continues to be tracked, and the position of the face (head) of the selected subject continues to be tracked. , visual information different from the second visual information is displayed (for example, the display is switched from the face/eye tracking frame 92 to the tracking frame 91). Due to such display control, the face or eyes of the selected subject were no longer detected, making it impossible to adjust the imaging control parameters based on the position information of the face or eyes. It is possible to intuitively and easily notify the user that tracking of the location) is continuing.

In the imaging control device according to the present technology described above, the subject may be a person.

This eliminates the need for the user to directly select a person's face or eyes when adjusting the imaging control parameters based on the person's face or eyes.

In the imaging control device according to the present technology described above, the subject may be an animal.

This eliminates the need for the user to directly select the face or eyes of an animal, such as a dog, cat, or bird, in order to adjust the imaging control parameters based on the face or eyes of the animal.

The imaging device according to the present technology also includes an imaging unit that obtains a captured image, and a face/pupil that determines the face or eyes of the selected subject in response to selection of a subject in the captured image by operation. The image forming apparatus includes a determination section, and an adjustment control section that performs adjustment control of imaging control parameters based on position information of the face or eyes determined by the face/pupil determination section.

Furthermore, the imaging control method according to the present technology determines the face or eyes of the selected subject in response to the selection of a subject in the captured image by an operation, and uses the position information of the determined face or pupils to determine the face or eyes of the selected subject. This is an imaging control method that performs adjustment control of imaging control parameters based on the above.

These imaging devices and imaging control methods can also provide the same effect as the imaging control device according to the present technology described above.

FIG. 1 is a perspective view of the exterior of an imaging device as an embodiment of the present technology. FIG. 2 is a rear view of an imaging device as an embodiment. FIG. 2 is a side view of a lens barrel that can be attached to an imaging device as an embodiment. FIG. 1 is a block diagram of the internal configuration of an imaging device as an embodiment. FIG. 3 is an explanatory diagram of the functional configuration of a camera control unit in the embodiment. FIG. 3 is an explanatory diagram of a display example during autofocus in the embodiment. FIG. 6 is an explanatory diagram of a display example during tracking in the embodiment. It is an explanatory diagram of operation of an embodiment. Similarly, it is an explanatory diagram of the operation of the embodiment. Similarly, it is an explanatory diagram of the operation of the embodiment. Similarly, it is an explanatory diagram of the operation of the embodiment. Similarly, it is an explanatory diagram of the operation of the embodiment. 3 is a flowchart illustrating main processing for realizing autofocus control on a subject's face or eyes. 7 is a flowchart illustrating an example of subject face determination processing. 7 is a flowchart showing a first example of subject face candidate determination processing. FIG. 3 is a diagram illustrating a relationship between a face detection frame and a face association range frame. 12 is a flowchart showing a second example of subject face candidate determination processing. FIG. 3 is a diagram illustrating an example of performing autofocus on a moving subject. FIG. 2 is a diagram showing an example of a scene to be imaged. FIG. 7 is a diagram illustrating a scene to be addressed in a modified example. It is a flowchart of subject face determination processing as a first modification. It is a diagram illustrating a normal face association range frame. It is a figure which illustrated the face association range frame after a change. FIG. 7 is a diagram illustrating a scene to be dealt with in a third modification. Similarly, it is a diagram illustrating a scene to be dealt with in a third modification. It is a flow chart which illustrated main processing as a third modification.

Hereinafter, embodiments will be described in the following order.

<1. Configuration of imaging device>
<2. Operation explanation of embodiment>
<3. Processing procedure＞
<4. Modified example>
<5. Other variations>
<6. Summary of embodiments>
<7. This technology>

Note that in the following description, the term "autofocus" is also written as "AF".
Moreover, autofocus control will be given as an example of adjustment control of imaging control parameters. Imaging control parameters refer to various control parameters related to imaging using an imaging device.

<1. Configuration of imaging device>

FIG. 1 shows a front perspective view of an imaging device 1 as an embodiment of the present technology, and FIG. 2 shows a rear view. Here, the imaging device 1 is assumed to be a so-called digital still camera, and can perform both still image imaging and video imaging by switching the imaging mode.
Note that in this embodiment, the imaging device 1 is not limited to a digital still camera, but may be a video camera that is mainly used for capturing moving images and can also capture still images.

In the imaging device 1, a lens barrel 2 (lens barrel 2A in FIG. 1) is disposed on the front side of a main body housing 100 constituting a camera body, or is detachable.
On the back side (user side) of the imaging device 1, a display panel 101 is provided, for example, a display device such as a liquid crystal display (LCD) or an organic EL (electro-luminescence) display.
Further, as the viewfinder 102, a display section formed using an LCD, an organic EL display, or the like is also provided.
The user can view images and various information using the display panel 101 and viewfinder 102.
In this example, the imaging device 1 is provided with both the display panel 101 and the viewfinder 102, but the present invention is not limited to this, and a configuration in which only one of the display panel 101 and the viewfinder 102 is provided, Both or one of the display panel 101 and the viewfinder 102 may be configured to be detachable.

Various operators 110 are provided on the main body casing 100 of the imaging device 1.
For example, various types of operators 110, such as keys, dials, and combination press/rotate operators, are provided to realize various operating functions. For example, menu operations, playback operations, mode selection operations, focus operations, zoom operations, and parameter selection operations such as shutter speed and F-number are possible. Although a detailed description of each operator 110 will be avoided, in this embodiment, among the operators 110, the shutter button 110S and the assignable button 110C are particularly shown.
The shutter button 110S is used for shutter operation (release operation) and AF operation by pressing halfway.
The assignable button 110C is an operator also called a custom button, and is a button to which the user can assign an arbitrary operation function.

The shutter button 110S is arranged on the upper right side of the main body housing 100, and can be pressed with the index finger of the right hand, for example, when the user is holding the grip part 103 with the right hand.
Further, the assignable button 110C is arranged, for example, at the upper part of the back side of the main body housing 100 as shown in FIG. 2, and can be pressed and operated by the user with the thumb of the right hand.
In other words, the shutter button 110S and the assignable button 110C are arranged at positions where they can be operated with the index finger and thumb when the user holds the main body casing 100 while gripping the grip portion 112 with the right hand. can be pressed individually or simultaneously.

Note that the shutter button 110S and the button for tracking operation are not necessarily limited to being able to be operated simultaneously with one hand. For example, the shutter button 110S may be pressed with the right hand and the tracking operation may be performed with the left hand.
Therefore, for example, a button on the lens barrel 2 may be used to perform the tracking operation.
For example, FIG. 3 shows an example of a lens barrel 2B that can be attached to the front side of the main body housing 100. Various operators 120 are also provided on the lens barrel 2B.
In this case, the assignable button 120C provided at a slightly lower position on the side surface is assigned to the tracking operation button.

Normally, the user grips the grip portion 103 with his right hand and uses his left hand to support the lens barrel 2 from below. In this case, the assignable button 120C can be pressed with the thumb of the left hand.
Even with this arrangement, the shutter button 110S and the assignable button 120C can be operated simultaneously or separately as desired.

FIG. 4 shows the internal configuration of the imaging device 1 including the lens barrel 2. As shown in FIG. Note that although an example will be described in which the imaging device 1 is configured to be divided into a main body casing 100 and a lens barrel 2, the portion corresponding to the lens barrel 2 is integrated with the main body casing 100. Good too.

The imaging device 1 includes a main body housing 100, an image sensor 12, a camera signal processing section 13, a recording section 14, a display section 15, an output section 16, an operation section 17, a power supply section 18, a camera control section 30, and a memory. It has a section 31.
Further, the lens barrel 2 includes a lens system 21, a driver section 22, a lens control section 23, and an operation section 24.

The lens system 21 in the lens barrel 2 includes lenses such as a cover lens, a zoom lens, and a focus lens, and an aperture (iris) mechanism. This lens system 21 guides light (incident light) from the subject and focuses it on the image sensor 12 in the image capture device 1 .

The image sensor 12 is configured, for example, as a CCD (Charge Coupled Device) type, a CMOS (Complementary Metal Oxide Semiconductor) type, or the like.
The image sensor 12 performs, for example, CDS (Correlated Double Sampling) processing, AGC (Automatic Gain Control) processing, etc. on the electrical signal obtained by photoelectrically converting the received light, and further performs A/D (Analog/Digital) processing. Perform the conversion process. The imaging signal as digital data is then output to the camera signal processing section 13 and camera control section 30 at the subsequent stage.

The camera signal processing unit 13 is configured as an image processing processor using, for example, a DSP (Digital Signal Processor). This camera signal processing section 13 performs various signal processing on the digital signal (captured image signal) from the image sensor 12. For example, the camera signal processing unit 13 performs preprocessing, synchronization processing, YC generation processing, resolution conversion processing, codec processing, and the like.

In the preprocessing, clamp processing for clamping the R, G, and B black levels to predetermined levels and correction processing between the R, G, and B color channels are performed on the captured image signal from the image sensor 12. .
In the synchronization process, a demosaic process is performed so that the image data for each pixel includes all R, G, and B color components.
In the YC generation process, a luminance (Y) signal and a color (C) signal are generated (separated) from R, G, and B image data.
In the resolution conversion process, the resolution conversion process is performed on image data that has been subjected to various types of signal processing.
In the codec processing, the resolution-converted image data is encoded for recording or communication, for example.

The recording unit 14 is made of, for example, a nonvolatile memory, and stores image files (content files) such as still image data and video data, attribute information of image files, thumbnail images, and the like.
The image file is stored in a format such as JPEG (Joint Photographic Experts Group), TIFF (Tagged Image File Format), or GIF (Graphics Interchange Format).
The actual form of the recording section 14 can be considered in various ways. For example, the recording unit 14 may be a flash memory built into the imaging device 1, or a memory card (for example, a portable flash memory) that can be attached to and removed from the imaging device 1, and a card recording/reproducing device that performs recording/reproduction access to the memory card. It may also be in the form of a section. Further, it may be implemented as a built-in form in the imaging device 1, such as an HDD (Hard Disk Drive).

The display unit 15 is a display unit that displays various displays to the photographer, and specifically indicates the display panel 101 and viewfinder 102 shown in FIG. 2.
The display unit 15 executes various displays on the display screen based on instructions from the camera control unit 30. For example, the display unit 15 displays a reproduced image of the image data read from the recording medium in the recording unit 14. Further, the display unit 15 is supplied with image data of a captured image whose resolution has been converted for display by the camera signal processing unit 13. The display unit 15 displays a so-called through image (monitoring image of a subject), which is a captured image during release standby, by displaying based on the image data of the captured image in accordance with instructions from the camera control unit 30. let
Further, the display unit 15 displays various operation menus, icons, messages, etc., ie, a GUI (Graphical User Interface) on the screen based on instructions from the camera control unit 30.

The output unit 16 performs wired or wireless data communication and network communication with external devices.
For example, captured image data (still image files and video files) is transmitted and output to an external display device, recording device, playback device, etc.
Further, the output unit 16 is a network communication unit, and performs communication via various networks such as the Internet, a home network, and a LAN (Local Area Network), and sends and receives various data to and from servers, terminals, etc. on the network. You can do it like this.

The operation unit 17 collectively represents input devices through which the user performs various operation inputs. Specifically, the operation unit 17 shows various operators 110 (including a shutter button 110S and an assignable button 110C) provided on the main body housing 100.
The operation unit 17 detects a user's operation, and a signal corresponding to the input operation is sent to the camera control unit 30.
As this operation unit 17, not only the operator 110 but also a touch panel may be used. For example, a touch panel may be formed on the display panel 101, and various operations may be made possible by touch panel operations using icons, menus, etc. displayed on the display panel 101.
Alternatively, the operation unit 17 may detect a user's tap operation using a touch pad or the like.
Furthermore, the operating section 17 may be configured as a receiving section for an external operating device such as a separate remote controller.

The power supply section 18 generates a power supply voltage Vcc necessary for each section from, for example, a battery loaded inside, and supplies it as an operating voltage.
When the lens barrel 2 is attached to the imaging device 1, the power supply voltage Vcc from the power supply unit 18 is also supplied to the circuit inside the lens barrel 2.
Note that the power supply unit 18 may be formed with a circuit that charges the battery or a circuit that generates the power supply voltage Vcc using a DC voltage that is converted and inputted by an AC adapter connected to a commercial AC power source as a power source. .

The camera control section 30 is constituted by a microcomputer (arithmetic processing unit) equipped with a CPU (Central Processing Unit).
The memory unit 31 stores information and the like used by the camera control unit 30 for processing. The illustrated memory section 31 includes, for example, a ROM (Read Only Memory), a RAM (Random Access Memory), and a flash memory.
The memory section 31 may be a memory area built into a microcomputer chip as the camera control section 30, or may be constituted by a separate memory chip.
The camera control unit 30 controls the entire imaging device 1 and lens barrel 2 by executing programs stored in the ROM, flash memory, etc. of the memory unit 31.
For example, the camera control unit 30 controls the shutter speed of the image sensor 12, instructs various signal processing in the camera signal processing unit 13, performs imaging and recording operations according to user operations, plays back recorded image files, and controls the lens mirror. It controls the operations of the necessary parts regarding the operations of the lens system 21 such as zoom, focus, and aperture adjustment in the barrel 2, and the operations of the user interface. Regarding aperture adjustment, the camera control unit 30 performs variable control of the F number in accordance with user operations and instructs the F number as automatic control (auto iris).

The RAM in the memory unit 31 is used to temporarily store data, programs, etc. as a work area when the CPU of the camera control unit 30 processes various data.
The ROM and flash memory (non-volatile memory) in the memory unit 31 contain an OS (Operating System) for the CPU to control each part, content files such as image files, application programs for various operations, and firmware. It is used to memorize things such as.

When the lens barrel 2 is attached to the main body housing 100, the camera control section 30 communicates with the lens control section 23 and issues various instructions.
The lens barrel 2 is equipped with a lens control section 23 made up of, for example, a microcomputer, and is capable of various data communications with the camera control section 30 . In the case of this embodiment, the camera control section 30 instructs the lens control section 23 to drive a zoom lens, a focus lens, an aperture mechanism, and the like. The lens control section 23 controls the driver section 22 according to these drive instructions, and causes the lens system 21 to operate.
Note that when the lens barrel 2 is attached to the main body housing 100, wired communication is performed between the camera control section 30 and the lens control section 23. However, the configuration may be such that the camera control section 30 and the lens control section 23 can communicate wirelessly.

The driver section 22 is provided with, for example, a motor driver for a zoom lens drive motor, a motor driver for a focus lens drive motor, a motor driver for an aperture mechanism motor, and the like.
These motor drivers apply drive currents to the corresponding drivers in response to instructions from the lens control unit 23 to move the focus lens and zoom lens, open and close the aperture blades of the aperture mechanism, and so on.

The operation unit 24 shows an operator 120 (including an assignable button 120C) provided on the lens barrel 2 side. Operation information from the operation unit 24 is supplied to the lens control unit 23 and notified to the camera control unit 30 via the lens control unit 23.
This allows the lens control unit 23 to control, for example, the operation of the lens system 21 in response to the operation of the operator 120, and the camera control unit 30 to perform various settings and operation control in response to the operation of the operator 120. .
Note that, like the operation section 17, the operation section 24 is not limited to keys, dials, etc., and may be configured as a touch panel, a touch pad, a receiving section of a remote controller, or the like.

FIG. 5 extracts a part of the configuration of FIG. 4 and shows the functions of the camera control unit 30 for performing the operations of this embodiment.
As functions of the camera control section 30, an AF distance measuring section 51, an AF control section 52, an image recognition section 53, an operation recognition section 54, a display control section 55, and an assignable button setting section 56 are provided.
As an example for explanation, it is assumed that these functions are provided in the camera control section 30 by software, but some or all of these functions may be provided in a DSP or a microcomputer used as the camera signal processing section 13, etc. Alternatively, it may be provided in another part, such as a microcomputer on a separate chip from the camera control section 30 or the camera signal processing section 13. In any case, a configuration for performing these functions may be provided in the imaging device 1.

The AF distance measuring section 51 performs distance measuring processing for autofocus. For example, a captured image signal obtained by the image sensor 12 is used to perform calculations for AF control using a so-called phase difference detection method, a contrast detection method, or a hybrid AF method having characteristics of both.
The AF control section 52 generates an AF control signal based on the control value obtained by the AF distance measuring section 51, and performs processing to execute an AF operation. The AF control signal from the AF control section 52 is transmitted to the lens control section 23. Then, the lens control section 23 controls the driver section 22 based on the AF control signal to drive the focus lens within the lens system 21, thereby performing an AF operation.

The image recognition unit 53 performs image analysis on the captured image signal obtained by the image sensor 12 and performs image content recognition processing. Particularly in the case of this embodiment, recognition processing is performed by a face/eye detection section 61, a tracking processing section 62, and a subject face determination section 63.
The face/eye detection unit 61 performs processing to detect the face and eyes of a person (or animal) as a subject within a frame of a captured image signal.
Here, in this embodiment, the subject whose face and eyes are to be detected by the face/eye detection unit is not limited to a living body such as a person or an animal that has a face and eyes, but also a face or eyes of a robot, etc. It may also include objects other than living bodies, including objects that have been exposed.

The tracking processing unit 62 performs processing for determining and tracking the position of a target object (for example, it may be a part of a person, animal, etc.) within the frame of the captured image signal. Note that the subject to be tracked is, for example, a subject subjected to AF. The selection of a tracking target object by the user (imager) may be performed by an operation of selecting an AF target object.

The subject face determination unit 63 determines the face of the subject. This may include determining the area of a person's face within a frame, or further processing to identify an individual face. For example, when the subject is a large number of people, it is assumed that the face of the targeted individual will also be recognized.
In particular, in this example, the subject face determination section 63 performs a process of determining the face and eyes of the subject selected by the user from among the faces and eyes detected within the frame of the captured image by the face/eye detection section 61. We will also do this, but this will be explained later.

The operation recognition unit 54 recognizes the operation performed by the operation unit 17. The camera control unit 30 performs necessary control processing according to the user operation recognized by the operation recognition unit 54.
Particularly related to the processing of the embodiment, the operation recognition unit 54 recognizes operations of the shutter button 110S and the assignable button 110C on the operation unit 17, and the assignable button 120C on the operation unit 24. Depending on the recognition of these operations, the AF distance measuring section 51, AF control section 52, and image recognition section 53 described above can perform corresponding processing.

The assignable button setting section 56 has a function of setting the operating functions of the assignable buttons 110C and 120C according to the user's setting operation. When the operation recognition unit 54 detects an operation of the assignable button 110C or the assignable button 120C, the camera control unit 30 recognizes that a predetermined function has been operated based on the settings of the assignable button setting unit 56.

The display control section 55 controls the display operation of the display section 15. For example, the display control unit 55 has a function of controlling the display of live view images and the display of various operation menus, icons, messages, etc. on the screen.

6 and 7 show display examples on the display unit 15. These are examples in which various icons are displayed superimposed on the through image.
In FIG. 6, a person HM playing a basketball is captured as a subject at approximately the center of the screen of the display unit 15 (that is, the angle of view being imaged).
Here, a focus frame 90 is displayed at the center of the screen. The focus frame 90 is a frame that indicates an AF target, and is a frame that indicates a so-called AF distance measurement point.
The display position of the focus frame 90 changes depending on various modes, and there are cases where a plurality of focus frames 90 are displayed, but here, one focus frame 90 is displayed in the center of the screen as an example.

FIG. 7 shows a state in which tracking is being performed. For example, assume that AF is applied to a person HM based on the focus frame 90 in FIG. 6 and that the person HM is set as a tracking target.
Assume that at the time of FIG. 7, the person HM is located on the left side of the screen. A tracking frame 91 is displayed for this person HM. The tracking frame 91 follows the target object and is displayed so as to overlap the object.
In this example, the focus frame 90 and the tracking frame 91 have different shapes. The timing at which this shape changes is, for example, when an AF operation (in this example, a half-press operation of the shutter button 110S) is performed and focus on the subject is completed.

Further, as shown in FIGS. 6 and 7, various icons and letters/numbers indicating the mode, operating state, etc. of the imaging device 1 are displayed on the screen.
For example, an ISO sensitivity 71, an exposure value 72, an aperture value 73, a shutter speed 74, and a focus mark 75 are displayed at the bottom of the screen.
The focus mark 75 indicates that the target is in focus, and FIG. 7 shows that the tracking target is in focus.

On the left side of the screen, a finder frame rate 76, shutter method 77, focus area 78, focus mode 79, drive mode 80, and shooting mode 81 are displayed.
The finder frame rate 76 is an icon indicating the frame rate of the image displayed on the viewfinder 102.
The shutter method 77 is an icon indicating mechanical shutter method, electronic shutter method, and auto. The one shown is auto, which is a mode in which the mechanical shutter method and electronic shutter method are automatically switched depending on the shooting situation and shutter speed.

The focus area 78 is an icon representing focus area setting as an area where focus control is performed. Focus area settings include "wide,""zone,""center," and "flexible spot."
"Wide" is a focus area mode that automatically focuses on the entire monitor as a through image.
"Zone" is a focus area mode that automatically focuses within a zone you select on the monitor.
"Center" is a focus area mode that automatically focuses on subjects near the center of the monitor.
"Flexible Spot" is a focus area mode that allows you to move the focus frame anywhere on the monitor and focus on very small objects or narrow areas.

Note that the icon of the focus area 78 is switched depending on the focus area setting at that time.
The image examples of the icons displayed as the focus area 78 shown in FIGS. 6 and 7 are illustrative examples and are not limited to these image designs.

A focus mode 79 displayed as shown in FIGS. 6 and 7 is an icon representing an AF operation mode. The AF operation modes include, for example, single AF (AF-S) and continuous AF (AF-C).
Single AF is a mode in which autofocus is activated when the shutter button 110S is pressed halfway, and once the focus is achieved, the focus is fixed there.
Continuous AF is a mode in which autofocus works and continues to focus on the subject while the shutter button 110S is pressed halfway. 6 and 7 are examples in which an icon indicating continuous AF is displayed.

The drive mode 80 is an icon indicating single-shot shooting, continuous shooting, bracket shooting, etc.
The shooting mode 81 is an icon indicating a shooting mode such as "program auto,""aperturepriority,""shutter speed priority,""manualexposure,""video," and "slow/quick motion."

In FIGS. 6 and 7, the number of storage media/sheets 82, aspect ratio 83, image size 84, image stabilization 85, image quality 86, and remaining battery power 87 are displayed at the top of the screen.
The number of pictures saved on the recording medium, the number of pictures that can be saved, etc. are displayed as the storage medium/number 82.
The aspect ratio 83 indicates the aspect ratio of the captured image.
Image size 84 displays the size of the captured image.
The image stabilization function 85 displays the on/off and mode of the image stabilization function.
Image quality 86 displays the image quality setting of the captured image.
The remaining battery amount 87 indicates the remaining battery amount using, for example, a picture and a percentage display.

Although various icons and the like displayed on the display unit 15 have been exemplified so far, other icons, numbers, characters, or graphics for presenting various information may also be displayed.

<2. Operation explanation of embodiment>

An example of the operation performed in the imaging device 1 having the above configuration for autofocusing on the face or eyes of a subject will be described.
First, in this example, the operation that the user should perform in order to cause the imaging device 1 to autofocus on the face or eyes of the subject is determined as follows.

1) Turn on the face or eye AF setting from the menu. 2) Set the AF operation mode setting to continuous AF (AF-C) from the menu. 3) Move the focus frame 90 to the desired subject (face or eye). AF operation (in this example, half-pressing the shutter button 110S)

FIGS. 8 to 12 are diagrams for explaining an example of an operation for autofocusing on a subject's face or eyes.
In FIG. 8, after the above-mentioned operations 1) and 2) have been performed, the person HM1 located on the front side of the person HM1 and person HM2 existing in the captured image is selected as the desired subject. ) shows a display example on the display unit 15 in a state where the AF operation is performed. Specifically, it is assumed that in the AF operation 3) in this case, the AF operation is performed with the focus frame 90 in contact with the torso of the person HM1.

First, as a premise, in this example, in response to the operation 1), the face/eye detection unit 61 starts detection processing of a face or eyes present in a captured image. This face or eye detection processing is performed every frame.

In the state of FIG. 8, AF is performed targeting the torso of the person HM1. Further, in accordance with the continuous AF setting, subject tracking by the tracking processing unit 62 with the torso as the tracking target subject is started.
At this time, on the screen of the display unit 15, the focus frame 90 becomes non-displayed as the tracking starts, and a tracking frame 91 is displayed instead. Thereafter, the tracking frame 91 is displayed at the tracking position by the tracking processing unit 62 (if the target subject moves, the tracking frame 91 also moves in conjunction with the movement of the subject).

In the state of FIG. 8, since the face of the person HM1 is facing backward, the face or eyes of the person HM1 are not detected in the detection processing by the face/eye detection section 61 described above. On the other hand, since the face of the person HM2 is facing forward, the face or eyes of the person HM2 are detected by the detection process of the face/eye detection unit 61.
However, in the face/eye determination processing in this embodiment, the method described below is adopted to prevent the face or eyes of the person HM2 facing forward from being erroneously determined as the face or eyes of the selected person HM1. is planned.

Assume that the face of the person HM1 turns forward from the state shown in FIG. 8 as shown in FIG.
In the state shown in FIG. 9, the face or eyes of the person HM1 is detected by the detection process of the face/eye detection unit 61, and AF is performed on the detected face or eyes of the person HM1.
Further, from now on, in order to track the face or eyes of the person HM1, a process is performed in which the tracking processing unit 62 updates the subject position to be tracked to the position of the detected face or eyes of the person HM1.

At this time, in order to notify the user that the AF or tracking target position is the face or eye position of the person HM1, that is, the selected subject, in this example, as shown in FIG. A face/eye tracking frame 92 is displayed at the face or pupil position, and the tracking frame 91 that had been displayed until then is hidden.
As a result, it is possible to intuitively and easily notify the user that tracking of the selected subject has been switched to tracking of the face or eyes, and it is possible to improve the accuracy of notification of information related to tracking.

Here, as shown in the transition from FIG. 10 to FIG. 11, it is assumed that the face of the person HM1 turns backward again.
In this case, the face or eyes of the person HM1 are not detected in the detection process of the face/eye detection section 61. In this way, once the face or eyes of the selected subject has been determined, if the determined face or eyes are no longer detected in the captured image, the tracking processing unit 62 changes the position of the subject to be tracked. The subject position that was being tracked until then is maintained. In other words, in the illustrated example, the head position of the person HM1 is maintained as it is.
Further, in this case, regarding AF, subject tracking AF is performed. That is, the tracking processing unit 62 performs AF on the subject position to be tracked. As a result, AF is performed on the head of the person HM1 while being tracked until the face of the person HM1 turns forward again.

In addition, as in the transition from FIG. 10 to FIG. The face/eye tracking frame 92 displayed at the position is hidden and a tracking frame 91 is displayed instead. At this time, the tracking frame 91 is displayed at the position of the subject being tracked by the tracking processing unit 62 (that is, the head of the person HM1 in the illustrated example).
In this way, by changing the display mode of the frame display related to tracking and AF on the screen in response to the fact that the determined face or eyes are no longer detected in the captured image, the face or eyes of the selected subject can be changed. Although AF based on face or eye position information is no longer possible due to no longer being detected, it is possible to intuitively and clearly notify the user that face position (head position) tracking continues. Therefore, it is possible to improve the accuracy of notification of information related to tracking and AF.

FIG. 12 shows an example of the screen in which the face of the person HM1 turns forward again from the state shown in FIG. 11.
In the state shown in FIG. 12, the face or eyes of the person HM1 is again detected by the face/eye detection unit 61, and as in the case of FIG. At the same time, a process is performed in which the tracking processing unit 62 updates the subject position to be tracked to the position of the face or eyes of the detected person HM1. Also in this case, similarly to FIG. 9, the previously displayed tracking frame 91 is hidden, and the face/pupil tracking frame 92 is displayed at the face or pupil position that is the tracking target.

As can be understood from the above series of operations, especially the operation described as the transition from FIG. 6 to FIG. , there is no need for the user to directly select the subject's face or eyes.
Therefore, it is possible to simplify the operation for realizing autofocus based on the position of the subject's face or eyes.

In addition, in the above example, the selection operation of the subject to be AFed to the face or eyes is an AF operation with the focus frame 90 applied, but the selection operation can be performed using, for example, a touch panel on the screen. The operation may be performed by touching the subject. Alternatively, the operation may be performed by placing the GUI cursor close to the subject and pressing a predetermined button. At this time, the cursor movement operation may be performed using a cross key or a touch operation.

<3. Processing procedure>

With reference to FIGS. 13 to 18, an example of processing to be executed to realize the operation of the embodiment described above will be described.
Note that the processes shown in the flowcharts below are processes executed by the camera control unit 30, and in particular, in this example, these processes are executed by a CPU included in the camera control unit 30 according to a program stored in the memory unit 31, for example. This is realized as software processing. However, all or part of these processes may be implemented as hardware processes.

Here, in the following explanation, for convenience, the face and the eyes are treated collectively as a face without distinguishing them.
Of course, we will treat the face and eyes separately, and if only the face can be detected, AF will be updated to the face position and the tracking target subject position will be updated, and if the eyes can be detected, AF will be updated to the eye position. It is also possible to update the position of the subject to be tracked.

FIG. 13 is a flowchart illustrating main processing for realizing AF control on the face or eyes of a subject. Note that, in this example, the process shown in FIG. 13 is started in response to the above-mentioned settings 1) and 2) being performed.

First, in step S101, the camera control unit 30 waits until a subject position is selected. That is, it waits until the AF operation as the operation 3) above is executed.
If it is determined that the AF operation has been performed and the subject position has been selected, the camera control unit 30 initializes the selected position as the subject position to be tracked in step S102. That is, the position overlapping the focus frame 90 at the time when the AF operation in step S101 is performed is initialized as the position of the subject to be tracked.

In step S103 following step S102, the camera control unit 30 executes subject face determination processing. This subject face determination process is a process for determining the face corresponding to the position of the subject to be tracked. Specifically, in this example, the process is to determine the face corresponding to the position of the subject to be tracked from among the faces detected in the captured image in the face detection process performed every frame.
Here, the processes up to step S110 including the face determination process of step S103 are executed for each frame of the captured image by providing a one-frame standby process of step S111, which will be described later.

An example of the subject face determination process in step S103 will be described with reference to the flowchart in FIG. 14.
In FIG. 14, first, the camera control unit 30 repeatedly executes the subject face candidate determination process in step S201 for all faces in the image by loop processing indicated by LP1 and LP2 in the figure.

The flowchart in FIG. 15 shows a first example of the subject face candidate determination process in step S201.
In FIG. 15, the camera control unit 30 sets a face association range frame Fr from the face detection frame Ff in step S210.
Here, the face detection frame Ff is a frame set as a frame representing the range of faces detected from within the captured image, and is set for each face when a plurality of faces are detected.
The face association range frame Fr is a frame that is set based on the face detection frame Ff, and when it is assumed that the face represented by the face detection frame Ff is the face corresponding to the selected subject position, This is a frame that defines a range that is expected to include the selected subject position. That is, if the selected subject position is within this face association range frame Fr, it can be estimated that the face represented by the face detection frame Ff is highly likely to be the face of the selected subject. .

FIG. 16 is a diagram illustrating the relationship between the face detection frame Ff and the face association range frame Fr.
In this example, the face detection frame Ff and the face association range frame Fr are each rectangular frames. The face association range frame Fr is set geometrically based on the face detection frame Ff.
Specifically, the camera control unit 30 sets the position of the face association range frame Fr based on the position of the face detection frame Ff in the X direction (horizontal direction) and the Y direction (vertical direction), for example. As shown in the figure, the position in the X direction is such that the center of the face association range frame Fr in the X direction matches the center of the face detection frame Ff in the X direction, and the position in the Y direction is such that the upper sides of the face detection frame Ff and Set as matching position.
Regarding the size of the face association range frame Fr, the camera control unit 30 sets the length of the side in the X direction to a length corresponding to the length of the side in the X direction of the face detection frame Ff, and The length of the side is set to correspond to the length of the side of the face detection frame Ff in the Y direction. For example, the length of the side in the X direction is set to twice the length of the side in the X direction of the face detection frame Ff, and the length of the side in the Y direction is set to the length of the side in the Y direction of the face detection frame Ff. Set it to 3 times the size.

Note that the above setting of the face association range frame Fr is merely an example. Further, regarding the face association range frame Fr, the position and size setting method may be different depending on when the imaging device 1 is held horizontally and when it is held vertically.

The explanation returns to FIG. 15.
The camera control unit 30 determines in step S211 whether the subject position exists within the face associating range frame Fr in response to the execution of the face associating range frame Fr setting process in step S210. If the subject position exists within the face association range frame Fr, the camera control unit 30 proceeds to step S212, and determines the DF (defocus amount) of the subject position (in this example, the tracking target subject position) and the face detection frame. It is determined whether the DF of Ff is close. Specifically, it is determined whether the difference between the DF of the subject position and the DF of the face detection frame Ff is less than or equal to a predetermined threshold. As the DF information here, DF information obtained by the AF ranging section 51 is used.
Note that the DF information may be obtained based on a detection signal of an image plane phase difference sensor embedded in the image sensor 12 (image sensor), or a separate phase difference sensor installed at a location different from the image sensor 12. It may be based on a detection signal obtained by

If it is determined in step S212 that the DF of the subject position is close to the DF of the face detection frame Ff, the camera control unit 30 proceeds to step S213, determines that the subject face is a candidate, and ends the subject face candidate determination process of step S201. . That is, among the faces detected in the captured image, the face to be processed is determined to be the subject face candidate, and the subject face candidate determination process is ended.

In this way, in this example, if the subject position with respect to the face to be processed satisfies the predetermined positional relationship conditions (in this example, the conditions for the position in the image plane direction and the position in the depth direction), the face to be processed is is determined as a candidate for the subject's face.

On the other hand, when the camera control unit 30 determines in step S211 that the subject position does not exist within the face association range frame Fr, and in step S212, the camera control unit 30 determines that the DF of the subject position and the DF of the face detection frame Ff are not close. If so, the process advances to step S214, where it is determined that the subject face is not a candidate, and the subject face candidate determination process of step S201 ends. That is, it is determined that the face targeted for processing among the faces detected in the captured image is not a subject face candidate, and the subject face candidate determination process is ended.

Note that the face association range frame Fr may be formed by recognizing human body parts through image recognition. However, generally, recognizing human body parts using image recognition requires large computational resources and tends to be unstable due to insufficient accuracy. It is more practical to seek it scientifically.

Further, the depth information of the subject position and the face detection frame Ff is not limited to the DF information. For example, depth information may be acquired using so-called Depth From Defocus, which is estimated from the difference in defocus that occurs when the lens is slightly shifted. Alternatively, the depth information may be estimated using an image processing technique such as deep learning, or may be calculated by stereo matching using images captured by two or more imaging units. Furthermore, depth information can also be calculated using dynamic object parallax. Note that dynamic parallax means that objects that are closer appear to move faster and objects that are farther away appear to move slower, and it is possible to grasp distance from near and far from this difference in speed.
Alternatively, the depth information of the subject position may be acquired from the information of the focus position of the lens, and the depth information of the face may be calculated and estimated from the size of the face on the imaging plane and the size of an average human face.
At this time, instead of the average face size, a manually set value of the actual size of the face may be used. In the case of animals, the type of animal may be set and the average size of the animal may be used.

The flowchart in FIG. 17 shows a second example of the subject face candidate determination process in step S201.
Here, in sports photography, etc., the subject takes a violent posture as shown in the upper row of FIG. 18, so the face is not always directly above the body. In the example in the upper part of FIG. 18, a person HM1 shown in white is a person who is desired to be targeted for AF, and a person HM2 shown in a matte background is a person who is not desired to be targeted for AF. In this case, as shown in the lower part of FIG. 18, it is assumed that the AF operation is performed by placing the focus frame 90 on the person HM1 to be the AF target at the timing when the person HM1 and the person HM2 intersect. At this time, if face candidate determination is performed based only on positional relationships as in the process of the first example in FIG. be.

Therefore, as the subject face candidate determination process in step S201, a second example of the process shown in FIG. 17 may be executed.
The difference from the first example in FIG. 15 is that, in addition to the conditions in steps S211 and S212, the condition in step S220 is added as a condition for determining a face candidate. Specifically, in step S220, the camera control unit 30 determines whether the movement of the subject and the movement of the face detection frame Ff are close to each other. Specifically, in the process of step S220, the movement of the face in the X, Y, and Z directions (the Z direction is the depth direction: the direction perpendicular to the XY plane) in the past few frames and the X, Y directions of the subject to be tracked are , and the Z-direction movements to determine whether their trajectories are similar. Regarding the movement in the Z direction, the above-mentioned DF information can be used.
Note that in the determination process of step S220, the movement of the face or subject may not be in all of the X, Y, and Z directions, but may be in only some directions.
The determination as to whether or not the trajectories are similar may be a rough determination, such as a binary determination of whether the trajectories are approaching or receding. Even being able to exclude faces that are moving in the opposite direction is considered to be of great benefit to the user.

As shown in the figure, if a positive result is obtained in all the determination processes in steps S211, S212, and S220, that is, the subject position exists within the face association range frame Fr, and the distance between the subject position and the face detection frame Ff is If it is determined that the DF is close and the movement of the subject and the face detection frame Ff is close, the camera control unit 30 performs a process of determining the subject face candidate in step S213.
On the other hand, if a negative result is obtained in any one of the determination processes in steps S211, S212, and S220, the camera control unit 30 performs a process in step S214 to determine that the face of the subject is not a candidate.

In the second example as well, the camera control unit 30 finishes the subject face candidate determination process in step S201 in response to executing the process in either step S213 or S214.

The explanation returns to FIG. 14.
The camera control unit 30 advances the process to step S202 in response to having performed the subject face candidate determination process as the first example or the second example described above for all faces in the image.
In step S202, the camera control unit 30 determines whether a subject face candidate exists.
If it is determined in step S202 that there is a subject face candidate, the camera control unit 30 proceeds to step S203, determines the subject face candidate closest to the tracking target subject position as the subject face, and performs the subject face determination process in step S104. finish.
On the other hand, if it is determined that the subject face candidate does not exist, the camera control unit 30 proceeds to step S204, determines that the subject face does not exist, and ends the subject face determination process of step S104.

In FIG. 13, after completing the subject face determination process in step S103, the camera control unit 30 advances the process to step S104 and determines whether or not the subject's face exists.
If the subject's face is present, the camera control unit 30 proceeds to step S105 to perform AF on the subject's face, and in subsequent step S106 performs processing to update the face position as the subject's position.
Here, the process of step S105 is a process of controlling the AF control unit 52 shown in FIG. 5 to perform AF on the subject's face. Further, the process of step S106 is a process of updating the subject position to be tracked by the tracking processing unit 62 to the position of the subject's face.

In step S107 following step S106, the camera control unit 30 executes face/pupil frame display processing. That is, regarding frame display on the screen of the display unit 15, processing is executed to hide the tracking frame 91 that had been displayed up to that point and to display the face/eye tracking frame 92 at the position of the subject's face.
As a result, the frame display can be switched in response to the determination of the subject's face, as described above as the transition from FIG. 9 to FIG. 10.
In response to executing the display process in step S107, the camera control unit 30 executes a one-frame standby process in step S111, and returns to the subject face determination process in step S103.

On the other hand, if it is determined in step S104 that the subject's face does not exist, the camera control unit 30 proceeds to step S108 and performs subject tracking AF. Specifically, processing is performed to perform AF on the position of the subject to be tracked.
Then, in the subsequent step S109, the camera control unit 30 performs a process of updating the subject tracking result as the subject position. That is, the position specified by the subject tracking process is updated as the subject position to be tracked.

Here, the subject position to be tracked is set (updated) in step S102, step S106, or step S109. If, after the subject is selected in step S101, the determination process in step S104 has never determined that the subject's face exists, the position of the subject to be tracked will be the position selected in step S101. (In other words, if the torso of the subject is selected, this is the position as the tracking result of the torso).
Furthermore, if the process in step S109 is performed once, it is determined that the subject's face exists in the determination process in step S104, and then it is determined that the subject's face does not exist in the determination process in step S104 performed for subsequent frames. When executed in accordance with , the subject position to be tracked becomes the position as a result of tracking the face position set in step S106. In the process of step S109 in this case, the process of updating the face position tracking result as the subject position of the tracking target is performed. Therefore, as shown in the transition from FIG. 10 to FIG. 11 above, if the face of the subject that has been determined turns backward and is no longer detected, the head of the subject will be tracked. (In other words, the tracking target does not switch to the selected body part).

In step S110 following step S109, the camera control unit 30 executes tracking frame display processing. That is, a process of displaying the tracking frame 91 as a frame display on the screen of the display unit 15 is executed. As a result, after the subject is selected in step S101, the tracking frame 91 is displayed as a frame display on the screen of the display unit 15 in a state where the determination result that the subject's face exists has not yet been obtained in step S104. continues to be. Furthermore, after the determination result that the subject's face exists is obtained in step S104, the tracking frame 91 will be displayed if it is determined that the subject's face does not exist in step S104 (see FIG. 10). (see transition to 11). That is, frame display switching from the face/eye tracking frame 92 to the tracking frame 91 as shown in the transition from FIG. 10 to FIG. 11 is realized.

In response to executing the tracking frame display process in step S110, the camera control unit 30 advances the process to one frame standby process in step S111.

Note that the focus frame 90, the tracking frame 91, and the face/eye tracking frame 92 are not limited to display in the frame shape, but can also adopt other shapes, and indicate the position targeted for AF or tracking. Any visual information that can be displayed may be displayed.

Further, regarding the display of information indicating the target position of AF or tracking, the following method may be adopted. In other words, when performing human body part analysis processing on a selected subject, if no body part is detected after selecting the subject, the tracking frame 91 is displayed at the subject position, and the tracking frame 91 is displayed when the body part is detected. If a face is detected, a face frame representing the range of the face is displayed, and if the pupil is detected, a face frame representing the range of the pupil is displayed. This method includes displaying a pupil frame representing the pupil frame.
Note that it is not essential to display visual information to indicate the position targeted for AF or tracking.

<4. Modified example>

Here, consider a scene as shown in FIG. 19 in which a person HMm as a mother is holding a person HMc as a child. At this time, it is assumed that the user wants to photograph the child rather than the mother. In this case, it can be assumed that the user performs the AF operation while placing the focus frame 90 on a relatively large area such as the child's torso as shown in the figure.
In the scene shown in Figure 19, when both the mother's and child's faces are detected, the child's face is closer to the user's selected position, so the child's face cannot be tracked correctly or AF can be performed. It can be carried out.

However, it is assumed that after that, the child becomes violent or the like and the face becomes undetected (see FIG. 20). In this state, the only face present in the image is the mother's face, and since the mother's face is highly likely to satisfy the conditions for the subject face candidate, the tracking target is switched to the mother's face, and the mother's face is There is a risk that AF may be performed.
Below, the first, second, and third variations as countermeasures for such situations will be described.
In the following description, parts that are similar to parts that have already been explained are given the same reference numerals and step numbers, and the description thereof will be omitted.

The first modification is a measure by introducing a non-subject face.
FIG. 21 is a flowchart of subject face determination processing (S103) as a first modification.
In the subject face determination process as the first modification, after the face of the selected subject has been determined once, all faces that have not been determined to be the face of the selected subject are excluded from the determination targets.
Specifically, the camera control unit 30 in this case first determines whether the detected flag is ON in step S301. This detected flag is a flag that is turned ON in response to determination of the subject's face in the process of step S302, which will be described later, and its initial value is OFF. In other words, the detected flag is a flag indicating whether the process of step S203 has been performed at least once after the subject is selected at step S101 (whether the face of the selected subject has been determined at least once).

If it is determined that the process of step S203 has not been executed yet after the selection of the subject in step S101 and the detected flag is not ON, the camera control unit 30 proceeds to the loop process of LP1 and LP2 in the figure, and The process of step S201 is repeatedly executed for all faces. Thereafter, in response to an affirmative result indicating that a subject face candidate exists in step S202 and the determination process in step S203 being executed, the camera control unit 30 turns on the detected flag in step S302, and then In step S303, all faces other than the subject face are set as non-subject faces, and the subject face determination process in step S103 is completed.

In subsequent frames, an affirmative result that the detected flag is ON is obtained in the determination process of step S301.
If a positive result is obtained in step S301, the camera control unit 30 proceeds to the loop processing of LP1' and LP2' in the figure, and performs the subject face candidate determination process of step S201 for faces other than non-subject faces in the image. Execute repeatedly.
Thereafter, the camera control unit 30 proceeds to step S202' and determines whether or not a subject face candidate exists. If a subject face candidate exists, the camera control unit 30 proceeds to step S203' to select a subject face candidate closest to the subject position to be tracked. A process of determining the subject's face is performed, and the subject face determining process of step S103 is completed. On the other hand, if the subject face candidate does not exist, a process is performed in step S204' to determine that the subject face does not exist, and the subject face determination process in step S103 ends.

Due to this process in the first modification, faces other than the selected subject's face will be incorrectly determined in the subject determination performed when the subject's face is lost after the subject's face has been determined even once. This will help prevent this.
Therefore, it is possible to improve the accuracy of determining the face or eyes of the subject.

Next, a second modification will be explained.
The second modification is a measure by changing the face association range frame Fr.
FIG. 22 is a diagram illustrating a normal face association range frame Fr for each person HMm and person HMc, and FIG. 23 is a diagram illustrating a changed face association range frame Fr for each person HMm and person HMc. This is a diagram.
In the second modification, once the subject's face has been determined, by narrowing the face linking range frame Fr, even if the subject's face cannot be detected afterwards, the face of another subject (mother) can be recognized as the subject's face. Aim to prevent misjudgment.
Although explanations using illustrations are omitted, in this case, when the subject's face has never been found, the camera control unit 30 uses the normal face association range frame Fr shown in FIG. 22 (for a specific example, see the description in FIG. ), the subject face determination process in step S104 is performed.
Thereafter, in response to the determination of the subject's face, the subject face determination process of step S104 is thereafter performed using the face association range frame Fr whose range is narrower than usual as shown in FIG. In particular, in this example, once the subject's face has been determined, the subject position to be tracked (that is, the position where it is determined whether or not it is located within the face association range frame Fr in step S211) is the position of the subject's face. Since it is located nearby, the face linking range frame Fr does not need to extend to the entire body, but is narrower than usual as shown in FIG. Note that "narrowing the range" here means narrowing the frame length in at least one of the X and Y directions.

Even in such a second modification, after the face of a subject (for example, a child's face) has been determined at least once, when the face is lost, a face other than the face of the selected subject (for example, a mother's face) is detected. It is possible to prevent the face of the subject from being incorrectly determined as the subject's face.
In particular, according to the second modification, even if the child's face is determined to be the subject face while the mother's face is not detected, when the child's face is not detected, the mother's It is possible to prevent a face from being incorrectly determined as the subject's face.

A third modification is a measure based on remembering the initially selected subject position.
For example, in the example of a mother and child, if the child is facing backwards when the user performs an operation to select the subject, as shown in FIG. 24, it is difficult to detect the child's face. If this is not possible, there is a risk that the mother's face may be incorrectly determined as the subject's face.
In the process explained so far, if the mother's face is determined as the subject's face in this way, the mother's face position is updated as the subject's position to be tracked. Even if the mother's face is detected, there is a possibility that the mother's face will continue to be determined as the subject's face as long as the mother's face is detected (see FIG. 25). That is, there is a possibility that AF or tracking cannot be performed on the face of the subject intended by the user.

FIG. 26 is a flowchart illustrating main processing as a third modification.
The difference from the main process shown in FIG. 13 is that the process in step S120 is executed instead of the process in step S106.
In step S120, the camera control unit 30 executes a process of tracking the subject from the subject position and updating the tracking result as the subject position. That is, even if it is determined in step S104 that the subject's face exists, the position of the subject to be tracked is not updated to the face position, but the previous tracking position is updated as the subject position to be tracked. Through this process, the position of the subject to be tracked (the position determined in step S211 as to whether or not it exists within the face association range frame Fr) is maintained as the position selected as the tracking result of the position selected in step S101. be done.

In this process as the third modified example, a subject (for example, a child) whose face is not detected because the face is turned backwards, etc. is selected by operation, and the face of a subject other than the selected subject (for example, Even if the mother's face) is incorrectly determined, subsequent determinations will be made based on the tracking position of the selected subject (for example, the tracking position of the child's torso). When the face of the selected subject is detected such as when the face of the subject is facing forward, it is possible to correctly determine the face of the subject.

Note that as a method for maintaining the position at which it is determined in step S211 whether or not the position exists within the face association range frame Fr as the subject position initially selected by the user, for example, a position indicated by an AF area such as a flexible spot can be used. A method may be adopted in which the position is regarded as the subject position selected by the user.

<5. Other variations>

Note that the above-described specific example is just an example, and the embodiment can take various configurations as modified examples.
For example, in the above, autofocus control was cited as an example of the adjustment control of the imaging control parameters, but there are other ways to adjust the imaging control parameters such as automatic exposure control (AE) and auto white balance control (AWB). ), etc. In other words, this technology can be used, for example, to control exposure so that the exposure of a selected subject's face or eyes is optimal, or to control white balance so that the color of a selected subject's face or eyes is optimal. It can also be suitably applied when carrying out.

In addition, in the above, on the condition that the selected subject exists within the search range (face association range frame Fr) set geometrically from the position of the detected face or eyes, the detected face or eyes are We have given an example of determining the face or eyes of the selected subject, but instead of this, for example, active segmentation can be performed based on the selected subject position, and the face or eyes belonging to the same segment as the selected position can be determined. It is also possible to adopt a method of determining the face or eyes of the selected subject.

In addition, above, we have given an example of AF using AF-C (AF that continues to focus on the face and eyes) as an example of AF for faces and eyes. You can also fix the focus after focusing on the eyes.

<6. Summary of embodiments>

As described above, the imaging control device (camera control unit 30) according to the embodiment determines the face or eyes of the selected subject in response to the selection of the subject in the captured image by operation. A determination unit (subject face determination unit 63), and an adjustment control unit (AF control unit 52) that performs adjustment control of imaging control parameters based on position information of the face or eyes determined by the face/eye determination unit. There is.

The imaging control parameters refer to various control parameters related to imaging using an imaging device, such as focus, exposure, and white balance. According to the above configuration, as long as an arbitrary part of the subject, such as the torso of the subject, is selected, the face or eyes of the subject are determined, and the imaging control is performed based on the determined position of the face or eyes. Parameter adjustment control is performed. That is, when adjusting imaging control parameters such as autofocus based on the subject's face or eyes, there is no need for the user to directly select the subject's face or eyes.
Therefore, it is possible to simplify the operation for adjusting the imaging control parameters based on the position of the subject's face and eyes.

Further, in the imaging control device according to the embodiment, the face/eye determination unit detects a face or eyes present in the captured image, and determines whether the detected face or eyes corresponds to the face or eyes of the subject. Judging.

As a method of determining the face or eyes of the selected subject, a method of performing image analysis to determine the human body part of the selected subject may be considered, but this type of image analysis processing has a relatively high calculation cost. According to the above configuration, it is possible to determine the face or eyes of the subject without performing such image analysis processing that requires high computational cost.
Therefore, it is possible to reduce the processing load when determining the face or eyes of the subject.

Further, in the imaging control device according to the embodiment, the face/eye determination section determines the face or eyes of the subject based on the positional relationship between the detected face or eyes and the subject.

This makes it possible to prevent faces or eyes detected in a captured image that are located far apart from each other from being determined as faces or eyes of the subject.
Therefore, the accuracy of determining the subject's face or eyes can be improved.

Furthermore, in the imaging control device as an embodiment, the face/eye determination unit detects the detected face or pupil on the condition that the subject is located within a geometrically set search range from the position of the detected face or pupil. The face or eyes are determined to be those of the subject.

Thereby, in determining the face or eyes of a subject, it is sufficient to perform at least a range setting process and a process of determining whether the selected subject is included within the set range for each detected face or eye.
Therefore, it is possible to reduce the processing load when determining the face or eyes of the subject.

Furthermore, in the imaging control device according to the embodiment, the face/eye determination unit determines the face or eyes of the subject based on the positional relationship in the depth direction between the detected face or eyes and the subject.

This makes it possible to determine the face or eyes of the subject based not only on the positional relationship in the image plane direction but also on the positional relationship in the depth direction.
Therefore, it is possible to improve the accuracy of determining the face or eyes of the subject.

Further, in the imaging control device according to the embodiment, the face/eye determination section determines the face or eyes of the subject based on consistency of movement between the detected face or eyes and the subject.

As a result, in response to cases where the face or eyes of the subject cannot be appropriately determined based on positional relationships alone, the face or eyes of the subject can be appropriately determined from among the detected faces or eyes based on consistency of movement. is possible.
Therefore, it is possible to improve the accuracy of determining the face or eyes of the subject.

Furthermore, in the imaging control device according to the embodiment, the face/eye determination unit determines the face or eyes of the subject based on the consistency of movement in the depth direction between the detected face or eyes and the subject.

This makes it possible to determine the face or eyes of the subject based not only on the consistency of movement in the plane direction of the image but also on the consistency of movement in the depth direction.
Therefore, it is possible to improve the accuracy of determining the face or eyes of the subject.

Further, in the imaging control device according to the embodiment, the adjustment control unit performs autofocus control as adjustment control of imaging control parameters, and the face/pupil determination unit performs autofocus control as depth information of the detected face or eyes and the subject. distance measurement information is used.

This eliminates the need to provide a separate distance measurement sensor other than the distance measurement sensor for autofocus control in the imaging device when determining the face or eyes of the subject based on the positional relationship in the depth direction and consistency of movement.
Therefore, it is possible to reduce the size and weight of the imaging device.

Furthermore, the imaging control device according to the embodiment includes a tracking unit (tracking processing unit 62) that tracks the target subject, and the face/pupil determination unit, when determining the face or eyes of the subject, The tracking unit updates the tracking target subject, which is the subject to be tracked, to the determined face or eyes, and in subsequent determinations, based on the position tracked by the tracking unit, the detected face or eyes are the subject's face. Or it is determined whether or not it is an pupil.

This makes it possible to track the subject's head even if the subject's face or eyes are no longer detected because the subject's face or eyes are turned backwards or the like after the subject's face or eyes have been determined.
Therefore, after the face or eyes of the subject that have been determined once are lost, the face or eyes of the subject can be easily determined again, and the accuracy of determining the face or eyes of the subject can be improved.

Furthermore, in the imaging control device according to the embodiment, after determining the face or eyes of the subject from among the detected faces or eyes, the face/eye determination unit may detect all of the detected faces or eyes that are not determined to be the face or eyes of the subject. face or eyes are excluded from the determination target (see the first modification).

This helps prevent a face or eyes other than the subject's face or eyes from being misjudged when the face or eyes of the subject are lost after the face or eyes of the subject have been determined even once. It will be done.
Therefore, it is possible to improve the accuracy of determining the face or eyes of the subject.

Further, in the imaging control device according to the embodiment, after determining the face or eyes of the subject from among the detected faces or eyes, the face/eye determination unit changes the search range from the first search range to the first search range. It is determined that the search range has been switched to a second search range different from the above (see the second modification).

This makes it possible to set the search range to a second search range, which is narrower than the first search range, in the judgment performed when the face or eyes of the subject are lost after the face or eyes of the subject have been determined at least once. Thus, it is possible to prevent faces or eyes other than those of the subject from being erroneously determined.
Therefore, it is possible to improve the accuracy of determining the face or eyes of the subject.

Furthermore, the imaging control device according to the embodiment includes a tracking unit that tracks the subject, and the tracking unit is configured to perform the following operations regardless of whether the face/eye determination unit has determined the face or eyes of the subject. The face/eye determination unit continues to track the subject selected by the operation, and determines whether the detected face or pupil is the face or pupil of the subject based on the position tracked by the tracking unit.

As a result, even if a subject whose face or eyes are not detected because the face is turned backwards etc. is selected and the face or eyes of a subject other than the selected subject is incorrectly determined, subsequent judgments will be made. Since the judgment is performed based on the selected subject position, when the face or eyes of the selected subject are detected, such as when the face of the selected subject is facing forward, the face or eyes of the selected subject are detected. It is possible to judge correctly.
Therefore, it is possible to improve the accuracy of determining the face or eyes of the subject.

Furthermore, in the imaging control device as an embodiment, in response to a subject being selected by an operation, the first visual information is displayed at the position of the subject on the display unit that displays the captured image, and face/pupil determination is performed. Display control that causes the display unit to display second visual information at the position of the determined face or eyes and hide the first visual information in response to the determination of the face or eyes of the subject by the unit. (see FIGS. 6 to 10).

This makes it possible to intuitively and easily notify the user that tracking of the selected subject has been switched to tracking of the face or eyes.
Therefore, it is possible to improve the accuracy of notification of information related to tracking.

Furthermore, in the imaging control device according to the embodiment, the display control unit displays information having a shape different from that of the first visual information as the second visual information.

This makes it possible to intuitively and more easily notify the user that tracking of the selected subject has been switched to tracking of the face or eyes.
Therefore, it is possible to further improve the accuracy of reporting information regarding tracking.

Furthermore, the imaging control device according to the embodiment includes a tracking unit that tracks the subject, and when the face/eye determination unit determines the face or eyes of the subject, the tracking unit determines that the subject is the subject of tracking. The tracking target subject, which is the subject to be photographed, is updated to the determined face or eyes, and if the determined face or eyes are no longer detected in the captured image, the tracking target subject is maintained as the previous tracking target subject. The adjustment control unit performs adjustment control of the imaging control parameters based on the information of the position tracked by the tracking unit, and the display control unit adjusts the imaging control parameters in response to the fact that the face or eyes determined by the face/pupil determination unit can no longer be detected. Control is performed to display visual information different from the second visual information at the position tracked by the tracking unit (see FIGS. 10 and 11).

According to the above configuration, after the face or eyes of the selected subject is determined, until the face or eyes of the selected subject becomes undetected, for example, when the face of the subject turns backward, the face or eyes of the selected subject are determined. The imaging control parameters are adjusted based on the position of . According to the above configuration, when the face or eyes of the selected subject is not detected, the face (head) position of the selected subject continues to be tracked, and the position of the face (head) of the selected subject continues to be tracked. , visual information different from the second visual information is displayed (for example, the display is switched from the face/eye tracking frame 92 to the tracking frame 91). Due to such display control, the face or eyes of the selected subject were no longer detected, making it impossible to adjust the imaging control parameters based on the position information of the face or eyes. It is possible to intuitively and easily notify the user that tracking of the location) is continuing.
Therefore, it is possible to improve the accuracy of notification of information related to tracking and imaging control parameter adjustment.

Furthermore, in the imaging control device according to the embodiment, the subject is a person.

This eliminates the need for the user to directly select a person's face or eyes when adjusting the imaging control parameters based on the person's face or eyes.
Therefore, it is possible to facilitate the operation for adjusting the imaging control parameters based on the positions of the person's face and eyes.

Furthermore, in the imaging control device according to the embodiment, the subject is an animal.

This eliminates the need for the user to directly select the face or eyes of an animal, such as a dog, cat, or bird, in order to adjust the imaging control parameters based on the face or eyes of the animal.
Therefore, it is possible to facilitate the operation for adjusting the imaging control parameters based on the position of the animal's face and eyes.

In addition, the imaging device (same 1) as an embodiment includes an imaging unit (imaging device 12) that obtains a captured image, and a face of the selected subject in response to selection of a subject in the captured image by operation. Alternatively, it includes a face/pupil determination section that determines the pupils, and an adjustment control section that performs adjustment control of the imaging control parameters based on the position information of the face or the pupils determined by the face/pupil determination section.

Further, in the imaging control method of the embodiment, in response to a subject being selected in a captured image by an operation, the face or eyes of the selected subject is determined, and based on the position information of the determined face or eyes. This is an imaging control method that performs adjustment control of imaging control parameters.

These imaging devices and imaging control methods can also provide the same operations and effects as the imaging control device of the above-described embodiment.

Note that the effects described in this specification are merely examples and are not limiting, and other effects may also exist.

<7. This technology＞

Note that the present technology can also adopt the following configuration.
(1)
a face/eye determination unit that determines the face or eyes of the selected subject in response to the selection of the subject in the captured image by an operation;
An imaging control device, comprising: an adjustment control unit that performs adjustment control of imaging control parameters based on position information of the face or pupils determined by the face/eye determination unit.
(2)
The face/eye determination unit includes:
The imaging control device according to (1) above, wherein a face or eyes present in the captured image is detected, and it is determined whether the detected face or eyes corresponds to the face or eyes of the subject.
(3)
The face/eye determination unit includes:
The imaging control device according to (2), wherein the face or eyes of the subject is determined based on the positional relationship between the detected face or eyes and the subject.
(4)
The face/eye determination unit includes:
The detected face or eyes are determined to be the face or eyes of the subject on the condition that the subject is located within a geometrically set search range from the position of the detected face or eyes. ).
(5)
The face/eye determination unit includes:
The imaging control device according to any one of (3) or (4), wherein the face or eyes of the subject is determined based on the positional relationship in the depth direction between the detected face or eyes and the subject.
(6)
The face/eye determination unit includes:
The imaging control device according to any one of (2) to (5), wherein the face or eyes of the subject is determined based on consistency of movement between the detected face or eyes and the subject.
(7)
The face/eye determination unit includes:
The imaging control device according to (6), wherein the face or eyes of the subject is determined based on consistency of movement in the depth direction between the detected face or eyes and the subject.
(8)
The adjustment control section includes:
Performing autofocus control as adjustment control of the imaging control parameter,
The face/eye determination unit includes:
The imaging control device according to (5) or (7), wherein distance measurement information for autofocus is used as depth information of the detected face or eyes and the subject.
(9)
Equipped with a tracking unit that tracks the targeted subject,
The face/eye determination unit includes:
When the face or eyes of the subject is determined, the tracking unit updates the tracking target subject, which is the subject to be tracked, to the determined face or eyes, and in subsequent determinations, the tracking unit The imaging control device according to any one of (2) to (8), wherein it is determined whether the detected face or eyes are the face or eyes of the subject based on the position to be tracked.
(10)
The face/eye determination unit includes:
After determining the face or eyes of the subject from among the detected faces or eyes, all faces or eyes that are not determined to be the face or eyes of the subject are excluded from the determination targets (2) to (9). ).
(11)
The face/eye determination unit includes:
After determining the face or eyes of the subject from among the detected faces or eyes, a determination is made in which the search range is switched from the first search range to a second search range different from the first search range. The imaging control device according to any one of 4) to (10).
(12)
Equipped with a tracking unit that tracks the targeted subject,
The tracking unit is
Continue to track the subject selected by the operation regardless of whether the face/eye determination unit has determined the face or eyes of the subject;
The face/eye determination unit includes:
The imaging control device according to any one of (2) to (8), wherein it is determined whether the detected face or pupil is the face or pupil of the subject based on the position tracked by the tracking unit.
(13)
In response to the selection of the subject by the operation, first visual information is displayed at the position of the subject on the display unit that displays the captured image, and the face or eyes of the subject is determined by the face/eye determination unit. a display control unit that displays second visual information on the display unit at the determined face or pupil position and hides the first visual information in accordance with the determined face or pupil position. The imaging control device according to any one of (12).
(14)
The display control section includes:
The imaging control device according to (13) above, wherein information having a shape different from that of the first visual information is displayed as the second visual information.
(15)
Equipped with a tracking unit that tracks the targeted subject,
The face/eye determination unit includes:
When the face or eyes of the subject is determined, the tracking unit updates the tracking target subject, which is the subject to be tracked, to the determined face or eyes, and the determined face or eyes are included in the captured image. If the tracking target is no longer detected, the tracking target is maintained as the tracking target until then,
The adjustment control section includes:
Adjustment control of the imaging control parameters is performed based on information on the position tracked by the tracking unit,
The display control section includes:
Control is performed to display visual information different from the second visual information at the position tracked by the tracking unit in response to the fact that the face/eye determination unit is no longer able to detect the determined face or eye. 13) or the imaging control device according to (14).
(16)
The imaging control device according to any one of (1) to (15) above, wherein the subject is a person.
(17)
The imaging control device according to any one of (1) to (15) above, wherein the subject is an animal.

1 Imaging device, 12 Imaging element, 30 Camera control unit, 51 AF ranging unit, 52 AF control unit, 53 Image recognition unit, 55 Display control unit, 61 Face/pupil detection unit, 62 Tracking processing unit, 63 Subject face determination part, 90 focus frame, 91 tracking frame, 92 face/eye tracking frame, 110S shutter button, 101 display panel, 102 viewfinder, Ff face detection frame, Fr face linking range frame

Claims (17)

a face/eye determination unit that determines the face or eyes of the selected subject in response to the selection of the subject in the captured image by an operation;
an adjustment control unit that performs adjustment control of imaging control parameters based on position information of the face or eyes determined by the face/eye determination unit ,
The face/eye determination unit includes:
Detecting a face or eyes existing in the captured image, and determining whether the detected face or eyes corresponds to the face or eyes of the subject based on the positional relationship between the detected face or eyes and the subject. judge
Imaging control device. The face/eye determination unit includes:
Claim 1 : The detected face or pupil is determined to be the face or pupil of the subject on the condition that the subject is located within a geometrically set search range from the position of the detected face or pupil. The imaging control device described in . The face/eye determination unit includes:
The imaging control device according to claim 1 , wherein the face or eyes of the subject is determined based on a positional relationship in the depth direction between the detected face or eyes and the subject. The face/eye determination unit includes:
The imaging control device according to claim 1 , wherein the face or eyes of the subject is determined based on consistency of movement between the detected face or eyes and the subject. The face/eye determination unit includes:
The imaging control device according to claim 4 , wherein the face or eyes of the subject is determined based on consistency of movement in the depth direction between the detected face or eyes and the subject. The adjustment control section includes:
Performing autofocus control as adjustment control of the imaging control parameter,
The face/eye determination unit includes:
The imaging control device according to claim 3 or 5 , wherein distance measurement information for autofocus is used as the depth information of the detected face or eyes and the subject. Equipped with a tracking unit that tracks the targeted subject,
The face/eye determination unit includes:
When the face or eyes of the subject is determined, the tracking unit updates the tracking target subject, which is the subject to be tracked, to the determined face or eyes, and in subsequent determinations, the tracking unit The imaging control device according to claim 1 , wherein it is determined whether the detected face or eyes are the face or eyes of the subject based on the position to be tracked. The face/eye determination unit includes:
Imaging according to claim 1 , wherein after determining the face or eyes of the subject from among the detected faces or eyes, all faces or eyes that are not determined to be the face or eyes of the subject are excluded from the determination targets. Control device. The face/eye determination unit includes:
After determining the face or eyes of the subject from among the detected faces or eyes, determination is made by switching the search range from the first search range to a second search range different from the first search range. 2. The imaging control device according to 2 . Equipped with a tracking unit that tracks the targeted subject,
The tracking unit is
Continue to track the subject selected by the operation regardless of whether the face/eye determination unit has determined the face or eyes of the subject;
The face/eye determination unit includes:
The imaging control device according to claim 1 , wherein it is determined whether the detected face or eyes are the face or eyes of the subject based on a position tracked by the tracking unit. In response to the selection of the subject by the operation, first visual information is displayed at the position of the subject on the display unit that displays the captured image, and the face or eyes of the subject is determined by the face/eye determination unit. according to claim 1, further comprising a display control unit that causes the display unit to display second visual information at the determined face or pupil position and to hide the first visual information in response to the determined face or pupil position. imaging control device. The display control section includes:
The imaging control device according to claim 11 , wherein information having a shape different from that of the first visual information is displayed as the second visual information. Equipped with a tracking unit that tracks the targeted subject,
The face/eye determination unit includes:
When the face or eyes of the subject is determined, the tracking unit updates the tracking target subject, which is the subject to be tracked, to the determined face or eyes, and the determined face or eyes are included in the captured image. If the tracking target is no longer detected, the tracking target is maintained as the tracking target until then,
The adjustment control section includes:
Adjustment control of the imaging control parameters is performed based on information on the position tracked by the tracking unit,
The display control section includes:
Control is performed to display visual information different from the second visual information at a position tracked by the tracking unit in response to the fact that the face/eye determination unit is unable to detect the determined face or eye. 12. The imaging control device according to 11 . The imaging control device according to claim 1, wherein the subject is a person. The imaging control device according to claim 1, wherein the subject is an animal. an imaging unit that obtains a captured image;
a face/eye determination unit that determines the face or eyes of the selected subject in response to the selection of the subject in the captured image by operation;
an adjustment control unit that performs adjustment control of imaging control parameters based on position information of the face or eyes determined by the face/eye determination unit ,
The face/eye determination unit includes:
Detecting a face or eyes existing in the captured image, and determining whether the detected face or eyes corresponds to the face or eyes of the subject based on the positional relationship between the detected face or eyes and the subject. judge
Imaging device. In response to the selection of a subject in the captured image by an operation, the face or eyes of the selected subject is determined, and the imaging control parameters are adjusted and controlled based on the determined positional information of the face or eyes. ,
In the face or eye determination,
Detecting a face or eyes existing in the captured image, and determining whether the detected face or eyes corresponds to the face or eyes of the subject based on the positional relationship between the detected face or eyes and the subject. judge
Imaging control method.

Images