JP2021078154A - Zoom control device, imaging apparatus, control method of zoom control device, and control program of zoom control device - Google Patents

Abstract

To perform auto zoom control with a composition suitable for a scene determined on the basis of photographic subject detection information without requiring a photographer to select a composition.SOLUTION: A zoom control device includes: a subject detection unit 123 for detecting a subject image from an image acquired by using an imaging optical system; and an AZ control unit 122, for setting reference size of the subject image used for control of zoom magnification and controlling the zoom magnification on the basis of the set reference size and size of subject images sequentially detected by the subject detection unit 123. The AZ control unit 122 changes the reference size on the basis of at least one of whether an imaging apparatus whose zoom magnification is controlled by the zoom control device is in a hand-held state, whether a self-photographing mode is set, and still time of the subject.SELECTED DRAWING: Figure 14

Description

The present invention relates to a technique of detecting a subject image from an image and controlling a zoom magnification.

Conventionally, a function of driving a zoom lens to perform optical scaling (hereinafter referred to as optical zoom) and electronic scaling (hereinafter referred to as electronic zoom) by enlarging a part of a captured image are performed. There is an imaging device that has a function. Furthermore, there is an auto zoom (automatic scaling) function that automatically changes the zoom magnification according to the detection information of the subject. Hereinafter, the optical zoom operation and the electronic zoom operation are collectively referred to as a zoom operation.

Patent Document 1 discloses a configuration of a camera equipped with an auto-zoom function that maintains a constant size of a subject. Patent Document 1 discloses a method in which a photographer selects a composition from a plurality of options such as "face up", "bust up", and "whole body" in the auto zoom function. In the auto zoom control that maintains the size of the subject image, the size of the subject image that is the reference for the zoom control is determined based on the selected composition. For example, when the photographer selects the composition of "whole body", the zoom operation is performed so that the image of the whole body of the person who is the subject is captured in the shooting screen.

Japanese Unexamined Patent Publication No. 2012-95019

However, in Patent Document 1, it is necessary to select in advance the composition that the photographer wants to shoot with the auto zoom function. Therefore, it is necessary for the photographer to change the setting every time the shooting scene or the state of the subject (movement, number, etc.) changes. Further, if the composition setting is not appropriate for the scene to be shot, it may not be possible to shoot the subject with an appropriate size. For example, if the composition is set to "face" in a scene where a moving subject or a distant subject is photographed, the subject may be out of frame. In addition, when the composition is set to "whole body" in a scene where a stationary subject is photographed, the composition of the subject may be too small.

In view of the above problems, it is an object of the present invention to perform auto zoom control with a composition suitable for a scene determined based on subject detection information without the photographer selecting a composition.

The zoom control device according to the present invention includes a subject detection means for detecting a subject image from an image acquired by using a photographing optical system, a setting means for setting a reference size of the subject image used for controlling the zoom magnification, and the above-mentioned setting. It has a control means for controlling the zoom magnification based on the reference size set by the means and the size of the subject image sequentially detected by the subject detection means. The setting means is based on at least one of whether or not the image pickup device whose zoom magnification is controlled by the zoom control device is in the handheld state, whether or not the self-portrait mode is set, and the still time of the subject. Change the standard size.

According to the present invention, the auto zoom control can be performed with a composition suitable for the scene determined based on the detection information of the subject without the photographer selecting the composition.

It is a block diagram which shows the structural example of the digital camera which concerns on embodiment of this invention. It is a figure which illustrated the relationship between the focal length and the focus lens position for each subject distance. It is explanatory drawing of the process which prevents the frame out of a subject (object) from the screen. It is explanatory drawing of the process which prevents the frame out of a subject (person) from the screen. It is explanatory drawing of the process which prevents the size change in the screen of a subject (person). It is a figure explaining composition setting when the subject is a person. It is a flowchart explaining the process flow of an auto zoom function. It is a flowchart explaining the subject search process. It is a figure explaining the search end area of the subject search process. It is a flowchart explaining the subject designation process. It is a flowchart explaining another example of subject designation processing. It is a flowchart explaining the setting process of a reference size. It is a flowchart explaining the calculation process of a screen size. It is a transition diagram explaining the automatic composition determination in composition setting "auto". It is a figure which shows the subject position and the screen size of the outermost periphery. It is a flowchart explaining the calculation process of a reference size in composition setting "auto". It is a flowchart explaining the calculation process of the horizontal subject position ratio. It is a flowchart explaining the calculation process of the vertical subject position ratio. It is a table which shows the number of faces for which the subject position is estimated. It is a flowchart explaining the process of auto zoom control. It is a flowchart explaining a zoom operation. It is a transition diagram explaining the automatic composition determination in composition setting "auto" in 2nd Embodiment.

(First Embodiment)
Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings. FIG. 1 is a block diagram showing a configuration example of a digital camera 100 having an auto-zoom function as an example of an image pickup apparatus provided with a zoom control device according to the present embodiment.

The lens barrel 101 holds a lens group inside the lens barrel 101. The zoom lens 102 adjusts the focal length by moving in the optical axis direction of the lens barrel 101, and optically changes the angle of view (moves the zoom position). The focus lens 103 adjusts the focus by moving in the optical axis direction of the lens barrel 101. The anti-vibration lens (lens for image stabilization) 104 corrects image blur caused by camera shake or the like. The aperture and shutter 105 for adjusting the amount of light are used for exposure control. In the present embodiment, the digital camera 100 is an imaging device in which the lens barrel 101 and the camera body are integrally configured, but the digital camera 100 is not limited thereto. This embodiment can also be applied to an imaging system including a camera body and an interchangeable lens that can be attached to and detached from the camera body.

The image sensor 106 receives the light that has passed through the lens barrel 101 and converts the subject image into an electric signal by photoelectric conversion to generate an image pickup signal. The image sensor 106 is a CCD (charge-coupled device) type or CMOS (complementary metal oxide semiconductor) type image sensor or the like. The image pickup signal from the image pickup element 106 is input to the image processing circuit 107, and various processes such as pixel interpolation processing and color conversion processing are performed. The image data after various processing is stored in the image memory 108. The image memory 108 is a storage device such as a DRAM (Dynamic Random Access Memory) or a SRAM (Static Random Access Memory).

The display unit 109 is configured to include a TFT type LCD (thin film transistor drive type liquid crystal display) or the like, and displays a photographed image (image data) or specific information (for example, photographed information). By displaying information such as a live view related to a captured image, it is possible to provide an electronic viewfinder (EVF) function for the photographer to adjust the angle of view.

The aperture shutter drive unit 110 calculates an exposure control value (aperture value and shutter speed) based on the brightness information obtained by image processing in the image processing circuit 107, and drives the aperture and shutter 105 based on the calculation result. .. As a result, automatic exposure (AE) control is performed. The anti-vibration lens drive unit 111 calculates the amount of vibration applied to the digital camera 100 based on the vibration detection information obtained by an angular velocity sensor such as a gyro sensor. The anti-vibration lens 104 is driven so as to cancel (reduce) the amount of vibration applied to the digital camera 100 according to the calculation result.

The focus lens driving unit 112 drives the focus lens 103. In the present embodiment, the digital camera 100 performs automatic focus adjustment (AF) control by a contrast method. That is, the focus lens driving unit 112 drives the focus lens 103 so as to focus on the subject based on the focus adjustment information (contrast evaluation value) of the photographing optical system obtained by the image processing in the image processing circuit 107. However, the AF control is not limited to this, and the phase difference AF method may be used as the AF control other than the contrast method, and AF control is performed by a plurality of methods such as a combination of the contrast method and another method. It may be configured.

The zoom lens driving unit 113 drives the zoom lens 102 according to the zoom operation instruction. The operation unit 117 includes a zoom lever, a zoom button, and the like as a zoom operation member for the photographer to instruct the camera to zoom. The system control unit 114 detects the operation amount and operation direction of the zoom operation member used for the zoom instruction operation, calculates the zoom drive speed and the zoom drive direction, and moves the zoom lens 102 along the optical axis according to the calculation result. Take control.

The image data generated by the shooting operation is sent to the recording unit 116 via the interface (I / F) unit 115 and recorded. The image data is recorded in an external recording medium, non-volatile memory 118, or both. The external recording medium is a memory card or the like that is used by being attached to the digital camera 100. The non-volatile memory 118 is a storage medium built in the digital camera 100. In addition to the program data and image data, the memory 118 stores information such as setting information of the digital camera 100 and a zoomed-in position in the auto zoom function described later.
In addition to the zoom operation member, the operation unit 117 includes a release switch for instructing the start of shooting, an auto zoom operation switch for instructing the start and end of the auto zoom function, and the like. The signal from the operation unit 117 is sent to the system control unit 114.

The system control unit 114 includes an arithmetic unit such as a CPU (central processing unit). The system control unit 114 controls the entire digital camera 100 by sending control commands to each unit according to the operation of the photographer. The system control unit 114 executes various control programs stored in the memory 118, for example, a program for controlling the image sensor 106, AE / AF control, zoom control (including auto zoom processing), and the like.

In order to maintain the in-focus state when the angle of view is changed by the optical zoom, when the lens barrel 101 is a rear focus type, it is necessary to move the focus lens 103 to an appropriate focus position according to the position of the zoom lens 102. .. Such control is called computer zoom (CZ) control, and is performed by the CZ control unit 119 in the system control unit 114. FIG. 2 is a diagram illustrating the relationship between the focal length of the zoom lens and the focus position for each subject distance. In FIG. 2, the relationship between the focal length of the zoom lens and the in-focus focus position is shown as a graph as a data table showing each distance to the subject. In this embodiment, this table is referred to as a focus cam table. In FIG. 2, the horizontal axis indicates the focal length corresponding to the zoom position, and the vertical axis indicates the focus position. Next to each graph line, the distance from the digital camera 100 to the subject (subject distance) is illustrated.

The system control unit 114 controls the focus lens driving unit 112 when performing AF control, and moves the focus lens 103 within a predetermined range to perform a scanning operation. The focus position, which is the in-focus position, is detected by a known method using the contrast evaluation value or the like obtained during the scanning operation. The subject distance is measured by referring to the focus cam table using the zoom position and the focus position at that time.

Next, the control related to the auto zoom function in the system control unit 114 will be described. As shown in FIG. 1, the system control unit 114 includes a CZ control unit 119, an electronic zoom control unit 120, an auto zoom control unit (hereinafter referred to as an AZ control unit) 122, and a subject detection unit 123.

The digital camera 100 has an optical zoom function and an electronic zoom function, and the CZ control unit 119 and the zoom lens drive unit 113 are in charge of controlling the optical zoom. The CZ control unit 119 detects the zoom position of the zoom lens 102 at predetermined control cycles during the zoom operation. Then, the CZ control unit 119 controls to drive the focus lens 103 so as to follow the focus cam table at the subject distance measured by the AF control according to the detected zoom position. This makes it possible to perform the optical zoom operation while maintaining the focused state.

On the other hand, the electronic zoom control unit 120 and the image memory 108 are in charge of controlling the electronic zoom. The electronic zoom control unit 120 realizes the electronic zoom function by cutting out the data in the target area from the image data transferred to the image memory 108. Further, the electronic zoom control unit 120 realizes a smooth electronic zoom display by displaying the image on the display unit 109 while gradually increasing the range to be cut out at the frame rate cycle of the image captured by the image sensor 106.
The subject detection unit 123 sequentially detects a desired subject area from the image data stored in the image memory 108. In the present embodiment, a subject detection method (face detection process, color detection process) for detecting a subject (face or object such as a person) based on face information or color information included in image data will be described.

The face detection process is a process of detecting a face region existing in image data by a known algorithm. For example, the subject detection unit 123 extracts a feature amount from a square-shaped partial region on the image data and compares the feature amount with a face feature amount prepared in advance. Then, when the correlation value between the two exceeds a predetermined threshold value, the subject detection unit 123 determines that the partial region is the face region. By repeating this determination process while changing the combination of the size, the arrangement position, and the arrangement angle of the partial area, various face areas existing in the image data can be detected.

In the color detection process, a process of storing the color information of the subject area designated according to the subject designation method described later as a feature color is executed. The color detection process is performed when the subject to be detected is an object (a "thing" other than a person). As the color information, RGB signals, luminance signals (Y signals), color difference (RY, BY) signals, etc., which are output signals from the image processing circuit 107, are used. At the time of subject detection, the subject detection unit 123 divides the image data into a plurality of partial regions and calculates the average value of the brightness and the color difference for each partial region. Further, the subject detection unit 123 compares the feature color information stored in advance with the color information of each region at the time of subject detection, and sets a partial region having a difference between the luminance and the color difference by a predetermined amount or less as a candidate for the subject region. A process is performed in which a group of adjacent partial regions in the candidate of the subject region is set as the same color region, and a region in which the same color region is in a predetermined size range is set as the final subject region.

The subject detection unit 123 calculates the size of the subject area on the image data by using the subject distance information measured by the CZ control unit 119 and the focal length information of the zoom lens 102 together with the face information and the color information. .. The posture detection unit 124 detects the posture (for example, normal position / above the grip / below the grip) of the digital camera 100 based on the information of the acceleration sensor. The shake detection unit 125 detects the shake state of the digital camera 100 by determination based on the angular velocity information and the like by the gyro sensor. The shaking detection unit 125 determines that the camera is in a handheld state when the amount of vibration (detection value) applied to the gyro sensor or the like is equal to or more than a predetermined amount (threshold value), and fixes it to a tripod or the like when the amount is less than the predetermined amount. It is determined that the state has been set. The acceleration sensor and the gyro sensor used for the attitude detection and the shaking detection may be configured to be also used as the sensor of the detection unit for acquiring the control information of the vibration isolation lens drive unit 111.

Next, the outline of the auto zoom function in the present embodiment and the AZ control unit 122 will be described. With a camera that does not have an auto-zoom function, the following operations are required when the subject moves and frames out while the photographer is framing in the telephoto state and waiting for a shutter chance.
First, the photographer performs a zoom-out operation by operating the zoom operation member, and then searches for the subject. After searching for the subject, the photographer adjusts the angle of view by performing a zoom operation until the desired angle of view is reached again. Further, even when the size of the subject image changes due to the movement of the subject, the photographer needs to operate the zoom operation member to adjust the size of the subject image.

On the other hand, in a camera equipped with an auto-zoom function, the photographer may set the auto-zoom function and then perform an operation of specifying a subject on a touch panel or the like to specify the subject to be photographed. When the auto-zoom function is set, the zoom operation is automatically performed so that the specified subject fits in a predetermined size near the center of the image. In addition to touch panel operation, the subject can be specified by specifying the subject near the center of the screen when the photographer operates a specific button, or by automatically selecting the main subject from the subjects detected by the camera. There is a method to select.

The subject detection unit 123 calculates the position and size of the subject area designated from the image memory 108 on the image data. By continuously performing this process on the sampled image data each time the image is displayed as a live view, it is possible to track the movement of the subject. When the image of the subject being tracked is detected in the zoom-out area described later, or when the image of the detected subject becomes larger than a predetermined size, the AZ control unit 122 starts the zoom-out operation. That is, the AZ control unit 122 instructs the CZ control unit 119 or the electronic zoom control unit 120 to zoom out in the wide-angle direction. When the subject is detected in the zoom-in area and the subject image is within the range of a predetermined size, the zoom-in operation is performed to the telephoto side. By such processing, the photographer does not have to worry about the zoom operation, and only needs to move the camera so that the image of the desired subject is contained in the screen. Even if the subject is about to be out of frame, the zoom magnification is automatically changed, so the angle of view can be adjusted more easily.

Next, the conditions for starting the zoom-out operation and the zoom-in operation will be described with reference to FIGS. 3 to 5. FIG. 3 is an explanatory diagram of a process for preventing a subject (object) from being framed out of the screen. FIG. 4 is an explanatory diagram of a process for preventing a subject (person) from being framed out of the screen.

In FIGS. 3 and 4, the frame 300a and the frame 300b are the first tracking frames (hereinafter referred to as mono tracking frames) for tracking the subject (object), and the frames 400a to 400 track the subject (person's face). This is the second tracking frame (hereinafter referred to as the face tracking frame). In the following, when the subject is applicable to both a person and an object, the mono tracking frame and the face tracking frame may be collectively referred to as a subject tracking frame. The subject tracking frame is displayed so as to surround the subject image on the EVF screen (electronic viewfinder screen) of the display unit 109 so that the subject designated by the photographer can be seen. The position and size of the subject tracking frame on the screen are sequentially calculated by the subject detection unit 123 based on the face information and the color information, and are updated at the frame rate cycle.

A process for preventing the subject (airplane) from being framed out of the screen will be described with reference to FIG. In FIG. 3A, the zoom-out area ZO (hereinafter referred to as the ZO area) indicates an area outside the predetermined ratio with respect to the entire angle of view (entire screen) displayed by the EVF. For example, assume that the center point of the screen is 0%, the entire screen is 100%, and the position that is 80% of the entire screen is set as the boundary of the ZO area. In this case, 80 to 100% of the entire screen is the ZO area. When a part of the mono tracking frame 300a enters the ZO region, the AZ control unit 122 controls to start the zoom-out operation. Further, the AZ control unit 122 stores the zoom magnification (corresponding to the zoom-in angle of view) before the zoom movement in the memory 118. The target zoom magnification and zoom speed during the zoom-out operation are preset according to the size and moving speed of the subject image. Further, the target zoom magnification and the zoom speed may be appropriately calculated according to the size and moving speed of the subject image. The zoom-out operation is executed according to the target zoom magnification and zoom speed. As a result, it is possible to effectively prevent the subject from being framed out.

FIG. 3B shows the angle of view when the zoom-out operation corresponding to a predetermined amount of zoom change is performed from the angle of view of FIG. 3A. The zoom-in area ZI (hereinafter referred to as ZI area) shown in FIG. 3B shows an area inside a predetermined ratio with respect to the zoom-in angle of view (angle of view before zoom-out) 301 in the subject search state. .. For example, assume that the center point of the screen is 0%, the zoom-in angle of view 301 is 100%, and the position that is 70% of the zoom-in angle of view 301 is set as the boundary of the ZI region. In this case, the area of 0 to 70% of the entire zoom-in angle of view 301 is the ZI area. At this time, for example, when the zoom-out magnification is 1/2 times, the zoom-in angle of view 301 is 50% of the entire screen. Therefore, it can be said that the boundary of the ZI region is 70% × (1/2) = 35% with respect to the entire screen, and the ZI region is an region of 0 to 35% with respect to the entire screen. When the photographer changes the direction of the camera so that the mono tracking frame 300b fits inside the ZI region, the AZ control unit 122 controls to start the zoom-in operation.

Next, with reference to FIG. 4, a process for preventing the subject (person) from being framed out of the screen will be described. When the subject is a person, the zoom-out operation starts when a part of the face tracking frame enters the ZO area, and the zoom-in operation is performed when the face tracking frame fits inside the ZI area. When the subject is a person, unlike the case of an object, the moving direction of the subject can be predicted to some extent, so that the ZO region and the ZI region are set according to the region of the predicted moving direction. Further, in handheld shooting, the subject may be out of frame due to the influence of camera shake or the like. However, when the subject is out of the frame due to camera shake or the like, it is possible to reframe in by the action of the photographer trying to frame in the subject. Here, when the ZO area is set at the upper part of the screen, the face tracking frame may enter the ZO area even when the person is placed near the center and photographed, and the person may unintentionally zoom out. Therefore, when the subject is a person and is in a handheld state (a state in which the photographer holds the camera), the ZO area is not set at the upper part of the screen in consideration of the framing operation of the photographer.

As described above, in the present embodiment, when the subject detection unit 123 detects the face, the AZ control unit 122 changes the ZO region and the ZI region according to the posture and the shaking state of the camera. The posture of the camera is detected by the posture detection unit 124, and the shaking state is determined from the detection result by the shaking detection unit 125. The detection result by the shaking detection unit 125 is a detection result of whether or not the camera is in a handheld state. Hereinafter, a specific description will be given with reference to FIGS. 4A to 4C.

FIG. 4A shows a ZO area and a ZI area that are set when the photographer holds the camera in the normal position by hand. In such a shooting scene, when the subject moves in the horizontal direction to frame out, the position of the subject in the screen moves in the horizontal direction (longitudinal direction) with respect to the screen at the normal position. Therefore, both the ZO region and the ZI region are arranged in a vertical band shape in the vertical direction (short direction) with respect to the screen at the normal position. The ZO area is set in the shape of a vertical band located near both ends in the horizontal direction on a horizontally long rectangular screen, and the ZI area is set in the shape of a vertical band located in the center of the screen. In this case, when the face tracking frame 400a enters the ZO region, the AZ control unit 122 determines the start of zoom-out and controls the zoom-out operation corresponding to a predetermined zoom magnification. Further, when the face tracking frame 400b is included in the ZI region, the AZ control unit 122 determines the start of zoom-in, and controls the zoom-in operation corresponding to a predetermined zoom magnification up to the zoom return position. By setting the ZO area and the ZI area in this way, it is possible to effectively prevent the flameout of the subject (person).

FIG. 4B shows the ZO area and the ZI area that are set when the camera posture is changed in the same shooting scene and the photographer holds the camera in the vertical position under the grip or on the grip. .. In this case, the ZO region and the ZI region are arranged in a vertical band shape in the vertical direction (longitudinal direction) with respect to the screen in the vertical position. That is, the ZO area is set in the shape of a vertical band located near both ends in the horizontal direction on a vertically long rectangular screen, and the ZI area is set in the shape of a vertical band located in the center of the screen. In this case, when the face tracking frame 400c enters the ZO region, the AZ control unit 122 determines the start of zoom-out and controls the zoom-out operation corresponding to a predetermined zoom magnification. Further, when the face tracking frame 400d is included in the ZI region, the AZ control unit 122 determines the start of zoom-in, and controls the zoom-in operation corresponding to a predetermined zoom magnification up to the zoom return position. By setting the ZO region and the ZI region in this way, it is possible to detect the movement of the subject in the horizontal direction and effectively prevent flameout.

FIG. 4C shows a ZO region and a ZI region that are set when the detection state of the shaking detection unit 125 is a fixed state. The fixed state is a state in which the camera is fixed by a tripod or the like, and there is no possibility of frame out due to camera shake. Furthermore, if the subject is not framed in near the center of the screen and the subject is zoomed in, the operation may cause the subject to be out of frame. Therefore, the ZO area is set on the entire peripheral portion of the screen, and the ZI area is set inside the zoom-in angle of view. That is, the ZO area is set in the shape of a rectangular band located near the edges in the vertical and horizontal directions of the screen, and the ZI area is set in the shape of a rectangle located in the center of the screen. In this case, when the face tracking frame 400e enters the ZO region, the AZ control unit 122 determines the start of zoom-out and controls the zoom-out operation corresponding to a predetermined zoom magnification. Further, when the face tracking frame 400f is included in the ZI region, the AZ control unit 122 determines the start of zoom-in and controls the zoom-in operation corresponding to a predetermined zoom magnification up to the zoom return position.

As described above, in the present embodiment, each range of the ZO region and the ZI region is dynamically changed according to the change in the posture of the camera and the shooting state (handheld state / fixed state). As a result, it is possible to effectively prevent the subject from being out of frame while preventing malfunction due to camera shake or the like. The ZO area or the ZI area may be changed according to either the posture of the camera or the shooting state (handheld state / fixed state), or only one of the ZO area and the ZI area may be changed. Good.

Next, a zoom operation for keeping the ratio of the subject image to the screen within a predetermined range will be described. In the present embodiment, when the size of the detected subject image changes more than a predetermined time of the reference size, the zoom operation is automatically performed to control the size of the subject image to be kept within a predetermined range from the reference size. (Size retention control) is performed. FIG. 5 is a diagram illustrating a process for maintaining the size of a subject (person) image on the screen. FIG. 6 is an explanatory diagram of a composition setting of a subject (person).

First, with reference to FIG. 6, the setting of the range (composition) within which the subject to be tracked is included in the screen will be described. FIG. 6A exemplifies the screen display when the composition setting is set to “manual”. When "manual" is set, the photographer performs a manual zoom operation by operating the zoom lever while looking at the image of a person on the screen to change the size of the face to be tracked. The size of the subject image at that time is stored in the memory 118 as a reference size. FIG. 6B exemplifies the screen display when the composition setting is set to “face”. When the composition of the "face" is set, the size of the face that fits on the screen is calculated as the reference size according to the posture of the camera and the direction of the face, and is stored in the memory 118. Similarly, FIG. 6 (C) illustrates the screen display when the composition setting is set to "upper body", and FIG. 6 (D) shows the screen display when the composition setting is set to "whole body". Illustrate. The reference size is calculated so as to be the size set on the screen, and stored in the memory 118.

FIG. 6E illustrates the screen display when the composition setting is set to “Auto”. When "auto" is set, the AZ control unit 122 determines an appropriate composition based on the subject position, subject size, number of subjects, camera posture, and the like on the screen. Further, the reference size is calculated and stored in the memory 118 so as to have the determined composition. The method of calculating the reference size will be described later with reference to FIG.

The photographer can perform an operation of changing the composition setting by operating the left and right buttons of the operation unit 117 or selecting on the setting menu from the shooting screen. When the composition setting change operation is performed, the AZ control unit 122 updates the composition setting information stored in the memory 118. Note that FIG. 6 shows an example of setting each composition of "manual", "face", "upper body", "whole body", and "auto" when the subject is a person, but the composition setting is limited to this. It's not something. Only a part of the five illustrated compositions may be set, or other compositions may be included. When the subject is an object, for example, "manual", "large", "medium", "small", and "auto" compositions may be set.

With reference to FIG. 5, the zoom operation for keeping the ratio of the subject image to the screen within a predetermined range will be described by taking the case where the composition setting is “manual” as an example. FIGS. 5A to 5C exemplify a zoom operation automatically performed by the camera when a subject (person) approaches the camera. This is a zoom-out operation for keeping the ratio of the subject image to the screen within a predetermined ratio. In FIG. 5, the face tracking frames 500a to ff are displayed so as to surround the face area as a feature area of the person who is the subject. Therefore, here, the size of the face tracking frame will be described as the subject size.

FIG. 5A shows the angle of view when the subject is designated according to the subject designation method described later. The size of the face tracking frame 500a when the subject is specified is stored in the memory 118 as a reference subject size (reference size). FIG. 5B shows the angle of view when the subject approaches the camera without changing the zoom magnification from the state of FIG. 5A. For example, a size that is 150% of the size of the face tracking frame 500a, which is the reference subject size, is set as the start size of the zoom-out operation. When the relationship of the subject tracking frame (face tracking frame) is "face tracking frame 500b> face tracking frame 500a x 150%", that is, the tracking frame has changed by exceeding a predetermined amount of change with respect to the reference size. Occasionally, the AZ control unit 122 determines the start of the zoom-out operation.

FIG. 5C shows an angle of view zoomed out from the angle of view 501 of FIG. 5B by a predetermined zoom magnification, and the face tracking frame 500c. Here, the predetermined zoom magnification is set to 1 / 1.5 times in consideration of the rate of change (150%) of the face tracking frame size with respect to the reference subject size when the zoom-out operation is started. After that, when the subject approaches the camera further, the subject image can be kept within a predetermined ratio by further zooming out to the wide-angle side. Therefore, the photographer can concentrate only on the operation of the release switch.

On the other hand, FIGS. 5 (D) to 5 (F) illustrate the zoom operation automatically performed by the camera when the person who is the subject moves away from the camera. This is a zoom-in operation for keeping the ratio of the subject image to the screen within a predetermined range. FIG. 5D shows the angle of view when the subject is designated according to the subject designation method described later. The size of the face tracking frame 500d when the subject is specified is stored in the memory 118 as the reference subject size (when the composition setting is "manual").

FIG. 5 (E) shows the angle of view when the subject moves away from the camera without changing the zoom magnification from the state of FIG. 5 (D). For example, a size that is 50% of the size of the face tracking frame 500d, which is the reference subject size, is set as the start size of the zoom-in operation. When the relationship of the face tracking frame is "face tracking frame 500e <face tracking frame 500d x 50%", it is determined that the tracking frame has changed by exceeding a predetermined amount of change with respect to the reference size. When this determination condition is satisfied and the face tracking frame 500e is included in the ZI region, the AZ control unit 122 determines the start of the zoom-in operation. Here, the ZI region is set inside the angle of view 502 zoomed in by a predetermined zoom magnification with respect to the angle of view of FIG. 5 (E).

FIG. 5 (F) shows an angle of view (corresponding to the angle of view 502) zoomed in by a predetermined zoom magnification from the angle of view of FIG. 5 (E) and the face tracking frame 500f. Here, the predetermined zoom magnification is set to 1 / 0.5 times in consideration of the rate of change (50%) of the face tracking frame size with respect to the reference subject size when the zoom-in operation is started.

FIG. 3 describes a process for preventing flameout when the subject is an object and FIG. 4 shows a process for preventing frame-out when the subject is a person. Further, in FIG. 5, when the subject is a person, the process of keeping the ratio of the size of the subject image to the screen within a predetermined range has been described. Even when the subject to be tracked is an object, the zoom operation start determination for controlling the size retention of the subject image shown in FIG. 5 may be performed as in the case where the subject is a person. Further, even when the composition setting is other than "manual", the zoom operation is automatically performed according to each reference size as in FIG.

Next, the processing of the auto zoom function will be described with reference to FIGS. 7 to 21. FIG. 7 is a flowchart showing an overall processing example of the auto zoom function. Unless otherwise specified, the following auto-zoom function shall be performed based on the command of the system control unit 114 (AZ control unit 122).
First, in S700, the AZ control unit 122 determines the operation state of the auto zoom operation switch of the operation unit 117. When the auto zoom operation switch is pressed, the process proceeds to S701, and when the switch is not pressed, the determination process of S700 is repeated. The subject search process is executed in S701, and the subject designation process in S702, the basic size setting process in S703, and the auto zoom control in S704 are sequentially executed.

Immediately after the processing of the auto zoom function is started, or when the subject search button of the operation unit 117 is pressed (yes in S705), the subject search processing of S701 is executed. The subject search process executed in the step of S701 will be described with reference to FIG. When the subject to be photographed is far away from the camera, even if the subject image exists on the screen, if the feature area of the subject is too small, the subject detection unit 123 may not be able to detect the subject. In such a scene, the auto zoom function based on the subject information cannot be executed. Therefore, a subject search process is executed in which the subject is zoomed in while detecting the subject up to a predetermined focal length.

In S800, the determination process is performed whether the subject search button is pressed during the auto zoom, not immediately after the start of the auto zoom or immediately after the start of the auto zoom. In S800, in the case of the former positive determination result (yes), the process proceeds to S801, and in the case of the negative determination result (no), the process proceeds to S803. In S801, a process of acquiring the subject search end focal length (hereinafter, referred to as search end focal length) is executed. In the present embodiment, the focal length at the end of the search is 85 mm in the 35 mm format immediately after the start of the auto zoom. This focal length is the focal length of a standard lens that is generally considered to be suitable for shooting a person, and even if there is no subject (person) or the face cannot be detected (in a backward facing state, etc.), after the search is completed. It is possible to wait for subject detection at an appropriate angle of view. However, the focal length is not limited to 85 mm, and other focal lengths (for example, 50 mm, 100 mm, etc.) may be used. Further, the search end focal length may be changed by the photographer from the menu setting, or the search end focal length may be changed according to the shooting mode or the subject to be photographed. When proceeding from S801 to S802, the focal length at the start of the mode is compared with the search end focal length, and it is determined whether the focal length is on the wide-angle side or the telephoto side of the search end focal length. When the focal length at the start of auto zoom is on the telephoto side of the focal length at the end of the search (no in S802), the subject search process ends. If the focal length at the start of auto zoom is wider than the focal length at the end of the search (yes in S802), the process proceeds to S805.

On the other hand, when it is determined in S800 that the subject search is performed by operating the subject search button, in S803, a process of acquiring the search end focal length is executed. In the present embodiment, the search end focal length by the operation of the subject search button is set to the focal length on the telephoto side by 2.5 times the focal length when the operation is executed. However, this magnification is an example, and other magnifications may be used. Further, the search end focal length may be changed by the photographer from the menu setting, or the search end focal length may be changed according to the shooting mode or the subject to be photographed. Proceeding from S803 to S804, the search end focal length acquired in S803 is compared with the focal length at the telephoto end (telephoto end), and it is determined whether the search end focal length is on the wide-angle side or the telephoto end side of the focal length at the telephoto end. .. When the search end focal length is on the telephoto side of the focal length at the telephoto end, the subject search process ends. If the search end focal length is on the wide-angle side of the focal length at the telephoto end, the process proceeds to S805. In S805, the AZ control unit 122 sets the zoom change amount corresponding to the search end focal length acquired in S801 or S803 to the CZ control unit 119 or the electronic zoom control unit 120, and instructs the CZ control unit 119 or the electronic zoom control unit 120 to perform the zoom-in operation.

Next, in S806, it is determined whether or not the current focal length has reached the search end focal length. When the current focal length reaches the search end focal length, the process proceeds to S809, the zoom-in operation is stopped, and then the subject search process ends. If it is determined in S806 that the current focal length has not reached the search end focal length, the process proceeds to S807. S807 is a process for determining whether or not a subject is detected by the subject detection unit 123. If it is determined that the subject has been detected in S807, the process proceeds to S808, and if it is not determined that the subject has been detected, the process returns to S806. In S808, it is determined whether or not the position where the subject is detected is within a predetermined search end area on the screen. The predetermined search end area shall be determined by a threshold value (position determination reference value) set in advance according to the subject. As a result of the determination, if the detection position of the subject detected in S807 is within the predetermined search end area on the screen, the process proceeds to S809, the zoom-in operation is stopped, and the subject search process is terminated. If it is determined in S808 that the detection position of the subject is not within the predetermined search end area on the screen, the process returns to S806 and the determination processing of S806 to S808 is continued.

Here, with reference to FIG. 9, the predetermined search end region described in S808 will be described. When shooting a specific subject (person or object), the subject to be shot is often placed near the center of the screen. Further, when a subject that is not the subject to be photographed is detected around the screen and the subject search process is terminated, the subject search process may be terminated before the subject to be photographed is detected. In this case, the auto zoom function cannot be executed as the photographer intended. Therefore, in the present embodiment, the search end area as the determination area is set so that the subject search process is terminated only when the subject is detected in the area near the center of the screen. FIG. 9A illustrates a search end area 900a that is set when the subject is an object (airplane). In the case of an object, a predetermined ratio, for example, 70% from the center of the screen is set as the search end area of the subject in any of the up, down, left, and right directions of the screen. On the other hand, FIG. 9B illustrates a search end area 900b that is set when the subject is a person. In the case of a person, the search ends so that the area at the top of the screen includes a range of 100% from the center of the screen, and the other directions (horizontal direction, downward) include a predetermined ratio from the center of the screen, for example, 70%. The area is set. When the subject is a person, the face tracking frame 902 of the subject to be photographed may be arranged at the upper part of the screen. Therefore, even when the subject is detected at the upper part of the screen, the search end area is set so that the subject search process ends.

Next, with reference to FIG. 10, the subject designation process shown in step S702 in FIG. 7 will be described. FIG. 10A is a flowchart showing a processing example in which the user specifies a desired subject using the touch panel constituting the operation unit 117. In this example, it is assumed that the user touches and specifies the subject image displayed on the display unit 109. In S1000, the AZ control unit 122 determines whether or not the touch panel is pressed. If the touch panel is pressed, the process proceeds to S1001, and if the touch panel is not pressed, the determination process of S1000 is repeated.

In S1001, the AZ control unit 122 acquires information on the position (touch position) touched by the user. In the next S1002, the AZ control unit 122 notifies the subject detection unit 123 of the touch position, and the subject detection unit 123 detects the face near the touch position. When a face is detected near the touch position in S1002, it is determined that the main subject is a person, and the process proceeds to S1003. If no face is detected near the touch position in S1002, it is determined that the main subject is an object other than a person, and the process proceeds to S1004.

In S1003, the AZ control unit 122 controls to store the face information of the person to be automatically tracked in the memory 118. Specific face information includes information such as the size of the face when the subject is specified, the detection position of the face, and the orientation of the face. Further, in a camera having a face recognition function, identification information such as an authentication ID is also stored in the memory 118. In S1004, the AZ control unit 122 controls to store the feature color near the touch position in the memory 118 as the color information to be automatically tracked. Specific color information includes information such as a feature color and its brightness when a subject is specified, a color difference value, a size of the same color region, and a position of the center of gravity of the same color region. In the following description, face information and color information are collectively referred to as subject information (including subject size, subject detection position, etc.).

After the subject information is stored in the memory 118 in S1003 or S1004, the process proceeds to S1005 and the subject tracking frame is displayed. In S1005, the AZ control unit 122 controls the display unit 109 to display a subject tracking frame (mono tracking frame or face tracking frame) having a size corresponding to the subject size, centering on the subject detection position. After that, the subject designation process ends.
In this way, in the system control unit 114, the subject detection unit 123 detects the subject at the position designated by the photographer on the display unit 109 or in the vicinity of the designated position. Then, the AZ control unit 122 causes the display unit 109 to display the subject tracking frame. According to the processing example of FIG. 10A, the photographer can easily specify the subject to be tracked by an intuitive method. However, the method of designating the subject is not limited to this method. Another method will be described with reference to FIG. 10 (B).

FIG. 10B is a flowchart showing a processing example when the user specifies a subject by using a switch (subject specification switch) different from the auto zoom operation switch as an operation member constituting the operation unit 117. is there. First, in S1006, the AZ control unit 122 displays a frame as a guide for specifying a subject near the center of the screen of the display unit 109. The photographer adjusts the orientation of the camera so that the image of the subject to be tracked is placed near the center of the screen using this frame as a guide. In the next S1007, the AZ control unit 122 determines whether or not the subject designation switch has been pressed. When the subject designation switch is pressed, the process proceeds to S1008, and when the switch is not pressed, the determination process of S1007 is repeated in the waiting state.

In S1008, the subject detection unit 123 detects the face near the center of the screen. When a face is detected near the center of the screen, it is determined that the main subject is a person, and the process proceeds to S1009. On the other hand, when the face is not detected near the center of the screen, it is determined that the main subject is an object other than a person, and the process proceeds to S1010. In S1009, the same processing as in S1003 of FIG. 10A is performed, and in S1010, the same processing as in S1004 of FIG. 10A is performed, so the description thereof will be omitted. After the subject information is stored in the memory 118 in S1009 or S1010, the process proceeds to S1011 and the subject tracking frame (mono tracking frame or face tracking frame) is displayed, and the subject designation process ends.

As described above, in the processing example of FIG. 10B, the subject detection unit 123 detects the subject at the screen center position or the vicinity of the screen center position of the display unit 109. Then, the AZ control unit 122 causes the display unit 109 to display a subject tracking frame indicating the position of the subject. Even in a camera that is not equipped with an operating member such as a touch panel, the user can easily specify the subject.

FIG. 11 is a flowchart showing a processing example of automatically selecting a subject to be tracked from the faces detected when the auto zoom operation switch constituting the operation unit 117 is pressed. First, in S1100, the subject detection unit 123 performs face detection on the entire screen and determines whether or not a person's face has been detected. If even one face is detected on the entire screen, it is determined that the main subject is a person, and the process proceeds to S1101. If no face is detected on the screen, the process proceeds to S1105.

In S1101, the subject detection unit 123 selects the face as the main face when there is only one person with the detected face. When there are a plurality of detected faces, the subject detection unit 123 selects the main face to be the subject to be tracked from the faces. As a criterion for selecting the main face, for example, there is a method of preferentially selecting a face whose face detection position is closer to the center of the screen. Further, when there are a plurality of faces at the same position, there is a method of selecting a face having a larger size as the main face. Further, in the case of a camera having a face recognition function, if there is a face registered for authentication, there is a method of giving priority to that face as the main face.

Proceeding from S1101 to S1102, the AZ control unit 122 stores the face information of the selected main face in the memory 118, and then proceeds to S1103 to display the face tracking frame on the screen. In the next S1104, the face change determination is performed. That is, when a plurality of faces are detected, the automatically selected main face is not always the face intended by the photographer. Therefore, the photographer can change the main face. At this time, when the photographer presses a switch (auto zoom operation switch, etc.) of the operation unit 117 to specify a desired face, another face that is not selected as the main face from the faces for which the face tracking frame is detected is specified. The process of changing the main face is performed. When the main face is changed in S1104, the process returns to S1102 again to update the face information to be stored in the memory 118. Further, in S1103, the face tracking frame is changed to the size and detection position of the newly selected main face.

On the other hand, when shifting from S1100 to S1105, it is determined that the main subject is an object other than a person, and the AZ control unit 122 stores the feature color near the center of the screen as color information to be automatically tracked in the memory 118. I do. In the next S1106, the subject designation process ends after the mono tracking frame is displayed on the screen. The basic processing in S1102, S1105, and S1103 (S1106) of FIG. 11 is the same as that of S1003, S1004, and S1005 of FIG. 10 (A), respectively.

As described above, in the processing example of FIG. 11, the system control unit 114 causes the subject detection unit 123 to detect the face on the entire screen of the display unit 109. When a plurality of faces are detected, the AZ control unit 122 controls the display unit 109 to display a subject tracking frame indicating the position of the first face as a subject from the plurality of faces. Further, the AZ control unit 122 controls the display unit 109 to display the subject tracking frame indicating the position of the second face when the first face as the subject is changed to the second face. The photographer can easily specify a desired subject with a smaller number of operations.

Regarding the subject designation process, in the case of a camera having both the touch panel constituting the operation unit 117 and the subject designation switch, both the method of FIG. 10 (A) and the method of FIG. 10 (B) are applied at the same time. You may. Further, after the determination of the subject change in S706 shown in FIG. 7 is made, the subject is designated by the method of FIG. 10A or FIG. 10B, and after the subject search process of S701, FIG. You can also specify the subject by the method. The subject designation process to be applied may be changed according to the process flow. When the subject designation process (S702 in FIG. 7) is completed, the process proceeds to S703 in FIG. 7 to execute a process of setting the size of the subject to be tracked as a reference size.

The reference size setting process of S703 shown in FIG. 7 will be described with reference to FIGS. 12 to 19. FIG. 12 is a flowchart showing an example of setting a reference size when the subject is a person. First, in S1200, the AZ control unit 122 determines whether or not the composition setting stored in the memory 118 is the above-mentioned "manual". If the composition setting is "manual", the process proceeds to S1201, and if the composition setting is other than "manual", the process proceeds to S1203.

In S1201, the AZ control unit 122 determines the subject detection. The AZ control unit 122 determines in the subject detection unit 123 whether or not the face is detected when the subject is a person, and whether or not the same feature color is detected when the subject is an object. The determination process of S1201 is repeated until the subject is detected, and when the subject is detected, the process proceeds to S1202. In S1202, the AZ control unit 122 sets the size of the detected subject image as a reference size, stores it in the memory 118, and ends the process. As a result, when the composition setting is "manual", the size (detection size) of the subject image first detected when the reference size setting process is started is stored in the memory 118 as the reference size.

On the other hand, in S1203, the AZ control unit 122 calculates the screen size based on the attitude information of the camera determined by the attitude detection unit 124. This screen size is used for calculating the reference size in S1211, S1704, S1705, S1806, and S1807, which will be described later. The screen size calculation process will be described with reference to FIG. In the present embodiment, a case where the screen size of the still image is set to long side x short side = 640 x 480 and the screen size of the moving image is set to long side x short side = 640 x 360 is illustrated, but the screen size is limited to this. It is not something that is done.

In S1300, the posture detection unit 124 determines whether the photographer is holding the camera in the normal position or in the vertical position. If it is determined that the photographer is holding the camera in the normal position, the process proceeds to S1301, but if the photographer is holding the camera in the vertical position, the process proceeds to S1306. In S1301, a process of setting the long side size 640 to the screen size in the horizontal direction is executed. In the next S1302, a process of determining whether or not the current setting mode is a mode in which a still image and a moving image can be simultaneously photographed (still image / moving image simultaneous shooting mode) is performed. In the still image / moving image simultaneous shooting mode, the photographer can simultaneously take a still image by pressing the release switch of the operation unit 117 during the moving image shooting, or the state of the framing operation immediately before the still image shooting is taken as a moving image. It is a mode that can be recorded automatically. When it is determined in S1302 that the still image / moving image simultaneous shooting mode is used, a process of comparing the short side size of the moving image with the short side size of the still image and setting a smaller size as the vertical screen size is executed. This is to prevent the subject from being framed out of the screen at both the angle of view of the still image and the moving image by performing the auto zoom according to the small screen size. Therefore, in the present embodiment, in the case of the still image / moving image simultaneous shooting mode, the process proceeds to S1304, and 360, which is the short side size of the moving image, is set as the vertical screen size. After that, the screen size calculation process ends.

If it is determined in S1302 that the mode is not the still image / moving image simultaneous shooting mode, the process proceeds to S1303. In S1303, it is determined whether the moving image is being recorded or the still image is being framed. If it is determined that the moving image is being recorded, the process proceeds to S1304, but if it is determined that the moving image is not being recorded, the process proceeds to S1305. In S1305, the short side size of the still image, 480, is set as the vertical screen size.

When it is determined in S1300 that the photographer is holding the photographer in the vertical position, the processes of S1306 to S1310 are executed. The processes of S1306 to S1310 are the processes in which the horizontal screen size and the vertical screen size are set in reverse with respect to the processes of S1301 to S1305, respectively. That is, since it may be read by exchanging "horizontal" and "vertical" in the above description, detailed description will be omitted. In this embodiment, the screen size of the still image is 4: 3 (see FIGS. 15A and 15B), and the screen size of the moving image is 16: 9 (see FIG. 15C). However, the screen size may be changed according to the aspect ratio setting. As a result, it is possible to keep the ratio of the image of a person to the screen constant regardless of the aspect ratio. Further, although the example in which the screen size is different between the still image and the moving image is shown as the still image / moving image simultaneous shooting mode, a form in which a plurality of still images or moving images having different screen sizes are simultaneously shot may be used.

When the screen size is determined in S1203 of FIG. 12, the process proceeds to S1204. When the composition setting stored in the memory 118 is any of "face", "upper body", and "whole body", in S1204 to S1209, the AZ control unit 122 occupies the face image on the screen according to the composition setting. Determine the size ratio (called the face ratio). The face ratio of the present embodiment indicates the ratio of the length of the face tracking frame to the screen in the vertical direction, but it may also be indicated by the ratio of the length in the horizontal direction, the area ratio, or the like. Specifically, for example, in S1204, it is determined whether or not the composition setting is "face", and in the case of the composition of "face" (yes in S1204), the face ratio is determined to be 27% (S1207). If the composition setting is not "face", the process proceeds to S1205, and it is determined whether or not the composition setting is "upper body". If the composition is "upper body" (yes in S1205), the face ratio becomes 16%. It is determined (S1208). If the composition setting is not "upper body", the process proceeds to S1206, and it is determined whether or not the composition setting is "whole body". In the case of "whole body" composition (yes in S1206), the face ratio is determined to be 7%. (S1209). After the face ratio is determined by each step of S1204 to S1209, the process proceeds to S1210.

In S1210, the AZ control unit 122 calculates the reference size by multiplying the face ratio determined in S1204 to S1209 with the vertical screen size calculated in S1203. This is a method of calculating the reference size from the face ratio to the screen size in the vertical direction on the premise of a subject (person) in a standing posture. If the subject detection unit 123 can determine the direction of the face, the direction of the screen size may be determined based on the vertical direction of the face on the screen. Further, after the process of storing the calculated reference size in the memory 118 is executed, the process of setting the reference size is terminated. As a result, when the composition setting is any of "face", "upper body", and "whole body", a size having a predetermined face ratio according to the composition setting with respect to the screen size is set as the reference size.

If the composition setting is not "whole body" in S1206, that is, if it is determined that the composition setting is "auto", the process proceeds to S1211, and the AZ control unit 122 sets the shooting state, the number of subjects, and the movement of the subjects. The appropriate composition is automatically determined according to the above. With reference to FIG. 14, the automatic composition determination and the composition selection process in the composition setting “auto” of S1211 will be described. The composition setting "auto" is a mode in which composition adjustment can be performed more easily by determining the shooting scene and automatically selecting an appropriate composition by the camera (automatic composition selection).

FIG. 14 shows a transition diagram of the composition automatically selected in the present embodiment. In the present embodiment, a composition selection process is performed in which an appropriate composition is selected according to the shooting scene from the five compositions of the upper body 1400, the whole body 1401, the face 1402, the plurality of people 1403, and the manual 1404. In the composition matching operation, when shooting a still image, it is necessary to instantly make the subject image an appropriate size before shooting. On the other hand, when shooting a moving image, it is necessary to keep the subject image at an appropriate size during shooting, such as keeping track of a moving subject or approaching a stationary subject. In addition, when shooting a moving image, image data during zoom-in operation or zoom-out operation due to a change in composition is also recorded, so that an effective image can be taken by changing the composition at an appropriate timing. In this way, the appropriate composition differs between the still image and the moving image. Therefore, a process of changing the composition to be selected depending on the shooting state is executed between the still image framing 1405 and the moving image recording 1406. That is, in the present embodiment, the shooting preparation state of the still image is exemplified as the first operating state, and the recording state of the moving image is exemplified as the second operating state. As the composition of the selection candidate belonging to the first composition group in the still image framing 1405, any composition is selected from the upper body 1400, the whole body 1401, and the plurality of people 1403. As the composition of the selection candidate belonging to the second composition group in 1406 during the moving image recording, any composition is selected from the upper body 1400, the whole body 1401, and the face 1402. That is, the compositions of the upper body 1400 and the whole body 1401 are common to the first composition group and the second composition group, the composition of the plurality of people 1403 is included only in the first composition group, and the composition of the face 1402 is the second. It is included only in the composition group of. It should be noted that three or more operating states and a plurality of composition groups (including compositions that can be selected in each operating state) corresponding to each of three or more operating states can be set according to the specifications of the imaging device and the like.

Here, the determination conditions of the shooting scene and the transition of each composition will be described. When the automatic composition determination process S1211 of the composition setting “auto” is started, the upper body 1400 is set as the composition in the initial state. The composition of the upper body 1400 is a composition selected when it is determined in 1405 during still image framing or 1406 during moving image recording that the scene is for shooting a single stationary subject. In such a scene, the composition can be set so that the upper body of the subject is contained in the screen so that the composition is appropriately close to the subject.

When the movement of the subject to be photographed is detected in the composition of the upper body 1400, or when it is determined that the distance from the camera to the subject is larger than the predetermined distance D1 (first threshold value), the composition is 1401 for the whole body. Is changed to. The composition of the whole body 1401 is when it is determined in 1405 during still image framing or 1406 during moving image recording that a moving subject or a distant subject, that is, a subject whose subject is likely to be out of the screen is photographed. This is the composition to be selected. In such a scene, the subject is set so as not to be framed out from the screen as much as possible by following the subject with a composition that fits the whole body of the subject. Further, when it is detected that the subject to be photographed has stopped for a predetermined time or longer in the composition of the whole body 1401 and the distance from the camera to the subject is determined to be smaller than the predetermined distance D2 (second threshold value). Is executed to return the composition to the upper body 1400. The predetermined time is a preset reference time for determination.

Hereinafter, a method for determining the movement and stillness of the subject will be described. When at least one of the amount of change in the position and size of the subject detected by the subject detection unit 123 or the amount of shaking detected by the shaking detection unit 125 while the subject is being detected is larger than a predetermined amount, It is determined that the subject to be photographed is moving. The amount of change in the position and size of the subject detected by the subject detection unit 123 and the amount of shaking detected by the shaking detection unit 125 while the subject was being detected continued to be smaller than the predetermined amount for a predetermined time or longer. In this case, it is determined that the subject to be photographed is stationary. The duration of the detection state shall be measured by the timekeeping timer.

Further, a method of calculating the distance from the camera to the subject will be described. First, the reference focal length and the subject distance with respect to the subject size are measured in advance, and the measurement data is stored in the memory 118. The subject distance is calculated by performing the calculation "subject distance = (reference subject distance x focal length x reference subject size) / (reference focal length x detection size)" with respect to the reference value stored in the memory 118. .. For example, it is assumed that when the reference values are a reference subject distance of 2 m, a reference focal length of 24 mm, and a reference subject size of 20 pix (pixels), a subject image having a detection size of 40 pix is detected at a focal length of 120 mm. In this case, the subject distance is calculated as (2 m × 120 mm × 20 pix) / (24 mm × 40 pix) = 5 m.

Next, the composition when a plurality of subjects are detected on the screen will be described. When a plurality of subjects are detected on the screen in the composition of the upper body 1400 or the whole body 1401 of 1405 during still image framing, a process of changing the composition to a plurality of 1403 is performed. At this time, the number of detected subjects is stored in the memory 118. The composition of the plurality of people 1403 is a composition selected when it is determined during the still image framing 1405 that the scene is a group photograph in which all the plurality of subjects are contained in a predetermined angle of view. In the state of the composition of the plurality of people 1403, the determination process of whether or not the number of detected subjects has changed is performed. When the number of detected subjects increases, a new subject enters the screen, so that the number of subjects stored in the memory 118 is immediately updated and the composition is changed. On the other hand, when the number of detected subjects decreases, there is a possibility that the detection cannot be performed temporarily, for example, when one of the subjects faces sideways. Therefore, if the composition is changed immediately, the subject that cannot be detected may be out of frame. Therefore, when the number of subjects decreases, it is determined whether or not the state continues for a predetermined time or longer. If the state continues for a predetermined time or longer, the number of subjects stored in the memory 118 is updated and the composition is changed. Further, when the subject is alone, the process of returning the composition to the upper body 1400 is executed.

If the composition of the 1406 during movie recording is such that all the subjects are within the angle of view, there is a concern that when a subject other than the shooting target is moving around, a malfunction of the zoom may occur in response to the subject. Therefore, even if a plurality of subjects are detected in 1406 during moving image recording, a process is performed in which only the main subject is taken as a shooting target and the composition is maintained as the upper body 1400 or the whole body 1401. When a face is detected near the center of the screen for a predetermined time or longer in the composition of the upper body 1400 at 1406 during the moving image recording, a process of changing the composition to the face 1402 is performed. The composition of the face 1402 is a composition selected when it is determined in 1406 during the moving image recording that the scene is the scene in which the photographer is paying attention to the face of the subject. In such a scene, the face of the subject of interest can be photographed in a larger size by making the composition closer to the face than the upper body. Further, during the zoom-in operation due to this composition change, a more effective image can be taken by changing the zoom speed to a significantly lower speed or a higher speed than the normal speed. Further, in the composition of the face 1402, if the face is detected near the periphery of the screen for a predetermined time or longer, or if it is determined that the photographer has changed the framing, the composition is returned to the upper body 1400. Processing is done. In the present embodiment, the peripheral area of the screen is illustrated as a specific area of the image, and the timekeeping timer measures the duration of the state in which the position of the face image of the subject detected by the subject detection unit 123 is the peripheral area. Further, as a method of determining the framing change, the photographer performs framing when the subject detection unit 123 does not detect the subject and the amount of shaking detected by the shaking detection unit 125 is larger than a predetermined amount. It is determined that the change has been made.

During still image framing 1405, the photographer may continue to wait for a shutter chance with the same composition until the orientation and facial expression of the person's face are in the desired state. In this case, if the composition is closer to the face, the composition may be different from the photographer's intention. Therefore, in the composition of the upper body 1400 of 1405 during still image framing, a process of maintaining the composition of the upper body 1400 is executed even if a face is detected near the center of the screen for a predetermined time or longer.

Next, a composition when the shooting state is changed depending on the start / end of moving image recording will be described. Each composition of the upper body 1400 and the whole body 1401 is a composition that can be selected in any case of 1405 during still image framing and 1406 during moving image recording. Therefore, in those compositions, the original composition is maintained even if the shooting state is changed. On the other hand, the composition of the plurality of people 1403 and the face 1402 is a composition selected only in the case of either 1405 during still image framing or 1406 during moving image recording. Therefore, when video recording is started with the composition of multiple people 1403 during still image framing 1405, or when video recording is completed with the composition of face 1402 at 1406 during video recording, the upper body 1400 has a common composition. It is possible to change it. However, in this case, since the zoom operation starts at the start and end of the moving image recording, there is a concern that the image due to the zooming operation is recorded at the beginning of the moving image or that the photographer feels annoyed by the framing of the still image. .. Therefore, when the shooting state is changed in such a state, a process of temporarily changing the composition to the manual 1404 is executed. The composition of the manual 1404 is a composition in which the size of the main subject image first detected after the composition change is used as a reference size. Therefore, the zoom operation does not start immediately just by changing the shooting state, and the photographer does not feel uncomfortable. Further, even when the subject starts to move, the size of the subject image can be maintained. Further, when a predetermined time elapses in the composition of the manual 1404, a process of returning the composition to the upper body 1400 is executed.

As described above, in S1211 of FIG. 12, when the composition setting is "auto", an appropriate composition is automatically determined according to the shooting state, the number of subjects, the movement of the subjects, and the like. The data indicating the composition after the determination is stored in the memory 118, and the process proceeds to S1212.

In S1212, whether or not the composition determined in S1211 has been changed with respect to the immediately preceding composition (whether the composition selected in FIG. 14 has changed, or whether the number of subjects has changed in the state of a plurality of people 1403). Is determined. If the composition has been changed, the process proceeds to S1213, and it is determined whether or not the selected composition is the composition of the manual 1404. If the selected composition is the composition of the manual 1404, the process proceeds to S1201 and a process of storing the size of the first detected subject image in the memory 118 as a reference size is executed. If the selected composition is not the composition of the manual 1404, the process proceeds to S1215, and the reference size in the composition setting “auto” is calculated. If it is determined in S1212 that the composition has not been changed, the process proceeds to S1214, and it is determined whether or not the subject image has moved in the screen. As a method of calculating the amount of movement of the subject image on the screen, after the reference size is set in S1215, the center position of the face tracking frame immediately after the zoom operation is performed to the angle of view that becomes the reference size is stored in the memory 118. It will be remembered. The distance from the position stored in the memory 118 to the center position of the detected face tracking frame is calculated as the movement amount. Further, the movement amount in the screen may be calculated by adding the center positions of the face tracking frames at predetermined cycles. When the calculated movement amount exceeds a predetermined amount, it is determined that the subject has moved. As a predetermined amount, by normalizing with the size of the detected face tracking frame (predetermined times the size of the face tracking frame), the actual movement amount of the subject does not depend on the size of the subject image on the screen. Can be about the same. If it is determined in S1214 that the subject is moving in the screen, the process proceeds to S1215, and the reference size in the composition setting "auto" is calculated. If it is determined in S1214 that the subject image has not moved in the screen, the reference size setting process is terminated without updating the reference size.

Hereinafter, the calculation process of the reference size in the composition setting “auto” will be described with reference to FIGS. 15 to 19. When the composition setting is any of "face", "upper body", and "whole body", the reference size is calculated so that the face has a constant ratio to the screen size in S1210 of FIG. That is, the same reference size is calculated regardless of the position of the subject image on the screen. Therefore, if the composition is changed so that the subject image becomes larger (zoomed in) when the subject is in the periphery of the screen, the subject may be out of frame. When calculating the reference size by this method, the photographer needs to frame the subject near the center of the screen. Therefore, it is not suitable for shooting with the background by shifting the subject from the center of the screen. Further, when there are a plurality of subjects, it is necessary to calculate the reference size by another method. Therefore, when the composition setting is "Auto", the size of the subject image is determined based on the composition determined in S1211, and the subject is set as a reference so that the subject fits in the screen regardless of the detected position of the subject. The process of calculating the size is performed.

FIG. 15 is a diagram showing the relationship between the position of the subject closest to the screen among the plurality of subjects and the screen size. 15 (A) and 15 (B) show a screen example of still image shooting, and FIG. 15 (C) shows a screen example of moving image shooting, exemplifying a plurality of subjects. Further, FIG. 16 is a flowchart showing the entire calculation process of the reference size in the composition setting “auto”.
First, in S1600 of FIG. 16, a process of calculating the subject position ratio Rh in the horizontal direction is executed. The horizontal subject position ratio Rh is the horizontal position near the shoulder of the subject closest to the center of the screen with respect to a predetermined ratio to the screen size in the horizontal direction (hereinafter referred to as shoulder position, referred to as Xsmax). It is the ratio occupied by the distance to x2. The predetermined ratio is, for example, 80% or 90% of the screen size in the horizontal direction, and is changed according to the number of people arranged in the horizontal direction in S1703, which will be described later. In FIG. 15A, among the subjects 1500a, 1501a, and 1502a, the subject 1500a that is closest to the center of the screen in the horizontal direction is the subject 1500a. The calculation process of the subject position ratio Rh in the horizontal direction will be described with reference to the flowchart of FIG.

In S1700, the shoulder position (denoted as Xs) of the subject is calculated based on the center position and size of the face tracking frame of the subject. The center of the screen is the origin (0,0), the center position of the face tracking frame is (Xc, Yc), and the size of the face tracking frame is S. The shoulder position Xs on the peripheral side of the screen can be calculated by "Xs = | Xc | + SxNs", where Ns is the number of face tracking frames occupying the distance from the center position to the shoulder position of the face tracking frame. FIG. 19 shows an example of setting Ns. The value of Ns is changed according to the composition determined in S1211 of FIG. For example, in the case of the composition of the face 1402, Ns = 1 (in this case, the horizontal position near the ear, not the shoulder position) is set. In other cases (“upper body”, “whole body”, “plurality”), Ns = 2 is set.

After calculating the shoulder position Xs of the subject, the process proceeds to S1701. In S1701, when the maximum value Xsmax of the shoulder position is not stored in the memory 118, the shoulder position Xs calculated in S1700 is stored as the maximum shoulder position Xsmax. When the maximum shoulder position Xsmax is stored in the memory 118, the shoulder position Xs calculated in S1700 and the maximum shoulder position Xsmax are compared. When the shoulder position Xs is larger than the maximum shoulder position Xsmax, a process of updating the maximum shoulder position Xsmax of the memory 118 at the shoulder position Xs is executed. After the update process of the maximum shoulder position Xsmax, the process proceeds to S1702. In S1702, it is determined whether or not the process of calculating the shoulder position Xs for all the subjects and updating the maximum shoulder position Xsmax is completed. If the calculation of the shoulder position Xs and the update of the maximum shoulder position Xsmax have not been completed, the process returns to S1700. When the determination of the maximum shoulder position Xsmax is completed for all the subjects and the update of the maximum shoulder position Xsmax is completed, the process proceeds to S1703.

In S1703, the determination process of the number of people whose subjects are lined up in the horizontal direction (the number of people lined up in the horizontal direction) is performed. The purpose of this is to create a composition in which the subject fills the entire screen when there are many subjects as in a group photograph, and a composition in which a margin is left around the screen when there are few subjects. Further, as a method of counting the number of people lined up in the horizontal direction, when the face tracking frames overlap in the vertical direction (vertical direction of the screen), the overlapping subjects are counted as one person. For example, if four subjects are detected on the screen and the face tracking frames of two of them overlap at the top and bottom of the screen, they are counted as three. In S1703, the number of people lined up in the horizontal direction is compared with the threshold value (for example, two people). If it is determined that the number of people lined up in the horizontal direction is two or less, the process proceeds to S1704, and if it is determined that the number of people lined up in the horizontal direction is three or more, the process proceeds to S1705. In S1704 and S1705, the horizontal subject position ratio Rh is calculated, respectively. The horizontal subject position ratio Rh in S1704 is calculated as the ratio of the maximum shoulder position Xsmax × 2 to 80% of the horizontal screen size. Further, the horizontal subject position ratio Rh in S1705 is calculated as the ratio of the maximum shoulder position Xsmax × 2 to 90% of the horizontal screen size. After S1704 or S1705, the calculation process of the subject position ratio Rh is completed.

Next, in S1601 of FIG. 16, a process of calculating the subject position ratio Rv in the vertical direction is executed. The vertical subject position ratio Rv is the vertical position of the head of the subject closest to the center of the screen (hereinafter referred to as the head position) or the vertical position of the body part with respect to a predetermined ratio of the screen size in the vertical direction. It is the ratio occupied by the distance to (hereinafter referred to as the body position) × 2. The predetermined ratio is, for example, 90% of the vertical screen size. In the following, the head position of the subject closest to the center of the screen will be referred to as Yhmax, and the body position of the subject closest to the center of the screen will be referred to as Ybmin. In FIG. 15B, among the subjects 1500b, 1501b, and 1502b, the subject having the head closest to the center of the screen in the vertical direction is the subject 1500b, and the subject having the body position closest to the center of the screen is the subject 1501b. Is. The calculation process of the subject position ratio Rv in the vertical direction will be described with reference to the flowchart of FIG.

In S1800 of FIG. 18, the head position (denoted as Yh) of the subject is calculated based on the center position and size of the face tracking frame of the subject. The head position Yh can be calculated by "Yh = Yc + S × Nh". Nh is the number of face tracking frames occupying the distance from the center position to the head position of the face tracking frame. FIG. 19 shows an example of setting Nh. The value of Nh is set to Nh = 1.5 regardless of the composition determined in S1211 of FIG. After calculating the head position Yh of the subject, the process proceeds to S1801.

In S1801, when the maximum value Yhmax of the head position is not stored in the memory 118, a process of storing the head position Yh calculated in S1800 as the maximum head position Yhmax is executed. When the maximum head position Yhmax is stored in the memory 118, the head position Yh calculated in S1800 and the maximum head position Yhmax are compared. When the head position Yh is larger than the maximum head position Yhmax, a process of updating the maximum head position Yhmax of the memory 118 at the head position Yh is performed. After the update process, the process proceeds to S1802.

In the next S1802, the body position (denoted as Yb) of the subject is calculated based on the center position and size of the face tracking frame of the subject. The body position Yb can be calculated by "Yb = Yc-S x Nb". Nb is the number of face tracking frames occupying the distance from the center position of the face tracking frame to the body position. FIG. 19 shows an example of setting Nb. The value of Nb is changed according to the composition determined in S1211 of FIG. For example, in the case of the composition of the face 1402, Nb = 1.5 is set. Nb = 5 is set in the case of the composition of the upper body 1400, Nb = 10 is set in the case of the composition of the whole body 1401, and Nb = 3.5 is set in the case of the composition of a plurality of people 1403. In each composition, the position is in the center of the screen, the chest for the face 1402, the lower back for the upper body 1400, the legs for the whole body 1401, and the upper waist for the plurality of people 1403. After calculating the body position Yb of the subject, the process proceeds to S1803.

In S1803, when the minimum value Ybmin of the body position is not stored in the memory 118, the process of storing the body position Yb calculated in S1802 as the minimum body position Ybmin is executed. When the minimum body position Ybmin is stored in the memory 118, the body position Yb calculated in S1802 and the minimum body position Ybmin are compared. When the body position Yb is smaller than the minimum body position Ybmin, a process of updating the minimum body position Ybmin of the memory 118 with the body position Yb is performed. After the update process, the process proceeds to S1804.

In S1804, the head position Yh and the body position Yb are calculated for all the subjects, and it is determined whether or not the process of updating the maximum head position Yhmax and the minimum body position Ybmin is completed. If the update of the maximum head position Yhmax and the minimum body position Ybmin has not been completed, the process returns to S1800. When the determination of the maximum head position Yhmax and the minimum body position Ybmin is completed for all the subjects and the update of the maximum head position Yhmax and the minimum body position Ybmin is completed, the process proceeds to S1805. In S1805, the absolute value of the maximum head position Yhmax and the absolute value of the minimum body position Ybmin are compared, and a process of determining which is located closer to the screen is performed. If it is determined that the maximum head position Yhmax is located closer to the screen, the process proceeds to S1806. If it is determined that the minimum body position Ybmin is located closer to the screen, the process proceeds to S1807. In S1806 and S1807, the subject position ratio Rv in the vertical direction is calculated, respectively. The vertical subject position ratio Rv in S1806 is calculated as the ratio of the absolute value of the maximum head position Yhmax × 2 to 90% of the vertical screen size. The vertical subject position ratio Rv in S1807 is calculated as the ratio of the absolute value of the minimum body position Ybmin × 2 to 90% of the vertical screen size. After S1806 or S1807, the calculation process of the vertical subject position ratio Rv is completed.

Subsequently, in S1602 of FIG. 16, the horizontal subject position ratio Rh and the vertical subject position ratio Rv are compared. As a result, it is possible to determine the position having the largest ratio of the distances from the center of the screen to each position of the subject with respect to the predetermined ratio of the screen size in each direction, that is, the position closest to the screen. it can. Each position is the maximum shoulder position Xsmax, the maximum head position Yhmax, and the minimum body position Ybmin. When it is determined that the horizontal subject position ratio Rh is larger than the vertical subject position ratio Rv, the process proceeds to S1603, and when it is determined that the horizontal subject position ratio Rh is equal to or less than the vertical subject position ratio Rv. To S1604. In S1603 and S1604, a process of calculating a reference size is executed based on the ratio of the detection size of the main subject and the position of the subject closest to the screen determined in S1602. The reference size is calculated by "detection size / horizontal subject position ratio Rh" in S1603, and is calculated by "detection size / vertical subject position ratio Rv" in S1604. That is, in S1603, the zoom operation is executed so that the main subject image has a reference size obtained by multiplying the size by "1 / horizontal subject position ratio Rh". In S1604, the zoom operation is executed so that the main subject image has a reference size obtained by multiplying the size by "1 / vertical subject position ratio Rv". By doing so, the angle of view based on the composition determined in S1211 can be set without frame-out of the subject in the screen.

In S1215 of FIG. 12, when the composition setting is "auto", the subject position that is the closest to the screen in the horizontal and vertical directions is determined based on the composition determined in S1211, and the position is referenced so as to fit on the screen. The size is calculated. As a result, the subject can be contained in an appropriate angle of view without frame-out regardless of the position of the subject image on the screen. Further, regardless of whether the number of subjects is singular or plural, the reference size can be calculated by the same processing. In the present embodiment, the direction of the screen size for calculating the reference size is determined on the premise of a person in a standing posture as a subject. Not limited to this, if the subject detection unit 123 can determine the direction of the face, the direction of the screen size may be determined based on the vertical direction of the face on the screen. Further, although the composition determination and the calculation of the reference size when the subject is a person have been described, the present invention can also be applied when the subject is an object. However, in this case, the selected composition is "large", "medium", "small", and "plural" instead of "face", "upper body", "whole body", and "plural". The number of mono-tracking frames (corresponding to Ns, Nh, Nb in the case of a person) when calculating the subject position may be set accordingly.

When the reference size setting process shown in FIG. 12 is completed, the process proceeds to S704 of FIG. 7. In S704, the AZ control unit 122 performs auto zoom control based on the subject information detected by the subject detection unit 123 and the reference size stored in the memory 118. The auto zoom control will be described later with reference to FIG. After the auto zoom control is completed, the process proceeds to S705, and the AZ control unit 122 determines the subject search instruction by the photographer. That is, it is determined whether or not the photographer has pressed the subject search button of the operation unit 117. When the subject search button is pressed, the process returns to S701 and the subject search process is executed. If the subject search button is not pressed, the process proceeds to S706. In S706, it is determined whether or not the subject to be auto-zoomed has been changed by operating the touch panel of the operation unit 117, the subject designation switch, or the like. When the subject is changed, the process returns to S702 and the subject designation process is executed. If the subject has not been changed, the process proceeds to S707.

In S707, it is determined whether or not the composition setting has been changed by operating the left and right buttons of the operation unit 117 from the shooting screen. When the composition setting is changed, the composition setting data stored in the memory 118 is updated, and then the process returns to S703 to execute the reference size setting process. If the composition setting has not been changed, the process proceeds to S708. In S708, it is determined whether or not the auto zoom operation switch of the operation unit 117 is pressed. When it is determined that the auto-zoom operation switch is pressed, the auto-zoom function is terminated. If it is determined that the auto zoom operation switch is not pressed, the process proceeds to S709. S709 is a process for determining whether or not the composition setting stored in the memory 118 is “auto”. When the composition setting is "auto", the process returns to S703 and the reference size setting process including the automatic composition determination is executed. If the composition setting is other than "auto", the process returns to S704 and the auto zoom control continues.

Next, the process of auto-zoom control in S704 will be described with reference to the flowchart of FIG. First, in S2000, the AZ control unit 122 determines whether or not the subject has been detected by the subject detection unit 123. If the subject is not detected in S2000, the auto zoom control is terminated. If the subject is detected in S2000, the process proceeds to S2001. The determination shown in each step of S2001 to S2003 is an auto-zoom start determination process for frame-out prevention control. That is, as described with reference to FIGS. 3 and 4, this process is performed to prevent the subject from being framed out of the screen. In S2001, the AZ control unit 122 determines whether or not the subject tracking frame related to the subject to be tracked has entered the ZO region. This ZO region corresponds to the ZO region described with reference to FIGS. 3 (A) and 4 (A) to (C). When the subject tracking frame enters the ZO region in S2001, that is, when there is a high possibility that the subject is out of the frame, the process proceeds to S2004 and the zoomout operation is started. The zoom-out operation here corresponds to a zoom-out operation for frame-out prevention control. After the zoom-out operation, the auto-zoom control is terminated.

On the other hand, if the subject tracking frame does not enter the ZO region in S2001, that is, if the camera captures the subject near the center of the screen, the process proceeds to S2002. In S2002, it is determined whether or not the immediately preceding zoom operation is a zoom-out operation due to entering the ZO region in S2001, that is, whether or not it is a zoom-out operation for frame-out prevention control. If it is determined in S2002 that the zoomout operation for the frameout prevention control has been performed, the process proceeds to S2003. If it is determined in S2002 that the zoom-out operation for frame-out prevention control has not been performed, the process proceeds to S2006.

In S2003, the AZ control unit 122 determines whether or not the subject tracking frame related to the subject to be tracked is contained within the ZI region (whether or not it is included). The ZI region here corresponds to the ZI region described with reference to FIGS. 3 (B) and 4 (A) to 4 (C). When it is determined in S2003 that the subject tracking frame does not fit in the ZI region, the auto zoom control process ends. On the other hand, if it is determined that the subject tracking frame is within the ZI region, the process proceeds to S2005. That is, when the camera captures the subject so as to be near the center of the screen and within the angle of view of the zoom return position, the zoom-in operation is started in S2005. The zoom-in operation here corresponds to the zoom-in operation for frame-out prevention control. After the zoom-in operation, the auto-zoom control is terminated.

In the present embodiment, in order to achieve both the flameout prevention control and the size retention control in the auto zoom, the frameout prevention control first captures the subject near the center of the screen, and then the size retention control can be executed. Therefore, in the state after the zoom-out operation for the frame-out prevention control, the auto-zoom processing (S2006 to S2010 below) for maintaining the subject size constant (size retention control) described later is not performed. .. In other words, when the frame-out prevention control is performed, the execution of the size holding control is restricted until the zoom-in operation for the frame-out prevention control is completed.

Next, each process of S2006 to S2010 will be described. If it is determined in S2002 that the zoom-out operation for frame-out prevention control has not been performed, the process proceeds to S2006, and the AZ control unit 122 sets the detection size of the subject and the reference size to a predetermined magnification (N1 and N1> 1. ) Multiply the size. The subject size of the reference subject information set in S1202, S1210, and S1215 of FIG. 12 is compared with the subject size detected in S2000. When the size of the subject detected in S2000 is larger than N1 times the size of the reference subject, that is, when the ratio of the subject image to the screen exceeds a predetermined value, the process proceeds to S2009. In S2009, the AZ control unit 122 starts the zoom-out operation. The zoom-out operation here corresponds to the zoom-out operation for size retention control. After the zoom-out operation, the auto-zoom control is terminated.

On the other hand, in S2006, when the size of the subject detected in S2000 is N1 times or less the size of the reference subject, the process proceeds to S2007. In S2007, the AZ control unit 122 compares the subject size of the reference subject information with the size of the subject detected in S2000. When the size of the detected subject is smaller than the predetermined magnification (denoted as N2 and N2 <1) with respect to the size of the reference subject (when it is less than N2 times), that is, the ratio of the subject image to the screen is a predetermined value. If it is less than, the process proceeds to S2008. On the other hand, when the size of the detected subject is N2 times or more the size of the reference subject, the auto zoom control is terminated.

In S2008, the AZ control unit 122 determines whether or not the subject tracking frame related to the subject to be tracked is contained within the ZI region (whether or not it is included). This is to prevent the subject from being out of the frame due to the zoom-in operation when the subject is around the screen. The ZI region here corresponds to the ZI region described with reference to FIG. 5 (E). If it is determined in S2008 that the subject tracking frame does not fit in the ZI region, the auto zoom control is terminated.

On the other hand, if it is determined in S2008 that the subject tracking frame is within the ZI region, the process proceeds to S2010. In S2010, the AZ control unit 122 starts the zoom-in operation. The zoom-in operation here corresponds to the zoom-in operation for size retention control. As described above, in the present embodiment, in order to prevent the subject from being framed out even in the zoom-in operation for size retention control, the zoom-in operation is started after the subject image is contained inside the ZI region. After the zoom-in operation, the auto-zoom control is terminated.

Next, the zoom operation will be described with reference to FIG. FIG. 21 is a flowchart illustrating a zoom-out operation or a zoom-in operation in S2004, S2005, S2009, and S2010 of FIG. First, in S2100, the AZ control unit 122 acquires a zoom change amount (change amount of zoom magnification) from the memory 118. In the case of the zoom-out operation for frame-out prevention control, the zoom-out change amount is set according to the detected subject information. Specifically, in the zoom-out operation for frame-out prevention control (S2004 in FIG. 20), the smaller the size of the subject, the smaller the zoom-out change amount is set. As a result, it is possible to prevent the subject from being undetectable because the size of the subject becomes too small due to the zoom-out operation. In consideration of the minimum size that can detect the subject, if the size of the subject is smaller than the predetermined size, the zoom-out operation may not be performed. Further, in the zoom-in operation for the frame-out prevention control, the zoom magnification before the start of the zoom-out operation is stored in the memory 118, and the zoom-in change amount is set so as to be the same zoom magnification as before the start of the zoom-out operation.

Further, in the zoom-out operation for size retention control (S2009 in FIG. 20), a zoom-out change amount (1 / N1 times) corresponding to a predetermined N1 times used for the determination of S2006 is set. As a result, even if the subject cannot be detected, the minimum zoom-out operation can be performed until the size of the subject becomes the size of the reference subject. Similarly, in the case of the zoom-in operation for size holding control (S2010 in FIG. 20), the zoom-in change amount (1 / N2 times) corresponding to the predetermined N2 times used for the determination in S2007 is set.

In S2101, the AZ control unit 122 sets the zoom change amount acquired in S2100 in the CZ control unit 119 or the electronic zoom control unit 120, and instructs the CZ control unit 119 or the electronic zoom control unit 120 to perform the scaling process. In the next S2102, the AZ control unit 122 determines which of the frame-out prevention control and the size retention control is in the zoom operation. If the current zoom operation is a zoom operation for flameout prevention control (S2004 and S2005 in FIG. 20), the process proceeds to S2105. If the current zoom operation is a zoom operation for size retention control (S2009, S2010 in FIG. 20), the process proceeds to S2103.

In S2103, the AZ control unit 122 determines whether or not the subject is detected by the subject detection unit 123. If the subject is detected, the process proceeds to S2104, and if the subject is not detected, the process proceeds to S2105. In S2104, the size of the subject indicated by the reference subject information is compared with the size of the subject detected in S2103. As a result of comparison, if the size of the subject detected in S2103 and the size of the reference subject do not fall within the range of the predetermined ratio (within the predetermined amount of change), the process proceeds to S2102 again to determine the zoom operation. continue. When the size of the subject detected in S2103 and the size of the reference subject are within the range of a predetermined ratio by the zoom operation, the process proceeds to S2106. After stopping the zoom operation in S2106, the AZ control unit 122 ends the zoom operation process.

If it is determined in S2102 that the zoom operation for frame-out prevention control is in progress, or if the subject is not detected in S2103 during the zoom operation for size retention control, the process proceeds to S2105. In S2105, the AZ control unit 122 determines whether or not the scaling process corresponding to a predetermined amount of zoom change amount corresponding to each zoom operation has been performed based on the zoom change amount acquired in S2100. If the scaling process of the predetermined zoom change amount has not been performed, the process returns to S2102 again to continue the process. Further, when the scaling process of the predetermined zoom change amount is performed, the process proceeds to S2106, and the AZ control unit 122 stops the zoom in operation and ends the zoom operation process.

In the present embodiment, a shooting scene is discriminated based on the movement and number of subjects, the detection time of the subject, the distance from the camera to the subject, and the like, and the camera automatically selects an appropriate composition according to the discriminated scene. Based on the composition selected by the composition selection process, the process of setting the reference size of the subject image is performed, and the zoom operation is executed so that the detected image of the subject becomes the reference size. According to the present embodiment, the auto-zoom control can be performed so that the composition is appropriate according to the shooting scene determined based on the detection information of the camera without the photographer selecting the composition.

(Second embodiment)
Next, a second embodiment of the present invention will be described with reference to FIG. FIG. 22 shows an example of a transition diagram having a composition different from that of FIG. 14 in the automatic composition determination process S1211 of the composition setting “auto” of FIG. The composition common to FIG. 14 is designated by the same reference numerals, and detailed description thereof will be omitted.

In the first embodiment, an example of determining a shooting scene when shooting a person different from the photographer as a subject has been described. The mode at this time is called a normal mode for convenience. In addition, the camera may be pointed at the photographer himself / herself, and one or a plurality of persons including the photographer himself / herself may be photographed as a subject. A mode suitable for such shooting is called a self-shooting mode. In the self-shooting mode, it is possible to perform effective portrait shooting by adjusting the angle of view suitable for self-shooting by auto-zoom control and applying a background blur effect, a skin-beautifying filter effect, and the like.
In the composition matching operation, it is necessary to make the subject image an appropriate size depending on whether the subject is stationary or moving in the normal mode. On the other hand, in the self-shooting mode, even if the subject is stationary, it is necessary to change the appropriate size depending on whether the camera is hand-held or placed. For example, when shooting by hand, the face is shot so as to fit in the angle of view, and when shooting with the camera placed, the whole body is shot so as to fit in the angle of view. In this way, the appropriate composition differs between shooting in the normal mode and shooting in the self-shooting mode.
Therefore, in the present embodiment, the process of changing the selected composition is executed in the normal mode 2200 and the self-portrait mode 2201 of FIG. That is, the normal mode is illustrated as the first mode, and the selfie mode is exemplified as the second mode. As the composition of the selection candidate belonging to the first composition group in the normal mode 2200, any composition is selected from the upper body 1400, the whole body 1401, and the plurality of people 1403. As the composition of the selection candidate belonging to the second composition group in the self-portrait mode 2201, any composition is selected from the self-portrait face 2202 and the self-portrait whole body 2203.

Here, the determination conditions of the shooting scene and the transition of each composition will be described. When the automatic composition determination process S1211 of the composition setting “auto” is started, the upper body 1400 is set as the composition in the initial state. When the mode is the normal mode 2200, the transition is the same as that of 1405 during the still image framing in FIG. 14, so the description thereof will be omitted.
When the mode is changed from the normal mode 2200 to the self-shooting mode 2201, the shake detection unit 125 determines whether or not the camera is in the handheld state based on the amount of shake applied to the gyro sensor or the like. When the mode is changed to the self-portrait mode 2201 and the amount of deflection is equal to or greater than a predetermined amount and it is determined that the camera is in hand, the composition is changed to the self-portrait face 2202. The composition of the self-portrait face 2202 is a composition when the self-portrait face is taken up by hand. Also, when shooting with multiple people including the photographer, adjust the zoom so that the faces of all the subjects fit in the angle of view as large as possible.

In the case of the self-shooting mode 2201, the same composition is selected regardless of whether the subject is only the photographer or a plurality of people. That is, the estimated number of face positions for the subject shown in FIG. 19 is set to the same value. As described above, the subject position estimated number of faces is the face tracking frame for estimating the shoulder position Xs, the head position Yh, and the body position Yb of the subject in the reference size calculation process S1215 of the composition setting “auto” in FIG. It is the number of tracking frames (Ns, Nh, Nb) from the center position. In particular, the number of tracking frames Ns up to the shoulder position and the number of tracking frames Nb up to the body position differ depending on the selected composition, so that the reference size for the composition is calculated. In the first embodiment, when a single subject is photographed with a “face” composition, the number of tracking frames from the center position (Xc, Yc) of the face tracking frame is the shoulder position Ns = 1 and the body position Nb = 1. As 5.5, the reference size was calculated so that the angle of view covers the subject's ears and chest. When shooting a plurality of subjects in a composition of "multiple people", the number of tracking frames from the center position (Xc, Yc) of the face tracking frame is set to shoulder position Ns = 2 and body position Nb = 3.5. The standard size was calculated so that the angle of view covers the shoulders and hips of all subjects. On the other hand, in the case of the self-portrait face 2202, the shoulder position Ns = 1 and the body position Nb = 1.5, which are the same number of tracking frames as the “face”, are set regardless of the number of subjects. As a result, the reference size is calculated so that the angle of view covers the entire subject's ears and chest, and the zoom is adjusted to an angle of view suitable for handheld selfie shooting.

When the face is changed to the self-portrait face 2202, the number of detected subjects is stored in the memory 118. In the state of the composition of the self-portrait face 2202, the determination process of whether or not the number of detected subjects has changed is performed. When the number of detected subjects increases, the number of subjects stored in the memory 118 is immediately updated, and the reference size is changed. When the number of subjects decreases, if the state continues for a predetermined time or longer, the number of subjects stored in the memory 118 is updated and the composition is changed. This is to prevent the subject from being out of frame by immediately changing the composition when the subject is temporarily turned sideways and cannot be detected as in the case of a plurality of people 1403.

When the mode is changed from the normal mode 2200 to the self-portrait mode 2201, the camera is placed (fixed) on the desk or the like in the shake detection unit 125 based on the amount of vibration applied to the gyro sensor or the like. Determine if it is in a state. When the mode is changed to the self-portrait mode 2201 and it is determined that the camera is placed with the amount of runout less than a predetermined amount, the composition is changed to the self-portrait whole body 2203. The composition of the self-portrait whole body 2203 is a composition when the camera is placed and the whole body is photographed. When shooting a plurality of people including the photographer, the zoom is adjusted so that the whole body of all the subjects fits in the angle of view. In the case of the self-portrait whole body 2203, the shoulder position Ns = 2 and the body position Nb = 10, which are the same number of tracking frames as the “whole body”, are set regardless of the number of subjects. As a result, the reference size is calculated so that the shoulders and feet of all the subjects are within the angle of view, and the zoom is adjusted to the angle of view suitable for taking a selfie with the camera placed. When the selfie is changed to 2203, the number of detected subjects is stored in the memory 118. Similarly to the self-portrait face 2202, in the self-portrait whole body 2203, when the number of subjects changes, the number of subjects stored in the memory 118 is updated and the reference size is changed.

In the present embodiment, the shooting scene to be determined is changed according to the set mode, and the camera automatically selects a more appropriate composition for each mode. Further, in the present embodiment, the case where an appropriate composition is selected from the selection candidates of a plurality of compositions in one mode is illustrated, but depending on the mode, only one composition may be a selection candidate. Further, although the case of the normal mode and the self-portrait mode has been illustrated as the mode, the mode is not limited to this mode. For example, in addition to the normal mode and the selfie mode, in the sports mode, the composition of the athlete's "whole body" is selected, and in the baby mode, the composition of the baby's "face" is selected. You may.
According to the present embodiment, it is possible to perform auto zoom control so that the composition is appropriate according to the shooting scene determined based on the set mode and the detection information of the camera without the photographer selecting the composition. it can.

(Other embodiments)
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

100 digital camera
102 Zoom lens 109 Display unit 113 Zoom lens drive unit 117 Operation unit 118 Memory 119 CZ control unit 120 Electronic zoom control unit 122 Auto zoom control unit 123 Subject detection unit

The zoom control device according to the present invention includes a subject detection means for detecting a subject image from an image acquired by using a photographing optical system, a reference size setting means for setting a reference size of the subject image used for controlling the zoom magnification, and a reference size setting means. A control means for controlling the zoom magnification based on the reference size set by the reference size setting means and the size of the subject image sequentially detected by the subject detection means, a shooting mode setting means for setting the shooting mode, and a shooting mode setting means for setting the shooting mode. Has. In the reference size setting means, when the self-shooting mode for shooting the photographer himself / herself is set by the shooting mode setting means, is the image pickup device whose zoom magnification is controlled by the zoom control device in a handheld state? The reference size is changed based on whether or not.

Claims (14)

It is a zoom control device
A subject detection means that detects a subject image from an image acquired using a photographing optical system, and
Setting means for setting the reference size of the subject image used to control the zoom magnification, and
It has a control means for controlling the zoom magnification based on the reference size set by the setting means and the size of the subject image sequentially detected by the subject detection means.
The setting means is based on at least one of whether or not the image pickup device whose zoom magnification is controlled by the zoom control device is in the handheld state, whether or not the self-portrait mode is set, and the still time of the subject. A zoom control device characterized by changing the reference size. The zoom control device according to claim 1, wherein the setting means changes the reference size based on the rest time of the subject. The setting means obtains the stationary time based on at least one of a change in the size of the subject image, a change in the position of the subject image, and a vibration detection result of the zoom control device. Item 2. The zoom control device according to item 2. The setting means is
The reference size is changed based on the change in the rest time, and the reference size is changed.
If it is determined that the subject is moving, a first reference size is set.
The zoom control device according to claim 2 or 3, wherein when it is determined that the rest time is longer than a predetermined time, a second reference size larger than the first reference size is set. .. The setting means changes the reference size based on the change in the rest time, and changes the reference size.
The control means controls the zoom magnification so that the zoom magnification is lower when it is determined that the subject is moving than when it is determined that the stationary time is longer than a predetermined time. The zoom control device according to any one of claims 2 to 4, wherein the zoom control device is characterized. The zoom control device according to any one of claims 1 to 5, wherein the setting means changes the reference size based on a determination result of whether or not the image pickup device is in a handheld state. The zoom control device according to claim 6, wherein the setting means determines whether or not the image pickup device is on hand based on a detection result of a shake amount of the zoom control device. The zoom control device according to any one of claims 1 to 7, wherein the setting means changes the reference size depending on whether or not the self-shooting mode is set. The setting means changes the reference size based on whether or not it is in the handheld state, whether or not the self-shooting mode is set, at least one of the still times of the subject, and the distance to the subject. The zoom control device according to any one of claims 1 to 8, wherein the zoom control device is characterized. It is a zoom control device
A subject detection means that detects a subject image from an image acquired using a photographing optical system, and
A composition selection means for selecting a composition based on at least one of whether or not the zoom control device is in a handheld state, whether or not a selfie mode is set, the distance to the subject, and the stationary time of the subject.
A setting means for setting a reference size of a subject image used for controlling the zoom magnification based on the selection result by the composition selection means, and a setting means.
It has a control means for controlling the zoom magnification based on the reference size set by the setting means and the size of the subject image sequentially detected by the subject detection means.
The setting means is a zoom control device characterized in that the reference size is set based on the composition selected by the composition selecting means and the size and position of the subject image detected by the subject detecting means. The zoom control device according to any one of claims 1 to 10.
An imaging means that converts a subject image by photoelectric conversion to generate an image,
It has a zoom means that changes the zoom magnification by driving the lens or processing the image.
The control means controls the zoom means, and controls the zoom magnification according to a change in the size of the subject image detected by the subject detection means based on the reference size set by the setting means. Imaging device. It is a control method executed by the zoom control device.
The acquisition step of acquiring information including the detection information of the subject image detected from the image acquired by using the photographing optical system, and the acquisition step.
A setting step to set the reference size of the subject image used to control the zoom magnification, and
It has a control step for controlling the zoom magnification based on the reference size set by the setting step and the size of the subject image indicated by the detection information.
The setting step is based on at least one of whether or not the image pickup device whose zoom magnification is controlled by the zoom control device is in the handheld state, whether or not the self-portrait mode is set, and the still time of the subject. A control method for a zoom control device, which comprises changing a reference size. It is a control method executed by the zoom control device.
The acquisition step of acquiring information including the detection information of the subject image detected from the image acquired by using the photographing optical system, and the acquisition step.
A composition selection step of selecting a composition based on at least one of whether or not the zoom control device is in the handheld state, whether or not the selfie mode is set, the distance to the subject, and the still time of the subject.
A setting step of setting a reference size of a subject image used for controlling the zoom magnification based on the composition selected by the composition selection step and the size and position of the subject image.
A control method for a zoom control device, comprising: a control step for controlling a zoom magnification based on the reference size set by the setting step and the size of a subject image indicated by the detection information. A control program for a zoom control device, which comprises causing a computer to execute each step of the control method for the zoom control device according to claim 12 or 13.

Images