JP2020039165A - Zoom control device, imaging apparatus, control method of zoom control device, control program of zoom control device - Google Patents

Abstract

To perform auto zoom control with a composition suitable for a scene determined on the basis of photographic subject detection information without requiring a photographer to select a composition.SOLUTION: The zoom control device includes: a subject detection unit 123 for detecting a subject image from an image acquired using an imaging optical system; an AZ control unit 122, for setting a reference size of the subject image used for controlling a zoom magnification and controlling the zoom magnification on the basis of the set reference size and sizes of subject images sequentially detected by the subject detection unit 123. The AZ control unit 122 changes the reference size on the basis of at least one of whether the zoom control device is in a hand-held state, whether a self-photographing mode is set, a distance to the subject, and a still time of the subject.SELECTED DRAWING: Figure 14

Description

  The present invention relates to a technology for detecting a subject image from an image and controlling a zoom magnification.

  2. Description of the Related Art Conventionally, a function of driving a zoom lens to perform optical zooming (hereinafter, referred to as optical zoom), and performing zooming electronically (hereinafter, electronic zooming) by enlarging a part of a captured image. Some imaging devices have a function. Further, there is an auto zoom (automatic scaling) function for automatically changing the zoom magnification according to the detection information of the subject. Hereinafter, the optical zoom operation and the electronic zoom operation are collectively referred to as a zoom operation.

  Patent Literature 1 discloses a configuration of a camera equipped with an auto-zoom function for maintaining a size of a subject constant. Patent Literature 1 discloses a method in which a photographer selects a composition from a plurality of options such as “face up”, “bust up”, and “whole body” in an auto zoom function. In the auto zoom control for maintaining the size of the subject image, the size of the subject image serving as a reference for the zoom control is determined based on the selected composition. For example, when the photographer selects the “whole body” composition, the zoom operation is performed so that the whole body image of the person who is the subject is included in the shooting screen.

JP 2012-95019 A

  However, in Patent Literature 1, it is necessary for the photographer to select a composition to be photographed in advance by using the auto zoom function. Therefore, every time the shooting scene or the state (movement, number, etc.) of the subject changes, the photographer needs to perform a setting change operation. In addition, if the composition setting is not appropriate for the scene to be photographed, the subject may not be photographed in an appropriate size. For example, when the composition of a “face” is set in a scene in which a moving subject or a distant subject is photographed, the subject may go out of frame. In addition, when the composition of a “still body” is set in a scene in which a still subject is photographed, the composition of the subject may be too small.

  In view of the above problems, an object of the present invention is to perform auto zoom control with a composition suitable for a scene determined based on subject detection information without a photographer selecting a composition.

An apparatus according to the present invention includes: a subject detection unit configured to detect a subject image from an image acquired using an imaging optical system; and a setting unit configured to set a reference size of the subject image used for controlling a zoom magnification.
Control means for controlling a zoom magnification based on the reference size set by the setting means and a size of a subject image sequentially detected by the subject detection means. The setting unit is configured to determine whether the zoom control device is in a hand-held state, whether a self-portrait mode is set, a distance to a subject, or at least one of a time during which the subject is at rest, to set the reference size. change.

  According to the present invention, auto-zoom control can be performed with a composition suitable for a scene determined based on subject detection information without a photographer selecting a composition.

FIG. 1 is a block diagram illustrating a configuration example of a digital camera according to an embodiment of the present invention. FIG. 3 is a diagram illustrating a relationship between a focal length and a focus lens position for each subject distance. FIG. 9 is an explanatory diagram of a process of preventing a subject (object) from going out of the frame; FIG. 9 is an explanatory diagram of a process of preventing a subject (person) from being out of the frame; FIG. 9 is an explanatory diagram of a process for preventing a change in size of a subject (person) in a screen. FIG. 3 is a diagram illustrating composition setting when a subject is a person. 5 is a flowchart illustrating a flow of processing of an auto zoom function. It is a flowchart explaining a subject search process. It is a figure explaining the search end area of subject search processing. It is a flowchart explaining a subject specification process. 13 is a flowchart illustrating another example of the subject specification process. It is a flowchart explaining a setting process of a reference size. It is a flowchart explaining the calculation process of a screen size. FIG. 8 is a transition diagram illustrating automatic composition determination in composition setting “auto”. FIG. 3 is a diagram showing the position of the subject at the outermost periphery and the screen size. 9 is a flowchart illustrating a reference size calculation process in a composition setting “auto”. 11 is a flowchart illustrating a calculation process of a horizontal subject position ratio. 9 is a flowchart illustrating a calculation process of a vertical subject position ratio. 9 is a table showing the number of faces for estimating a subject position. 6 is a flowchart illustrating a process of automatic zoom control. 5 is a flowchart illustrating a zoom operation. FIG. 11 is a transition diagram illustrating automatic composition determination in composition setting “auto” according to the second embodiment.

(First embodiment)
Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings. FIG. 1 is a block diagram illustrating a configuration example of a digital camera 100 having an automatic zoom function as an example of an imaging apparatus including a zoom control device according to the present embodiment.

  The lens barrel 101 holds a lens group inside. The zoom lens 102 adjusts the focal length by moving in the optical axis direction of the lens barrel 101, and optically changes the angle of view (moves the zoom position). The focus lens 103 performs focus adjustment by moving in the optical axis direction of the lens barrel 101. An anti-shake lens (image blur correction lens) 104 corrects image blur caused by camera shake or the like. A diaphragm and a shutter 105 for adjusting the amount of light are used for exposure control. Note that, in the present embodiment, the digital camera 100 is an imaging device in which the lens barrel 101 and the camera body are integrally configured, but is not limited to this. The present embodiment is also applicable to an imaging system including a camera body and an interchangeable lens that is detachable from the camera body.

  The image sensor 106 receives light that has passed through the lens barrel 101 and generates an image signal by converting a subject image into an electric signal by photoelectric conversion. The image sensor 106 is a CCD (charge coupled device) type or CMOS (complementary metal oxide semiconductor) type image sensor or the like. An image signal from the image sensor 106 is input to an image processing circuit 107, and various processes such as a pixel interpolation process and a color conversion process are performed. The image data after various processes is stored in the image memory 108. The image memory 108 is a storage device such as a DRAM (Dynamic Random Access Memory) or an SRAM (Static Random Access Memory).

  The display unit 109 includes a TFT LCD (thin film transistor driven liquid crystal display) and the like, and displays a captured image (image data) and specific information (for example, imaging information). By displaying information such as a live view on a captured image, an electronic viewfinder (EVF) function for a photographer to adjust the angle of view can be provided.

  The aperture shutter drive unit 110 calculates an exposure control value (aperture value and shutter speed) based on luminance information obtained by the image processing in the image processing circuit 107, and drives the aperture and the shutter 105 based on the calculation result. . Thereby, automatic exposure (AE) control is performed. The image stabilizing lens driving unit 111 calculates a shake amount applied to the digital camera 100 based on shake detection information from an angular velocity sensor such as a gyro sensor. The image stabilizing lens 104 is driven to cancel (reduce) the amount of shake applied to the digital camera 100 according to the calculation result.

  The focus lens driving unit 112 drives the focus lens 103. In the present embodiment, the digital camera 100 performs automatic focus adjustment (AF) control by a contrast method. That is, the focus lens driving unit 112 drives the focus lens 103 based on the focus adjustment information (contrast evaluation value) of the photographing optical system obtained by the image processing in the image processing circuit 107 so that the subject is in focus. However, the present invention is not limited to this, and as the AF control other than the contrast method, a phase difference AF method may be used, or AF control is performed by a plurality of methods such as a combination of a contrast method and another method. A configuration may be used.

  The zoom lens driving unit 113 drives the zoom lens 102 according to a zoom operation instruction. The operation unit 117 includes a zoom lever or a zoom button as a zoom operation member for the photographer to instruct the camera to perform zooming. The system control unit 114 detects the operation amount and operation direction of the zoom operation member used for the zoom instruction operation, calculates the zoom drive speed and the zoom drive direction, and moves the zoom lens 102 along the optical axis according to the calculation result. Perform control.

The image data generated by the photographing operation is sent to the recording unit 116 via the interface (I / F) unit 115 and recorded. The image data is recorded on an external recording medium, the nonvolatile memory 118, or both. The external recording medium is a memory card or the like used by being attached to the digital camera 100. The non-volatile memory 118 is a storage medium built in the digital camera 100. The memory 118 stores, in addition to the program data and the image data, setting information of the digital camera 100 and information such as a zoom-in position in an auto-zoom function described later.
The operation unit 117 includes, in addition to the zoom operation members, a release switch for instructing the start of photographing, an auto zoom operation switch for instructing start and end of the auto zoom function, and the like. A signal from the operation unit 117 is sent to the system control unit 114.

  The system control unit 114 includes an arithmetic device such as a CPU (Central Processing Unit). The system control unit 114 controls the entire digital camera 100 by sending a control command to each unit according to the operation of the photographer. The system control unit 114 executes various control programs stored in the memory 118, for example, a program for performing control of the image sensor 106, AE / AF control, zoom control (including automatic zoom processing), and the like.

  In order to maintain a focused state when the angle of view is changed by optical zoom, when the lens barrel 101 is a rear focus type, it is necessary to move the focus lens 103 to an appropriate focus position according to the position of the zoom lens 102. . Such control is called computer zoom (CZ) control, and is performed by the CZ control unit 119 in the system control unit 114. FIG. 2 is a diagram illustrating the relationship between the focal length of the zoom lens and the focus position for each subject distance. FIG. 2 is a graph showing the relationship between the focal length of the zoom lens and the focus position where focus is achieved, as a data table indicating the distance to the subject. In the present embodiment, this table is called a focus cam table. In FIG. 2, the horizontal axis indicates the focal length corresponding to the zoom position, and the vertical axis indicates the focus position. A distance (subject distance) from the digital camera 100 to the subject is illustrated next to each graph line.

  The system control unit 114 controls the focus lens driving unit 112 when performing the AF control, and performs a scanning operation by moving the focus lens 103 within a predetermined range. The focus position, which is the focal point, is detected by a known method using a contrast evaluation value or the like obtained during the scanning operation. The subject distance is measured by referring to the focus cam table using the zoom position and the focus position at that time.

  Next, control related to the auto zoom function in the system control unit 114 will be described. As shown in FIG. 1, the system control unit 114 includes a CZ control unit 119, an electronic zoom control unit 120, an auto zoom control unit (hereinafter, referred to as an AZ control unit) 122, and a subject detection unit 123.

  The digital camera 100 has an optical zoom function and an electronic zoom function, and the CZ control unit 119 and the zoom lens driving unit 113 control the optical zoom. The CZ control unit 119 detects a zoom position of the zoom lens 102 at a predetermined control cycle during a zoom operation. Then, the CZ control unit 119 performs control to drive the focus lens 103 so as to follow the focus cam table at the subject distance measured by the AF control according to the detected zoom position. This makes it possible to perform the optical zoom operation while maintaining the in-focus state.

On the other hand, the electronic zoom control unit 120 and the image memory 108 are in charge of controlling the electronic zoom. The electronic zoom control unit 120 realizes an electronic zoom function by cutting out data in the target area from the image data transferred to the image memory 108. In addition, the electronic zoom control unit 120 realizes a smooth electronic zoom display by causing the display unit 109 to display the image to be captured by the image sensor 106 while gradually increasing the range to be cut out at the frame rate cycle.
The subject detection unit 123 sequentially detects a desired subject area from the image data stored in the image memory 108. In the present embodiment, a subject detection method (face detection processing, color detection processing) for detecting a subject (a face such as a person or an object) based on face information or color information included in image data will be described.

  The face detection process is a process of detecting a face region existing in image data by a known algorithm. For example, the subject detection unit 123 extracts a feature amount from a square partial region on the image data, and compares the feature amount with a previously prepared face feature amount. When the correlation value between the two exceeds a predetermined threshold, the subject detection unit 123 determines that the partial area is a face area. This determination processing is repeated while changing the combination of the size, arrangement position, and arrangement angle of the partial area, so that various face areas existing in the image data can be detected.

  In the color detection process, a process of storing color information of a subject area designated according to a subject designation method described later as a characteristic color is executed. The color detection process is performed when the subject to be detected is an object (a “thing” other than a person). As the color information, an RGB signal, a luminance signal (Y signal), a color difference (RY, BY) signal, which is an output signal from the image processing circuit 107, and the like are used. When detecting a subject, the subject detection unit 123 divides the image data into a plurality of partial areas, and calculates an average value of luminance and color difference for each partial area. In addition, the subject detection unit 123 compares the previously stored characteristic color information with the color information of each area at the time of subject detection, and sets a partial area in which the difference between the luminance and the color difference is equal to or less than a predetermined amount as a subject area candidate. A process is performed in which a group of adjacent partial regions in the candidate of the subject region is set as the same color region, and a region where the same color region has a predetermined size range is set as a final subject region.

  The subject detection unit 123 calculates the size of the subject area on the image data by using the subject distance information measured by the CZ control unit 119 and the focal length information of the zoom lens 102 together with the face information and the color information. . The posture detection unit 124 detects the posture (for example, the normal position / above the grip / below the grip) of the digital camera 100 based on the information of the acceleration sensor. The shake detection unit 125 determines the shake state of the digital camera 100 based on angular velocity information and the like from the gyro sensor. The shake detection unit 125 determines that the camera is in a hand-held state when the shake amount (detected value) applied to the gyro sensor or the like is equal to or more than a predetermined amount (threshold), and fixes the camera to a tripod or the like when the shake amount (detection value) is less than the predetermined amount. It is determined that the state has been performed. The acceleration sensor and the gyro sensor used for the posture detection and the shake detection may be configured to also serve as the sensor of the detection unit for acquiring the control information of the image stabilizing lens driving unit 111.

Next, an overview of the auto zoom function and the AZ control unit 122 according to the present embodiment will be described. In a camera not equipped with an auto-zoom function, the following operation is required when a photographer moves in a frame while moving in a telephoto state while waiting for a photo opportunity.
First, the photographer searches for a subject after performing a zoom-out operation by operating the zoom operation member. After searching for the subject, the photographer performs a zoom operation again to adjust the angle of view until the desired angle of view is obtained. In addition, even when the size of the subject image changes due to the movement of the subject, the photographer has to operate the zoom operation member to adjust the size of the subject image.

  On the other hand, in a camera equipped with an auto-zoom function, after the photographer sets the auto-zoom function, an operation of designating a subject with a touch panel or the like may be performed to designate a subject to be photographed. When the auto-zoom function is set, the zoom operation is automatically performed so that the designated subject fits in a predetermined size near the center of the image. In addition to the touch panel operation, the subject can be specified by specifying the subject near the center of the screen when the photographer operates a specific button, or automatically selecting the main subject from among the subjects detected by the camera. There is a selection method.

  The subject detection unit 123 calculates the position and size of the subject area specified from the image memory 108 on the image data. This processing is continuously performed on the sampled image data every time an image is displayed as a live view, so that the movement of the subject can be tracked. The AZ control unit 122 starts a zoom-out operation when an image of the tracked subject is detected in a zoom-out area described later or when the detected image of the subject becomes larger than a predetermined size. That is, the AZ control unit 122 instructs the CZ control unit 119 or the electronic zoom control unit 120 to zoom out in the wide angle direction. When the subject is detected in the zoom-in area and the subject image falls within a predetermined size range, a zoom-in operation is performed on the telephoto side. By such a process, the photographer does not care about the zoom operation, but only needs to move the camera so that the image of the desired subject is included in the screen. Even if the subject is about to go out of frame, the zoom magnification is automatically changed, so that the angle of view can be adjusted more easily.

  Next, a start condition of the zoom-out operation or the zoom-in operation will be described with reference to FIGS. FIG. 3 is an explanatory diagram of a process for preventing a subject (object) from going out of the frame. FIG. 4 is an explanatory diagram of a process for preventing a subject (person) from going out of the frame.

  3 and 4, frames 300a and 300b are first tracking frames (hereinafter referred to as mono-tracking frames) for tracking a subject (object), and frames 400a to 400f track the subject (human face). This is a second tracking frame (hereinafter, referred to as a face tracking frame). Hereinafter, when a subject can be applied to both a person and an object, the mono tracking frame and the face tracking frame may be collectively referred to as a subject tracking frame. The subject tracking frame is displayed on the EVF screen (electronic viewfinder screen) of the display unit 109 so as to surround the subject image so that the subject specified by the photographer can be recognized. The position and size of the subject tracking frame on the screen are sequentially calculated by the subject detection unit 123 based on the face information and the color information, and are updated at the frame rate cycle.

  With reference to FIG. 3, a process for preventing the subject (airplane) from going out of the frame out of the screen will be described. In FIG. 3A, a zoom-out area ZO (hereinafter, referred to as a ZO area) indicates an area outside a predetermined ratio with respect to the entire angle of view (entire screen) displayed by the EVF. For example, assume that the center point of the screen is 0%, the entire screen is 100%, and a position that is 80% of the entire screen is set as a boundary of the ZO region. In this case, an area of 80 to 100% of the entire screen is a ZO area. When a part of the mono tracking frame 300a enters the ZO area, the AZ control unit 122 performs control to start a zoom-out operation. In addition, the AZ control unit 122 stores the zoom magnification (corresponding to the zoomed-in angle of view) before the zoom movement in the memory 118. The target zoom magnification and zoom speed during the zoom-out operation are set in advance according to the size and moving speed of the subject image. Further, the target zoom magnification and the zoom speed may be appropriately calculated according to the size and the moving speed of the subject image. The zoom-out operation is executed according to the target zoom magnification and zoom speed. This makes it possible to effectively prevent the subject from going out of the frame.

  FIG. 3B illustrates an angle of view when a zoom-out operation corresponding to a predetermined zoom change amount is performed from the angle of view in FIG. A zoom-in area ZI (hereinafter, referred to as a ZI area) illustrated in FIG. 3B indicates an area inside a predetermined ratio with respect to a zoom-in angle of view (an angle of view before zoom-out) 301 in a subject search state. . For example, assume that the center point of the screen is 0%, the zoom-in angle of view 301 is 100%, and a position that is 70% of the zoom-in angle of view 301 is set as the boundary of the ZI area. In this case, a region of 0 to 70% of the entire zoomed-in angle of view 301 is a ZI region. At this time, for example, when the zoom-out magnification is 1/2, the zoom-in angle of view 301 is 50% of the entire screen. Therefore, the boundary of the ZI area is 70% × (１ ／) = 35% with respect to the entire screen, and the ZI area can be said to be an area of 0 to 35% with respect to the entire screen. When the photographer changes the direction of the camera so that the mono tracking frame 300b falls within the ZI area, the AZ control unit 122 performs control to start a zoom-in operation.

  Next, a process for preventing a subject (person) from going out of the frame outside the screen will be described with reference to FIG. When the subject is a person, a zoom-out operation starts when a part of the face tracking frame enters the ZO region, and a zoom-in operation is performed when the face tracking frame is within the ZI region. When the subject is a person, unlike the case of the object, the moving direction of the subject can be predicted to some extent, so that the ZO region and the ZI region are set according to the region of the predicted moving direction. Also, in hand-held shooting, the subject may go out of frame due to the influence of camera shake or the like. However, when the subject goes out of the frame due to camera shake or the like, the photographer can re-frame in by performing an operation of putting the subject into the frame. Here, when the ZO area is set in the upper part of the screen, the face tracking frame enters the ZO area even when a person is placed near the center and shooting is performed, and there is a case where the zoom-out is performed unintentionally. Therefore, when the subject is a person and is in a hand-held state (a state in which the photographer holds the camera), a ZO area is not set at the top of the screen in consideration of the framing operation of the photographer.

  As described above, in the present embodiment, when the face is detected by the subject detection unit 123, the AZ control unit 122 changes the ZO area and the ZI area according to the posture and the shaking state of the camera. The posture of the camera is detected by the posture detection unit 124, and the swing state is determined from the detection result by the shake detection unit 125. The detection result by the shake detection unit 125 is a detection result of whether the camera is in a hand-held state. Hereinafter, a specific description will be given with reference to FIGS.

  FIG. 4A shows a ZO area and a ZI area which are set when the photographer holds the camera in a normal position by hand. In such a shooting scene, when the subject moves in the horizontal direction and goes out of the frame, the position of the subject in the screen moves in the horizontal direction (longitudinal direction) with respect to the screen at the normal position. Therefore, both the ZO area and the ZI area are arranged in a vertical band (short direction) with respect to the screen at the normal position. The ZO area is set as a vertical band located near both ends in the horizontal direction on a horizontally long rectangular screen, and the ZO area is set as a vertical band located at the center of the screen. In this case, when the face tracking frame 400a enters the ZO area, the AZ control unit 122 determines the start of zoom-out, and performs zoom-out operation control corresponding to a predetermined zoom magnification. When the face tracking frame 400b is included in the ZI area, the AZ control unit 122 determines the start of zoom-in, and performs zoom-in operation control corresponding to a predetermined zoom magnification up to the zoom return position. By setting the ZO area and the ZI area in this way, it is possible to effectively prevent the subject (person) from going out of the frame.

  FIG. 4B shows the ZO area and the ZI area set when the camera posture is changed in the same shooting scene and the photographer holds the camera in the vertical position below or above the grip. . In this case, the ZO area and the ZI area are arranged in a vertical band shape in the vertical direction (longitudinal direction) with respect to the screen at the vertical position. That is, the ZO area is set in a vertical band shape located near both ends in the horizontal direction on a vertically long rectangular screen, and the ZI area is set in a vertical band shape positioned at the center of the screen. In this case, when the face tracking frame 400c enters the ZO area, the AZ control unit 122 determines the start of zoom-out, and performs zoom-out operation control corresponding to a predetermined zoom magnification. When the face tracking frame 400d is included in the ZI area, the AZ control unit 122 determines the start of zoom-in, and performs zoom-in operation control corresponding to a predetermined zoom magnification up to the zoom return position. By setting the ZO area and the ZI area in this manner, the movement of the subject in the horizontal direction can be detected, and frame out can be effectively prevented.

  FIG. 4C shows the ZO area and the ZI area set when the detection state of the shake detection unit 125 is the fixed state. The fixed state is a state in which the camera is fixed by a tripod or the like, and there is no possibility of frame out due to camera shake. Furthermore, if the subject zooms in when the subject is not in the frame near the center of the screen, the operation may cause a frame out. Therefore, a ZO area is set on the entire peripheral portion of the screen, and a ZI area is set inside the zoomed-in angle of view. That is, the ZO area is set in a rectangular band shape located near the vertical and horizontal edges of the screen, and the ZI area is set in a rectangular shape located in the center of the screen. In this case, when the face tracking frame 400e enters the ZO area, the AZ control unit 122 determines the start of zoom-out, and performs zoom-out operation control corresponding to a predetermined zoom magnification. When the face tracking frame 400f is included in the ZI area, the AZ control unit 122 determines the start of zoom-in, and performs zoom-in operation control corresponding to a predetermined zoom magnification up to the zoom return position.

  As described above, in the present embodiment, the respective ranges of the ZO region and the ZI region are dynamically changed according to changes in the posture of the camera and the shooting state (hand-held state / fixed state). As a result, it is possible to effectively prevent the subject from going out of the frame while preventing malfunction due to camera shake or the like. Note that the ZO area or the ZI area may be changed according to one of the posture of the camera and the shooting state (hand-held state / fixed state), or only one of the ZO area and the ZI area may be changed. Good.

  Next, a zoom operation for maintaining the ratio of the subject image to the screen within a predetermined range will be described. In the present embodiment, when the size of the detected subject image changes more than a predetermined multiple of the reference size, the zoom operation is automatically performed so that the size of the subject image is kept within a predetermined range from the reference size. (Size holding control) is performed. FIG. 5 is a diagram illustrating a process for maintaining the size of the subject (person) image in the screen. FIG. 6 is an explanatory diagram of composition setting of a subject (person).

  First, the setting of the range (composition) within which the subject to be tracked falls within the screen will be described with reference to FIG. FIG. 6A illustrates a screen display when the composition setting is set to “manual”. When "manual" is set, the photographer changes the size of the face to be tracked by performing a manual zoom operation by operating the zoom lever while viewing the image of a person on the screen. The size of the subject image at that time is stored in the memory 118 as a reference size. FIG. 6B illustrates a screen display when the composition setting is set to “face”. When the composition of the “face” is set, the size of the face that fits on the screen is calculated as the reference size according to the posture of the camera and the direction of the face, and stored in the memory 118. Similarly, FIG. 6C illustrates a screen display when the composition setting is set to “upper body”, and FIG. 6D illustrates a screen display when the composition setting is set to “whole body”. For example. A reference size is calculated so as to have a size set on the screen, and stored in the memory 118.

  FIG. 6E illustrates a screen display when the composition setting is set to “auto”. When “auto” is set, the AZ control unit 122 determines an appropriate composition based on the position of the subject in the screen, the size of the subject, the number of subjects, the posture of the camera, and the like. Further, a reference size is calculated so that the determined composition is obtained and stored in the memory 118. The method of calculating the reference size will be described later with reference to FIG.

  The photographer can perform an operation of changing the composition setting by operating the left and right buttons of the operation unit 117 or making a selection on the setting menu from the shooting screen. When a composition setting change operation is performed, the AZ control unit 122 updates the composition setting information stored in the memory 118. Although FIG. 6 shows an example in which the composition of “manual”, “face”, “upper body”, “whole body”, and “auto” is set when the subject is a person, the composition setting is not limited to this. Not something. Only a part of the five exemplified compositions may be set, or another composition may be included. When the subject is an object, for example, each composition of “manual”, “large”, “medium”, “small”, and “auto” may be set.

  With reference to FIG. 5, a description will be given of a zoom operation for keeping the ratio of the subject image occupying the screen within a predetermined range, taking a case where the composition setting is “manual” as an example. FIGS. 5A to 5C illustrate a zoom operation automatically performed by a camera when a subject (person) approaches the camera. This is a zoom-out operation for keeping the ratio of the subject image to the screen within a predetermined ratio. In FIG. 5, the face tracking frames 500a to 500f are displayed so as to surround the face area as the characteristic area of the person who is the subject. Therefore, the size of the face tracking frame will be described here as the subject size.

  FIG. 5A shows an angle of view when a subject is designated according to a subject designation method described later. The size of the face tracking frame 500a when the subject is specified is stored in the memory 118 as a reference subject size (reference size). FIG. 5B shows the angle of view when the subject approaches the camera without changing the zoom magnification from the state of FIG. 5A. For example, a size that is 150% of the size of the face tracking frame 500a that is the reference subject size is set as the start size of the zoom-out operation. When the relationship between the subject tracking frame (face tracking frame) is “face tracking frame 500b> face tracking frame 500a × 150%”, that is, the tracking frame has changed beyond a predetermined change amount with respect to the reference size. At this time, the AZ control unit 122 determines the start of the zoom-out operation.

  FIG. 5C shows an angle of view obtained by zooming out from the angle of view 501 of FIG. 5B by a predetermined zoom magnification, and a face tracking frame 500c. Here, the predetermined zoom magnification is set to 1 / 1.5 in consideration of the change rate (150%) of the face tracking frame size to the reference subject size when starting the zoom-out operation. Thereafter, when the subject further approaches the camera, the subject image can be kept within a predetermined ratio by zooming out further to the wide angle side. Therefore, the photographer can concentrate on only the operation of the release switch.

  On the other hand, FIGS. 5D to 5F illustrate a zoom operation performed automatically by the camera when a person who is the subject moves away from the camera. This is a zoom-in operation for keeping the ratio of the subject image on the screen within a predetermined range. FIG. 5D shows the angle of view when a subject is designated according to a subject designation method described later. The size of the face tracking frame 500d when the subject is designated is stored in the memory 118 as the reference subject size (when the composition setting is "manual").

  FIG. 5E shows the angle of view when the subject moves away from the camera without changing the zoom magnification from the state of FIG. 5D. For example, a size that is 50% of the size of the face tracking frame 500d, which is the reference subject size, is set as the start size of the zoom-in operation. When the relationship between the face tracking frames becomes “face tracking frame 500e <face tracking frame 500d × 50%”, it is determined that the tracking frame has changed by more than a predetermined change amount with respect to the reference size. When this determination condition is satisfied and the face tracking frame 500e is included in the ZI region, the AZ control unit 122 determines the start of the zoom-in operation. Here, the ZI area is set inside the angle of view 502 obtained by zooming in by a predetermined zoom magnification with respect to the angle of view in FIG.

  FIG. 5F shows an angle of view (corresponding to the angle of view 502) obtained by zooming in from the angle of view of FIG. 5E by a predetermined zoom magnification, and a face tracking frame 500f. Here, the predetermined zoom magnification is set to 1 / 0.5 times in consideration of the rate of change (50%) of the face tracking frame size to the reference subject size when starting the zoom-in operation.

  FIG. 3 illustrates the process of preventing frame-out when the subject is an object, and FIG. 4 illustrates the case where the subject is a person. Further, FIG. 5 has described the process of setting the ratio of the size of the subject image on the screen to a predetermined range when the subject is a person. Note that, even when the subject to be tracked is an object, the zoom operation start determination for controlling the holding of the size of the subject image shown in FIG. 5 may be performed as in the case where the subject is a person. Further, even when the composition setting is other than "manual", the zoom operation is automatically performed according to the respective reference sizes, similarly to FIG.

Next, the processing of the auto zoom function will be described with reference to FIGS. FIG. 7 is a flowchart illustrating an example of the entire processing of the auto zoom function. The following auto zoom function is performed based on a command from the system control unit 114 (AZ control unit 122) unless otherwise specified.
First, in S700, the AZ control unit 122 determines the operation state of the auto zoom operation switch of the operation unit 117. When the auto zoom operation switch is pressed, the process proceeds to S701, and when the switch is not pressed, the determination process of S700 is repeated. A subject search process is executed in S701, and a subject designation process in S702, a basic size setting process in S703, and an auto zoom control in S704 are sequentially executed.

  Immediately after the processing of the auto zoom function is started, or when the subject search button of the operation unit 117 is pressed (Yes in S705), the subject search processing of S701 is executed. With reference to FIG. 8, the subject search processing executed in step S701 will be described. When the subject to be photographed is far away from the camera, the subject detection unit 123 may not be able to detect the subject if the feature area of the subject is too small even if the subject image exists in the screen. In such a scene, the automatic zoom function based on the subject information cannot be executed. Therefore, a subject search process of performing a zoom-in operation while detecting a subject up to a predetermined focal length is executed.

  In S800, a process is performed to determine whether the subject search button has been pressed immediately after the start of the auto zoom, or not immediately after the start of the auto zoom, but during the auto zoom. In the case of the former positive determination result (yes) in S800, the process proceeds to S801, and in the case of a negative determination result (no), the process proceeds to S803. In S801, a process of acquiring a subject search end focal length (hereinafter, referred to as a search end focal length) is performed. In this embodiment, immediately after the start of the auto zoom, the search end focal length is set to a focal length of 85 mm in 35 mm format conversion. This focal length is a focal length of a standard lens which is generally considered to be suitable for photographing a person. Even when there is no subject (person) or a face cannot be detected (in a backward state, etc.), the search is performed after the search is completed. A waiting state for subject detection can be set at an appropriate angle of view. However, the focal length is not limited to 85 mm, and may be another focal length (for example, 50 mm or 100 mm). Further, a configuration in which the photographer can change the search end focal length from the menu setting, or a configuration in which the search end focal length can be changed according to the shooting mode or the subject to be shot may be adopted. When the process proceeds from step S801 to step S802, the focal length at the start of the mode is compared with the search end focal length, and it is determined whether the focal length is wider than the search end focal length or on the telephoto side. If the focal length at the start of the auto-zoom is on the telephoto side of the search end focal length (no in S802), the processing of the subject search ends. If the focal length at the start of the auto zoom is wider than the search end focal length (Yes in S802), the process proceeds to S805.

  On the other hand, if it is determined in S800 that the object search is performed by operating the object search button, a process of acquiring the search end focal length is executed in S803. In the present embodiment, the search end focal length in the operation of the subject search button is set to a focal length on the telephoto side by 2.5 times the focal length when the operation is performed. However, this magnification is an example, and another magnification may be used. Further, a configuration in which the photographer can change the search end focal length from the menu setting, or a configuration in which the search end focal length can be changed according to the shooting mode or the subject to be shot may be adopted. Proceeding from S803 to S804, the search end focal length obtained in S803 is compared with the focal length at the tele end (telephoto end), and it is determined whether the search end focal length is wider or telephoto than the focal length at the tele end. . If the search end focal length is on the telephoto side of the focal length at the telephoto end, the subject search process ends. If the search end focal length is on the wider angle side than the focal length at the tele end, the process proceeds to S805. In step S805, the AZ control unit 122 sets the zoom change amount corresponding to the search end focal length acquired in step S801 or S803 in the CZ control unit 119 or the electronic zoom control unit 120, and instructs the zoom in operation.

  Next, in S806, it is determined whether or not the current focal length has reached the search end focal length. If the current focal length has reached the search end focal length, the process advances to step S809 to stop the zoom-in operation and then end the subject search process. If it is determined in step S806 that the current focal length has not reached the search end focal length, the process proceeds to step S807. Step S807 is processing for determining whether or not a subject has been detected by the subject detection unit 123. If it is determined in S807 that the subject has been detected, the process proceeds to S808, and if it is not determined that the subject has been detected, the process returns to S806. In S808, it is determined whether or not the position where the subject is detected is within a predetermined search end area on the screen. It is assumed that the predetermined search end area is determined by a threshold value (position determination reference value) set in advance according to the subject. If the result of determination is that the detection position of the subject detected in S807 is within the predetermined search end area on the screen, the flow advances to S809 to stop the zoom-in operation and end the subject search processing. If it is determined in step S808 that the detected position of the subject is not within the predetermined search end area on the screen, the process returns to step S806, and the determination processing of steps S806 to S808 is continued.

  Here, the predetermined search end area described in S808 will be described with reference to FIG. In photographing a specific subject (person or object), the subject to be photographed is often arranged near the center of the screen. Further, when a subject not to be photographed is detected around the screen and the subject search processing is terminated, the subject search processing may be completed before the subject to be photographed is detected. In this case, the auto zoom function as intended by the photographer cannot be executed. Therefore, in the present embodiment, a search end area as a determination area is set so that the object search processing is ended only when an object is detected in an area near the center of the screen. FIG. 9A illustrates a search end area 900a set when the subject is an object (airplane). In the case of an object, a predetermined ratio, for example, a range of 70% from the center of the screen is set as the subject search end area in any of the vertical and horizontal directions of the screen. On the other hand, FIG. 9B illustrates a search end area 900b set when the subject is a person. In the case of a person, the search is terminated so that the upper region of the screen includes a range of 100% from the center of the screen, and the other directions (left and right directions, downwards) include a predetermined ratio from the center of the screen, for example, a range of 70%. The area is set. When the subject is a person, there is a possibility that the face tracking frame 902 of the subject to be photographed is also arranged at the top of the screen. Therefore, even when a subject is detected in the upper part of the screen, the search end area is set so that the subject search processing ends.

  Next, with reference to FIG. 10, the subject designation processing shown in step S702 in FIG. 7 will be described. FIG. 10A is a flowchart illustrating an example of processing in which a user specifies a desired subject using a touch panel included in the operation unit 117. In this example, it is assumed that the user performs an operation of touching and specifying a subject image displayed on the display unit 109. In S1000, AZ control unit 122 determines whether or not the touch panel has been pressed. When the touch panel is pressed, the process proceeds to S1001, and when the touch panel is not pressed, the determination process of S1000 is repeated.

  In S1001, the AZ control unit 122 acquires information on the position touched by the user (touch position). In the next step S1002, the AZ control unit 122 notifies the subject detection unit 123 of the touch position, and the subject detection unit 123 performs face detection near the touch position. If a face is detected near the touch position in S1002, it is determined that the main subject is a person, and the process proceeds to S1003. If no face is detected near the touch position in S1002, the main subject is determined to be an object other than a person, and the process proceeds to S1004.

  In S1003, the AZ control unit 122 performs control to store the face information of the person to be automatically tracked in the memory 118. Specific face information includes information such as a face size when a subject is specified, a face detection position, and a face direction. In a camera having a face authentication function, identification information such as an authentication ID is also stored in the memory 118. In step S <b> 1004, the AZ control unit 122 performs control to store the characteristic color near the touch position in the memory 118 as color information of an automatic tracking target. As specific color information, there are information such as a characteristic color at the time of subject designation, its luminance, a color difference value, the size of the same color area, the position of the center of gravity of the same color area, and the like. In the following description, face information and color information are collectively referred to as subject information (including subject size, subject detection position, and the like).

After the subject information is stored in the memory 118 in S1003 or S1004, the process proceeds to S1005, and the subject tracking frame is displayed. In step S <b> 1005, the AZ control unit 122 controls the display unit 109 to display a subject tracking frame (mono tracking frame or face tracking frame) having a size corresponding to the subject size around the subject detection position. Thereafter, the subject designation processing ends.
As described above, in the system control unit 114, the subject detection unit 123 detects the subject at the position specified by the photographer on the display unit 109 or in the vicinity of the specified position. Then, the AZ control unit 122 causes the display unit 109 to display the subject tracking frame. According to the processing example of FIG. 10A, the subject that the photographer wants to track can be easily specified in an intuitive manner. However, the method of designating the subject is not limited to this method. Another method will be described with reference to FIG.

  FIG. 10B is a flowchart illustrating a processing example in which the user designates a subject by using a switch (subject designation switch) different from the auto-zoom operation switch as an operation member configuring the operation unit 117. is there. First, in step S <b> 1006, the AZ control unit 122 displays a frame serving as an indication of subject specification near the center of the screen of the display unit 109. The photographer adjusts the direction of the camera such that the image of the subject to be tracked is positioned near the center of the screen using the frame as a guide. In the next step S1007, the AZ control unit 122 determines whether the subject designation switch has been pressed. If the subject specifying switch is pressed, the process proceeds to S1008. If the switch is not pressed, the determination process of S1007 is repeated in a waiting state.

  In step S1008, the subject detection unit 123 performs face detection near the center of the screen. If a face is detected near the center of the screen, it is determined that the main subject is a person, and the process proceeds to S1009. On the other hand, if no face is detected near the center of the screen, the main subject is determined to be an object other than a person, and the process proceeds to S1010. In S1009, the same processing as in S1003 in FIG. 10A is performed, and in S1010, the same processing as in S1004 in FIG. 10A is performed, and thus description thereof will be omitted. After the subject information is stored in the memory 118 in S1009 or S1010, the process advances to S1011 to display a subject tracking frame (mono tracking frame or face tracking frame), and the subject specification processing ends.

  As described above, in the processing example of FIG. 10B, the subject detection unit 123 detects the subject at or near the center position of the screen of the display unit 109. Then, the AZ control unit 122 causes the display unit 109 to display a subject tracking frame indicating the position of the subject. The user can easily specify a subject even in a camera without an operation member such as a touch panel.

  FIG. 11 is a flowchart illustrating an example of processing for automatically selecting a subject to be tracked from faces detected at the time when an auto zoom operation switch constituting the operation unit 117 is pressed. First, in step S1100, the subject detection unit 123 performs face detection on the entire screen and determines whether a human face has been detected. If at least one face is detected on the entire screen, it is determined that the main subject is a person, and the process proceeds to S1101. If no face is detected in the screen, the flow shifts to S1105.

  In step S1101, the subject detection unit 123 selects the face as the main face when the detected face is a single person. When there are a plurality of detected faces, the subject detection unit 123 selects a main face as a subject to be tracked from the faces. As a criterion for selecting the main face, for example, there is a method of preferentially selecting a face whose face detection position is located closer to the center of the screen. In addition, when there are a plurality of faces at substantially the same position, there is a method of selecting a face having a larger size as a main face. Further, in the case of a camera having a face authentication function, there is a method in which, when there is a face registered for authentication, that face is preferentially set as the main face.

  Proceeding from S1101 to S1102, the AZ control unit 122 stores the face information of the selected main face in the memory 118, and then proceeds to S1103 to display a face tracking frame on the screen. In the next step S1104, a face change determination is made. That is, when a plurality of faces are detected, the main face automatically selected is not necessarily the face intended by the photographer. Therefore, the photographer can change the main face. At this time, if the photographer presses a switch (auto-zoom operation switch or the like) of the operation unit 117 to specify a desired face, another face which is not selected as the main face from the faces whose face tracking frames have been detected is selected. Is performed to change the main face. When the main face is changed in S1104, the process returns to S1102 again, and the face information to be stored in the memory 118 is updated. In step S1103, the face tracking frame is changed to the size and detection position of the newly selected main face.

  On the other hand, when the process proceeds from S1100 to S1105, it is determined that the main subject is an object other than a person, and the AZ control unit 122 stores the characteristic color near the center of the screen in the memory 118 as color information for automatic tracking. I do. In the next step S1106, after the mono tracking frame is displayed on the screen, the subject specifying process ends. The basic processes in S1102, S1105, and S1103 (S1106) in FIG. 11 are the same as those in S1003, S1004, and S1005 in FIG.

  As described above, in the processing example of FIG. 11, the subject detection unit 123 in the system control unit 114 performs face detection on the entire screen of the display unit 109. When a plurality of faces are detected, the AZ control unit 122 controls the display unit 109 to display a subject tracking frame indicating the position of the first face as a subject from the plurality of faces. In addition, when the first face is changed to the second face as the subject, the AZ control unit 122 performs control to cause the display unit 109 to display a subject tracking frame indicating the position of the second face. The photographer can easily specify a desired subject with a smaller number of operations.

  Regarding the subject designating process, in the case of a camera having both a touch panel and a subject designating switch constituting the operation unit 117, both the method of FIG. 10A and the method of FIG. You may. After the subject change is determined in S706 shown in FIG. 7, the subject is designated by the method shown in FIG. 10A or FIG. The subject can also be specified by a method. The subject specifying process to be applied may be changed according to the flow of the process. When the subject specifying process (S702 in FIG. 7) is completed, the process proceeds to S703 in FIG. 7, and a process of setting the size of the tracked subject as a reference size is executed.

  The reference size setting process in S703 shown in FIG. 7 will be described with reference to FIGS. FIG. 12 is a flowchart illustrating an example of a reference size setting process when a subject is a person. First, in S1200, the AZ control unit 122 determines whether or not the composition setting stored in the memory 118 is the “manual” described above. If the composition setting is “manual”, the flow proceeds to S1201; otherwise, the flow proceeds to S1203.

  In S1201, the AZ control unit 122 makes a determination of subject detection. The AZ control unit 122 determines whether or not a face is detected when the subject is a person, and whether or not the same characteristic color is detected when the subject is an object. The determination process of S1201 is repeated until a subject is detected, and the process proceeds to S1202 when a subject is detected. In step S1202, the AZ control unit 122 sets the size of the detected subject image as a reference size, stores the reference size in the memory 118, and ends the process. Thus, when the composition setting is “manual”, the size (detection size) of the subject image first detected when the reference size setting process is started is stored in the memory 118 as the reference size.

  On the other hand, in step S1203, the AZ control unit 122 calculates the screen size based on the posture information of the camera determined by the posture detection unit 124. This screen size is used for calculating a reference size in S1211, S1704, S1705, S1806, and S1807, which will be described later. With reference to FIG. 13, the process of calculating the screen size will be described. In the present embodiment, a case where the screen size of a still image is long side × short side = 640 × 480 and the screen size of a moving image is long side × short side = 640 × 360, but the screen size is not limited to this. It is not done.

  In S1300, the posture detection unit 124 determines whether the photographer holds the camera in the normal position or in the vertical position. If it is determined that the photographer holds the camera at the normal position, the process proceeds to S1301, whereas if the photographer holds the camera at the vertical position, the process proceeds to S1306. In S1301, a process of setting the long side size 640 to the horizontal screen size is performed. In the next step S1302, a determination process is performed to determine whether or not the current setting mode is a mode in which a still image and a moving image can be simultaneously shot (still image / moving image simultaneous shooting mode). In the still image / moving image simultaneous shooting mode, a photographer can simultaneously shoot a still image by pressing a release switch of the operation unit 117 during shooting of a moving image, or a framing operation immediately before shooting a still image as a moving image. This is a mode that can be recorded automatically. If it is determined in step S1302 that the mode is the still image / moving image simultaneous shooting mode, a process of comparing the short side size of the moving image with the short side size of the still image and setting a smaller size as the vertical screen size is executed. This is to prevent the subject from going out of the frame at any angle of view of the still image and the moving image by performing the auto zoom in accordance with the small screen size. Accordingly, in the present embodiment, in the case of the still image / moving image simultaneous shooting mode, the process proceeds to S1304, and the short side size 360 of the moving image is set as the vertical screen size. Thereafter, the screen size calculation processing ends.

  If it is determined in step S1302 that the mode is not the still image / moving image simultaneous shooting mode, the process advances to step S1303. In step S1303, it is determined whether moving image recording is in progress or still image framing is in progress. If it is determined that the moving image is being recorded, the process proceeds to S1304, but if it is determined that the moving image is not being recorded, the process proceeds to S1305. In S1305, 480 which is the short side size of the still image is set as the vertical screen size.

  If it is determined in S1300 that the photographer is in the vertical position, the processing from S1306 to S1310 is performed. The processing from S1306 to S1310 is processing in which the setting of the horizontal screen size and the vertical screen size are reversed from the processing of S1301 to S1305, respectively. In other words, the above description may be replaced by replacing “horizontal” and “vertical”, and a detailed description thereof will be omitted. In the present embodiment, the screen size of the still image is 4: 3 (see FIGS. 15A and 15B), and the screen size of the moving image is 16: 9 (see FIG. 15C). However, the screen size may be changed according to the setting of the aspect ratio. This makes it possible to make the ratio of the image of the person to the screen constant regardless of the aspect ratio. Further, an example in which the screen size is different between the still image and the moving image in the still image / moving image simultaneous shooting mode has been described, but a mode in which a plurality of still images and moving images with different screen sizes are simultaneously shot may be adopted.

  When the screen size is determined in S1203 in FIG. 12, the process proceeds to S1204. If the composition setting stored in the memory 118 is any one of “face”, “upper body”, and “whole body”, the AZ control unit 122 determines in step S1204 to S1209 the face image occupied on the screen according to the composition setting. Determine the size ratio (called face ratio). The face ratio in the present embodiment indicates the ratio of the length of the face tracking frame to the screen in the vertical direction, but may be indicated by the ratio of the length in the horizontal direction or the area ratio. More specifically, for example, it is determined whether the composition setting is “face” in S1204. If the composition is “face” (yes in S1204), the face ratio is determined to be 27% (S1207). If the composition setting is not “face”, the process proceeds to S1205, and it is determined whether the composition setting is “upper body”. If the composition setting is “upper body” (yes in S1205), the face ratio is reduced to 16%. It is determined (S1208). If the composition setting is not “upper body”, the process proceeds to S1206, and it is determined whether the composition setting is “whole body”. If the composition setting is “whole body” (yes in S1206), the face ratio is determined to be 7%. (S1209). After the face ratio is determined in each step from S1204 to S1209, the process proceeds to S1210.

  In S1210, the AZ control unit 122 calculates the reference size by multiplying the face ratio determined in S1204 to S1209 by the vertical screen size calculated in S1203. This is a method of calculating a reference size from a face ratio with respect to a screen size in the vertical direction, assuming a subject (person) in a standing posture. If the direction of the face can be determined by the subject detection unit 123, the direction of the screen size may be determined based on the vertical direction of the face on the screen. Further, after the process of storing the calculated reference size in the memory 118 is performed, the process of setting the reference size ends. Thus, when the composition setting is any one of “face”, “upper body”, and “whole body”, a size having a predetermined face ratio according to the composition setting with respect to the screen size is set as the reference size.

  If the composition setting is not “whole body” in S1206, that is, if it is determined that the composition setting is “auto”, the process proceeds to S1211, and the AZ control unit 122 determines the shooting state, the number of subjects, and the movement of the subjects. An appropriate composition is automatically determined according to the above. The automatic composition determination and composition selection processing in the composition setting “auto” in S1211 will be described with reference to FIG. The composition setting “auto” is a mode in which the composition can be further easily adjusted by the camera automatically selecting an appropriate composition by selecting a shooting scene (automatic composition selection).

  FIG. 14 shows a transition diagram of a composition automatically selected in the present embodiment. In the present embodiment, a composition selection process is performed to select an appropriate composition from the five compositions of the upper body 1400, the whole body 1401, the face 1402, the plural persons 1403, and the manual 1404 according to the shooting scene. In the composition adjustment operation, when photographing a still image, it is necessary to instantaneously make the subject image an appropriate size before photographing. On the other hand, when capturing a moving image, it is necessary to always keep the subject image in an appropriate size while capturing the moving subject, such as following a moving subject or approaching a stationary subject. Further, at the time of shooting a moving image, image data during a zoom-in operation or a zoom-out operation due to a change in composition is also recorded, so that an effective video can be shot by changing the composition at an appropriate timing. As described above, an appropriate composition differs between a still image and a moving image. Thus, during still image framing 1405 and during moving image recording 1406, a process of changing the selected composition depending on the shooting state is executed. That is, in the present embodiment, the first operation state exemplifies a still image shooting preparation state, and the second operation state exemplifies a moving image recording state. As the composition of the selection candidate belonging to the first composition group during the still image framing 1405, any composition is selected from the upper body 1400, the whole body 1401, and the plurality 1403. As the composition of the selection candidate belonging to the second composition group during the moving image recording 1406, any one of the upper body 1400, the whole body 1401, and the face 1402 is selected. That is, the respective compositions of the upper body 1400 and the whole body 1401 are common to the first composition group and the second composition group, the composition of a plurality of persons 1403 is included only in the first composition group, and the composition of the face 1402 is the second composition group. Are included only in the composition group. Note that three or more operation states and a plurality of composition groups (including compositions that can be selected in each operation state) can be set in accordance with the specifications and the like of the imaging device.

  Here, the determination conditions of the shooting scene and the transition of each composition will be described. When the automatic composition determination process S1211 of the composition setting “auto” starts, the upper body 1400 is set as the composition in the initial state. The composition of the upper body 1400 is a composition selected when it is determined that a still image of a single subject is captured during still image framing 1405 or moving image recording 1406. In such a scene, the composition can be appropriately adjusted to the subject by setting the composition of the upper body of the subject within the screen.

  When the movement of the subject to be photographed is detected in the composition of the upper body 1400, or when it is determined that the distance from the camera to the subject is larger than a predetermined distance D1 (first threshold), the composition is set to the whole body 1401. Is changed to The composition of the whole body 1401 is determined when a moving object or a distant object is determined during still image framing 1405 or moving image recording 1406, that is, a scene in which the subject easily shoots out of the screen. It is a composition to be selected. In such a scene, the subject is set such that the subject does not go out of the frame as much as possible by following the subject with a composition that accommodates the whole body of the subject. Further, when the composition of the whole body 1401 detects that the subject to be photographed has stopped for a predetermined time or more, and it is determined that the distance from the camera to the subject is smaller than a predetermined distance D2 (second threshold). Is executed to return the composition to the upper body 1400. The predetermined time is a reference time for determination set in advance.

  Hereinafter, a method for determining whether the subject is moving or stationary will be described. When at least one of the change amount of the position or size of the subject detected by the subject detection unit 123 or the swing amount detected by the shake detection unit 125 while the subject is being detected is larger than a predetermined amount, It is determined that the subject to be photographed is moving. A state in which the amount of change in the position or size of the subject detected by the subject detection unit 123 and the amount of shake detected by the shake detection unit 125 while the subject is being detected are all smaller than a predetermined amount for a predetermined time or more. In this case, it is determined that the subject to be photographed is stationary. The duration of the detection state is measured by a timer.

  Further, a method of calculating the distance from the camera to the subject will be described. First, the reference focal length and the subject distance with respect to the subject size are measured in advance, and the measurement data is stored in the memory 118. The subject distance is calculated by performing an operation of “subject distance = (reference subject distance × focal length × reference subject size) / (reference focal length × detection size)” on the reference value stored in the memory 118. . For example, it is assumed that when the reference values are a reference subject distance of 2 m, a reference focal length of 24 mm, and a reference subject size of 20 pix (pixels), a subject image having a focal length of 120 mm and a detection size of 40 pix is detected. In this case, the subject distance is calculated as (2 m × 120 mm × 20 pix) / (24 mm × 40 pix) = 5 m.

  Next, the composition when a plurality of subjects are detected in the screen will be described. When a plurality of subjects are detected on the screen in the composition of the upper body 1400 or the whole body 1401 during the still image framing 1405, a process of changing the composition to the plurality of persons 1403 is performed. At this time, the number of detected subjects is stored in the memory 118. The composition of the plurality of persons 1403 is selected when it is determined during the still image framing 1405 that the scene is a group photograph shooting scene in which all the plurality of subjects are included in a predetermined angle of view. In the composition state of the plurality of persons 1403, a process of determining whether or not the number of detected subjects has changed is performed. When the number of detected subjects increases, the number of subjects stored in the memory 118 is immediately updated because a new subject has entered the screen, and the composition is changed. On the other hand, when the number of detected subjects decreases, for example, there is a possibility that a state where detection is temporarily impossible, such as when one of the subjects faces sideways. Therefore, if the composition is changed immediately, the subject that cannot be detected may go out of the frame. Therefore, when the number of subjects decreases, it is determined whether the state has continued for a predetermined time or more. If the state continues for a predetermined time or more, the number of subjects stored in the memory 118 is updated, and the composition is changed. Further, when the subject is alone, a process of returning the composition to the upper body 1400 is performed.

  If a composition is adopted in which all subjects are included in the angle of view during recording of a moving image 1406, when a subject other than the subject to be photographed is moving around, there is a concern that malfunction of the zoom may occur in response to the subject. Therefore, even when a plurality of subjects are detected during the recording of a moving image 1406, a process of maintaining the composition as the upper body 1400 or the whole body 1401 with only the main subject as a shooting target is performed. In the composition of the upper body 1400 during recording of a moving image 1406, when a face is detected near the center of the screen for a predetermined time or more, a process of changing the composition to the face 1402 is performed. The composition of the face 1402 is a composition selected when it is determined during recording of a moving image 1406 that the scene focuses on the face of the subject. In such a scene, by making the composition closer to the face than the upper body, the face of the subject of interest can be photographed in a larger size. Further, during the zoom-in operation due to the composition change, by changing the zoom speed to be significantly lower or higher than the normal speed, a more effective image can be captured. Further, in the composition of the face 1402, when the face is detected for a predetermined time or more near the periphery of the screen, or when it is determined that the photographer has changed the framing, the composition is returned to the upper body 1400. Processing is performed. In the present embodiment, the peripheral area of the screen is exemplified as the specific area of the image, and the timer for measuring the duration of the state where the position of the face image of the subject detected by the subject detection unit 123 is the peripheral area is measured. In addition, as a framing change determination method, when the subject detection unit 123 is not detecting a subject and the shake amount detected by the shake detection unit 125 is larger than a predetermined amount, the photographer performs framing. It is determined that it has been changed.

  During still image framing 1405, the photographer may continue to wait for a photo opportunity with the same composition until the orientation and expression of the person's face are in the desired state. In this case, if the composition is close to the face, the composition may be different from the photographer's intention. Therefore, in the composition of the upper body 1400 during the still image framing, even if a face is detected for a predetermined time or more near the center of the screen, a process of maintaining the composition of the upper body 1400 is executed.

  Next, a composition in a case where the shooting state is changed according to the start / end of the moving image recording will be described. Each composition of the upper body 1400 and the whole body 1401 is a composition that can be selected in any case of 1405 during still image framing and 1406 during moving image recording. Therefore, in those compositions, the original composition is maintained even if the shooting state is changed. On the other hand, the composition of the plurality of persons 1403 and the face 1402 is a composition selected only in one of the cases of 1405 during still image framing and 1406 during moving image recording. Therefore, when moving image recording is started with a composition of a plurality of persons 1403 during still image framing 1405 or when moving image recording is ended with a composition of a face 1402 during moving image recording 1406, the upper body 1400, which is a common composition, is added. It can be changed. However, in this case, when the zoom operation starts at the same time as the start and end of the moving image recording, there is a concern that an image due to the zoom operation is recorded at the beginning of the moving image or that the photographer feels troublesome when framing a still image. . Therefore, when the shooting state is changed in such a state, a process of temporarily changing the composition to the manual 1404 is executed. The composition of the manual 1404 is a composition in which the size of the main subject image detected first after the composition is changed is set as the reference size. Therefore, the zoom operation does not immediately start just by changing the photographing state, and the photographer does not feel uncomfortable. Further, even when the subject starts moving, the size of the subject image can be maintained. Further, when a predetermined time has elapsed in the composition of the manual 1404, a process of returning the composition to the upper body 1400 is performed.

  As described above, in S1211 of FIG. 12, when the composition setting is “auto”, an appropriate composition is automatically determined according to the shooting state, the number of subjects, the movement of the subjects, and the like. Data indicating the composition after the determination is stored in the memory 118, and the process proceeds to S1212.

  In step S1212, it is determined whether the composition determined in step S1211 has been changed with respect to the immediately preceding composition (whether the composition selected in FIG. 14 has transitioned or the number of subjects has changed in a state of a plurality of persons 1403). Is determined. If the composition has been changed, the process advances to step S1213 to determine whether the selected composition is a composition of the manual 1404. If the selected composition is the composition of the manual 1404, the process advances to step S1201 to execute a process of storing the size of the first detected subject image in the memory 118 as a reference size. If the selected composition is not the composition of the manual 1404, the process advances to step S1215 to calculate the reference size in the composition setting "auto". If it is determined in step S1212 that the composition has not been changed, the process advances to step S1214 to determine whether the subject image has moved on the screen. As a method of calculating the moving amount of the subject image in the screen, after the reference size is set in S1215, the center position of the face tracking frame immediately after performing the zoom operation to the angle of view that becomes the reference size is stored in the memory 118. It is memorized. The distance from the position stored in the memory 118 to the center position of the detected face tracking frame is calculated as the movement amount. Further, the movement amount in the screen may be calculated by adding the center position of the face tracking frame at every predetermined cycle. It is determined that the subject has moved when the calculated movement amount is equal to or more than a predetermined amount. The predetermined amount is normalized by the size of the detected face tracking frame (to be a predetermined multiple of the size of the face tracking frame), so that the actual moving amount of the subject regardless of the size of the subject image on the screen Can be made comparable. If it is determined in step S1214 that the subject is moving on the screen, the process advances to step S1215 to calculate a reference size in the composition setting “auto”. If it is determined in step S1214 that the subject image has not moved within the screen, the reference size setting process ends without updating the reference size.

  Hereinafter, the calculation process of the reference size in the composition setting “auto” will be described with reference to FIGS. If the composition setting is any one of “face”, “upper body”, and “whole body”, the reference size is calculated in S1210 of FIG. 12 so that the face has a fixed ratio to the screen size. That is, the same reference size is calculated regardless of the position of the subject image on the screen. Therefore, if the composition is changed so that the subject image becomes large (zoom in) when the subject is around the screen, the subject may be out of frame. When calculating the reference size by this method, the photographer needs to frame the subject near the center of the screen. Therefore, it is not suitable for a case where the subject is shifted from the center of the screen and photographed together with the background. When there are a plurality of subjects, it is necessary to calculate the reference size by another method. Therefore, when the composition setting is “auto”, the size of the subject image is determined based on the composition determined in S1211 and the reference is set so that the subject fits within the screen regardless of the position of the detected subject. Processing for calculating the size is performed.

FIG. 15 is a diagram illustrating the relationship between the position of the subject closest to the screen among the plurality of subjects and the screen size. FIGS. 15A and 15B show an example of a still image shooting screen, and FIG. 15C shows an example of a moving image shooting screen, illustrating a plurality of subjects. FIG. 16 is a flowchart showing the entire process of calculating the reference size in the composition setting “auto”.
First, in S1600 of FIG. 16, a process of calculating the horizontal subject position ratio Rh is performed. The horizontal subject position ratio Rh is a horizontal position in the vicinity of the shoulder of the subject located nearest from the center of the screen with respect to a predetermined ratio with respect to the horizontal screen size (hereinafter, referred to as a shoulder position and described as Xsmax). Distance x 2 is the ratio occupied. The predetermined ratio is, for example, 80% or 90% of the screen size in the horizontal direction, and is changed according to the number of people arranged in the horizontal direction in S1703 described later. In FIG. 15A, among the subjects 1500a, 1501a, and 1502a, the subject that is closest to the periphery from the center of the screen in the horizontal direction is the subject 1500a. The calculation process of the horizontal subject position ratio Rh will be described with reference to the flowchart in FIG.

  In S1700, the shoulder position of the subject (denoted as Xs) is calculated based on the center position and size of the face tracking frame of the subject. The center of the screen is the origin (0, 0), the center position of the face tracking frame is (Xc, Yc), and the size of the face tracking frame is S. Assuming that the number of face tracking frames occupying the distance from the center position of the face tracking frame to the shoulder position is Ns, the shoulder position Xs on the peripheral side of the screen can be calculated by “Xs = | Xc | + S × Ns”. FIG. 19 shows an example of setting Ns. The value of Ns is changed according to the composition determined in S1211 of FIG. For example, in the case of the composition of the face 1402, Ns = 1 (in this case, not the shoulder position but the horizontal position near the ear) is set. In other cases (“upper body”, “whole body”, “plurality”), Ns = 2 is set.

  After calculating the shoulder position Xs of the subject, the process proceeds to S1701. In S1701, when the maximum value Xsmax of the shoulder position is not stored in the memory 118, the shoulder position Xs calculated in S1700 is stored as the maximum shoulder position Xsmax. If the maximum shoulder position Xsmax is stored in the memory 118, the shoulder position Xs calculated in S1700 is compared with the maximum shoulder position Xsmax. If the shoulder position Xs is larger than the maximum shoulder position Xsmax, a process of updating the maximum shoulder position Xsmax in the memory 118 with the shoulder position Xs is executed. After updating the maximum shoulder position Xsmax, the process proceeds to S1702. In S1702, it is determined whether or not the processing of calculating the shoulder position Xs for all subjects and updating the maximum shoulder position Xsmax has been completed. If the calculation of the shoulder position Xs and the update of the maximum shoulder position Xsmax have not been completed, the process returns to S1700. If the determination of the maximum shoulder position Xsmax has been completed for all the subjects and the update of the maximum shoulder position Xsmax has been completed, the process proceeds to S1703.

  In S1703, a determination process of the number of persons in which the subjects are arranged in the horizontal direction (the number of persons arranged in the horizontal direction) is performed. The purpose of this is to provide a composition in which the subject fills the entire screen when there are many subjects, such as a group photo, and a composition in which a margin is left around the screen when there are few subjects. When the face tracking frames overlap in the vertical direction (the vertical direction of the screen), the number of persons arranged in the horizontal direction is counted as one person including the overlapping subjects. For example, if four subjects are detected in the screen, and two face tracking frames of the two subjects overlap the top and bottom of the screen, they are counted as three. In S1703, the number of persons arranged in the horizontal direction is compared with a threshold value (for example, two persons). If it is determined that the number of people arranged in the horizontal direction is two or less, the process proceeds to S1704, and if it is determined that the number of people is three or more, the process proceeds to S1705. In S1704 and S1705, the subject position ratio Rh in the horizontal direction is calculated, respectively. The horizontal subject position ratio Rh in S1704 is calculated as a ratio of the maximum shoulder position Xsmax × 2 to 80% of the horizontal screen size. The horizontal subject position ratio Rh in S1705 is calculated as a ratio of the maximum shoulder position Xsmax × 2 to 90% of the horizontal screen size. After S1704 or S1705, the calculation processing of the subject position ratio Rh ends.

  Next, in S1601 of FIG. 16, a process of calculating the subject position ratio Rv in the vertical direction is executed. The vertical subject position ratio Rv is the vertical position of the head of the subject closest to the center of the screen (hereinafter referred to as head position) or the vertical position of the body part for a predetermined ratio of the screen size in the vertical direction. (Hereinafter referred to as body position) × 2. The predetermined ratio is, for example, 90% of the screen size in the vertical direction. Hereinafter, the head position of the subject closest to the center of the screen is referred to as Yhmax, and the body position of the subject closest to the center of the screen is referred to as Ybmin. In FIG. 15B, of the subjects 1500b, 1501b, and 1502b, the subject having the head at the periphery from the center of the screen in the vertical direction is the subject 1500b, and the subject at the body position at the periphery from the center of the screen is the subject 1501b. It is. With reference to the flowchart of FIG. 18, a description will be given of a process of calculating the subject position ratio Rv in the vertical direction.

  In S1800 of FIG. 18, the head position (denoted as Yh) of the subject is calculated based on the center position and size of the face tracking frame of the subject. The head position Yh can be calculated from “Yh = Yc + S × Nh”. Nh is the number of face tracking frames occupying the distance from the center position of the face tracking frame to the head position. FIG. 19 shows an example of setting Nh. The value of Nh is set at Nh = 1.5 regardless of the composition determined in S1211 of FIG. After the calculation of the head position Yh of the subject, the process proceeds to S1801.

  In S1801, when the maximum value Yhmax of the head position is not stored in the memory 118, a process of storing the head position Yh calculated in S1800 as the maximum head position Yhmax is executed. If the maximum head position Yhmax is stored in the memory 118, the head position Yh calculated in S1800 is compared with the maximum head position Yhmax. When the head position Yh is larger than the maximum head position Yhmax, a process of updating the maximum head position Yhmax in the memory 118 with the head position Yh is performed. After the update processing, the process proceeds to S1802.

  In the next step S1802, the body position of the subject (referred to as Yb) is calculated based on the center position and size of the face tracking frame of the subject. The body position Yb can be calculated from “Yb = Yc−S × Nb”. Nb is the number of face tracking frames occupying the distance from the center position of the face tracking frame to the body position. FIG. 19 shows an example of setting Nb. The value of Nb is changed according to the composition determined in S1211 of FIG. For example, in the case of the composition of the face 1402, Nb = 1.5 is set. Nb = 5 is set for the composition of the upper body 1400, Nb = 10 is set for the composition of the whole body 1401, and Nb = 3.5 is set for the composition of a plurality of persons 1403. This setting is such that the position is in the center of the screen in each composition, the position of the chest in the face 1402, the position of the lower back in the upper body 1400, the position of the feet in the whole body 1401, and the position in the vicinity of the upper waist of the plural persons 1403. After calculating the body position Yb of the subject, the process advances to S1803.

  In S1803, when the minimum value Ybmin of the body position is not stored in the memory 118, a process of storing the body position Yb calculated in S1802 as the minimum body position Ybmin is executed. If the minimum body position Ybmin is stored in the memory 118, the body position Yb calculated in S1802 is compared with the minimum body position Ybmin. When the body position Yb is smaller than the minimum body position Ybmin, a process of updating the minimum body position Ybmin in the memory 118 with the body position Yb is performed. After the update processing, the process advances to step S1804.

  In S1804, the head position Yh and the body position Yb are calculated for all the subjects, and it is determined whether or not the process of updating the maximum head position Yhmax and the minimum body position Ybmin has been completed. If the update of the maximum head position Yhmax and the minimum body position Ybmin has not been completed, the process returns to S1800. If the determination of the maximum head position Yhmax and the minimum body position Ybmin has been completed for all the subjects and the update of the maximum head position Yhmax and the minimum body position Ybmin has been completed, the process proceeds to S1805. In S1805, the absolute value of the maximum head position Yhmax and the absolute value of the minimum body position Ybmin are compared, and processing for determining which is located closer to the screen is performed. If it is determined that the maximum head position Yhmax is located closer to the screen periphery, the process proceeds to S1806. If it is determined that the minimum body position Ybmin is located closer to the periphery of the screen, the process proceeds to S1807. In S1806 and S1807, the subject position ratio Rv in the vertical direction is calculated. The vertical subject position ratio Rv in S1806 is calculated as a ratio of the absolute value of the maximum head position Yhmax × 2 to 90% of the vertical screen size. The vertical subject position ratio Rv in S1807 is calculated as a ratio of the absolute value of the minimum body position Ybmin to 90% of the vertical screen size × 2. After S1806 or S1807, the calculation process of the vertical subject position ratio Rv ends.

  Subsequently, in S1602 of FIG. 16, the subject position ratio Rh in the horizontal direction and the subject position ratio Rv in the vertical direction are compared. With this, it is possible to determine the position having the largest ratio, that is, the position closest to the screen, among the ratios of the distance from the center of the screen to each position of the subject with respect to the predetermined ratio of the screen size in each direction. it can. The positions are a maximum shoulder position Xsmax, a maximum head position Yhmax, and a minimum body position Ybmin. If it is determined that the horizontal direction subject position ratio Rh is greater than the vertical direction subject position ratio Rv, the process proceeds to S1603, and it is determined that the horizontal direction subject position ratio Rh is equal to or less than the vertical direction subject position ratio Rv. , The process proceeds to S1604. In S1603 and S1604, a process of calculating the reference size is executed based on the detected size of the main subject and the ratio of the position of the subject closest to the screen determined in S1602. The reference size is calculated based on “detected size / horizontal subject position ratio Rh” in S1603, and calculated based on “detected size / vertical subject position ratio Rv” in S1604. That is, in S1603, the zoom operation is performed such that the main subject image has a reference size obtained by multiplying the size by “1 / horizontal subject position ratio Rh”. In S1604, the zoom operation is performed such that the main subject image has a reference size obtained by multiplying the size by “1 / vertical subject position ratio Rv”. By doing so, the angle of view based on the composition determined in S1211 can be set without causing the subject in the screen to go out of the frame.

  In S1215 of FIG. 12, when the composition setting is “auto”, the subject position that is the closest to the screen in the horizontal and vertical directions is determined based on the composition determined in S1211, and the reference position is set so that the position fits on the screen. The size is calculated. Thus, regardless of the position of the subject image on the screen, the subject can be set at an appropriate angle of view without going out of the frame. Further, the reference size can be calculated by the same processing regardless of whether the number of subjects is singular or plural. In the present embodiment, the direction of the screen size for calculating the reference size is determined on the assumption that the subject is a person in a standing posture. The present invention is not limited to this, and when the subject detection unit 123 can determine the direction of the face, the direction of the screen size may be determined based on the vertical direction of the face on the screen. Furthermore, the example of composition determination and calculation of the reference size when the subject is a person has been described, but the present invention can also be applied to the case where the subject is an object. However, in this case, the composition to be selected is “large”, “medium”, “small”, and “plural” instead of “face”, “upper body”, “whole body”, and “plurality”. The number of mono tracking frames (corresponding to Ns, Nh, and Nb in the case of a person) when calculating the subject position may be set accordingly.

  When the reference size setting process shown in FIG. 12 ends, the process proceeds to S704 in FIG. In step S704, the AZ control unit 122 performs automatic zoom control based on the subject information detected by the subject detection unit 123 and the reference size stored in the memory 118. The automatic zoom control will be described later with reference to FIG. After the end of the auto zoom control, the process proceeds to S705, where the AZ control unit 122 determines a subject search instruction by the photographer. That is, it is determined whether or not the photographer has pressed the subject search button of the operation unit 117. If the subject search button has been pressed, the process returns to S701, and the subject search processing is executed. If the subject search button has not been pressed, the process proceeds to S706. In step S706, it is determined whether the subject to be subjected to the auto-zoom has been changed by operating the touch panel or the subject specifying switch of the operation unit 117. If the subject has been changed, the process returns to S702 and the subject specifying process is executed. If the subject has not been changed, the process proceeds to S707.

  In S707, it is determined whether the composition setting has been changed by operating the left and right buttons of the operation unit 117 from the shooting screen. When the composition setting is changed, after the composition setting data stored in the memory 118 is updated, the process returns to S703 to execute the reference size setting processing. If the composition setting has not been changed, the process proceeds to S708. In S708, it is determined whether the auto zoom operation switch of the operation unit 117 has been pressed. If it is determined that the auto zoom operation switch has been pressed, the auto zoom function ends. If it is determined that the auto zoom operation switch has not been pressed, the process proceeds to S709. S709 is a process of determining whether or not the composition setting stored in the memory 118 is “auto”. If the composition setting is “auto”, the process returns to step S703, and the reference size setting processing including automatic composition determination is performed. If the composition setting is other than “auto”, the process returns to step S704 to continue the automatic zoom control.

  Next, the processing of the auto zoom control in S704 will be described with reference to the flowchart in FIG. First, in S2000, the AZ control unit 122 determines whether or not a subject has been detected by the subject detection unit 123. If no subject is detected in S2000, the automatic zoom control ends. If a subject is detected in S2000, the process proceeds to S2001. The determination shown in each step of S2001 to S2003 is a process of determining the start of auto zoom for frame-out prevention control. That is, this processing is performed to prevent the subject from going out of the frame outside the screen as described with reference to FIGS. In S2001, the AZ control unit 122 determines whether or not the subject tracking frame relating to the subject to be tracked has entered the ZO area. This ZO area corresponds to the ZO area described in FIGS. 3A and 4A to 4C. If the subject tracking frame has entered the ZO area in S2001, that is, if there is a high possibility that the subject will be out of the frame, the process proceeds to S2004, and a zoom-out operation is started. The zoom-out operation here corresponds to a zoom-out operation for frame-out prevention control. After the zoom-out operation, the automatic zoom control ends.

  On the other hand, if the subject tracking frame has not entered the ZO area in S2001, that is, if the camera is capturing the subject near the center of the screen, the process proceeds to S2002. In S2002, it is determined whether or not the immediately preceding zoom operation is a zoom-out operation due to entry into the ZO region in S2001, that is, whether or not the zoom-out operation is for frame-out prevention control. If it is determined in S2002 that the zoom-out operation for frame-out prevention control has been performed, the process proceeds to S2003. If it is determined in S2002 that the zoom-out operation for frame-out prevention control has not been performed, the process proceeds to S2006.

  In S2003, the AZ control unit 122 determines whether or not the subject tracking frame of the subject to be tracked falls within the ZI region (whether or not it is included). The ZI region here corresponds to the ZI region described with reference to FIGS. 3B and 4A to 4C. If it is determined in S2003 that the subject tracking frame does not fall within the ZI region, the auto zoom control process ends. On the other hand, if it is determined that the subject tracking frame falls within the ZI area, the process proceeds to S2005. That is, when the camera is capturing the subject so that the subject size is near the center of the screen and within the angle of view at the zoom return position, the zoom-in operation is started in S2005. The zoom-in operation here corresponds to a zoom-in operation for frame-out prevention control. After the zoom-in operation, the automatic zoom control ends.

  In the present embodiment, in order to achieve both the frame-out prevention control and the size holding control in the auto-zoom, the subject is first captured near the center of the screen by the frame-out prevention control, and then the size holding control can be executed. For this reason, in the state after the zoom-out operation for the frame-out prevention control, the auto-zoom processing (hereinafter, S2006 to S2010) for maintaining the subject size constant (of size holding control) described below is not performed. . In other words, when the frame-out prevention control is performed, the execution of the size holding control is restricted until the zoom-in operation for the frame-out prevention control is completed.

  Next, each processing of S2006 to S2010 will be described. If it is determined in S2002 that the zoom-out operation has not been performed for the frame-out prevention control, the process proceeds to S2006, in which the AZ control unit 122 sets the subject detection size and the reference size to a predetermined magnification (N1 and N1> 1). ) Is compared with the size. The subject size of the reference subject information set in S1202, S1210, and S1215 of FIG. 12 is compared with the subject size detected in S2000. If the size of the subject detected in S2000 is larger than N1 times the size of the reference subject, that is, if the ratio of the subject image to the screen exceeds a predetermined value, the process proceeds to S2009. In step S2009, the AZ control unit 122 starts a zoom-out operation. The zoom-out operation here corresponds to a zoom-out operation for size holding control. After the zoom-out operation, the automatic zoom control ends.

  On the other hand, in S2006, if the size of the subject detected in S2000 is smaller than or equal to N1 times the size of the reference subject, the process proceeds to S2007. In S2007, the AZ control unit 122 compares the subject size of the reference subject information with the size of the subject detected in S2000. When the detected size of the subject is smaller than a predetermined magnification (described as N2 and N2 <1) with respect to the size of the reference subject (less than N2 times), that is, the ratio of the subject image to the screen is a predetermined value. If it is less, the process proceeds to S2008. On the other hand, if the detected size of the subject is equal to or more than N2 times the size of the reference subject, the automatic zoom control ends.

  In S2008, the AZ control unit 122 determines whether or not the subject tracking frame of the subject to be tracked falls within the ZI region (whether or not it is included). This is to prevent the subject from being framed out by the zoom-in operation when the subject is around the screen. The ZI region here corresponds to the ZI region described with reference to FIG. If it is determined in S2008 that the subject tracking frame does not fall within the ZI area, the auto zoom control ends.

  On the other hand, if it is determined in step S2008 that the subject tracking frame is within the ZI region, the process advances to step S2010. In S2010, the AZ control unit 122 starts the zoom-in operation. The zoom-in operation here corresponds to a zoom-in operation for size holding control. As described above, in the present embodiment, in order to prevent the subject from going out of the frame even in the zoom-in operation for the size holding control, the zoom-in operation is started after the subject image falls within the ZI area. After the zoom-in operation, the automatic zoom control ends.

  Next, the zoom operation will be described with reference to FIG. FIG. 21 is a flowchart illustrating the zoom-out operation or the zoom-in operation in S2004, S2005, S2009, and S2010 of FIG. First, in step S2100, the AZ control unit 122 acquires a zoom change amount (zoom magnification change amount) from the memory 118. In the case of a zoom-out operation for frame-out prevention control, the zoom-out change amount is set according to the detected subject information. Specifically, in the zoom-out operation for frame-out prevention control (S2004 in FIG. 20), the smaller the size of the subject, the smaller the zoom-out change amount is set. Thus, it is possible to prevent the subject from being undetectable because the size of the subject becomes too small due to the zoom-out operation. If the size of the subject is smaller than the predetermined size in consideration of the minimum size in which the subject can be detected, the zoom-out operation may not be performed. In the zoom-in operation for frame-out prevention control, the zoom magnification before the start of the zoom-out operation is stored in the memory 118, and the zoom-in change amount is set so as to be the same zoom magnification as before the start of the zoom-out operation.

  In the zoom-out operation for size holding control (S2009 in FIG. 20), a zoom-out change amount (1 / N1 times) corresponding to a predetermined N1 times used in the determination in S2006 is set. Thereby, even when the subject cannot be detected, it is possible to perform the minimum zoom-out operation until the size of the subject becomes the size of the reference subject. Similarly, in the case of the zoom-in operation for size holding control (S2010 in FIG. 20), the zoom-in change amount (1 / N2 times) corresponding to the predetermined N2 times used for the determination in S2007 is set.

  In step S2101, the AZ control unit 122 sets the zoom change amount acquired in step S2100 in the CZ control unit 119 or the electronic zoom control unit 120, and instructs the zoom control unit 120 to perform a scaling process. In the next step S2102, the AZ control unit 122 determines which zoom operation is being performed, either for frame-out prevention control or for size holding control. If the current zoom operation is a frame-out prevention control zoom operation (S2004, S2005 in FIG. 20), the process proceeds to S2105. If the current zoom operation is a zoom operation for size holding control (S2009, S2010 in FIG. 20), the process proceeds to S2103.

  In step S2103, the AZ control unit 122 determines whether the subject is detected by the subject detection unit 123. If the subject has been detected, the process proceeds to S2104, and if the subject has not been detected, the process proceeds to S2105. In S2104, the size of the subject indicated by the reference subject information is compared with the size of the subject detected in S2103. As a result of the comparison, if the size of the subject detected in S2103 and the size of the reference subject do not fall within the range of a predetermined ratio (within a predetermined change amount), the process proceeds to S2102 again to determine the zoom operation. continue. If the size of the subject detected in S2103 and the size of the reference subject are within the range of the predetermined ratio by the zoom operation, the process proceeds to S2106. After stopping the zoom operation in S2106, the AZ control unit 122 ends the zoom operation processing.

  If it is determined in step S2102 that a zoom operation for frame-out prevention control is being performed, or if a subject is not detected in step S2103 during a zoom operation for controlling size retention, the process advances to step S2105. In step S2105, the AZ control unit 122 determines based on the zoom change amount acquired in step S2100 whether or not a scaling process corresponding to a predetermined amount of zoom change according to each zoom operation has been performed. If the zoom process for the predetermined zoom change amount has not been performed, the process returns to step S2102 to continue the process. If the zoom process of the predetermined zoom change amount has been performed, the process proceeds to S2106, where the AZ control unit 122 stops the zoom in operation and ends the zoom operation process.

  In the present embodiment, the shooting scene is determined based on the movement and the number of the subjects, the detection time of the subject, the distance from the camera to the subject, and the like, and the camera automatically selects an appropriate composition according to the determined scene. Based on the composition selected by the composition selection processing, processing for setting the reference size of the subject image is performed, and the zoom operation is performed so that the detected image of the subject has the reference size. According to the present embodiment, it is possible to perform auto zoom control so that an appropriate composition is obtained according to a shooting scene determined based on camera detection information without a photographer selecting a composition.

(Second embodiment)
Next, a second embodiment of the present invention will be described with reference to FIG. FIG. 22 shows an example of a transition diagram of a composition different from that in FIG. 14 in the automatic composition determination processing S1211 of the composition setting “auto” in FIG. Note that the same components as those in FIG. 14 are denoted by the same reference numerals, and detailed description thereof will be omitted.

In the first embodiment, an example has been described in which a photographing scene is determined when a person different from the photographer is photographed as a subject. The mode at this time is called a normal mode for convenience. In some cases, the camera is pointed at the photographer himself, and one or more persons including the photographer himself are photographed as subjects. A mode suitable for such shooting is called a self-portrait mode. In the self-portrait mode, it is possible to perform effective portrait photographing by adjusting the angle of view suitable for self-portrait by auto zoom control and applying a background blurring effect, a beautiful skin filter effect, and the like.
In the composition adjustment operation, in the normal mode, it is necessary to set the subject image to an appropriate size depending on whether the subject is stationary or moving. On the other hand, in the self-portrait mode, even if the subject is stationary, it is necessary to change an appropriate size depending on whether the camera is hand-held or placed and photographed. For example, when taking a picture by hand, the face is photographed so as to be large in the angle of view, and when photographing with the camera placed, the whole body is photographed so as to be within the angle of view. As described above, the appropriate composition differs between the shooting in the normal mode and the shooting in the self-portrait mode.
Therefore, in the present embodiment, a process of changing the composition to be selected is performed between the normal mode 2200 and the self-portrait mode 2201 in FIG. That is, the normal mode is illustrated as the first mode, and the self-portrait mode is illustrated as the second mode. As the composition of the selection candidate belonging to the first composition group in the normal mode 2200, any composition is selected from the upper body 1400, the whole body 1401, and the plurality 1403. As the composition of the selection candidate belonging to the second composition group in the self-portrait mode 2201, any composition from the self-portrait face 2202 and the self-portrait whole body 2203 is selected.

Here, the determination conditions of the shooting scene and the transition of each composition will be described. When the automatic composition determination process S1211 of the composition setting “auto” starts, the upper body 1400 is set as the composition in the initial state. When the mode is the normal mode 2200, the transition is the same as during the still image framing 1405 in FIG. 14, and the description is omitted.
When the mode is changed from the normal mode 2200 to the self-portrait mode 2201, the shake detecting unit 125 determines whether or not the camera is in a hand-held state based on the shake amount applied to the gyro sensor or the like. When the mode is changed to the self-portrait mode 2201 and the shake amount is equal to or more than the predetermined amount and it is determined that the camera is hand-held, the composition is changed to the self-portrait face 2202. The composition of the self-portrait face 2202 is a composition in a case where the user takes a picture with his / her face up by hand. Also, when photographing by a plurality of persons including the photographer, the zoom is adjusted so that the faces of all the subjects are included in the angle of view as large as possible.

  In the case of the self-portrait mode 2201, the same composition is selected whether the subject is a photographer or a plurality of subjects. That is, the number of estimated subject position faces shown in FIG. 19 is set to the same value. As described above, the number of subject position estimated faces is a face tracking frame for estimating a shoulder position Xs, a head position Yh, and a body position Yb in the reference size calculation processing S1215 of the composition setting “auto” in FIG. The number of tracking frames from the center position (Ns, Nh, Nb). In particular, since the number Ns of tracking frames up to the shoulder position and the number Nb of tracking frames up to the body position are different depending on the selected composition, the reference size for the composition is calculated. In the first embodiment, when one subject is photographed in the “face” composition, the number of tracking frames from the center position (Xc, Yc) of the face tracking frame is determined by the shoulder position Ns = 1 and the body position Nb = 1. As a reference size, a reference size was calculated to be a range within the angle of view including the vicinity of the ear and chest of the subject. Further, when photographing a plurality of subjects with a composition of “multiple persons”, the number of tracking frames from the center position (Xc, Yc) of the face tracking frame is set as shoulder position Ns = 2 and body position Nb = 3.5. The reference size was calculated so that the angle of view included the vicinity of the shoulders and hips of all the subjects within the angle of view. On the other hand, in the case of the self-portrait face 2202, the shoulder position Ns = 1 and the body position Nb = 1.5, which are the same number of tracking frames as “face”, are set regardless of the number of subjects. As a result, the reference size is calculated so that the angle of view covers the ears and chests of all the subjects within the angle of view, and the zoom is adjusted to an angle of view suitable for hand-held self-portrait shooting.

  When the face is changed to the self-portrait face 2202, the number of detected subjects is stored in the memory 118. In the composition state of the self-portrait face 2202, a process of determining whether or not the number of detected subjects has changed is performed. When the number of detected subjects increases, the number of subjects stored in the memory 118 is immediately updated, and the reference size is changed. When the number of subjects decreases, when the state continues for a predetermined time or more, the number of subjects stored in the memory 118 is updated, and the composition is changed. This is to prevent the subject from going out of the frame by immediately changing the composition when the subject is temporarily undetectable due to turning sideways as in the case of the plurality of persons 1403.

  When the mode is changed from the normal mode 2200 to the self-portrait mode 2201, the shake detection unit 125 sets the camera on a desk or the like (fixed) based on the shake amount applied to the gyro sensor or the like. It is determined whether or not it is in the state. If the mode is changed to the self-portrait mode 2201 and it is determined that the shake amount is less than the predetermined amount and the camera is placed, the composition is changed to the self-portrait whole body 2203. The composition of the self-portrait whole body 2203 is a composition in a case where a camera is put and the whole body is photographed. Further, when photographing a plurality of persons including the photographer, the zoom is adjusted so that the whole body of all subjects falls within the angle of view. In the case of the self-portrait whole body 2203, the shoulder position Ns = 2 and the body position Nb = 10, which are the same number of tracking frames as “whole body”, are set regardless of the number of subjects. As a result, the reference size is calculated so that the shoulders and feet of all subjects fall within the angle of view, and the zoom is adjusted to an angle of view suitable for performing self-portrait shooting with the camera placed. When the self-portrait whole body 2203 is changed, the number of detected subjects is stored in the memory 118. As in self-portrait face 2202, when the number of subjects changes in self-portrait whole body 2203, the number of subjects stored in memory 118 is updated and the reference size is changed.

In the present embodiment, the shooting scene to be determined is changed according to the set mode, and the camera automatically selects a more appropriate composition for each mode. Further, in the present embodiment, an example has been described in which an appropriate composition is selected from a plurality of composition selection candidates in one mode. However, depending on the mode, only one composition may be a selection candidate. Furthermore, the case of the normal mode and the self-portrait mode has been described as an example, but the mode is not limited to this mode. For example, in addition to the normal mode and the self-portrait mode, an embodiment in which the composition of the athlete's "whole body" is selected in the sports mode, and an embodiment in which the composition of the "face" of the baby is selected in the baby mode. You may.
According to the present embodiment, it is possible to perform auto zoom control so that a photographer can select an appropriate composition according to a set mode and a shooting scene determined based on camera detection information without selecting a composition. it can.

(Other embodiments)
The present invention supplies a program for realizing one or more functions of the above-described embodiments to a system or an apparatus via a network or a storage medium, and one or more processors in a computer of the system or the apparatus read and execute the program. This processing can be realized. Further, it can also be realized by a circuit (for example, an ASIC) that realizes one or more functions.

100 digital camera
102 zoom lens 109 display unit 113 zoom lens drive unit 117 operation unit 118 memory 119 CZ control unit 120 electronic zoom control unit 122 auto zoom control unit 123 subject detection unit

Claims (13)

A zoom control device,
Subject detection means for detecting a subject image from an image obtained by using the imaging optical system,
Setting means for setting a reference size of the subject image used for controlling the zoom magnification;
Control means for controlling a zoom magnification based on the reference size set by the setting means and the size of the subject image sequentially detected by the subject detection means,
The setting unit is configured to determine whether the zoom control device is in a hand-held state, whether a self-portrait mode is set, a distance to a subject, or at least one of a time during which the subject is at rest, to set the reference size. A zoom control device characterized by changing.   The apparatus according to claim 1, wherein the setting unit acquires the still time based on at least one of a change in the size of the subject image, a change in the position of the subject image, and a shake detection result of the zoom control device. Item 2. The zoom control device according to item 1.   The setting unit determines whether or not the zoom control device is hand-held based on a detection result of a shake amount of the zoom control device, and changes the reference size based on a result of the determination. The zoom control device according to claim 1.   The setting unit acquires a distance between the zoom control device and the subject based on a focus position of the photographing optical system and the zoom magnification, and changes the reference size based on the distance. The zoom control device according to claim 1.   The imaging apparatus according to claim 1, wherein the setting unit changes the reference size according to whether the self-portrait mode is set. Composition selecting means for selecting a composition based on at least one of whether the zoom control device is in a hand-held state, whether a self-portrait mode is set, a distance to a subject, and a still time of the subject. And
The zoom control device according to claim 1, wherein the setting unit changes the reference size based on a result of the selection by the composition selecting unit.   The zoom control device according to claim 6, wherein the composition selecting unit selects the composition based on the number of subject images detected by the subject detecting unit.   8. The apparatus according to claim 6, wherein the setting unit sets the reference size based on a composition detected by the composition selection unit and a size and a position of a subject image detected by the subject detection unit. The zoom control device according to item 1. The setting means,
Changing the reference size based on the change in the stationary time,
If it is determined that the subject is moving, a first reference size is set,
9. The method according to claim 1, wherein when it is determined that the stationary time is longer than a predetermined time, a second reference size larger than the first reference size is set. The zoom control device according to any one of the preceding claims. The setting means changes the reference size based on the change in the stationary time,
The control means controls the zoom magnification such that the zoom magnification is lower when it is determined that the subject is moving than when the stationary time is longer than a predetermined time. The zoom control device according to claim 1, wherein: A zoom control device according to any one of claims 1 to 10,
Imaging means for photoelectrically converting a subject image to generate an image;
A zoom unit for changing a zoom magnification by driving a lens or image processing;
The control unit controls the zoom unit, and controls a zoom magnification according to a change in size of a subject image detected by the subject detection unit based on the reference size set by the setting unit. Imaging device. A control method executed by a zoom control device,
An acquisition step of acquiring information including detection information of a subject image detected from an image acquired using the imaging optical system,
A setting step of setting a reference size of a subject image used for controlling a zoom magnification;
A control step of controlling a zoom magnification based on the size of the subject image indicated by the reference size and the detection information set by the setting step,
The setting step determines whether or not the zoom control device is in a hand-held state, whether or not a self-portrait mode is set, a distance to a subject, and at least one of a time during which the subject is at rest and the reference size. A control method of a zoom control device, wherein the control method is changed.   A control program for a zoom control device, which causes a computer to execute each step of the control method for a zoom control device according to claim 12.

Images