JP2016122030A - Zoom control device, imaging device, zoom control device control method, zoom control device control program, and storage medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To exert an auto-zoom control such that a detected subject image has a reference size set relatively to the number of subjects and a position of the subject in an image plane on the basis of the reference size and a selected composition.SOLUTION: A subject detection unit of an imaging device detects a subject image from an image, and an auto-zoom control unit automatically selects a specific composition from among a plurality of compositions, calculates a position of the subject image relative to a center of the image, and calculates a ratio occupied by the position of the subject image to a predetermined field angle in a horizontal direction or a perpendicular direction. The auto-zoom control unit determines a ratio occupied by the position of the subject located most closely around the field angle in the horizontal direction or the perpendicular direction to a plurality of subject images. A reference size of the subject image for use to control a zoom magnification is set from the determined ratio, the selected composition, and a size of a main subject image selected from among the plurality of subject images, and an auto-zoom control is exerted.SELECTED DRAWING: Figure 16

Description

  The present invention relates to a technique for detecting a subject image from an image and controlling a zoom magnification.

  Conventionally, a zoom lens is driven to perform optical zooming (hereinafter referred to as optical zoom), and a part of a captured image is enlarged to perform electronic zooming (hereinafter referred to as electronic zoom). There is an imaging device having a function. In addition, there is an auto-zoom (automatic scaling) function that automatically changes the zoom magnification in accordance with subject detection information. Hereinafter, the optical zoom operation and the electronic zoom operation are collectively referred to as a zoom operation.

  Patent Document 1 discloses a configuration of a camera equipped with an auto-zoom function that keeps the size of a subject constant. Patent Document 1 discloses a method in which a photographer selects a composition from a plurality of options such as “face up”, “bust up”, and “whole body” in an auto zoom function. In the auto zoom control that maintains the size of the subject image, the size of the subject image serving as a reference for zoom control is determined based on the selected composition. For example, when the photographer selects the “whole body” composition, the zoom operation is performed so that the whole body image of the person who is the subject falls within the shooting screen. Patent Document 2 discloses a configuration of a camera equipped with an auto zoom function for accommodating a plurality of subjects within a shooting screen. In Patent Document 2, a method for setting an automatic zoom frame so as to enclose a plurality of subjects and setting the ratio of the long side of the automatic zoom frame to the size of the corresponding screen to a predetermined ratio to fit the subject in the screen Is disclosed.

JP 2012-95019 A JP 2011-130382 A

  However, in Patent Document 1, it is necessary to select in advance the composition that the photographer wants to shoot with the auto zoom function. For this reason, it is necessary for the photographer to perform a setting change operation each time the shooting scene or the state of the subject (movement, number, etc.) changes. Further, if the composition setting is not appropriate for the scene to be shot, there is a possibility that the subject cannot be shot with an appropriate size. For example, if a “face” composition is set in a scene where a moving subject or a distant subject is photographed, the subject may be out of frame. In addition, when the composition of “whole body” is set in a scene for shooting a stationary subject, the subject may be too small.

In Patent Document 2, a zoom operation is performed based on the ratio between the distance between a plurality of subjects and the size of the screen. Therefore, when the plurality of subjects are not in a symmetric position on the screen, the image intended by the photographer is taken. There may be no corners. Further, when the photographer intends to compose a composition including a background building or object by removing the subject from the center of the screen, the subject around the screen may not fit within the screen.
In view of the above problems, the present invention provides an automatic zoom control based on the reference size set for the number of subjects and the position of the subject in the screen and the selected composition so that the detected subject image becomes the reference size. The purpose is to do.

  An apparatus according to the present invention includes a subject detection unit that detects a subject image from an image, a composition selection unit that automatically selects a specific composition from a plurality of compositions, and a position of the subject image based on the center of the image. Calculating means for calculating the ratio of the position of the subject image in the horizontal or vertical direction with respect to the angle of view of the image, and the subject detecting means When a plurality of subject images are detected by the above, the calculation unit determines the ratio for each subject image calculated in the second process with respect to the plurality of subject images, and the angle of view of the image is determined. Determining means for determining the ratio of the position of the subject image located closest to the periphery in the horizontal or vertical direction, the composition selected by the composition selecting means, the ratio determined by the determining means, and the subject Setting means for setting a reference size of the subject image used for controlling the zoom magnification from the size of the main subject image selected from the plurality of subject images detected by the output means; and a reference size set by the setting means. And a control means for controlling the zoom magnification in accordance with a change in the size of the subject image detected by the subject detection means.

  According to the present invention, based on the reference size set for the number of subjects and the position of the subject in the screen and the selected composition, the auto zoom control can be performed so that the detected subject image becomes the reference size. it can.

It is a block diagram which shows the structural example of the digital camera which concerns on embodiment of this invention. It is the figure which illustrated the relationship between a focal distance and a focus lens position for every subject distance. It is explanatory drawing of the process which prevents the frame out of the screen of a to-be-photographed object (object). It is explanatory drawing of the process which prevents the frame out of the screen of a to-be-photographed object (person). It is explanatory drawing of the process which prevents the size change in the screen of a to-be-photographed object (person). It is a figure explaining composition setting when a subject is a person. It is a flowchart explaining the flow of a process of an auto zoom function. It is a flowchart explaining a subject search process. It is a figure explaining the search end field of subject search processing. It is a flowchart explaining a subject designation process. It is a flowchart explaining another example of subject specification processing. It is a flowchart explaining the setting process of a reference | standard size. It is a flowchart explaining the calculation process of a screen size. FIG. 6 is a transition diagram for explaining automatic composition determination in composition setting “auto”. It is a figure which shows the outermost subject position and screen size. 10 is a flowchart for explaining a reference size calculation process in composition setting “auto”. It is a flowchart explaining the calculation process of a horizontal direction subject position ratio. It is a flowchart explaining the calculation process of a vertical direction subject position ratio. It is a table | surface which shows the number of faces which estimates a to-be-photographed object position. It is a flowchart explaining the process of auto zoom control. It is a flowchart explaining a zoom operation.

  Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings. FIG. 1 is a block diagram illustrating a configuration example of a digital camera 100 having an auto zoom function as an example of an imaging apparatus including a zoom control device according to the present embodiment.

  The lens barrel 101 holds a lens group therein. The zoom lens 102 adjusts the focal length by moving in the optical axis direction of the lens barrel 101, and optically changes the angle of view (moves the zoom position). The focus lens 103 performs focus adjustment by moving in the optical axis direction of the lens barrel 101. The anti-vibration lens (image blur correction lens) 104 corrects image blur caused by camera shake or the like. A diaphragm and shutter 105 for adjusting the amount of light are used for exposure control. In the present embodiment, the digital camera 100 is an imaging device in which the lens barrel 101 and the camera main body are integrally configured, but is not limited thereto. The present embodiment can also be applied to an imaging system including a camera body and an interchangeable lens that can be attached to and detached from the camera body.

  The image sensor 106 receives light that has passed through the lens barrel 101, and generates an imaging signal by converting a subject image into an electrical signal by photoelectric conversion. The image sensor 106 is a CCD (Charge Coupled Device) type or CMOS (Complementary Metal Oxide Semiconductor) type image sensor or the like. An imaging signal from the imaging element 106 is input to the image processing circuit 107, and various processes such as a pixel interpolation process and a color conversion process are performed. The image data after various processing is stored in the image memory 108. The image memory 108 is a storage device such as a DRAM (Dynamic Random Access Memory) or an SRAM (Static Random Access Memory).

  The display unit 109 includes a TFT type LCD (Thin Film Transistor Driven Liquid Crystal Display) and the like, and displays captured images (image data) and specific information (for example, captured information). An electronic viewfinder (EVF) function for the photographer to adjust the angle of view can be provided by displaying information such as a live view related to the photographed image.

  The aperture shutter driver 110 calculates an exposure control value (aperture value and shutter speed) based on the luminance information obtained by the image processing in the image processing circuit 107, and drives the aperture and shutter 105 based on the calculation result. . Thereby, automatic exposure (AE) control is performed. The anti-vibration lens driving unit 111 calculates a shake amount applied to the digital camera 100 based on shake detection information obtained by an angular velocity sensor such as a gyro sensor. The anti-vibration lens 104 is driven so as to cancel (reduce) the shake amount applied to the digital camera 100 according to the calculation result.

  The focus lens driving unit 112 drives the focus lens 103. In the present embodiment, the digital camera 100 performs automatic focus adjustment (AF) control by a contrast method. That is, the focus lens driving unit 112 drives the focus lens 103 so that the subject is focused based on the focus adjustment information (contrast evaluation value) of the photographing optical system obtained by the image processing in the image processing circuit 107. However, the present invention is not limited to this, and a phase difference AF method may be used as AF control other than the contrast method, and AF control is performed by a plurality of methods such as a combination of the contrast method and another method. It may be configured.

  The zoom lens driving unit 113 drives the zoom lens 102 in accordance with a zoom operation instruction. The operation unit 117 includes a zoom lever or a zoom button as a zoom operation member for the photographer to instruct the camera to perform zooming. The system control unit 114 detects the operation amount and operation direction of the zoom operation member used for the zoom instruction operation, calculates the zoom drive speed and the zoom drive direction, and moves the zoom lens 102 along the optical axis according to the calculation result. Take control.

The image data generated by the shooting operation is sent to the recording unit 116 via the interface (I / F) unit 115 and recorded. The image data is recorded on an external recording medium, a nonvolatile memory 118, or both. The external recording medium is a memory card or the like used by being attached to the digital camera 100. The nonvolatile memory 118 is a storage medium built in the digital camera 100. In addition to program data and image data, the memory 118 stores setting information of the digital camera 100 and information such as a zoom-in position in an auto zoom function described later.
In addition to the zoom operation member, the operation unit 117 includes a release switch for instructing start of photographing, an auto zoom operation switch for instructing start and end of an auto zoom function, and the like. A signal from the operation unit 117 is sent to the system control unit 114.

  The system control unit 114 includes an arithmetic device such as a CPU (Central Processing Unit). The system control unit 114 controls the entire digital camera 100 by sending a control command to each unit in accordance with a photographer's operation. The system control unit 114 executes various control programs stored in the memory 118, for example, programs for performing control of the image sensor 106, AE / AF control, zoom control (including auto zoom processing), and the like.

  In order to maintain the in-focus state when the angle of view is changed by the optical zoom, when the lens barrel 101 is a rear focus type, it is necessary to move the focus lens 103 to an appropriate focus position according to the position of the zoom lens 102. . Such control is called computer zoom (CZ) control, and is performed by the CZ control unit 119 in the system control unit 114. FIG. 2 is a diagram illustrating the relationship between the focal length of the zoom lens and the focus position for each subject distance. In FIG. 2, the relationship between the focal length of the zoom lens and the focus position in focus is shown as a graph as a data table for each distance to the subject. In the present embodiment, this table is referred to as a focus cam table. In FIG. 2, the horizontal axis indicates the focal length corresponding to the zoom position, and the vertical axis indicates the focus position. Next to each graph line, the distance from the digital camera 100 to the subject (subject distance) is illustrated.

  The system control unit 114 controls the focus lens driving unit 112 when performing AF control, and performs a scanning operation by moving the focus lens 103 within a predetermined range. A focus position which is a focal point is detected by a known method using a contrast evaluation value or the like obtained during the scanning operation. The subject distance is measured by referring to the focus cam table using the zoom position and the focus position at that time.

  Next, control related to the auto zoom function in the system control unit 114 will be described. As shown in FIG. 1, the system control unit 114 includes a CZ control unit 119, an electronic zoom control unit 120, an auto zoom control unit (hereinafter referred to as AZ control unit) 122, and a subject detection unit 123.

  The digital camera 100 has an optical zoom function and an electronic zoom function, and the CZ control unit 119 and the zoom lens driving unit 113 are in charge of controlling the optical zoom. The CZ control unit 119 detects the zoom position of the zoom lens 102 at every predetermined control period during the zoom operation. The CZ control unit 119 performs control to drive the focus lens 103 so as to follow the focus cam table at the subject distance measured by the AF control according to the detected zoom position. This makes it possible to perform an optical zoom operation while maintaining the in-focus state.

On the other hand, the electronic zoom control unit 120 and the image memory 108 are in charge of controlling the electronic zoom. The electronic zoom control unit 120 implements an electronic zoom function by cutting out data in the target area from the image data transferred to the image memory 108. In addition, the electronic zoom control unit 120 realizes a smooth electronic zoom display by displaying the display unit 109 while gradually increasing the range to be extracted at the frame rate period of the image to be captured by the image sensor 106.
The subject detection unit 123 detects a desired subject region from the image data stored in the image memory 108. In the present embodiment, a subject detection method (face detection process, color detection process) for detecting a subject (a face or object such as a person) based on face information or color information included in image data will be described.

  The face detection process is a process for detecting a face area existing in image data by a known algorithm. For example, the subject detection unit 123 extracts a feature amount from a square partial region on the image data, and compares the feature amount with a facial feature amount prepared in advance. If the correlation value between the two exceeds a predetermined threshold, the subject detection unit 123 determines that the partial area is a face area. This determination process is repeated while changing the combination of the size of the partial area, the arrangement position, and the arrangement angle, so that various face areas existing in the image data can be detected.

  In the color detection process, a process of storing color information of a subject area designated according to a subject designation method described later as a characteristic color is executed. The color detection process is performed when the subject to be detected is an object (a “thing” other than a person). As the color information, an RGB signal, a luminance signal (Y signal), a color difference (R−Y, B−Y) signal, and the like, which are output signals from the image processing circuit 107 are used. At the time of subject detection, the subject detection unit 123 divides the image data into a plurality of partial areas, and calculates an average value of luminance and color difference for each partial area. In addition, the subject detection unit 123 compares the characteristic color information stored in advance with the color information of each region at the time of subject detection, and sets a partial region in which the difference in luminance and color difference is equal to or less than a predetermined amount as a subject region candidate. A process is performed in which a group of adjacent partial areas as candidates for the subject area is set as the same color area, and an area where the same color area falls within a predetermined size range is set as a final subject area.

  The subject detection unit 123 calculates the size of the subject area on the image data by using the subject distance information measured by the CZ control unit 119 and the focal length information of the zoom lens 102 together with the face information and the color information. . The posture detection unit 124 detects the posture of the digital camera 100 (for example, normal position / above grip / below the grip) based on information from the acceleration sensor. The shake detection unit 125 detects the shake state of the digital camera 100 by determination based on angular velocity information or the like by a gyro sensor. The shake detection unit 125 determines that the camera is in a hand-held state when the shake amount (detection value) applied to the gyro sensor or the like is greater than or equal to a predetermined amount (threshold value), and if it is less than the predetermined amount, the shake detection unit 125 is fixed to a tripod or the like. It is determined that it has been performed. The acceleration sensor and the gyro sensor used for the posture detection and the shake detection may be configured to also serve as the sensor of the detection unit for acquiring the control information of the anti-vibration lens driving unit 111.

Next, the outline of the auto zoom function in the present embodiment and the AZ control unit 122 will be described. In a camera not equipped with an auto zoom function, the following operations are required when the subject moves out of the frame while framing in the telephoto state and waiting for a photo opportunity, etc.
First, the photographer searches for a subject after performing a zoom-out operation by operating the zoom operation member. After searching for the subject, the photographer adjusts the angle of view by zooming until the desired angle of view is reached again. Even when the size of the subject image changes due to the movement of the subject, the photographer needs to adjust the size of the subject image by operating the zoom operation member.

  On the other hand, in a camera equipped with an auto-zoom function, after the photographer sets the auto-zoom function, an operation for designating a subject with a touch panel or the like may be performed to designate the subject to be photographed. When the auto-zoom function is set, the zoom operation is automatically performed so that the designated subject is within a predetermined size near the center of the image. In addition to the touch panel operation, the subject can be specified by specifying the subject near the center of the screen when the photographer operates a specific button, or automatically selecting the main subject from the subjects detected by the camera. There are methods to select.

  The subject detection unit 123 calculates the position and size of the subject region designated from the image memory 108 on the image data. By performing this process continuously on the sampled image data each time an image is displayed as a live view, the movement of the subject can be tracked. When an image of the tracked subject is detected in a zoom-out area described later, or when the detected image of the subject becomes larger than a predetermined size, the AZ control unit 122 starts a zoom-out operation. That is, the AZ control unit 122 instructs the CZ control unit 119 or the electronic zoom control unit 120 to zoom out in the wide angle direction. When the subject is detected within the zoom-in area and the subject image falls within a predetermined size range, a zoom-in operation is performed toward the telephoto side. With such a process, the photographer need only move the camera so that the image of the desired subject is within the screen, without worrying about the zoom operation. Even if the subject is about to be out of frame, the zoom magnification is automatically changed, so that the angle of view can be adjusted more easily.

  Next, the start conditions for the zoom-out operation and the zoom-in operation will be described with reference to FIGS. FIG. 3 is an explanatory diagram of processing for preventing the subject (object) from being out of the frame. FIG. 4 is an explanatory diagram of processing for preventing a subject (person) from going out of the frame.

  3 and 4, a frame 300a and a frame 300b are first tracking frames (hereinafter referred to as mono tracking frames) that track a subject (object), and frames 400a to 400f track a subject (person's face). This is a second tracking frame (hereinafter referred to as a face tracking frame). Hereinafter, when the subject is applicable to both a person and an object, the mono tracking frame and the face tracking frame may be collectively referred to as a subject tracking frame. The subject tracking frame is displayed so as to surround the subject image on the EVF screen (electronic viewfinder screen) of the display unit 109 so that the subject specified by the photographer can be seen. The position and size of the subject tracking frame on the screen are calculated by the subject detection unit 123 based on the face information and the color information, and are updated at a frame rate cycle.

  With reference to FIG. 3, a process for preventing the subject (airplane) from going out of the frame will be described. In FIG. 3A, a zoom-out area ZO (hereinafter referred to as a ZO area) indicates an area outside a predetermined ratio with respect to the entire angle of view (entire screen) displayed by EVF. For example, assume a case where the center point of the screen is 0%, the entire screen is 100%, and a position that is 80% of the entire screen is set as the boundary of the ZO region. In this case, 80% to 100% of the entire screen is the ZO area. When a part of the mono tracking frame 300a enters the ZO area, the AZ control unit 122 performs control to start the zoom-out operation. Further, the AZ control unit 122 stores in the memory 118 the zoom magnification before zoom movement (corresponding to the zoom-in angle of view). The target zoom magnification and zoom speed during the zoom-out operation are set in advance according to the size and movement speed of the subject image. Further, the target zoom magnification and zoom speed may be appropriately calculated according to the size and moving speed of the subject image. The zoom-out operation is executed according to the target zoom magnification and zoom speed. Thereby, it is possible to effectively prevent the subject from being out of frame.

  FIG. 3B shows the angle of view when a zoom-out operation corresponding to a predetermined zoom change amount is performed from the angle of view of FIG. A zoom-in area ZI (hereinafter referred to as a ZI area) shown in FIG. 3B indicates an area inside a predetermined ratio with respect to the zoom-in angle of view (view angle before zoom-out) 301 in the subject search state. . For example, assume that the center point of the screen is 0%, the zoom-in angle of view 301 is 100%, and a position that is 70% of the zoom-in angle of view 301 is set as the boundary of the ZI region. In this case, an area of 0 to 70% in the entire zoom-in angle of view 301 is the ZI area. At this time, for example, when the zoom-out magnification is ½, the zoom-in angle of view 301 is 50% of the entire screen. Therefore, the boundary of the ZI area is 70% × (½) = 35% with respect to the entire screen, and it can be said that the ZI area is an area of 0 to 35% with respect to the entire screen. When the photographer changes the orientation of the camera so that the mono tracking frame 300b fits inside the ZI area, the AZ control unit 122 performs control to start a zoom-in operation.

  Next, with reference to FIG. 4, a process for preventing the subject (person) from going out of the frame will be described. When the subject is a person, the zoom-out operation starts when a part of the face tracking frame enters the ZO region, and the zoom-in operation is performed when the face tracking frame falls within the ZI region. When the subject is a person, unlike the case of the object, the moving direction of the subject can be predicted to some extent, so the ZO region and the ZI region are set according to the region of the predicted moving direction. In hand-held shooting, the subject may be out of frame due to the influence of camera shake or the like. However, when the subject is out of frame due to camera shake or the like, it is possible for the photographer to re-frame in by an operation to try to frame the subject. Here, when the ZO area is set at the top of the screen, the face tracking frame may enter the ZO area and zoom out unintentionally even when a person is placed near the center for shooting. Therefore, when the subject is a person and is in a hand-held state (a state where the photographer is holding the camera), the ZO region is not set at the top of the screen in consideration of the framing operation of the photographer.

  As described above, in the present embodiment, when the face is detected by the subject detection unit 123, the AZ control unit 122 changes the ZO region and the ZI region according to the posture and shaking state of the camera. The posture of the camera is detected by the posture detection unit 124, and the shaking state is determined from the detection result by the shaking detection unit 125. The detection result by the shake detection unit 125 is a detection result of whether or not the camera is in a handheld state. Hereinafter, a specific description will be given with reference to FIGS.

  FIG. 4A shows a ZO region and a ZI region that are set when the photographer holds the camera in a normal position. In such a shooting scene, when the subject moves in the horizontal direction and goes out of the frame, the position of the subject in the screen moves in the horizontal direction (longitudinal direction) with respect to the screen at the normal position. Therefore, both the ZO region and the ZI region are arranged in a vertical band shape in the vertical direction (short direction) with respect to the screen at the normal position. The ZO area is set in a vertical band shape positioned near both ends in the horizontal direction on a horizontally long rectangular screen, and the ZO area is set in a vertical band shape positioned in the center of the screen. In this case, when the face tracking frame 400a enters the ZO region, the AZ control unit 122 determines the start of zoom-out and performs zoom-out operation control corresponding to a predetermined zoom magnification. When the face tracking frame 400b is included in the ZI region, the AZ control unit 122 determines the start of zoom-in, and performs zoom-in operation control corresponding to a predetermined zoom magnification up to the zoom return position. By setting the ZO region and the ZI region in this way, the subject (person) can be effectively prevented from being out of frame.

  FIG. 4B shows the ZO region and the ZI region that are set when the camera posture is changed in the same shooting scene and the photographer holds the camera in a vertical position under the grip or on the grip. . In this case, the ZO region and the ZI region are arranged in a vertical band shape in the vertical direction (longitudinal direction) with respect to the screen at the vertical position. In other words, the ZO region is set to a vertical band shape that is positioned near both ends in the horizontal direction on a vertically long rectangular screen, and the ZI region is set to a vertical band shape that is positioned at the center of the screen. In this case, when the face tracking frame 400c enters the ZO region, the AZ control unit 122 determines the start of zoom-out and performs zoom-out operation control corresponding to a predetermined zoom magnification. When the face tracking frame 400d is included in the ZI region, the AZ control unit 122 determines the start of zoom-in, and performs zoom-in operation control corresponding to a predetermined zoom magnification up to the zoom return position. By setting the ZO area and the ZI area in this way, it is possible to detect the movement of the subject in the horizontal direction and effectively prevent frame-out.

  FIG. 4C illustrates a ZO region and a ZI region that are set when the detection state in the shake detection unit 125 is a fixed state. The fixed state is a state in which the camera is fixed by a tripod or the like, and there is no possibility of frame-out due to camera shake. Furthermore, if the subject zooms in when the subject is not framed in the vicinity of the center of the screen, there is a possibility that the frame will be out due to the operation. Therefore, a ZO region is set for the entire peripheral portion of the screen, and a ZI region is set inside the zoom-in angle of view. That is, the ZO area is set to a rectangular band located near the edges in the vertical and horizontal directions of the screen, and the ZI area is set to a rectangular shape located at the center of the screen. In this case, when the face tracking frame 400e enters the ZO region, the AZ control unit 122 determines the start of zoom-out and performs zoom-out operation control corresponding to a predetermined zoom magnification. When the face tracking frame 400f is included in the ZI region, the AZ control unit 122 determines the zoom-in start and performs zoom-in operation control corresponding to a predetermined zoom magnification up to the zoom return position.

  As described above, in the present embodiment, the ranges of the ZO region and the ZI region are dynamically changed according to changes in the posture of the camera and the photographing state (hand-held state / fixed state). Accordingly, it is possible to effectively prevent the subject from being out of frame while preventing malfunction due to camera shake or the like. Note that the ZO area or the ZI area may be changed according to either the camera posture or the shooting state (handheld state / fixed state), or only one of the ZO area and the ZI area may be changed. Good.

  Next, a zoom operation for keeping the ratio of the subject image to the screen within a predetermined range will be described. In this embodiment, control is performed so as to keep the size of the subject image within the predetermined range from the reference size by automatically performing a zoom operation when the size of the detected subject image changes more than a predetermined multiple of the reference size. (Size holding control) is performed. FIG. 5 is a diagram illustrating a process for maintaining the size of the subject (person) image in the screen. FIG. 6 is an explanatory diagram of composition setting of a subject (person).

  First, with reference to FIG. 6, the setting of a range (composition) in which a subject to be tracked is included in the screen will be described. FIG. 6A illustrates a screen display when the composition setting is set to “manual”. When “manual” is set, the photographer changes the size of the face to be tracked by performing a manual zoom operation by operating the zoom lever while looking at the image of the person on the screen. The size of the subject image at that time is stored in the memory 118 as a reference size. FIG. 6B illustrates a screen display when the composition setting is set to “face”. When the “face” composition is set, the size that fits the face on the screen according to the camera posture and the face direction is calculated as the reference size and stored in the memory 118. Similarly, FIG. 6C illustrates a screen display when the composition setting is set to “upper body”, and FIG. 6D illustrates a screen display when the composition setting is set to “whole body”. Illustrate. A reference size is calculated so as to have a size set on the screen and stored in the memory 118.

  FIG. 6E illustrates a screen display when the composition setting is set to “auto”. When “Auto” is set, the AZ control unit 122 determines an appropriate composition based on the position of the subject, the size of the subject, the number of subjects, the posture of the camera, and the like. Further, a reference size is calculated and stored in the memory 118 so as to achieve the determined composition. A method for calculating the reference size will be described later with reference to FIG.

  The photographer can perform an operation of changing the composition setting by operating the left and right buttons of the operation unit 117 or selecting on the setting menu from the shooting screen. When the composition setting change operation is performed, the AZ control unit 122 updates the composition setting information stored in the memory 118. FIG. 6 shows an example in which “manual”, “face”, “upper body”, “whole body”, and “auto” composition are set when the subject is a person, but the composition setting is limited to this. It is not something. Only a part of the five illustrated compositions may be set, or other compositions may be included. Further, when the subject is an object, for example, “manual”, “large”, “medium”, “small”, and “auto” compositions may be set.

  With reference to FIG. 5, the zoom operation for keeping the ratio of the subject image to the screen within a predetermined range will be described as an example in which the composition setting is “manual”. FIGS. 5A to 5C illustrate a zoom operation automatically performed by the camera when a subject (person) approaches the camera. This is a zoom-out operation for keeping the ratio of the subject image on the screen within a predetermined ratio. In FIG. 5, the face tracking frames 500a to 500f are displayed so as to surround the face area as the characteristic area of the person who is the subject. Therefore, here, the size of the face tracking frame will be described as the subject size.

  FIG. 5A shows the angle of view when a subject is designated according to a subject designation method described later. The size of the face tracking frame 500a at the time of subject designation is stored in the memory 118 as a reference subject size (reference size). FIG. 5B shows an angle of view when the subject approaches the camera without changing the zoom magnification from the state of FIG. For example, a size that is 150% of the size of the face tracking frame 500a that is the reference subject size is set as the start size of the zoom-out operation. When the relationship of the subject tracking frame (face tracking frame) is “face tracking frame 500b> face tracking frame 500a × 150%”, that is, the tracking frame has changed beyond a predetermined change amount with respect to the reference size. Sometimes, the AZ control unit 122 determines the start of the zoom-out operation.

  FIG. 5C shows an angle of view zoomed out from the angle of view 501 of FIG. 5B by a predetermined zoom magnification, and a face tracking frame 500c. Here, the predetermined zoom magnification is set to 1 / 1.5 in consideration of the rate of change (150%) of the face tracking frame size with respect to the reference subject size when starting the zoom-out operation. Thereafter, when the subject further approaches the camera, the subject image can be kept within a predetermined ratio by further zooming out to the wide angle side. Therefore, the photographer can concentrate only on the operation of the release switch.

  On the other hand, FIGS. 5D to 5F illustrate a zoom operation that is automatically performed by the camera when a subject person moves away from the camera. This is a zoom-in operation for keeping the ratio of the subject image on the screen within a predetermined range. FIG. 5D shows the angle of view when a subject is designated according to a subject designation method described later. The size of the face tracking frame 500d at the time of subject designation is stored in the memory 118 as a reference subject size (when the composition setting is “manual”).

  FIG. 5E shows the angle of view when the subject moves away from the camera without changing the zoom magnification from the state of FIG. For example, a size that is 50% of the size of the face tracking frame 500d that is the reference subject size is set as the start size of the zoom-in operation. When the relationship of the face tracking frame becomes “face tracking frame 500e <face tracking frame 500d × 50%”, it is determined that the tracking frame has changed beyond a predetermined change amount with respect to the reference size. When this determination condition is satisfied and the face tracking frame 500e is included in the ZI region, the AZ control unit 122 determines the start of the zoom-in operation. Here, the ZI region is set inside the angle of view 502 zoomed in by a predetermined zoom magnification with respect to the angle of view of FIG.

  FIG. 5F shows an angle of view (corresponding to the angle of view 502) zoomed in by a predetermined zoom magnification from the angle of view of FIG. 5E, and a face tracking frame 500f. Here, the predetermined zoom magnification is set to 1 / 0.5 in consideration of the rate of change (50%) of the face tracking frame size with respect to the reference subject size when starting the zoom-in operation.

  In FIG. 3, the processing for preventing frame-out when the subject is an object and FIG. 4 when the subject is a person has been described. Further, FIG. 5 illustrates the processing for keeping the ratio of the size of the subject image in the screen within a predetermined range when the subject is a person. Note that when the subject to be tracked is an object, the zoom operation start determination for controlling the size of the subject image shown in FIG. 5 may be performed as in the case where the subject is a person. Even when the composition setting is other than “manual”, the zoom operation is automatically performed in the same manner as in FIG. 5 according to each reference size.

Next, processing of the auto zoom function will be described with reference to FIGS. FIG. 7 is a flowchart illustrating an example of overall processing of the auto zoom function. The following auto-zoom functions are performed based on commands from the system control unit 114 (AZ control unit 122) unless otherwise specified.
First, in S <b> 700, the AZ control unit 122 determines the operation state of the auto zoom operation switch of the operation unit 117. When the auto zoom operation switch is pressed, the process proceeds to S701, and when the switch is not pressed, the determination process of S700 is repeated. In step S701, subject search processing is executed, and further subject specification processing in step S702, basic size setting processing in step S703, and auto zoom control in step S704 are sequentially executed.

  Immediately after the processing of the auto zoom function is started, or when the subject search button of the operation unit 117 is pressed (Yes in S705), the subject search processing in S701 is executed. With reference to FIG. 8, the subject search process executed in step S701 will be described. When the subject to be photographed is far from the camera, the subject detection unit 123 may not be able to detect if the subject feature area is too small even if the subject image exists in the screen. In such a scene, the auto zoom function based on the subject information cannot be executed. Therefore, subject search processing is performed in which a zoom-in operation is performed while detecting the subject up to a predetermined focal length.

  In S800, it is determined whether or not the subject search button has been pressed immediately after the start of auto-zoom, or not just after the start of auto-zoom but during auto-zoom. In S800, in the case of the former positive determination result (yes), the process proceeds to S801, and in the case of a negative determination result (no), the process proceeds to S803. In step S801, processing for acquiring the subject search end focal length is executed. The subject search end focal length is a focal length at the end zoom position when the search for the subject image in the image is ended, and is hereinafter referred to as a search end focal length. In this embodiment, immediately after the start of auto zoom, the search end focal length is set to 85 mm in terms of 35 mm. This focal length is the focal length of a standard lens that is generally suitable for photographing a person. Even if there is no subject (person) or a face cannot be detected (such as when facing backwards), after the search is completed. It is possible to enter a subject detection waiting state with an appropriate angle of view. However, the focal length is not limited to 85 mm, and may be other focal lengths (for example, 50 mm, 100 mm, etc.). Further, a configuration in which the photographer can change the search end focal length from the menu setting, or a configuration in which the search end focal length can be changed in accordance with the shooting mode or the subject to be shot or the like. When the processing proceeds from S801 to S802, the focal length at the start of the mode is compared with the search end focal length, and it is determined whether the focal length is wider than the search end focal length or on the telephoto side. If the focal length at the start of auto-zoom is closer to the telephoto side than the search end focal length (no in S802), the subject search process is terminated. If the focal length at the start of auto-zoom is on the wide-angle side from the search end focal length (yes in S802), the process proceeds to S805.

  On the other hand, when it is determined in S800 that the subject search is performed by operating the subject search button, in S803, a process of acquiring the search end focal length is executed. In the present embodiment, the search end focal length when the subject search button is operated is set to the focal length on the telephoto side by 2.5 times the focal length when the operation is executed. However, this magnification is an example, and other magnifications may be used. Further, a configuration in which the photographer can change the search end focal length from the menu setting, or a configuration in which the search end focal length can be changed in accordance with the shooting mode or the subject to be shot or the like. Proceeding from S803 to S804, the search end focal length acquired in S803 is compared with the telephoto end (telephoto end) focal length, and it is determined whether the search end focal length is wider or telephoto than the telephoto end focal length. . If the search end focal length is closer to the telephoto side than the telephoto end focal length, the subject search processing is ended. If the search end focal length is on the wide-angle side with respect to the telephoto end focal length, the process advances to step S805. In S805, the AZ control unit 122 sets the zoom change amount corresponding to the search end focal length acquired in S801 or S803 in the CZ control unit 119 or the electronic zoom control unit 120, and instructs to perform a zoom-in operation.

  Next, in S806, it is determined whether or not the current focal length has reached the search end focal length. When the current focal length reaches the search end focal length, the process proceeds to S809, and after the zoom-in operation is stopped, the subject search processing is ended. If it is determined in S806 that the current focal length has not reached the search end focal length, the process proceeds to S807. S807 is a process for determining whether or not a subject has been detected by the subject detection unit 123. If it is determined in S807 that the subject has been detected, the process proceeds to S808. If it is not determined that the subject has been detected, the process returns to S806. In S808, it is determined whether or not the position where the subject is detected is within a predetermined search end area on the screen. The predetermined search end region is determined by a threshold value (position determination reference value) set in advance according to the subject. As a result of the determination, if the subject detection position detected in S807 is within a predetermined search end area on the screen, the process proceeds to S809, the zoom-in operation is stopped, and the subject search process is ended. If it is determined in S808 that the detection position of the subject is not within the predetermined search end area on the screen, the process returns to S806, and the determination processing from S806 to S808 is continued.

  Here, the predetermined search end region described in S808 will be described with reference to FIG. When photographing a specific subject (person or object), the subject to be photographed is often arranged near the center of the screen. Further, when a subject not to be photographed is detected around the screen and the subject search process is terminated, the subject search process may be terminated before the subject to be photographed is detected. In this case, the auto zoom function as intended by the photographer cannot be executed. Therefore, in the present embodiment, the search end region is set as the determination region so that the subject search process is ended only when the subject is detected in the region near the center of the screen. FIG. 9A illustrates a search end area 900a that is set when the subject is an object (airplane). In the case of an object, a predetermined ratio, for example, a range of 70% from the center of the screen is set as the subject search end region in any of the top, bottom, left and right directions of the screen. On the other hand, FIG. 9B illustrates the search end area 900b set when the subject is a person. In the case of a person, the search ends so that the area at the top of the screen includes a range of 100% from the center of the screen and the other direction (left and right direction, downward) includes a predetermined ratio from the center of the screen, for example, a range of 70%. An area is set. When the subject is a person, there is a possibility that the face tracking frame 902 of the subject to be photographed is also arranged at the top of the screen. Therefore, even when a subject is detected at the top of the screen, the search end area is set so that the subject search process is completed.

  Next, with reference to FIG. 10, the subject designation process shown in step S702 in FIG. 7 will be described. FIG. 10A is a flowchart illustrating an example of processing in which a user specifies a desired subject using the touch panel constituting the operation unit 117. In this example, it is assumed that the user performs an operation of specifying by touching the subject image displayed on the display unit 109. In S1000, the AZ control unit 122 determines whether the touch panel has been pressed. When the touch panel is pressed, the process proceeds to S1001, and when the touch panel is not pressed, the determination process of S1000 is repeated.

  In S1001, the AZ control unit 122 acquires information on a position touched by the user (touch position). In next step S1002, the AZ control unit 122 notifies the subject detection unit 123 of the touch position, and the subject detection unit 123 performs face detection near the touch position. If a face is detected in the vicinity of the touch position in S1002, it is determined that the main subject is a person, and the process proceeds to S1003. If no face is detected in the vicinity of the touch position in S1002, it is determined that the main subject is an object other than a person, and the process proceeds to S1004.

  In S1003, the AZ control unit 122 performs control to store the face information of the person who is the target of automatic tracking in the memory 118. Specific face information includes information such as the face size, face detection position, and face orientation when the subject is designated. In addition, in a camera having a face authentication function, identification information such as an authentication ID is also stored in the memory 118. In step S <b> 1004, the AZ control unit 122 performs control to store the characteristic color in the vicinity of the touch position in the memory 118 as color information for the automatic tracking target. Specific color information includes information such as the characteristic color at the time of subject designation and its luminance, the value of the color difference, the size of the same color area, and the barycentric position of the same color area. In the following description, face information and color information are collectively referred to as subject information (including subject size, subject detection position, etc.).

After the subject information is stored in the memory 118 in S1003 or S1004, the process proceeds to S1005 and a subject tracking frame is displayed. In step S1005, the AZ control unit 122 performs control for causing the display unit 109 to display a subject tracking frame (mono tracking frame or face tracking frame) having a size corresponding to the subject size around the subject detection position. Thereafter, the subject designation process ends.
As described above, in the system control unit 114, the subject detection unit 123 detects the subject at the position designated by the photographer on the display unit 109 or in the vicinity of the designated position. The AZ control unit 122 displays the subject tracking frame on the display unit 109. According to the processing example in FIG. 10A, the subject that the photographer wants to track can be easily specified by an intuitive method. However, the method for specifying the subject is not limited to this method. Another method will be described with reference to FIG.

  FIG. 10B is a flowchart showing a processing example when the user designates a subject using a switch (subject designation switch) different from the auto zoom operation switch as an operation member constituting the operation unit 117. is there. First, in step S <b> 1006, the AZ control unit 122 displays a frame serving as a guide for specifying a subject near the center of the screen of the display unit 109. The photographer adjusts the direction of the camera so that the image of the subject to be tracked is placed near the center of the screen using this frame as a guide. In next step S1007, the AZ control unit 122 determines whether or not the subject specifying switch has been pressed. If the subject designation switch is pressed, the process proceeds to S1008. If the switch is not pressed, the determination process of S1007 is repeated in a waiting state.

  In step S1008, the subject detection unit 123 performs face detection near the center of the screen. If a face is detected near the center of the screen, the main subject is determined to be a person, and the process proceeds to S1009. On the other hand, if no face is detected near the center of the screen, the main subject is determined to be an object other than a person, and the process proceeds to S1010. In S1009, the same processing as S1003 in FIG. 10A is performed, and in S1010, the same processing as S1004 in FIG. 10A is performed. After the subject information is stored in the memory 118 in S1009 or S1010, the process proceeds to S1011 where the subject tracking frame (mono tracking frame or face tracking frame) is displayed, and the subject designation process is terminated.

  10B, the subject detection unit 123 detects the subject at the screen center position of the display unit 109 or in the vicinity of the screen center position. Then, the AZ control unit 122 causes the display unit 109 to display a subject tracking frame indicating the position of the subject. Even in a camera not equipped with an operation member such as a touch panel, the user can easily specify a subject.

  FIG. 11 is a flowchart illustrating an example of processing for automatically selecting a subject to be tracked from faces detected when an auto zoom operation switch constituting the operation unit 117 is pressed. First, in step S1100, the subject detection unit 123 performs face detection on the entire screen and determines whether a human face has been detected. If even one person's face is detected on the entire screen, the main subject is determined to be a person, and the process proceeds to S1101. If no face is detected in the screen, the process proceeds to S1105.

  In step S1101, the subject detection unit 123 selects the face as the main face when there is only one detected face person. When there are a plurality of detected faces, the subject detection unit 123 selects a main face as a subject to be tracked from the faces. As a determination criterion for main face selection, for example, there is a method of preferentially selecting a face whose face detection position is located near the center of the screen. In addition, when there are a plurality of faces at the same position, there is a method of selecting a larger face as the main face. In the case of a camera having a face authentication function, if there is a face registered for authentication, there is a method in which the face is given priority as a main face.

  Proceeding from S1101 to S1102, the AZ control unit 122 stores the face information of the selected main face in the memory 118, and then proceeds to S1103 to display a face tracking frame on the screen. In next step S1104, face change determination is performed. That is, when a plurality of faces are detected, the main face automatically selected is not necessarily the face intended by the photographer. Therefore, the photographer can change the main face. At this time, when the photographer presses a switch (such as an auto zoom operation switch) on the operation unit 117 to designate a desired face, another face that is not selected as the main face from the faces in which the face tracking frame is detected. A process of changing the main face is performed. When the main face is changed in S1104, the process returns to S1102 again, and the face information to be stored in the memory 118 is updated. In S1103, the face tracking frame is changed to the size and detection position of the newly selected main face.

  On the other hand, when the process proceeds from S1100 to S1105, it is determined that the main subject is an object other than a person, and the AZ control unit 122 performs control to store the feature color near the center of the screen in the memory 118 as color information of the automatic tracking target. I do. In the next step S1106, after the mono tracking frame is displayed on the screen, the subject designation process ends. Basic processes in S1102, S1105, and S1103 (S1106) in FIG. 11 are the same as those in S1003, S1004, and S1005 in FIG.

  As described above, in the processing example of FIG. 11, the subject detection unit 123 performs face detection on the entire screen of the display unit 109 in the system control unit 114. When a plurality of faces are detected, the AZ control unit 122 performs control for causing the display unit 109 to display a subject tracking frame indicating the position of the first face as a subject from among the plurality of faces. In addition, when the first face is changed to the second face as the subject, the AZ control unit 122 performs control to display the subject tracking frame indicating the position of the second face on the display unit 109. The photographer can easily specify a desired subject with fewer operations.

  For the subject designation process, in the case of a camera having both the touch panel constituting the operation unit 117 and the subject designation switch, both the method of FIG. 10A and the method of FIG. 10B are applied simultaneously. May be. Further, after the subject change determination in S706 shown in FIG. 7 is made, the subject is designated by the method of FIG. 10A or FIG. 10B, and after the subject search processing of S701, FIG. The subject can also be specified by the method. The subject designation process to be applied may be changed according to the process flow. When the subject designation process (S702 in FIG. 7) is completed, the process proceeds to S703 in FIG. 7, and a process for setting the size of the subject to be tracked as a reference size is executed.

  With reference to FIGS. 12 to 19, the reference size setting process in S <b> 703 illustrated in FIG. 7 will be described. FIG. 12 is a flowchart illustrating an example of a reference size setting process when the subject is a person. First, in S1200, the AZ control unit 122 determines whether or not the composition setting stored in the memory 118 is the above-described “manual”. If the composition setting is “manual”, the process advances to step S1201. If the composition setting is other than “manual”, the process advances to step S1203.

  In step S1201, the AZ control unit 122 determines subject detection. The AZ control unit 122 determines whether or not the subject detection unit 123 detects a face when the subject is a person and whether or not the same characteristic color is detected when the subject is an object. The determination process in S1201 is repeated until a subject is detected, and if a subject is detected, the process proceeds to S1202. In S1202, the AZ control unit 122 sets the size of the detected subject image as a reference size, stores it in the memory 118, and ends the process. Accordingly, when the composition setting is “manual”, the size (detected size) of the subject image detected first when the reference size setting process is started is stored in the memory 118 as the reference size.

  On the other hand, in S <b> 1203, the AZ control unit 122 calculates the screen size based on the camera posture information determined by the posture detection unit 124. This screen size is used to calculate the reference size in S1211, S1704, S1705, S1806, and S1807, which will be described later. The screen size calculation process will be described with reference to FIG. In this embodiment, the case where the screen size of the still image is long side × short side = 640 × 480 and the screen size of the moving image is long side × short side = 640 × 360 is exemplified, but the screen size is limited to this. Is not to be done.

  In S1300, the posture detection unit 124 determines whether the photographer holds the camera at the normal position or the vertical position. If it is determined that the photographer holds the camera in the normal position, the process proceeds to S1301. If the photographer holds the camera in the vertical position, the process proceeds to S1306. In step S1301, processing for setting the long side size 640 to the horizontal screen size is executed. In next step S1302, it is determined whether or not the current setting mode is a mode (still image / moving image simultaneous shooting mode) in which a still image and a moving image can be simultaneously captured. The still image / video simultaneous shooting mode is a mode in which a photographer can simultaneously take a still image by pressing the release switch of the operation unit 117 during movie shooting, or a framing operation immediately before shooting a still image as a movie. It is a mode that can be recorded automatically. If it is determined in S1302 that the still image / moving image simultaneous shooting mode is set, the processing compares the short side size of the moving image with the short side size of the still image and sets the smaller size as the vertical screen size. This is to prevent the subject from being framed out of the screen at any angle of view of still images and moving images by performing auto-zoom in accordance with the small screen size. Therefore, in this embodiment, in the still image / moving image simultaneous shooting mode, the process proceeds to S1304, and 360, which is the short side size of the moving image, is set as the vertical screen size. Thereafter, the screen size calculation process ends.

  If it is determined in S1302 that the mode is not the still image / moving image simultaneous shooting mode, the process proceeds to S1303. In step S1303, it is determined whether moving image recording is in progress or still image framing is in progress. If it is determined that the moving image is being recorded, the process proceeds to S1304. If it is determined that the moving image is not being recorded, the process proceeds to S1305. In S1305, the short side size 480 of the still image is set as the vertical screen size.

  If it is determined in S1300 that the photographer is in the vertical position, the processing from S1306 to S1310 is executed. The processes of S1306 to S1310 are processes in which the settings of the horizontal screen size and the vertical screen size are reversed with respect to the processes of S1301 to S1305. That is, in the above description, “horizontal” and “vertical” may be interchanged so that the detailed description is omitted. In this embodiment, the still image screen size is 4: 3 (see FIGS. 15A and 15B), and the moving image screen size is 16: 9 (see FIG. 15C). The screen size may be changed according to the aspect ratio setting. Accordingly, it is possible to make the ratio of the person image to the screen constant regardless of the aspect ratio. Further, as an example of the still image / moving image simultaneous shooting mode, an example in which the screen size is different between the still image and the moving image has been shown.

  When the screen size is determined in S1203 of FIG. 12, the process proceeds to S1204. When the composition setting stored in the memory 118 is any one of “face”, “upper body”, and “whole body”, in steps S1204 to S1209, the AZ control unit 122 determines the face image occupied on the screen according to the composition setting. Determine the size ratio (called face ratio). The face ratio of the present embodiment indicates the ratio of the length of the face tracking frame in the vertical direction relative to the screen, but may be indicated by the ratio of the length in the horizontal direction, the area ratio, or the like. Specifically, for example, it is determined whether or not the composition setting is “face” in S1204. If the composition setting is “face” (yes in S1204), the face ratio is determined to be 27% (S1207). If the composition setting is not “face”, the process advances to step S1205 to determine whether the composition setting is “upper body”. If the composition setting is “upper body” (yes in step S1205), the face ratio is 16%. It is determined (S1208). If the composition setting is not “upper body”, the process proceeds to S1206, and it is determined whether or not the composition setting is “whole body”. If the composition setting is “whole body” (yes in S1206), the face ratio is determined to be 7%. (S1209). After the face ratio is determined in steps S1204 to S1209, the process proceeds to S1210.

  In S1210, the AZ control unit 122 calculates the reference size by multiplying the face ratio determined in S1204 to S1209 by the vertical screen size calculated in S1203. This is a method of calculating the reference size from the face ratio with respect to the screen size in the vertical direction on the premise of a subject (person) in a standing posture. When the subject detection unit 123 can determine the face direction, the screen size direction may be determined based on the vertical direction of the face on the screen. Further, after the process of storing the calculated reference size in the memory 118 is executed, the reference size setting process is terminated. Accordingly, when the composition setting is any one of “face”, “upper body”, and “whole body”, a size that is a predetermined face ratio corresponding to the composition setting with respect to the screen size is set as the reference size.

  If the composition setting is not “whole body” in S1206, that is, if it is determined that the composition setting is “auto”, the process proceeds to S1211, and the AZ control unit 122 moves the shooting state, the number of subjects, and the movement of the subject. An appropriate composition is automatically determined according to the above. With reference to FIG. 14, the automatic composition determination and composition selection processing in the composition setting “auto” in S1211 will be described. The composition setting “Auto” is a mode in which the composition can be further easily adjusted by discriminating a shooting scene and automatically selecting an appropriate composition (automatic composition selection).

  FIG. 14 shows a transition diagram of the composition automatically selected in the present embodiment. In the present embodiment, composition selection processing is performed in which an appropriate composition is selected from the five compositions of the upper body 1400, the whole body 1401, the face 1402, the plurality of people 1403, and the manual 1404 according to the shooting scene. In the composition adjustment operation, when shooting a still image, it is necessary to instantaneously set the subject image to an appropriate size before shooting. On the other hand, at the time of shooting a moving image, it is necessary to always set the subject image to an appropriate size while shooting, such as following a moving subject or approaching a stationary subject. In addition, since the image data at the time of zoom-in operation or zoom-out operation by changing the composition is recorded at the time of moving image shooting, an effective video can be taken by changing the composition at an appropriate timing. Thus, the appropriate composition differs between the still image and the moving image. Therefore, processing for changing the composition to be selected depending on the shooting state is executed during still image framing 1405 and during moving image recording 1406. That is, in the present embodiment, a still image shooting preparation state is illustrated as the first operation state, and a moving image recording state is illustrated as the second operation state. As a composition of a selection candidate belonging to the first composition group in the still image framing 1405, any composition is selected from the upper body 1400, the whole body 1401, and a plurality of people 1403. As the composition of the selection candidate belonging to the second composition group in the moving image recording 1406, any one of the upper body 1400, the whole body 1401, and the face 1402 is selected. That is, the composition of the upper body 1400 and the whole body 1401 is common to the first composition group and the second composition group, the composition of the plurality of persons 1403 is included only in the first composition group, and the composition of the face 1402 is the second composition group. It is included only in the composition group. Note that three or more operation states and a plurality of composition groups (including compositions that can be selected in each operation state) can be set in accordance with the specifications of the imaging apparatus.

  Here, the determination conditions of the shooting scene and the transition of each composition will be described. When the automatic composition determination process S1211 of the composition setting “auto” starts, the upper body 1400 is set as the composition in the initial state. The composition of the upper body 1400 is a composition that is selected when it is determined during the still image framing 1405 or the moving image recording 1406 that the scene is a single object that is still. In such a scene, by composing the upper body of the subject within the screen, it is possible to achieve a composition that is reasonably close to the subject.

  When the movement of the subject to be imaged is detected in the composition of the upper body 1400, or when it is determined that the distance from the camera to the subject is greater than the predetermined distance D1 (first threshold), the composition is the whole body 1401. Changed to When the composition of the whole body 1401 is determined to be a scene for shooting a moving subject or a distant subject, that is, a subject subject to being easily framed out of the screen, during still image framing 1405 or during moving image recording 1406. The composition to be selected. In such a scene, the subject is set so as not to be out of the frame as much as possible by following the subject with a composition that fits the whole body of the subject. Further, when the composition of the whole body 1401 detects that the subject to be photographed is stationary for a predetermined time or more and it is determined that the distance from the camera to the subject is smaller than the predetermined distance D2 (second threshold). The process of returning the composition to the upper body 1400 is executed. The predetermined time is a predetermined reference time for determination.

  Hereinafter, a method for determining the movement and stillness of a subject will be described. When at least one of the amount of change in the position and size of the subject detected by the subject detection unit 123 or the amount of shake detected by the shake detection unit 125 while detecting the subject is larger than a predetermined amount, It is determined that the subject to be photographed is moving. The amount of change in the position and size of the subject detected by the subject detection unit 123 and the amount of shake detected by the shake detection unit 125 while detecting the subject all continue to be smaller than a predetermined amount for a predetermined time or more. In this case, it is determined that the subject to be photographed is stationary. The duration of the detection state is measured by a timer for timekeeping.

  Further, a method for calculating the distance from the camera to the subject will be described. First, a reference focal length and a subject distance with respect to a subject size are measured in advance, and measurement data is stored in the memory 118. The subject distance is calculated by performing a calculation of “subject distance = (reference subject distance × focal length × reference subject size) / (reference focal length × detection size)” with respect to the reference value stored in the memory 118. . For example, it is assumed that a subject image having a detection size of 40 pix is detected at a focal length of 120 mm when the reference value is a reference subject distance of 2 m, a reference focal length of 24 mm, and a reference subject size of 20 pix (pixel). In this case, the subject distance is calculated as (2 m × 120 mm × 20 pix) / (24 mm × 40 pix) = 5 m.

  Next, a composition when a plurality of subjects are detected in the screen will be described. When a plurality of subjects are detected in the screen in the composition of the upper body 1400 or the whole body 1401 during the still image framing 1405, processing for changing the composition to the plurality of people 1403 is performed. At this time, the number of detected subjects is stored in the memory 118. The composition of the plurality of persons 1403 is a composition that is selected when it is determined that a group photo shooting scene in which all of a plurality of subjects fall within a predetermined angle of view during 1405 during still image framing. In the composition state of the plurality of people 1403, a determination process is performed as to whether or not the number of detected subjects has changed. When the number of detected subjects increases, a new subject has entered the screen, so the number of subjects stored in the memory 118 is immediately updated, and the composition is changed. On the other hand, when the number of detected subjects decreases, there is a possibility that a state in which the subject cannot be detected temporarily, for example, when any subject faces sideways, may occur. For this reason, if the composition is changed immediately, a subject that can no longer be detected may be out of frame. Therefore, when the number of subjects decreases, it is determined whether or not the state has continued for a predetermined time or more. When the state continues for a predetermined time or more, the number of subjects stored in the memory 118 is updated and the composition is changed. Further, when the subject is alone, a process of returning the composition to the upper body 1400 is executed.

  If the composition is such that all the subjects fall within the angle of view during moving image recording 1406, there is a concern that when a subject other than the subject is moving, a zoom malfunction may occur in response to the subject. Therefore, even when a plurality of subjects are detected during moving image recording 1406, processing is performed to keep the composition of the upper body 1400 or the whole body 1401 with only the main subject as the shooting target. In the composition of the upper body 1400 during moving image recording 1406, when a face is detected for a predetermined time or more near the center of the screen, processing for changing the composition to the face 1402 is performed. The composition of the face 1402 is a composition that is selected when it is determined during the moving image recording 1406 that the photographer is paying attention to the face of the subject. In such a scene, the composition of the face closer to the face than the upper body allows the face of the subject of interest to be photographed in a larger size. Further, during the zoom-in operation by this composition change, a more effective image can be taken by changing the zoom speed to a significantly lower speed or higher speed than the normal speed. Furthermore, when the face 1402 is detected in the vicinity of the screen for a predetermined time or more in the composition of the face 1402, or when it is determined that the photographer has changed the framing, the composition is returned to the upper body 1400. Processing is performed. In the present embodiment, the peripheral area of the screen is exemplified as the specific area of the image, and the timer for measuring the duration of the state where the position of the face image of the subject detected by the subject detection unit 123 is the peripheral area. Further, as a method for determining framing change, when the subject detection unit 123 has not detected a subject and the shaking amount detected by the shaking detection unit 125 is larger than a predetermined amount, the photographer performs framing. It is determined that it has changed.

  During still image framing 1405, the photographer may continue to wait for a photo opportunity with the same composition until the face orientation and facial expression of the person are in a desired state. In this case, a composition close to the face may result in a composition different from the photographer's intention. Therefore, even if a face is detected in the vicinity of the center of the screen for a predetermined time or more in the composition of the upper body 1400 during the still image framing 1405, processing for maintaining the composition of the upper body 1400 is executed.

  Next, a composition in the case where the shooting state is changed by starting / ending moving image recording will be described. Each composition of the upper body 1400 and the whole body 1401 is a composition that can be selected in either case of the still image framing 1405 or the moving image recording 1406. Therefore, in those compositions, the original composition is maintained even if the shooting state is changed. On the other hand, the composition of the plurality of people 1403 and the face 1402 is selected only in any of the cases of the still image framing 1405 and the moving image recording 1406. Therefore, when the moving image recording is started with the composition of a plurality of people 1403 during the still image framing 1405, or when the moving image recording is ended with the composition of the face 1402 during the moving image recording 1406, the upper body 1400, which is a common composition, is displayed. It is possible to change. However, in this case, there is a concern that the zoom operation starts at the start and end of the moving image recording, so that an image by the zoom operation is recorded at the beginning of the moving image, or the photographer feels troublesome due to the framing of the still image. . Therefore, when the shooting state is changed in such a state, processing for temporarily changing the composition to the manual 1404 is executed. The composition of the manual 1404 is a composition in which the size of the main subject image detected first after the composition change is used as a reference size. Therefore, the zoom operation does not start immediately after changing the shooting state, and the photographer does not feel uncomfortable. Even when the subject starts to move, the size of the subject image can be maintained. Further, when a predetermined time elapses in the composition of the manual 1404, processing for returning the composition to the upper body 1400 is executed.

  As described above, in S <b> 1211 of FIG. 12, when the composition setting is “auto”, an appropriate composition is automatically determined according to the shooting state, the number of subjects, the movement of subjects, and the like. Data indicating the composition after the determination is stored in the memory 118, and the process proceeds to S1212.

  In S1212, whether the composition determined in S1211 has been changed from the previous composition (whether the composition selected in FIG. 14 has changed, or the number of subjects has changed in the state of multiple people 1403). Is determined. If the composition has been changed, the process advances to step S1213 to determine whether or not the selected composition is the composition of the manual 1404. If the selected composition is the composition of the manual 1404, the process proceeds to S1201, and a process of storing the size of the first detected subject image in the memory 118 as a reference size is executed. If the selected composition is not the composition of the manual 1404, the process proceeds to S1215, and the reference size in the composition setting “auto” is calculated. If it is determined in S1212 that the composition has not been changed, the process proceeds to S1214 to determine whether or not the subject image has moved within the screen. As a method of calculating the amount of movement of the subject image within the screen, the center position of the face tracking frame immediately after the zoom operation is performed up to the angle of view that becomes the reference size in the memory 118 after the reference size is set in S1215. Remembered. The distance from the position stored in the memory 118 to the center position of the detected face tracking frame is calculated as the movement amount. Alternatively, the amount of movement within the screen may be calculated by adding the center position of the face tracking frame at every predetermined period. When the calculated movement amount is equal to or greater than a predetermined amount, it is determined that the subject has moved. The predetermined amount is normalized by the size of the detected face tracking frame (set to a predetermined multiple of the size of the face tracking frame), so that the actual amount of movement of the subject regardless of the size of the subject image on the screen Can be comparable. If it is determined in S1214 that the subject is moving within the screen, the process proceeds to S1215, and a reference size in the composition setting “auto” is calculated. If it is determined in S1214 that the subject image has not moved within the screen, the reference size setting process is terminated without updating the reference size.

  Hereinafter, with reference to FIGS. 15 to 19, the reference size calculation process in the composition setting “auto” will be described. When the composition setting is any one of “face”, “upper body”, and “whole body”, the reference size is calculated so that the face has a constant ratio to the screen size in S1210 of FIG. That is, the same reference size is calculated regardless of the position of the subject image on the screen. Therefore, if the composition is changed so that the subject image becomes larger (zoom in) when the subject is in the periphery of the screen, the subject may be out of frame. When the reference size is calculated by this method, the photographer needs to frame the subject near the center of the screen. Therefore, it is not suitable for a case where the subject is shifted from the center of the screen and taken together with the background. Further, when there are a plurality of subjects, it is necessary to calculate the reference size by another method. Therefore, when the composition setting is “auto”, the size of the subject image is determined based on the composition determined in step S1211, and the reference is set so that the subject is within the screen regardless of the position of the detected subject. A process of calculating the size is performed.

FIG. 15 is a diagram showing the relationship between the position of the subject closest to the screen among a plurality of subjects and the screen size. 15A and 15B show examples of still image shooting screens, and FIG. 15C shows an example of moving image shooting screens, illustrating a plurality of subjects. FIG. 16 is a flowchart showing the entire reference size calculation process in composition setting “auto”.
First, in S1600 of FIG. 16, processing for calculating the subject position ratio Rh in the horizontal direction is executed. The subject position ratio Rh in the horizontal direction is the horizontal position near the shoulder of the subject closest to the screen center (hereinafter referred to as the shoulder position, referred to as Xsmax) with respect to a predetermined ratio to the screen size in the horizontal direction. It is the ratio which distance x2 occupies. The predetermined ratio is, for example, 80% or 90% of the screen size in the horizontal direction, and is changed according to the number of people arranged in the horizontal direction in S1703 to be described later. In FIG. 15A, among the subjects 1500a, 1501a, and 1502a, the subject that is closest to the screen center in the horizontal direction is the subject 1500a. With reference to the flowchart of FIG. 17, the calculation processing of the subject position ratio Rh in the horizontal direction will be described.

  In S1700, the shoulder position of the subject (denoted as Xs) is calculated based on the center position and size of the face tracking frame of the subject. The center of the screen is the origin (0, 0), the center position of the face tracking frame is (Xc, Yc), and the size of the face tracking frame is S. The shoulder position Xs on the peripheral side of the screen can be calculated by “Xs = | Xc | + S × Ns”, where Ns is the number of face tracking frames occupying the distance from the center position of the face tracking frame to the shoulder position. FIG. 19 shows a setting example of Ns. The value of Ns is changed according to the composition determined in S1211 of FIG. For example, in the case of the composition of the face 1402, Ns = 1 (in this case, the horizontal position near the ear, not the shoulder position) is set. In other cases (“upper body”, “whole body”, “multiple people”), Ns = 2 is set.

  After calculating the shoulder position Xs of the subject, the process proceeds to S1701. In S1701, when the maximum value Xsmax of the shoulder position is not stored in the memory 118, the shoulder position Xs calculated in S1700 is stored as the maximum shoulder position Xsmax. When the maximum shoulder position Xsmax is stored in the memory 118, the shoulder position Xs calculated in S1700 is compared with the maximum shoulder position Xsmax. When the shoulder position Xs is larger than the maximum shoulder position Xsmax, a process of updating the maximum shoulder position Xsmax in the memory 118 with the shoulder position Xs is executed. After updating the maximum shoulder position Xsmax, the process proceeds to S1702. In S1702, it is determined whether or not the process of calculating the shoulder position Xs and updating the maximum shoulder position Xsmax is completed for all the subjects. If the calculation of the shoulder position Xs and the update of the maximum shoulder position Xsmax are not completed, the process returns to S1700. If the determination of the maximum shoulder position Xsmax is completed for all the subjects and the update of the maximum shoulder position Xsmax is completed, the process proceeds to S1703.

  In S1703, determination processing is performed for the number of people whose subjects are arranged in the horizontal direction (the number of people arranged in the horizontal direction). The purpose of this is to create a composition that fills the screen when there are many subjects, such as a group photo, and to leave a margin around the screen when there are few subjects. As a method of counting the number of people arranged in the horizontal direction, if the face tracking frames overlap in the vertical direction (up and down direction of the screen), the overlapping subjects are counted as one person. For example, if four subjects are detected in the screen and the face tracking frames of two of them are overlapped on the top and bottom of the screen, the number is counted as three. In S1703, the number of people arranged in the horizontal direction is compared with a threshold value (for example, two people). If it is determined that the number of people arranged in the horizontal direction is two or less, the process proceeds to S1704. If it is determined that there are three or more persons, the process proceeds to S1705. In S1704 and S1705, the subject position ratio Rh in the horizontal direction is calculated. The horizontal subject position ratio Rh in S1704 is calculated as the ratio of the maximum shoulder position Xsmax × 2 to 80% of the horizontal screen size. The horizontal subject position ratio Rh in S1705 is calculated as the ratio of the maximum shoulder position Xsmax × 2 to 90% of the horizontal screen size. After S1704 or S1705, the subject position ratio Rh calculation process is terminated.

  Next, in S1601 of FIG. 16, processing for calculating the subject position ratio Rv in the vertical direction is executed. The subject position ratio Rv in the vertical direction is the vertical position of the head of the most peripheral subject from the center of the screen (hereinafter referred to as the head position) or the vertical position of the body part with respect to a predetermined ratio of the screen size in the vertical direction. It is the ratio occupied by the distance x2 (hereinafter referred to as body position). The predetermined ratio is 90% of the screen size in the vertical direction, for example. Hereinafter, the head position of the most peripheral subject from the center of the screen will be denoted as Yhmax, and the body position of the most peripheral subject from the center of the screen will be denoted as Ybmix. In FIG. 15B, of the subjects 1500b, 1501b, and 1502b, the subject with the head closest to the center from the center of the screen in the vertical direction is the subject 1500b, and the subject at the most peripheral body position from the center of the screen is the subject 1501b. It is. With reference to the flowchart of FIG. 18, the calculation processing of the subject position ratio Rv in the vertical direction will be described.

  In S1800 of FIG. 18, the head position of the subject (denoted as Yh) is calculated based on the center position and size of the face tracking frame of the subject. The head position Yh can be calculated by “Yh = Yc + S × Nh”. Nh is the number of face tracking frames in the distance from the center position of the face tracking frame to the head position. FIG. 19 shows a setting example of Nh. The value of Nh is set to Nh = 1.5 regardless of the composition determined in S1211 of FIG. After calculating the head position Yh of the subject, the process proceeds to S1801.

  In S1801, when the maximum value Yhmax of the head position is not stored in the memory 118, a process of storing the head position Yh calculated in S1800 as the maximum head position Yhmax is executed. When the maximum head position Yhmax is stored in the memory 118, the head position Yh calculated in S1800 is compared with the maximum head position Yhmax. When the head position Yh is larger than the maximum head position Yhmax, processing for updating the maximum head position Yhmax in the memory 118 with the head position Yh is performed. After the update process, the process proceeds to S1802.

  In next step S1802, the body position of the subject (denoted as Yb) is calculated based on the center position and size of the face tracking frame of the subject. The body position Yb can be calculated by “Yb = Yc−S × Nb”. Nb is the number of face tracking frames in the distance from the center position of the face tracking frame to the body position. FIG. 19 shows a setting example of Nb. The value of Nb is changed according to the composition determined in S1211 of FIG. For example, in the case of the composition of the face 1402, Nb = 1.5 is set. In the case of the composition of the upper body 1400, Nb = 5 is set, in the case of the composition of the whole body 1401, Nb = 10, and in the case of the composition of a plurality of people 1403, Nb = 3.5 is set. This setting is the center of the screen in each composition, the face 1402 is the chest, the upper body 1400 is below the waist, the whole body 1401 is the foot, and the plurality of people 1403 is near the waist. After calculating the body position Yb of the subject, the process proceeds to S1803.

  In S1803, when the minimum body position value Ybmin is not stored in the memory 118, a process of storing the body position Yb calculated in S1802 as the minimum body position Ybmin is executed. When the minimum body position Ybmin is stored in the memory 118, the body position Yb calculated in S1802 is compared with the minimum body position Ybmin. When the body position Yb is smaller than the minimum body position Ybmin, a process of updating the minimum body position Ybmin in the memory 118 with the body position Yb is performed. After the update process, the process proceeds to S1804.

  In step S1804, the head position Yh and the body position Yb are calculated for all the subjects, and it is determined whether or not the processing for updating the maximum head position Yhmax and the minimum body position Ybmin is completed. If the update of the maximum head position Yhmax and the minimum body position Ybmin has not been completed, the process returns to S1800. If the determination of the maximum head position Yhmax and the minimum body position Ybmin is completed for all the subjects and the update of the maximum head position Yhmax and the minimum body position Ybmin is completed, the process proceeds to S1805. In S1805, the absolute value of the maximum head position Yhmax and the absolute value of the minimum body position Ybmin are compared, and processing for determining which is located closer to the periphery of the screen is performed. If it is determined that the maximum head position Yhmax is located closer to the screen periphery, the process proceeds to S1806. If it is determined that the minimum body position Ybmin is located closer to the periphery of the screen, the process proceeds to S1807. In S1806 and S1807, the subject position ratio Rv in the vertical direction is calculated. The vertical subject position ratio Rv in S1806 is calculated as a ratio of the absolute value of the maximum head position Yhmax × 2 with respect to 90% of the vertical screen size. The vertical subject position ratio Rv in S1807 is calculated as a ratio of the absolute value of the minimum body position Ybmin × 2 to 90% of the vertical screen size. After S1806 or S1807, the calculation process of the vertical subject position ratio Rv ends.

  Subsequently, in S1602 of FIG. 16, the subject position ratio Rh in the horizontal direction is compared with the subject position ratio Rv in the vertical direction. Thus, it is possible to determine the position having the largest ratio among the ratios of the distance from the center of the screen to each position of the subject, that is, the position closest to the periphery of the screen with respect to the predetermined ratio of the screen size in each direction. it can. Each position is a maximum shoulder position Xsmax, a maximum head position Yhmax, and a minimum body position Ybmin. If it is determined that the horizontal subject position ratio Rh is greater than the vertical subject position ratio Rv, the process proceeds to S1603, and it is determined that the horizontal subject position ratio Rh is less than or equal to the vertical subject position ratio Rv. In step S1604, the process proceeds to step S1604. In S1603 and S1604, a process of calculating a reference size is executed based on the detected size of the main subject and the ratio of the subject position closest to the screen determined in S1602. The reference size is calculated by “detected size / horizontal subject position ratio Rh” in S1603, and calculated by “detected size / vertical direction subject position ratio Rv” in S1604. That is, in S1603, the zoom operation is executed so that the main subject image has a reference size obtained by multiplying the size of the main subject image by “1 / horizontal subject position ratio Rh”. In S1604, the zoom operation is executed so that the main subject image has a reference size obtained by multiplying the size of the main subject image by “1 / vertical subject position ratio Rv”. In this way, the angle of view based on the composition determined in S1211 can be set without causing the subject in the screen to be out of frame.

  In S1215 of FIG. 12, when the composition setting is “Auto”, the subject position that is the most peripheral in the horizontal and vertical directions is determined based on the composition determined in S1211, and the reference is set so that the position fits on the screen. The size is calculated. As a result, the subject can be held at an appropriate angle of view without being out of the frame no matter where the subject image is in the screen. In addition, whether the number of subjects is singular or plural, the reference size can be calculated by the same process. In the present embodiment, the direction of the screen size for calculating the reference size is determined on the assumption that the subject is in a standing posture. Not limited to this, when the subject detection unit 123 can determine the face direction, the screen size direction may be determined based on the vertical direction of the face on the screen. Furthermore, the composition determination and the reference size calculation example when the subject is a person have been described, but the present invention can also be applied when the subject is an object. However, in this case, the selected composition is “large”, “medium”, “small”, and “plurality” instead of “face”, “upper body”, “whole body”, and “multiple people”. Accordingly, the number of mono tracking frames when calculating the subject position (corresponding to Ns, Nh, Nb in the case of a person) may be set.

  When the reference size setting process illustrated in FIG. 12 is completed, the process proceeds to S704 in FIG. In step S <b> 704, the AZ control unit 122 performs auto zoom control based on the subject information detected by the subject detection unit 123 and the reference size stored in the memory 118. The auto zoom control will be described later with reference to FIG. After completion of the auto zoom control, the process proceeds to S705, where the AZ control unit 122 determines a subject search instruction from the photographer. That is, it is determined whether or not the photographer has pressed the subject search button of the operation unit 117. If the subject search button is pressed, the process returns to S701 and subject search processing is executed. If the subject search button has not been pressed, the process proceeds to S706. In step S <b> 706, it is determined whether or not the subject to be subjected to auto zooming has been changed by the operation of the touch panel of the operation unit 117 or the subject designation switch. If the subject has been changed, the process returns to S702 and subject designation processing is executed. If the subject has not been changed, the process proceeds to S707.

  In S707, it is determined whether or not the composition setting has been changed by operating the left and right buttons of the operation unit 117 from the shooting screen. When the composition setting is changed, the composition setting data stored in the memory 118 is updated, and then the process returns to S703 to execute the reference size setting process. If the composition setting has not been changed, the process proceeds to S708. In S708, it is determined whether or not the auto zoom operation switch of the operation unit 117 has been pressed. If it is determined that the auto zoom operation switch has been pressed, the auto zoom function is terminated. If it is determined that the auto zoom operation switch has not been pressed, the process proceeds to S709. S709 is a process for determining whether or not the composition setting stored in the memory 118 is “auto”. When the composition setting is “auto”, the process returns to S703 and the reference size setting process including the automatic composition determination is executed. If the composition setting is other than “auto”, the process returns to S704 and the auto zoom control is continued.

  Next, the auto zoom control process of S704 will be described with reference to the flowchart of FIG. First, in S2000, the AZ control unit 122 determines whether or not a subject is detected by the subject detection unit 123. If no subject is detected in S2000, the auto zoom control is terminated. If a subject is detected in S2000, the process proceeds to S2001. The determination shown in each step of S2001 to S2003 is an automatic zoom start determination process for frame-out prevention control. That is, this process is performed to prevent the subject from being out of the screen as described with reference to FIGS. In S2001, the AZ control unit 122 determines whether the subject tracking frame related to the subject to be tracked has entered the ZO region. This ZO region corresponds to the ZO region described with reference to FIGS. 3A and 4A to 4C. If the subject tracking frame enters the ZO area in S2001, that is, if the possibility that the subject is out of frame is high, the process proceeds to S2004, and the zoom-out operation is started. The zoom-out operation here corresponds to a zoom-out operation for frame-out prevention control. After the zoom-out operation, the auto zoom control is terminated.

  On the other hand, if the subject tracking frame has not entered the ZO area in S2001, that is, if the camera catches the subject near the center of the screen, the process proceeds to S2002. In S2002, it is determined whether or not the previous zoom operation is a zoom-out operation due to entering the ZO region in S2001, that is, whether or not it is a zoom-out operation for frame-out prevention control. If it is determined in S2002 that the zoom-out operation for frame-out prevention control has been performed, the process proceeds to S2003. If it is determined in S2002 that the zoom-out operation for the frame-out prevention control has not been performed, the process proceeds to S2006.

  In S2003, the AZ control unit 122 determines whether or not the subject tracking frame related to the subject to be tracked is within the ZI area (whether or not included). The ZI region here corresponds to the ZI region described with reference to FIGS. 3B and 4A to 4C. If it is determined in S2003 that the subject tracking frame does not fit in the ZI area, the auto zoom control process ends. On the other hand, if it is determined that the subject tracking frame is within the ZI region, the process proceeds to S2005. That is, if the camera captures the subject so that the subject size is near the center of the screen and within the angle of view at the zoom return position, a zoom-in operation is started in S2005. The zoom-in operation here corresponds to a zoom-in operation for frame-out prevention control. After zooming in, the auto zoom control is terminated.

  In this embodiment, in order to achieve both frame-out prevention control and size retention control in auto-zoom, first, the subject is captured near the center of the screen by frame-out prevention control, and then size retention control can be executed. For this reason, in the state after the zoom-out operation for the frame-out prevention control, the auto-zoom processing (the following S2006 to S2010) for keeping the subject size constant (the size holding control) described below is not performed. . In other words, when the frame-out prevention control is performed, the execution of the size holding control is limited until the zoom-out operation for the frame-out prevention control is completed.

  Next, each process of S2006 to S2010 will be described. If it is determined in S2002 that the zoom-out operation has not been performed for the frame-out prevention control, the process proceeds to S2006, where the AZ control unit 122 sets a predetermined magnification (N1 and N1> 1 for the detected size of the subject and the reference size). ) And the size multiplied. The subject size of the reference subject information set in S1202, S1210, and S1215 of FIG. 12 is compared with the subject size detected in S2000. If the size of the subject detected in S2000 is larger than N1 times the size of the reference subject, that is, if the ratio of the subject image to the screen exceeds a predetermined value, the process proceeds to S2009. In S2009, the AZ control unit 122 starts a zoom-out operation. The zoom-out operation here corresponds to a zoom-out operation for size retention control. After the zoom-out operation, the auto zoom control is terminated.

  On the other hand, in S2006, when the size of the subject detected in S2000 is N1 times or less than the size of the reference subject, the process proceeds to S2007. In S2007, the AZ control unit 122 compares the subject size of the reference subject information with the size of the subject detected in S2000. When the size of the detected subject is smaller than a predetermined magnification (denoted N2 and N2 <1) with respect to the size of the reference subject (less than N2 times), that is, the ratio of the subject image to the screen is a predetermined value If it is less, the process proceeds to S2008. On the other hand, when the size of the detected subject is N2 times or more than the size of the reference subject, the auto zoom control is terminated.

  In S2008, the AZ control unit 122 determines whether or not the subject tracking frame related to the subject to be tracked is within the ZI region (whether or not included). This is to prevent the subject from being out of frame by the zoom-in operation when the subject is in the periphery of the screen. The ZI region here corresponds to the ZI region described with reference to FIG. If it is determined in S2008 that the subject tracking frame is not within the ZI area, the auto zoom control is terminated.

  On the other hand, if it is determined in S2008 that the subject tracking frame is within the ZI area, the process proceeds to S2010. In S2010, the AZ control unit 122 starts a zoom-in operation. The zoom-in operation here corresponds to a zoom-in operation for size retention control. Thus, in this embodiment, in order to prevent the subject from being out of frame even in the zoom-in operation for size retention control, the zoom-in operation is started after the subject image is within the ZI region. After zooming in, the auto zoom control is terminated.

  Next, the zoom operation will be described with reference to FIG. FIG. 21 is a flowchart for explaining the zoom-out operation or zoom-in operation in S2004, S2005, S2009, and S2010 of FIG. First, in step S <b> 2100, the AZ control unit 122 acquires the zoom change amount (change amount of zoom magnification) from the memory 118. In the case of the zoom-out operation for the frame-out prevention control, the zoom-out change amount is set according to the detected subject information. Specifically, in the zoom-out operation for frame-out prevention control (S2004 in FIG. 20), the smaller the subject size, the smaller the zoom-out change amount is set. Thereby, it is possible to prevent the subject from becoming undetectable because the size of the subject becomes too small due to the zoom-out operation. In consideration of the minimum size that can be detected by the subject, the zoom-out operation may not be performed when the size of the subject is smaller than a predetermined size. Also, in the zoom-in operation for frame-out prevention control, the zoom magnification before the start of the zoom-out operation is stored in the memory 118, and the zoom-in change amount is set so as to be the same zoom magnification as that before the start of the zoom-out operation.

  In the zoom-out operation for size retention control (S2009 in FIG. 20), a zoom-out change amount (1 / N1 times) corresponding to the predetermined N1 times used for the determination in S2006 is set. Thereby, even when the subject cannot be detected, a minimum zoom-out operation until the size of the subject reaches the size of the reference subject can be performed. Similarly, in the case of the zoom-in operation for size retention control (S2010 in FIG. 20), the zoom-in change amount (1 / N2 times) corresponding to the predetermined N2 times used for the determination in S2007 is set.

  In step S2101, the AZ control unit 122 sets the zoom change amount acquired in step S2100 in the CZ control unit 119 or the electronic zoom control unit 120, and instructs the zoom processing to be performed. In the next step S2102, the AZ control unit 122 determines which zoom operation is being performed for frame-out prevention control or size retention control. If the current zoom operation is a zoom-out prevention zoom operation (S2004, S2005 in FIG. 20), the process proceeds to S2105. If the current zoom operation is a zoom operation for size maintenance control (S2009, S2010 in FIG. 20), the process proceeds to S2103.

  In step S <b> 2103, the AZ control unit 122 determines whether the subject is detected by the subject detection unit 123. If a subject has been detected, the process proceeds to S2104. If a subject has not been detected, the process proceeds to S2105. In S2104, the size of the subject indicated by the reference subject information is compared with the size of the subject detected in S2103. As a result of the comparison, if the size of the subject detected in S2103 and the size of the reference subject do not fall within a predetermined ratio range (within a predetermined change amount), the process proceeds to S2102 again to determine the zoom operation. continue. If the size of the subject detected in S2103 and the size of the reference subject are within a predetermined range due to the zoom operation, the process proceeds to S2106. In S2106, the AZ control unit 122 stops the zoom operation, and then ends the zoom operation process.

  If it is determined in S2102 that the zoom-out operation for frame-out prevention control is being performed, or if the subject is not detected in S2103 during the zoom operation for size retention control, the process proceeds to S2105. In S2105, the AZ control unit 122 determines based on the zoom change amount acquired in S2100 whether or not a scaling process corresponding to a predetermined amount of zoom change amount corresponding to each zoom operation has been performed. If the zooming process for the predetermined zoom change amount has not been performed, the process returns to S2102 again to continue the process. If zoom processing of a predetermined zoom change amount has been performed, the process proceeds to S2106, where the AZ control unit 122 stops zooming in operation and ends the zoom operation processing.

As described above, in the present embodiment, the following calculation process is executed.
First process: a process of calculating the position of the subject image with reference to the center of the image.
Second process: a process of calculating the ratio of the position of the subject image in the horizontal direction or the vertical direction with respect to a predetermined angle of view.
When a plurality of subject images are detected in the image, a ratio for each subject image is calculated in the second process for the plurality of subject images, and the horizontal or vertical direction with respect to the angle of view is determined by the determination process. The ratio occupied by the position of the subject image located nearest to the direction is determined. The reference size is set from this ratio, the selected composition, and the size of the main subject image selected from the plurality of subject images. That is, based on the composition selected by the composition selection process, the number of subjects, and the reference size set for the position of the subject in the screen, the zoom operation is executed so that the detected subject image becomes the reference size. . According to the present embodiment, it is possible to perform an automatic zoom control that achieves an appropriate composition regardless of the number of subjects and the position of the subject in the screen.

(Other embodiments)
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

DESCRIPTION OF SYMBOLS 100 Digital camera 102 Zoom lens 109 Display part 113 Zoom lens drive part 117 Operation part 118 Memory 119 CZ control part 120 Electronic zoom control part 122 Auto zoom control part 123 Subject detection part

Claims (14)

Subject detection means for detecting a subject image from the image;
Composition selecting means for automatically selecting a specific composition from a plurality of compositions;
A first process for calculating the position of the subject image with reference to the center of the image; and a second process for calculating a ratio of the position of the subject image in the horizontal direction or the vertical direction with respect to the angle of view of the image. Calculating means for performing processing;
When a plurality of subject images are detected by the subject detection unit, the calculation unit determines the ratio for each subject image calculated in the second process with respect to the plurality of subject images, and Determination means for determining a ratio occupied by the position of the subject image located closest to the angle of view in the horizontal or vertical direction;
The zoom magnification is controlled from the composition selected by the composition selection means, the ratio determined by the determination means, and the size of the main subject image selected from the plurality of subject images detected by the subject detection means. Setting means for setting a reference size of the subject image to be used;
Zoom control, comprising: control means for controlling the zoom magnification according to a change in the size of the subject image detected by the subject detection means based on the reference size set by the setting means. apparatus.   The calculation means calculates the position of the subject image from the center position and size of the subject image and the composition selected by the composition selection means in the first processing. The zoom control device described in 1. The calculating unit calculates a ratio of the positions of the plurality of subject images in the horizontal direction or the vertical direction with respect to the angle of view of the image in the second processing;
The determination means determines a maximum ratio from a plurality of ratios calculated by the calculation means as a ratio occupied by a position of a subject image located closest to the angle of view of the image. Item 3. The zoom control device according to Item 1 or 2. The calculation means calculates a ratio of the position of the subject image in the horizontal direction and the vertical direction with respect to the angle of view of the image in the second process,
The determination means compares the ratio of the position of the subject image in the horizontal direction with respect to the angle of view of the image to the ratio of the position of the subject image in the direction perpendicular to the angle of view of the image. The zoom control apparatus according to claim 1, wherein the larger ratio is determined.   5. The zoom control apparatus according to claim 1, wherein the setting unit changes the reference size when the composition selected by the composition selection unit is changed. 6.   6. The zoom control apparatus according to claim 1, wherein the setting unit changes the reference size when the number of subject images detected by the subject detection unit increases.   The said setting means changes the said reference | standard size, when the state in which the number of the subjects detected by the said subject detection means decreased continues for more than the set time. 2. A zoom control apparatus according to item 1.   The said setting means changes the said reference | standard size, when the center position of the to-be-photographed image detected by the to-be-photographed object detection means moves more than a threshold value, The said reference size is changed. Zoom control device.   The angle of view is determined by the control unit from the direction in which the ratio determined by the determination unit is calculated and the number of subject images detected by the subject detection unit. The zoom control device according to any one of the above. A zoom control device according to any one of claims 1 to 9,
Imaging means for photoelectrically converting a subject image to generate an image;
Comprising zoom means for changing the zoom magnification by driving the lens or image processing;
The control means controls the zoom means, and controls a zoom magnification according to a change in the size of the subject image detected by the subject detection means based on the reference size set by the setting means. An imaging device. It has a mode to record multiple images with different aspect ratios,
The calculation unit according to claim 10, wherein, when calculating the ratio, the ratio is calculated using a size having the smallest image size for each of a horizontal direction and a vertical direction among the plurality of images. The imaging device described. A control method executed by a zoom control device,
An acquisition step of acquiring detection information of the subject image detected from the image;
A composition selection step for automatically selecting a specific composition from a plurality of compositions;
A first process for calculating the position of the subject image with reference to the center of the image; and a second process for calculating a ratio of the position of the subject image in the horizontal direction or the vertical direction with respect to the angle of view of the image. A calculation step for performing processing;
When a plurality of subject images are detected in the detection information obtained by the obtaining step, the ratio for each subject image calculated in the second process by the calculating step is determined for the plurality of subject images. A determination step of determining a ratio occupied by the position of the subject image located closest to the periphery in the horizontal direction or the vertical direction with respect to the angle of view of the image;
Based on the composition selected in the composition selection step, the ratio determined in the determination step, and the size of the main subject image selected from the plurality of subject images, a reference size of the subject image used for zoom magnification control is determined. A setting step to set;
And a control step of controlling a zoom magnification according to a change in the size of the subject image indicated by the detection information acquired by the acquisition step based on the reference size set by the setting step. Control method of the control device.   A control program for a zoom control apparatus, which causes a computer to execute each step of the control method for a zoom control apparatus according to claim 12. 14. A storage medium storing a control program for the zoom control apparatus according to claim 13.

Images