JP2020198536A - Controller, imaging apparatus, control method, and program - Google Patents

Abstract

To provide a controller capable of executing appropriate automatic zooming according to the responsiveness of a zooming operation.SOLUTION: A controller (125), controlling zooming means (124, 113) changing an imaging magnification, includes: calculation means (201) of calculating an object size on the basis of a picked-up image; first determination means (202) of determining a target object size; and second determination means (203) of determining a zooming speed on the basis of the object size and the target object size. The calculation means changes a calculation method of the subject size on the basis of responsiveness of a zooming operation by the zooming means.SELECTED DRAWING: Figure 6

Description

The present invention relates to a control device and an imaging device that detect a subject and control a zoom operation.

Conventionally, an imaging device having a function of driving a zoom lens to perform optical scaling (optical zoom) and a function of enlarging a part of a captured image to perform electronic scaling (electronic zoom) has been known. Has been done. Further, there is known an imaging device having an auto-zoom function that automatically changes the zoom magnification according to the detection information of a subject. Patent Document 1 has an auto-zoom function that automatically zooms while keeping the size of a moving subject constant, and is a difference between a subject size that is a reference for zoom control selected by the user and a detected subject size. A configuration is disclosed in which the zoom speed is changed according to the above.

Japanese Unexamined Patent Publication No. 2016-25376

By the way, there is a shooting method (dolly zoom shooting) in which a user (camera) zooms so as to keep the subject size constant while approaching or moving away from the subject. In order to keep the size of the subject constant in Dolly Zoom shooting, it is necessary to quickly adjust the angle of view according to the change in the size of the subject. Therefore, it is necessary to improve the responsiveness of the zoom operation to a change in the subject size.

However, in the configuration disclosed in Patent Document 1, in order to prevent a malfunction of the zoom due to an erroneous detection of the subject size, the movement start determination of the zoom is performed. As a result, the responsiveness of the zoom operation is lowered, and it is not possible to shoot a desired image when performing dolly zoom shooting. In this way, the responsiveness of the zoom operation required differs depending on the shooting conditions, so the responsiveness of a single zoom operation alone can simultaneously prevent zoom malfunctions and quickly adjust the angle of view in response to changes in the size of the subject. Can not do it.

Therefore, an object of the present invention is to provide a control device, an image pickup device, a control method, and a program capable of performing an appropriate auto zoom according to the responsiveness of a zoom operation.

The control device as one aspect of the present invention is a control device that controls the zoom means for changing the imaging magnification, and is a calculation means for calculating the subject size based on the captured image and a first method for determining the target subject size. It has a determination means and a second determination means for determining the zoom speed based on the subject size and the target subject size, and the calculation means is based on the responsiveness of the zoom operation by the zoom means. Change the calculation method of the subject size.

Other objects and features of the present invention will be described in the following embodiments.

According to the present invention, it is possible to provide a control device, an image pickup device, a control method, and a program capable of performing an appropriate auto zoom according to the responsiveness of a zoom operation.

It is a block diagram of the image pickup apparatus in each Example. It is a block diagram of the AZ control unit in each embodiment. It is a figure which shows an example of the auto-zoom photography in Example 1. FIG. It is a figure which shows an example of the auto-zoom photography in Example 1. FIG. It is a flowchart which shows the auto zoom control in Example 1. FIG. It is a flowchart which shows the averaging process of the subject size in Example 1. FIG. It is a figure which shows the calculation method of the zoom speed in Example 1. FIG. It is a flowchart which shows the averaging process of the subject size in Example 2. FIG. It is a flowchart which shows the averaging process of the subject size in Example 2. FIG. It is a flowchart which shows the averaging process of the subject size in Example 2. FIG. It is a flowchart which shows the thinning-out process of the subject size in Example 3. It is a flowchart which shows the thinning-out process of the subject size in Example 4. It is a flowchart which shows the thinning-out process of the subject size in Example 4. It is a flowchart which shows the thinning-out process of the subject size in Example 4. It is a flowchart which shows the filtering process of the subject size in Example 5. It is a flowchart which shows the filtering process of the subject size in Example 6. It is a flowchart which shows the filtering process of the subject size in Example 6. It is a flowchart which shows the filtering process of the subject size in Example 6.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

First, the configuration of the imaging device according to the present embodiment will be described with reference to FIG. FIG. 1 is a block diagram of the image pickup apparatus 100, and shows the configuration of a digital camera having a subject tracking function as an example.

The lens barrel (lens device) 101 holds a lens group inside the lens barrel (lens device) 101. The zoom lens 102 adjusts the focal length by moving in the optical axis direction of the lens barrel 101, and optically changes the angle of view (moves the zoom position). The focus lens 103 adjusts the focus by moving in the optical axis direction of the lens barrel 101. The anti-vibration lens (lens for correcting image blur) 104 corrects image blur caused by camera shake or the like. The aperture and shutter 105 for adjusting the amount of light are used for exposure control. In the present embodiment, the image pickup device 100 is an image pickup device in which the lens barrel 101 and the camera body are integrally configured, but the present invention is not limited to this. This embodiment can also be applied to an imaging system including a camera body and an interchangeable lens (lens device) that can be attached to and detached from the camera body.

The image sensor 106 receives the light that has passed through the lens barrel 101 and converts the subject image into an electric signal by photoelectric conversion to generate an image pickup signal. The image pickup device 106 is a CCD (charge-coupled device) type or CMOS (complementary metal oxide semiconductor) type image sensor or the like. The image pickup signal from the image pickup element 106 is input to the image processing unit 107, and various processes such as pixel interpolation processing and color conversion processing are performed. The image data after various processing is stored in the image memory 108. The image memory 108 is a storage device such as a DRAM (Dynamic Random Access Memory) or a SRAM (Static Random Access Memory).

The display unit 109 is configured to include a TFT type LCD (thin film transistor drive type liquid crystal display) or the like, and displays a photographed image (image data) or specific information (for example, photographed information). By displaying information such as a live view related to the captured image, it is possible to provide an electronic viewfinder (EVF) function for the user to adjust the angle of view.

The image data generated by the photographing process is sent to the recording unit 116 via the interface (I / F) unit 115 and recorded. The image data is recorded in an external recording medium, a non-volatile memory 117, or both. The external recording medium is a memory card or the like that is used by being attached to the image pickup apparatus 100. The non-volatile memory 117 is a storage medium built in the image pickup apparatus 100. The memory 117 stores information such as settings of the image pickup apparatus 100 in addition to program data and image data. The operation unit 118 includes a zoom operation member, a release switch for instructing the start of shooting, a touch panel for designating a subject and setting the image pickup device 100, an operation switch, and the like. The operation signal of the operation unit 118 is sent to the system control unit 114.

The system control unit 114 includes an arithmetic unit such as a CPU (central processing unit). The system control unit 114 controls the entire image pickup apparatus 100 by sending control commands to each unit in response to a user operation. The system control unit 114 performs various control programs stored in the memory 117, for example, control of the image sensor 106, AE / AF control, optical / electronic vibration isolation control, zoom control (including auto zoom processing), and the like. Run the program.

The camera control unit 120 gives instructions for controlling the entire camera function. For example, when the release switch is pressed halfway, a shooting preparation operation such as AE / AF is performed, and when the release switch is fully pressed, an instruction is given to perform a shooting operation of a still image or a moving image. Further, in the subject tracking function described later, the camera control unit 120 also instructs the zoom operation of the optical zoom and the start / stop of the electronic vibration isolation.

The AE control unit (automatic exposure control unit) 121 calculates an exposure control value (aperture value and shutter speed) based on a photometric value obtained by image processing by the image processing unit 107. The AE control unit 121 calculates the brightness value (photometric value) in an arbitrary area (photometric frame) in the screen according to the set photometric method. As the metering method, spot metering that measures at a specific set position, evaluation metering that the camera automatically determines the position of the metering frame according to the scene, and average metering over the entire screen with an emphasis on the center of the screen There are metering methods such as center-weighted average metering. The AE control unit 121 controls the aperture shutter drive unit 110 based on the photometric result. The aperture shutter drive unit 110 drives the aperture and the shutter 105 to realize automatic exposure adjustment. The aperture shutter drive unit 110 is composed of actuators such as a stepping motor and an electromagnetic plunger and a drive circuit for driving them, and drives the aperture and the shutter 105.

The optical vibration isolation control unit 122 calculates the amount of vibration applied to the image pickup device 100 based on the vibration detection information obtained by the angular velocity sensor such as the gyro sensor of the vibration detection unit 119. Optical vibration isolation is realized by driving the vibration isolation lens 104 so as to cancel (or reduce) the amount of vibration applied to the image pickup apparatus 100 according to the calculation result. The optical anti-vibration control unit 122 calculates a target position for controlling the anti-vibration lens 104 in a predetermined calculation cycle based on the amount of shaking, and issues a drive instruction to the anti-vibration lens drive unit 111. Based on the instruction, the anti-vibration lens driving unit 111 drives the anti-vibration lens 104. The anti-vibration lens drive unit 111 is composed of an actuator such as a voice coil motor (VCM), a drive circuit, and a position detection sensor such as a Hall element. The optical anti-vibration control unit 122 feeds back the position of the anti-vibration lens 104 detected by the position detection sensor of the anti-vibration lens drive unit 111, and feedback control is executed so as to hold the anti-vibration lens 104 at the target position. ..

Similar to the optical vibration isolation control unit 122, the electronic vibration isolation control unit 127 calculates the amount of vibration applied to the image pickup device 100 based on the vibration detection information obtained by the angular velocity sensor such as the gyro sensor of the vibration detection unit 119. The range of the image stored in the image memory 108 to be displayed on the display unit 109 or recorded on the recording unit 116 is framed so as to cancel (or reduce) the amount of runout applied to the image pickup apparatus 100 according to the calculation result. Electronic vibration isolation is realized by changing each image. The amount of vibration that can be vibration-isolated by optical vibration isolation is determined by the movable range of the vibration-proof lens 104. That is, in order to prevent large camera shake caused by the subject designation operation, it is necessary to increase the movable range of the vibration isolation lens 104. Therefore, it is necessary to increase the diameter of the lens barrel 101, and the image pickup device 100 itself may also be increased in size. On the other hand, in the electronic vibration isolation, the vibration isolation effect can be enhanced by increasing the control range used for the electronic vibration isolation control with respect to the image memory 108. In recent years, the number of pixels of the image sensor 106 has been increased, and the number of pixels is sufficient for the resolution of the LCD. Therefore, even when the range of the image displayed on the display unit 109 is reduced and the range of the electronic vibration isolation control is increased, a sufficient resolution can be obtained as a viewfinder function. However, since the recorded image may be displayed on a large screen monitor or the like, a higher resolution is required.

The AF control unit (autofocus control unit) 123 performs automatic focus adjustment (AF) control by a contrast method. That is, the AF control unit 123 controls the focus lens drive unit 112 so as to focus on the subject based on the focus adjustment information (contrast evaluation value) of the photographing optical system obtained by the image processing by the image processing unit 107. .. The focus lens drive unit 112 is composed of an actuator such as a voice coil motor (VCM), a position detection sensor for feeding back the position, and a drive circuit for driving the actuator, and drives the focus lens 103. Further, as another configuration, there is also a configuration that does not require a position detection sensor like a stepping motor. In the present embodiment, the focus lens driving unit 112 may have any configuration. Further, as the AF control other than the contrast method, the phase difference AF method may be used, or the AF control may be performed by a plurality of methods such as a combination of the contrast method and another method. The AF control unit 123 controls the focus lens driving unit 112 when performing AF control, and moves the focus lens 103 within a predetermined range to perform a scanning operation. The focus position, which is the in-focus point, is detected by using the contrast evaluation value obtained during the scanning operation.

The zoom lens driving unit 113 drives the zoom lens 102 according to the zoom operation instruction. The zoom lens drive unit 113 includes an actuator such as a direct current (DC) motor that drives the zoom lens 102, a position detection sensor such as a rotary encoder that detects the motor position, and a drive circuit that drives them. Further, as another configuration, there is also a configuration that does not require a position detection sensor like a stepping motor. In a retractable lens barrel in which the lens barrel 101 is extended from the camera when the power is turned on, it is necessary to detect an error when the lens barrel is pressed during the extension operation of the lens barrel. Often adopted. Further, in the inner type lens barrel in which the zoom lens 102 is driven inside the lens barrel 101, the latter, which is advantageous for noise reduction, is often adopted. The configuration of the zoom lens driving unit 113 in the present embodiment may be any configuration. The operation unit 118 includes a zoom lever, a zoom button, and the like as zoom operation members for instructing the camera to zoom and instructing the start and end of auto zoom. The camera control unit 120 detects the operation amount and operation direction of the zoom operation member used for the zoom instruction operation, calculates the zoom drive speed and the zoom drive direction, and moves the zoom lens 102 along the optical axis according to the calculation result. Take control.

In order to maintain the in-focus state when the angle of view is changed by the optical zoom, when the lens barrel 101 is a rear focus type, it is necessary to move the focus lens 103 to an appropriate focus position according to the position of the zoom lens 102. .. Such control is called computer zoom (CZ) control, and is performed by the CZ control unit (computer zoom control unit) 124 in the system control unit 114. The CZ control unit 124 drives the focus lens based on the correspondence between the focal length of the zoom lens and the focus position for each subject distance (focus cam table). The CZ control unit 124 detects the zoom position of the zoom lens 102 at predetermined control cycles during the zoom operation. Then, the focus lens 103 is controlled to be driven so as to follow the focus cam table at the subject distance measured by the AF control unit 123 with respect to the detected zoom position. This makes it possible to perform the optical zoom operation while maintaining the focused state.

The electronic zoom control unit 130 realizes the electronic zoom function by cutting out the data in the target area from the image data transferred to the image memory 108 in accordance with the instruction of the camera control unit 120. The AZ control unit (auto zoom control unit) 125 controls the CZ control unit 124 so that the size of the subject on the screen falls within a predetermined range based on the subject detection information obtained by the subject detection unit 126 described later. To do. As a result, automatic angle adjustment (auto zoom) is realized. The details of the auto zoom process in the AZ control unit 125 will be described later.

The zoom responsiveness setting unit 131 includes a touch panel, an operation switch, an operation bar, and the like for switching the responsiveness of the zoom operation. The operation signal of the zoom responsiveness setting unit 131 is sent to the AZ control unit 125 via the system control unit 114.

Next, the subject detection process in the present embodiment will be described. The subject detection unit 126 detects a desired subject area from the image data stored in the image memory 108. In the present embodiment, a subject detection method (face detection process, color detection process) for detecting a subject (face or object such as a person) based on face information or color information included in image data will be described.

First, the face detection process and the color detection process in the subject detection unit 126 will be described. The face detection process is a process of detecting a face region existing in image data by a known algorithm. For example, the subject detection unit 126 extracts a feature amount from a square-shaped partial region on the image data and compares the feature amount with a face feature amount prepared in advance. Then, when the correlation value between the two exceeds a predetermined threshold value, the subject detection unit 126 determines that the partial region is the face region. By repeating this determination process while changing the combination of the size, the arrangement position, and the arrangement angle of the partial area, various face areas existing in the image data can be detected. Further, when the face recognition function is provided, pattern matching is performed by a known algorithm between the feature amount of the face image registered in advance and the feature amount of the detected face area, and the registered face image having the highest correlation value is executed. Authenticate as the face that is detecting. If the correlation values for all the registered face images are less than a predetermined value, it is determined that the detected face area is an unregistered face.

In the color detection process, a process of storing the color information of the subject area designated according to the subject designation method described later as a feature color is executed. The color detection process is performed when the subject to be detected is an object. As the color information, RGB signals, luminance signals (Y signals), color difference (RY, BY) signals, etc., which are output signals from the image processing unit 107, are used. At the time of subject detection, the subject detection unit 126 divides the image data into a plurality of partial regions and calculates the average value of the brightness and the color difference for each partial region. Further, the subject detection unit 126 compares the feature color information stored in advance with the color information of each region at the time of subject detection, and sets a partial region having a difference between the luminance and the color difference by a predetermined amount or less as a candidate for the subject region. Processing is performed in which a group of adjacent partial regions in the candidate of the subject region is set as the same color region, and the region in which the same color region is in a predetermined size range is set as the final subject region. When the subject is an object, the shape of the subject area may differ depending on the object. Therefore, the smallest quadrangular region that includes all the same color regions is defined as the object region. Therefore, the object area may be a rectangular area having different sizes in the horizontal direction and the vertical direction. The position of the object area is the center position of the square area.

The subject detection unit 126 detects the subject (subject information) by face detection processing or color detection processing, and passes the detected subject information to the subject size extraction unit 132. The subject size extraction unit 132 extracts the subject size information from the subject information detected by the subject detection unit 126. The subject detection unit 126 and the subject size extraction unit 132 constitute acquisition means for sequentially acquiring a plurality of subject size information from the captured image. The frame display unit 128 displays the face area and the object area detected by the subject detection process on the display unit 109 as a frame (subject detection frame).

Next, the tracking process of the subject detection frame detected by the subject detection unit 126 will be described. The subject detection frame tracking process is a process of continuously tracking a subject area that moves between continuous frame images. When the subject tracking function is started, the area of a person or an object to be the main subject is designated according to the subject designation method described later. When the specified subject is a person, that is, a face, all faces in the frame image are detected. When a plurality of faces are detected in the same image, the subject detection unit 126 determines that the person to be tracked is the face closest to the position of the face in the immediately preceding frame image. If the detected face is a face registered by the face recognition function, it is determined that the authenticated face is preferentially tracked regardless of the positional relationship between the frame images.

When the designated subject is an object, the start position of the feature color information comparison process is started from the position of the object area of the immediately preceding frame image. If the frame rate is sufficiently higher than the speed of the moving object, it is likely that the same object will be detected in the neighboring area between the previous and next frame images. Therefore, it is possible to track the moving object region by performing the matching process of the same color region while shifting the partial region to be compared left, right, up and down from the position of the immediately preceding object region by the subject detection unit 126. The frame display unit 128 displays the face area and the object area tracked by the tracking process of the subject detection frame on the display unit 109 as the subject tracking frame.

Next, the control related to the subject tracking function in the camera control unit 120 will be described. In the present embodiment, the functions of measuring / ranging the metering / AF in conjunction with the position of the subject tracking frame with the AE / AF controlled metering frame and the ranging frame are referred to as a tracking AE function and a tracking AF function. Further, the functions of driving the zoom lens 102 and the vibration isolation lens 104 according to the size and position of the subject tracking frame are called the tracking AZ function and the tracking vibration isolation function. These functions are collectively called the subject tracking function. The user specifies the main subject by operating the touch panel of the operation unit 118 or the operation switch for specifying the subject. Each tracking function is executed so that the exposure, focus, and angle of view of the specified main subject are appropriate. These functions may be executed at the same time, or only one of the functions may be executed.

Next, with reference to FIG. 2, a configuration for realizing the AZ function in the AZ control unit 125 will be described. FIG. 2 is a block diagram of the AZ control unit 125. Auto-zoom can perform zoom operation with both optical zoom and electronic zoom. The auto zoom is started by a user operation on the operation unit 118. When the auto zoom starts, the image information is passed from the image memory 108 to the subject detection unit 126, and the subject is detected. The subject size extraction unit 132 extracts the detected size information of the subject, and the extracted subject size information is stored in the memory 117. The subject size information stored in the memory 117 is passed to the subject size calculation unit (calculation means) 201 and the target subject size determination unit (first determination means) 202.

The zoom responsiveness setting unit 131 can switch the responsiveness of the zoom operation according to the shooting intention of the user. The subject size calculation unit 201 acquires subject size information from the memory 117 during a predetermined period, and performs arithmetic processing for smoothing the subject size information according to the responsiveness of the zoom operation set by the zoom responsiveness setting unit 131. (Smoothing process) is performed to calculate the current subject size. The smoothing method in the subject size calculation unit 201 will be described later. The target subject size determination unit 202 determines the subject size information at the start of the auto zoom acquired from the memory 117 as the target subject size during the auto zoom operation. The zoom speed determination unit (second determination means) 203 determines the zoom speed in order to match the subject size obtained by the subject size calculation unit 201 with the target subject size determined by the target subject size determination unit 202. To do. When performing optical zooming, the speed determined by the zoom speed determining unit 203 is passed to the zoom lens driving unit 113 via the CZ control unit 124, and the zoom lens 102 is driven at the determined speed. When the electronic zoom is performed, the data in the target area is cut out from the image data acquired from the image memory 108 so that the image can be magnified at a determined speed. The method of determining the zoom speed by the zoom speed determining unit 203 will be described later.

Hereinafter, each embodiment will be specifically described.

First, the imaging apparatus according to the first embodiment will be described. The auto-zoom function of this embodiment has two functions, and is a "low" mode (first responsiveness) that lowers the responsiveness of the zoom operation based on the setting information by the user operation in the zoom responsiveness setting unit 131. It is possible to switch to a "high" mode (second responsiveness) that increases responsiveness.

FIG. 3 is a diagram showing an example of auto-zoom shooting, and shows an example when auto-zoom shooting is performed with the zoom responsiveness set to the “low” mode (first responsiveness). In the auto-zoom shooting, the zoom may malfunction due to the influence of the detection error of the face size information of the main subject detected by the subject detection unit 126. The angle of view may fluctuate due to the malfunction of the zoom, which may impair the quality of the image. When tracking a main subject in this way, it is more important to suppress sudden fluctuations in the angle of view (prevent malfunction of the zoom) than to follow the subject. Therefore, as a measure to prevent the malfunction of the zoom, the influence of the false detection of the face size is suppressed by setting the zoom response to the change in the face size to be low. Further, FIG. 3 illustrates a zoom operation automatically performed by the camera when the subject approaches the camera by setting the zoom responsiveness to be low. This is a zoom-out operation for keeping the ratio of the subject image to the screen within a predetermined ratio.

FIG. 3A shows the angle of view when the user shoots a distant subject. At this time, the user is in a state where the zoom is moved in the telephoto direction in order to fit the subject in a desired angle of view. The subject is detected by the subject detection unit 126 at the start of shooting, and the face size information of the main subject extracted by the subject size extraction unit 132 is stored in the memory 117. At this time, the frame display unit 128 displays a frame representing the face area of the subject on the display unit 109.

FIG. 3B shows the angle of view before the zoom operation when the subject approaches the optical axis direction from the state of FIG. 3A. Since the subject is closer to the camera, the face size is larger than that in FIG. 3 (A). FIG. 3C shows the zoom speed determined by the zoom speed determination method described later so as to keep the face size in the state of FIG. 3A from the state of FIG. 3B in which the face size has increased. It shows the angle of view when zooming out. In this way, when the subject moves in the optical axis direction, the zoom operates in order to maintain the face size of the subject. Although FIG. 3 describes the zoom operation when the subject approaches the camera, the zoom is performed by the speed determination method described later in order to maintain the face size of the subject even when the subject moves away from the camera. Zoom in at the determined speed. When performing auto-zoom shooting on a moving subject as shown in FIG. 3, it is important to prevent a malfunction of the zoom due to a change in face size and to follow the subject smoothly.

FIG. 4 is a diagram showing an example of auto-zoom shooting, and shows an example when auto-zoom shooting is performed with the zoom responsiveness set to the “high” mode. When the zoom responsiveness is set high, the user can adjust the angle of view while approaching the still subject to keep the face size of the main subject constant, and obtain an effect like Dolly Zoom. In order to keep the face size constant, it is necessary to immediately adjust the angle of view by zooming when the face size fluctuates. Therefore, during Dolly Zoom, it is desirable that the zoom response to changes in the face size of the subject is higher. Further, FIG. 4 illustrates a zoom operation automatically performed by the camera when the camera is moved in the optical axis direction with respect to a stationary subject with a high zoom responsiveness.

FIG. 4A shows the angle of view when the user shoots a distant subject. The user has moved the zoom in the telephoto direction in order to fit the subject in a desired angle of view. The subject is detected by the subject detection unit 126 at the start of shooting, and the face size information of the main subject extracted by the subject size extraction unit 132 is stored in the memory 117. At this time, the frame display unit 128 displays a frame representing the face area of the subject on the display unit 109.

FIG. 4B shows the angle of view when the camera approaches the subject in the optical axis direction from the state of FIG. 4A. At this time, the zoom is zoomed out at a speed determined by the speed determination method described later so as to maintain the size of the face in the state of FIG. 4 (A). FIG. 4C represents the time when the camera approaches the subject from the state of FIG. 4B, and the zoom will be described later so as to maintain the size of the face in the state of FIG. 4A. Zoom out at the speed determined by the speed determination method. When auto-zooming is performed on a stationary subject as shown in FIG. 4, it is important to keep the face size, which changes according to the movement of the camera, always the same size. Therefore, the zoom responsiveness to changes in face size needs to be high. FIG. 4 describes the zooming operation when the camera approaches the subject, but the same applies when the camera moves away from the subject, except that when the camera moves away, the camera zooms in instead of zooming out. ..

The zoom control responsiveness setting in the auto zoom of this embodiment can be switched by a user operation on the zoom responsiveness setting unit 131. The responsiveness of zoom control can be set in multiple steps. The zoom responsiveness setting unit 131 notifies the subject size calculation unit 201 of the set zoom control responsiveness information each time the setting is changed.

Next, the auto zoom control process will be described with reference to FIG. FIG. 5 is a flowchart showing auto zoom control. Each step of FIG. 5 is mainly executed by the system control unit 114 (AZ control unit 125).

When the auto zoom is started, first, in step S501, the system control unit 114 determines whether or not there is an auto zoom start instruction. If there is no auto zoom start instruction, the next process will not proceed until the start instruction is given. Subsequently, in step S502, the target subject size determination unit 202 determines the target face size. The face size determined here can be determined from the composition setting selected by the user via the operation unit 118, or automatically determined from the subject detection information when the auto zoom is activated. These two determination methods can be selected by setting the operation unit 118 of the user. The selected method is stored in memory 117.

Here, a method of determining the target face size from the composition setting set by the user will be described. A target face size suitable for composition is stored in memory 117 in advance. There are three compositions that the user can select, "face", "upper body", and "whole body", and the proportion of the subject's face on the screen is different. The target face size is stored in the memory 117 in a form corresponding to the three compositions of "face", "upper body", and "whole body". The target subject size determination unit 202 acquires face size information according to the selected composition from the memory 117, and determines the target face size to be used when determining the zoom speed. Here, an example of setting the target face size based on the information set by the user by the operation unit 118 is shown. By user operation, the face size of the subject occupying the screen when the auto zoom is activated may be set automatically as the target face size.

Subsequently, in step S503, the subject size calculation unit 201 calculates the face size information to be used for determining the zoom speed among the face size information acquired from the memory 117 (smoothing processing such as averaging processing) described later. Ask for. Subsequently, in step S504, the zoom speed determination unit 203 determines the zoom speed based on the target face size information determined in step S502 and the face size information selected in step S503. The specific method for determining the zoom speed will be described later. Subsequently, in step S505, the zoom lens driving unit 113 zooms at the zoom speed determined in step S504. Subsequently, in step S506, the system control unit 114 determines whether or not there is an instruction to end the auto zoom. If there is no instruction to end the auto zoom, the process of step S503 is continued in order. On the other hand, when there is an instruction to end the auto zoom, a series of processes are terminated.

Next, with reference to FIG. 6, step S503 (averaging process, that is, smoothing process) will be described in detail. FIG. 6 is a flowchart showing the subject size averaging process in this embodiment. In the example of FIG. 6, the subject size calculation unit 201 performs the averaging process of the face size of the subject according to the setting of the responsiveness of the zoom operation.

First, in step S601, the subject size calculation unit 201 determines the responsiveness of the current zoom operation set by the zoom responsiveness setting unit 131. In this embodiment, the case where the zoom responsiveness is switched in two steps will be described, but the present invention is not limited to this. If it is determined in step S601 that the zoom responsiveness setting is "low", the process proceeds to step S602. The responsiveness of the zoom is lowered when shooting is performed by preventing the malfunction of the zoom in order to track the subject. Therefore, in step S602, the subject size calculation unit 201 acquires the number N of face size information used for averaging to suppress the fluctuation of the zoom speed due to the change in face size from the memory 117. The number N used for averaging is stored in the memory 117 in advance. The method for determining the number N will be described later. Subsequently, in step S603, the subject size calculation unit 201 averages N past face size information from the current face size information and sets the face size to be used for determining the zoom speed.

On the other hand, if the zoom responsiveness is determined to be "high" in step S601, the process proceeds to step S604. The responsiveness of the zoom is enhanced when shooting with the face size kept constant at all times, such as dolly zoom shooting. In this case, it is assumed that there is almost no fluctuation in the face size, and it is ideal that the zoom operates while keeping the face size constant, so that high responsiveness is required. Therefore, in step S604, the subject size calculation unit 201 sets the latest face size information as the face size used for determining the zoom speed without performing the averaging process. When the face size information for determining the zoom speed is selected in steps S603 and S604, the processing in the subject size calculation unit 201 ends.

Next, the reason for smoothing the face size information as a means for changing the zoom responsiveness will be described. Since the zoom speed is determined based on the information on the face size of the subject, the amount of change in the face size affects the zoom speed.

Since a moving subject is targeted when tracking the subject, there is a high possibility that the face size will vary as compared with the case of shooting a stationary subject due to the influence of the movement of the subject or the change in the face orientation. If the zoom speed is determined based on the detected face size information, malfunctions such as repeated zooming in and out may occur due to the influence of the variation in face size. Therefore, by using the smoothed face size information, it is possible to suppress the malfunction of the zoom. On the other hand, smoothing has the demerit that the responsiveness of the zoom becomes low and a delay occurs.

On the other hand, when performing Dolly Zoom shooting, since a stationary subject is targeted for shooting, it is unlikely that the face size will vary. Therefore, by not performing smoothing or suppressing smoothing, the responsiveness of the zoom is improved, and the current face size can always be maintained at the target face size.

Here, a method of determining the number N of face size information used for smoothing in step S603 will be described. When the subject detection unit 126 detects the face size, the amount of variation in the face size information changes depending on conditions such as "brightness", "focus state (focus)", "camera shake", and "object orientation". To do. The variation of face size is measured by combining these multiple conditions, and the distribution of variation is calculated. Next, the standard deviation σ of the variation and the mean value A are obtained. It is determined whether or not the fluctuation of the face size generated in the distribution of the average value A ± 2σ can be visually recognized by the human eye. If it is a visually recognizable amount, the average face size information is increased by one and the average value A and the standard deviation σ are recalculated. Finally, the number N of face information to be averaged is determined so that the variation of the variation generated in the distribution of the average value A ± 2σ cannot be seen by the human eye.

Next, a method of determining the zoom speed by the zoom speed determination unit 203 will be described with reference to FIG. 7. The zoom speed determination unit 203 calculates the zoom speed from the amount of change in face size. FIG. 7 is a diagram showing a method of calculating the zoom speed, showing the positional relationship between the imaging device 100 and the subject, and the subject to be captured on the imaging screen when the positional relationship is such.

FIG. 7A is a diagram showing the relationship between the subject distance d and the focal length f when the subject is located at a position separated by the shooting distance L in the optical axis direction from the image sensor 106 provided in the image pickup device 100. The focal length f indicates the focal length of the lens barrel 101, and the subject distance d is the distance obtained by subtracting the focal length f from the shooting distance L. Further, the size of the imaging surface of the image sensor 106 is α, the face size on the imaging surface is β, and the face size with respect to the imaging angle of view range A is B.

FIG. 7B is an explanatory diagram of a state in which the subject approaches the image pickup apparatus 100 in the optical axis direction from the state of FIG. 7A. Only the subject approaches the image pickup apparatus 100 without changing the focal length f, and the relationship of subject distance dx <subject distance d is established. Therefore, the ratio of the face size B to the imaging angle of view range A becomes relatively large, and as a result, the face size on the imaging surface becomes large (face size β ′> face size β).

FIG. 7 (C) is focused so that the face size on the imaging surface becomes the same size (face size β) as in FIG. 7 (A) without changing the shooting distance L'from the state of FIG. 7 (B). It is explanatory drawing of the state (focal length f'<focal length f) when the distance f is changed to the focal length f'. As a result, the imaged angle of view range A'in FIG. 7C is wider than the imaged angle of view range A by the amount of change in the focal length (imaging angle of view range A'> imaged angle of view range A). It means that the size of the face on the imaging surface (face size B) is the same as that of the captured image of FIG. 7A, but changes so that the background range of the subject reflected in the captured image becomes wider. To do. On the other hand, since the auto-zoom function is a function that zooms while keeping the face size of the subject at the same size, it is necessary to automatically operate the zoom from the state of FIG. 7 (A) to the state of FIG. 7 (C). There is. Therefore, a method of calculating the movement amount of the zoom lens will be described below.

When performing auto-zooming from the state shown in FIG. 7A, it is necessary to zoom while keeping the face size B and the face size β on the imaging surface constant. Therefore, even in FIG. 7C, the face size and the face size on the imaging surface are controlled to be the same as those in FIG. 7A. From this, the ratio of the subject distance d and the focal length f in FIG. 7 (A) is the same as the ratio of the subject distance d'and the focal length f'in FIG. 7 (C), so that d: f = d': The relationship f'establishes. As a result, the focal length f'of the lens barrel 101 of the image pickup apparatus 100 can be calculated by the formula f'= (d'/ d) f.

Here, since the focal length f'for keeping the face size of the subject constant is calculated, it corresponds to the focal lengths f and f'to convert the amount of change in the focal length into the amount of movement of the zoom lens position. It can be calculated from the difference in the zoom lens position. As an example of the process, although not shown, the focal length may be converted into the zoom lens position by using the table information of the zoom lens position according to the focal length stored in the memory 117.

The amount of movement of the zoom lens position indicates the amount of movement to the zoom lens position after the next predetermined time when the moving speed of the subject is constant. Therefore, it is possible to calculate the zoom speed as the amount of movement per predetermined time from the amount of movement of the zoom lens position. In this embodiment, the zoom speed is determined from the amount of change in the face size, but the table information for determining the zoom speed is stored in advance from the difference from the current face size with respect to the target face size. The zoom speed may be determined based on this table information. The auto zoom process that enables the responsiveness of the zoom operation to be changed according to the shooting conditions has been described above.

In this embodiment, the auto-zoom processing is performed based on the amount of change in the face size, but the auto-zoom is performed based not only on the face size but also on the size change other than the face obtained by the subject detection unit 126. You may. Further, if a subject other than a person can be detected and size information can be acquired, auto-zooming may be performed based on the information.

Further, in this embodiment, the method for driving the optical zoom during the auto zoom process has been described, but the angle of view may be adjusted by the electronic zoom. In that case, the zoom speed determination unit 203 determines the magnification change speed for changing the magnification in the electronic zoom. After that, the electronic zoom control unit 130 cuts out the data in the target area from the image data acquired from the image memory 108 so that the image can be magnified at a determined speed.

Next, Example 2 will be described. In this embodiment, an example of the face size averaging process of the subject by the subject size calculation unit 201 will be described.

First, with reference to FIG. 8, step S503 (averaging process, that is, smoothing process) of FIG. 5 will be described in detail. FIG. 8 is a flowchart showing the subject size averaging process. FIG. 8 describes a case where the zoom responsiveness is switched in two steps.

First, in step S801, the subject size calculation unit 201 determines the responsiveness (whether the responsiveness is "high" or "low") of the current zoom operation set by the zoom responsiveness setting unit 131. To do. If it is determined in step S801 that the zoom responsiveness setting is "low", the process proceeds to step S802. In step S802, the subject size calculation unit 201 acquires the number N of face size information used for averaging the face size from the memory 117. The number N used for averaging is stored in the memory 117 in advance. The method for determining the number N is the same as in the first embodiment. Subsequently, in step S803, the subject size calculation unit 201 averages N past face size information from the current face size information and sets the face size to be used for determining the zoom speed.

On the other hand, if the zoom responsiveness is determined to be "high" in step S801, the process proceeds to step S804. In step S804, when it is necessary to reduce the influence of the face size variation, the subject size calculation unit 201 calculates the average face size with M face size information that does not impair the responsiveness of the zoom. The number M is a value stored in advance in the memory 117, and the relationship of M <N is established in this embodiment. Subsequently, in step S805, the subject size calculation unit 201 averages M past face size information from the current face size information and sets the face size to be used for determining the zoom speed. Face size information for determining the zoom speed is selected in steps S803 and S805, and the process in the subject size calculation unit 201 ends.

Next, with reference to FIG. 9, a case where the zoom responsiveness is switched in three or more stages will be described. FIG. 9 is a flowchart showing a subject size averaging process as a modification. FIG. 9 describes a flow of the face size averaging process of the subject when the zoom responsiveness has three stages of “high / medium / low”.

First, in step S901, the subject size calculation unit 201 determines whether the responsiveness of the zoom operation set by the zoom responsiveness setting unit 131 is other than "low" or "low". If it is determined in step S901 that the zoom responsiveness setting is "low", the process proceeds to step S902. In step S902, the subject size calculation unit 201 acquires the number N of face size information used for averaging the face size from the memory 117. The number N used for averaging is stored in the memory 117 in advance. Subsequently, in step S903, the subject size calculation unit 201 averages N past face size information from the current face size information and sets the face size to be used for determining the zoom speed.

On the other hand, when the zoom responsiveness is determined to be other than "low" in step S901, the subject size calculation unit 201 determines in step S904 whether the zoom responsiveness is "medium" or "high". .. If the zoom responsiveness is determined to be "medium" in step S904, the process proceeds to step S905. In step S905, the subject size calculation unit 201 acquires the number M of face size information used for averaging the face size from the memory 117. The number M used for averaging is stored in the memory 117 in advance. The number M is a value stored in advance in the memory 117. Subsequently, in step S906, the subject size calculation unit 201 averages M past face size information from the current face size information to obtain the face size used for determining the zoom speed. The method for determining the number N and the number M is the same as in the first embodiment.

On the other hand, if the zoom responsiveness is determined to be "high" in step S904, the process proceeds to step S907. In step S907, the subject size calculation unit 201 uses the latest face size information as the face size for determining the zoom speed without averaging the face size in order to increase the zoom response to the change in face size. To do. The face size information for determining the zoom speed is selected in steps S903, S906, and S907, and the processing in the subject size calculation unit 201 ends.

Further, when the zoom responsiveness can be set to two or more steps, the flowchart described with reference to FIG. 10 may be followed. FIG. 10 is a flowchart showing a subject size averaging process as another modification of the present embodiment.

First, in step S1001, the subject size calculation unit 201 acquires the responsiveness of the current zoom operation set by the zoom responsiveness setting unit 131. In the memory 117, the responsiveness setting of the zoom operation and the number of tables used for averaging corresponding to the setting are stored in advance. Subsequently, in step S1002, the subject size calculation unit 201 acquires the number X to be used for averaging from the correspondence table between the responsiveness of the zoom operation of the memory 117 and the number of averaging. In this embodiment, the number X has a relationship of either M ≦ X ≦ N or 1 ≦ X ≦ N. Subsequently, in step S1003, the subject size calculation unit 201 uses the number X acquired in step S1002 to average X past face size information from the current face size information, and the face size used for determining the zoom speed. And. The method for determining the number N and the number M is as described above. Further, when the number X becomes 1, the latest face size information is used as the face size used for determining the zoom speed without averaging.

In this embodiment, the configurations other than the smoothing (averaging) calculation processing in the subject size calculation unit 201 are the same as those in the first embodiment, and thus the description thereof will be omitted.

Next, Example 3 will be described. In this embodiment, an example of thinning out the face size of the subject by the subject size calculation unit 201 will be described. The thinning process is a process for removing face size information that may be erroneously detected when the amount of change in face size does not fall within a predetermined range.

First, with reference to FIG. 11, step S503 (thinning process, that is, smoothing process) of FIG. 5 will be described in detail. FIG. 11 is a flowchart showing the thinning process of the subject size. FIG. 11 describes a case where the zoom responsiveness is switched in two steps.

First, in step S1101, the subject size calculation unit 201 determines the responsiveness of the current zoom operation set by the zoom responsiveness setting unit 131. If it is determined in step S1101 that the zoom responsiveness setting is "low", the process proceeds to step S1102. In step S1102, the subject size calculation unit 201 acquires the threshold value T used for the thinning process for suppressing the fluctuation of the zoom speed due to the change in face size from the memory 117. The threshold value T used for the thinning process is stored in the memory 117 in advance. The method of determining the threshold value T will be described later. Subsequently, in step S1103, the subject size calculation unit 201 thins out the latest face size information when the absolute value of the amount of change between the latest face size information and the previous face size information exceeds the threshold value T. Then, the subject size calculation unit 201 uses the previous face size information for determining the zoom speed. On the other hand, when the absolute value of the amount of change between the latest face size information and the previous face size information does not exceed the threshold value T, the subject size calculation unit 201 determines the zoom speed without thinning out the latest face size information. ..

On the other hand, if the zoom responsiveness is determined to be "high" in step S1101, the process proceeds to step S1104. When increasing the responsiveness of the zoom, it is assumed that there is almost no fluctuation in the face size, and it is ideal that the zoom operates while keeping the face size constant, so that high responsiveness is required. Therefore, in step S1104, the subject size calculation unit 201 does not perform the thinning process, and sets the latest face size information as the face size used for determining the zoom speed. Face size information for determining the zoom speed is selected in steps S1103 and S1104, and the process in the subject size calculation unit 201 ends.

Next, a method of determining the threshold value T used for the thinning process in step S1103 will be described. The subject size calculation unit 201 measures in advance the fluctuation of the face size when the subject actually moves, and obtains the average value of the fluctuation of the face size at that time. Then, a value that is a predetermined multiple of the average value of the face size fluctuation due to the movement of the subject is set as the threshold value T. Face size fluctuations larger than the threshold value T may cause false detection of face size. Therefore, the subject size calculation unit 201 uses the face size information when the fluctuation of the face size is equal to or less than the threshold value T. In addition, the face size also changes as the subject's face changes from the front facing state to the sideways direction. Therefore, the change in face size due to the change in face orientation may be reduced by using the face orientation information.

Since the configurations other than the smoothing (averaging) calculation processing in the subject size calculation unit 201 are the same as those in the first embodiment, their description will be omitted.

Next, Example 4 will be described. In this embodiment, an example of thinning out the face size of the subject by the subject size calculation unit 201 will be described. First, with reference to FIG. 12, step S503 (thinning process, that is, smoothing process) of FIG. 5 will be described in detail. FIG. 12 is a flowchart showing the thinning process of the subject size in this embodiment. FIG. 12 describes a case where the zoom responsiveness is switched in two steps.

First, in step S1201, the subject size calculation unit 201 determines the responsiveness of the current zoom operation set by the zoom responsiveness setting unit 131. If it is determined in step S1201 that the zoom responsiveness setting is "low", the process proceeds to step S1202. In step S1202, the subject size calculation unit 201 acquires the threshold value T used for the thinning process for suppressing the fluctuation of the zoom speed due to the change in face size from the memory 117. The threshold value T used for the thinning process is stored in the memory 117 in advance. The method for determining the threshold value T is as described in Example 3. Subsequently, in step S1203, the subject size calculation unit 201 thins out the latest face size information when the absolute value of the amount of change between the latest face size information and the previous face size information exceeds the threshold value T. Then, the subject size calculation unit 201 uses the previous face size information for determining the zoom speed. On the other hand, when the absolute value of the amount of change between the latest face size information and the previous face size information does not exceed the threshold value T, the subject size calculation unit 201 determines the zoom speed without thinning out the latest face size information. To do.

On the other hand, if the zoom responsiveness is determined to be "high" in step S1201, the process proceeds to step S1204. In step S1204, the subject size calculation unit 201 performs thinning processing using the threshold value S when it is necessary to reduce the influence of the face size variation. The threshold value S is a value that does not impair the responsiveness of the zoom, and S> T holds. Subsequently, in step S1205, when the absolute value of the amount of change between the latest face size information and the previous face size information exceeds the threshold value S, the subject size calculation unit 201 thins out the latest face size information. Then, the subject size calculation unit 201 uses the previous face size information for determining the zoom speed. On the other hand, when the absolute value of the amount of change between the latest face size information and the previous face size information does not exceed the threshold value S, the subject size calculation unit 201 determines the zoom speed without thinning out the latest face size information. To do. Face size information for determining the zoom speed is selected in steps S1203 and S1205, and the process in the subject size calculation unit 201 ends.

Next, with reference to FIG. 13, a case where the zoom responsiveness is switched in three or more stages will be described. FIG. 13 is a flowchart showing a subject size thinning process as a modification of the present embodiment. FIG. 13 describes a flow of thinning out the face size of the subject when the zoom responsiveness has three stages of “high / medium / low”.

First, in step S1301, the subject size calculation unit 201 determines whether the responsiveness of the current zoom operation set by the zoom responsiveness setting unit 131 is other than "low" or "low". If it is determined in step S1301 that the zoom responsiveness setting is "low", the process proceeds to step S1302. In step S1302, the subject size calculation unit 201 acquires the threshold value T used for the thinning process for suppressing the fluctuation of the zoom speed due to the change in face size from the memory 117. The threshold value T used for the thinning process is stored in the memory 117 in advance. Subsequently, in step S1303, when the absolute value of the amount of change between the latest face size information and the previous face size information exceeds the threshold value T, the subject size calculation unit 201 thins out the latest face size information. Then, the subject size calculation unit 201 uses the previous face size information for determining the zoom speed. On the other hand, when the absolute value of the amount of change between the latest face size information and the previous face size information does not exceed the threshold value T, the subject size calculation unit 201 determines the zoom speed without thinning out the latest face size information. To do.

On the other hand, when it is determined in step S1301 that the zoom responsiveness is other than "low", in step S1304, the subject size calculation unit 201 determines whether the zoom responsiveness is "medium" or "high". judge. If the zoom responsiveness is determined to be "medium" in step S1304, the process proceeds to step S1305. In step S1305, the subject size calculation unit 201 acquires the threshold value S of the face size information used for the thinning process for suppressing the fluctuation of the zoom speed due to the change in the face size from the memory 117. The threshold value S used for the thinning process is stored in the memory 117 in advance. Subsequently, in step S1306, when the absolute value of the amount of change between the latest face size information and the previous face size information exceeds the threshold value S, the subject size calculation unit 201 thins out the latest face size information. Then, the subject size calculation unit 201 uses the previous face size information for determining the zoom speed. On the other hand, when the absolute value of the amount of change between the latest face size information and the previous face size information does not exceed the threshold value S, the subject size calculation unit 201 determines the zoom speed without thinning out the latest face size information. To do. The method for determining the threshold value T and the threshold value S is as described above.

On the other hand, if the zoom responsiveness is determined to be "high" in step S1304, the process proceeds to step S1307. In step S1307, the subject size calculation unit 201 does not perform the thinning process of the face size information and uses the latest face size information for determining the zoom speed in order to increase the responsiveness of the zoom to the change of the face size. The face size information for determining the zoom speed is selected in steps S1303, S1306, and S1307, and the processing in the subject size calculation unit 201 ends.

Further, when the zoom responsiveness can be set to two or more steps, the flow described with reference to FIG. 14 may be followed. FIG. 14 is a flowchart showing a subject size thinning process as another modification of the present embodiment.

First, in step S1401, the subject size calculation unit 201 acquires the responsiveness of the current zoom operation set by the zoom responsiveness setting unit 131. In the memory 117, a table of responsiveness of the zoom operation and a threshold value used for the thinning process corresponding to the setting is stored in advance. Subsequently, in step S1402, the subject size calculation unit 201 acquires a threshold value Y for use in the thinning process from the correspondence table between the responsiveness of the zoom operation of the memory 117 and the threshold value of the thinning process. In this embodiment, the threshold value Y has a relationship of either T ≦ Y ≦ S or 0 ≦ Y ≦ S.

Subsequently, in step S1403, the subject size calculation unit 201 uses the threshold value Y acquired in step S1402 when the absolute value of the amount of change between the latest face size information and the previous face size information exceeds the threshold value Y. , Thin out face size information. Then, the subject size calculation unit 201 uses the previous face size information for determining the zoom speed. On the other hand, when the absolute value of the amount of change between the latest face size information and the previous face size information does not exceed the threshold value Y, the subject size calculation unit 201 performs the latest face size information without thinning out the face size information. Used to determine the zoom speed. When the threshold value becomes 0, the subject size calculation unit 201 does not perform the thinning process and uses the latest face size information for determining the zoom speed. The method for determining the threshold value T and the threshold value S is as described above.

In this embodiment, the threshold value for the thinning process is changed based on the responsiveness of the zoom operation, but the sampling rate may be changed. Since the configurations other than the smoothing (averaging) calculation processing in the subject size calculation unit 201 are the same as those in the first embodiment, their description will be omitted.

Next, Example 5 will be described. In this embodiment, an example of the face size filtering process of the subject by the subject size calculation unit 201 will be described. The filtering process is a process for gradually reducing the face size that may have been erroneously detected and removing the face size information that may have been erroneously detected when the frequency of change in face size is equal to or higher than a predetermined frequency.

First, with reference to FIG. 15, step S503 (filtering process, that is, smoothing process) of FIG. 5 will be described in detail. FIG. 15 is a flowchart showing a subject size filtering process in this embodiment. FIG. 15 describes a case where the zoom responsiveness is switched in two steps.

First, in step S1501, the subject size calculation unit 201 determines the responsiveness of the current zoom operation set by the zoom responsiveness setting unit 131. If it is determined in step S1501 that the zoom responsiveness setting is "low", the process proceeds to step S1502. In step S1502, the subject size calculation unit 201 acquires the frequency F used for the filtering process for suppressing the fluctuation of the zoom speed due to the change in the face size from the memory 117. The frequency F used for the filtering process is stored in the memory 117 in advance. The method for determining the frequency F will be described later. Subsequently, in step S1503, the subject size calculation unit 201 first passes the face size information through a digital low-pass filter designed using the frequency F to remove high frequency components. After that, the face size information passed through the filter is used to determine the zoom speed.

On the other hand, if the zoom responsiveness is determined to be "high" in step S1501, the process proceeds to step S1504. When increasing the responsiveness of the zoom, it is assumed that there is almost no fluctuation in the face size, and it is ideal that the zoom operates while keeping the face size constant, so that high responsiveness is required. Therefore, in step S1504, the subject size calculation unit 201 does not perform the filtering process and uses the latest face size information for determining the zoom speed. Face size information for determining the zoom speed is selected in steps S1503 and S1504, and the process in the subject size calculation unit 201 ends.

Next, a method of determining the frequency F used for the filtering process in step S1503 will be described. The subject size calculation unit 201 measures in advance the fluctuation of the face size when the subject actually moves, and obtains the frequency characteristic of the fluctuation of the face size from the measurement result. Since the frequency of face size fluctuation due to the movement of the subject can be determined from the frequency characteristics, that frequency is set to frequency F. A digital low-pass filter designed using the frequency F is used because a face size fluctuation having a frequency faster than the frequency F may cause a false detection of the face size.

Since the configurations other than the smoothing (averaging) calculation processing in the subject size calculation unit 201 are the same as those in the first embodiment, their description will be omitted.

Next, Example 6 will be described. In this embodiment, an example of the face size filtering process of the subject by the subject size calculation unit 201 will be described. First, with reference to FIG. 16, step S503 (filtering process, that is, smoothing process) of FIG. 5 will be described in detail. FIG. 16 is a flowchart showing a subject size filtering process in this embodiment. FIG. 16 describes a case where the zoom responsiveness is switched in two steps.

First, in step S1601, the subject size calculation unit 201 determines the responsiveness of the current zoom operation set by the zoom responsiveness setting unit 131. If it is determined in step S1601 that the zoom responsiveness setting is "low", the process proceeds to step S1602. In step S1602, the subject size calculation unit 201 acquires the frequency F used for the filtering process for suppressing the fluctuation of the zoom speed due to the change in the face size from the memory 117. The frequency F used for the filtering process is stored in the memory 117 in advance. The method for determining the frequency F is as described in Example 5. Subsequently, in step S1603, the subject size calculation unit 201 first passes the face size information through a digital low-pass filter created using the frequency F to remove high-frequency components. After that, the subject size calculation unit 201 uses the face size information passed through the filter to determine the zoom speed.

On the other hand, if the zoom responsiveness is determined to be "high" in step S1601, the process proceeds to step S1604. In step S1604, when it is necessary to reduce the influence of face size variation, a filtering process is performed. Therefore, in step S1604, the subject size calculation unit 201 acquires the frequency G used for the filtering process for suppressing the fluctuation of the zoom speed due to the change in the face size from the memory 117. The frequency G used for the filtering process is stored in the memory 117 in advance. The frequency G is set to a value that does not impair the responsiveness of the zoom, and G> F holds. Subsequently, in step S1605, the subject size calculation unit 201 passes the face size information through a digital low-pass filter created using the frequency G to remove the high frequency component. After that, the subject size calculation unit 201 uses the face size information passed through the filter to determine the zoom speed. Face size information for determining the zoom speed is selected in steps S1603 and S1605, and the process in the subject size calculation unit 201 ends.

Next, with reference to FIG. 17, a case where the zoom responsiveness is switched in three or more stages will be described. FIG. 17 is a flowchart showing a subject size filtering process as a modification of the present embodiment. FIG. 17 describes a flow of filtering processing of the face size of the subject when the zoom responsiveness has three stages of “high / medium / low”.

First, in step S1701, the subject size calculation unit 201 determines whether the responsiveness of the current zoom operation set by the zoom responsiveness setting unit 131 is other than "low" or "low". If it is determined in step S1701 that the zoom responsiveness setting is "low", the process proceeds to step S1702. In step S1702, the subject size calculation unit 201 acquires the frequency F used for the filtering process for suppressing the fluctuation of the zoom speed due to the change in the face size from the memory 117. The frequency F used for the filtering process is stored in the memory 117 in advance. Subsequently, in step S1703, the subject size calculation unit 201 passes the face size information through a digital low-pass filter created by using the frequency F to remove the high frequency component. After that, the subject size calculation unit 201 uses the face size information passed through the filter to determine the zoom speed.

On the other hand, when it is determined in step S1701 that the zoom responsiveness is other than "low", in step S1704, the subject size calculation unit 201 has either "medium" or "high" zoom responsiveness. Is determined. If the zoom responsiveness is determined to be "medium" in step S1704, the process proceeds to step S1705. In step S1705, the subject size calculation unit 201 acquires the frequency G used for the filtering process for suppressing the fluctuation of the zoom speed due to the change in the face size from the memory 117. The frequency G used for the filtering process is stored in the memory 117 in advance. Subsequently, in step S1706, the subject size calculation unit 201 passes the face size information through a digital low-pass filter created using the frequency G to remove the high frequency component. After that, the subject size calculation unit 201 uses the face size information passed through the filter to determine the zoom speed.

On the other hand, if the zoom responsiveness is determined to be "high" in step S1704, the process proceeds to step S1707. In step S1707, the subject size calculation unit 201 does not perform the filtering process of the face size information and uses the latest face size information for determining the zoom speed in order to increase the responsiveness of the zoom to the change of the face size. The method for determining the frequency F and the frequency G is as described above. The face size information for determining the zoom speed is selected in steps S1703, S1706, and S1707, and the processing in the subject size calculation unit 201 ends.

Further, when the zoom responsiveness can be set to two or more steps, the flow described with reference to FIG. 18 may be followed. FIG. 18 is a flowchart showing a subject size filtering process as another modification of the present embodiment.

First, in step S1801, the subject size calculation unit 201 acquires the responsiveness of the current zoom operation set by the zoom responsiveness setting unit 131. In the memory 117, a setting of responsiveness of the zoom operation and a table of frequencies used for the filtering process corresponding to the setting are stored in advance. Subsequently, in step S1802, the subject size calculation unit 201 acquires the frequency Z to be used for the filtering process from the correspondence table between the responsiveness of the zoom operation of the memory 117 and the threshold value of the filtering process. The frequency Z has a relationship of either F ≦ Z ≦ G or 0 ≦ Z ≦ G. Subsequently, in step S1803, the subject size calculation unit 201 uses the frequency Z acquired in step S1802 and uses the latest face size information for determining the zoom speed if the fluctuation of the face size is frequency Z or less. On the other hand, when the face size fluctuates above the frequency Z, the subject size calculation unit 201 determines that the face size is erroneously detected. Then, the subject size calculation unit 201 corrects the falsely detected face size so that the frequency is Z or less, and determines the zoom speed. When the frequency becomes 0, the subject size calculation unit 201 does not perform the filtering process and uses the latest face size for determining the zoom speed. The method for determining the frequency F and the frequency G is as described above.

Since the configurations other than the smoothing (averaging) calculation processing in the subject size calculation unit 201 are the same as those in the first embodiment, their description will be omitted.

As described above, in each embodiment, the control device (AZ control unit 125) is a control device that controls the zoom means (124, 113) for changing the imaging magnification. The control device also includes a calculation means (subject size calculation unit 201), a first determination unit (target subject size determination unit 202), and a second determination unit (zoom speed determination unit 203). The calculation means calculates the subject size based on the captured image. The first determining means determines the target subject size. The second determining means determines the zoom speed based on the subject size and the target subject size. The calculation means changes the method of calculating the subject size based on the responsiveness of the zoom operation by the zoom means.

Preferably, the control device includes acquisition means (subject detection unit 126, subject size extraction unit 132) for sequentially acquiring a plurality of subject size information from the captured image. More preferably, the calculation means calculates the subject size by performing smoothing processing of a plurality of subject size information. More preferably, when the responsiveness is the first responsiveness (when the responsiveness is low), the calculation means performs smoothing processing of a plurality of subject size information to calculate the subject size. On the other hand, when the responsiveness is the second responsiveness higher than the first responsiveness (when the responsiveness is high), the calculation means calculates the subject size without smoothing a plurality of subject size information. To do.

Preferably, the calculation means performs a plurality of subject size information averaging processes as a smoothing process (Examples 1 and 2). More preferably, when the responsiveness is the first responsiveness, the calculation means performs the averaging process of the first number (N) of the subject size information to calculate the subject size. On the other hand, when the responsiveness is the second responsiveness, the calculation means averages the subject size information of the second number (M (<N)) smaller than the first number to obtain the subject size. calculate.

Preferably, as the smoothing process, the calculation means performs a thinning process of removing at least one subject size information from the plurality of subject size information (Examples 3 and 4). More preferably, the calculation means means that the absolute value of the amount of change between the first subject size information and the second subject size information acquired continuously among the plurality of subject size information exceeds a predetermined threshold value (T). If so, the thinning process is performed. On the other hand, the calculation means does not perform the thinning process when the absolute value of the amount of change between the first subject size information and the second subject size information does not exceed a predetermined threshold value.

Preferably, the calculation means has the first responsiveness, and the absolute amount of change between the first subject size information and the second subject size information acquired continuously among the plurality of subject size information is absolute. When the value exceeds the first threshold value (T), the subject size is calculated by performing the thinning process. On the other hand, the calculation means has a second threshold value (the responsiveness is the second responsiveness and the absolute value of the amount of change between the first subject size information and the second subject size information is larger than the first threshold value. When S (> T)) is exceeded, the subject size is calculated by performing the thinning process (Example 4).

Preferably, the calculation means performs a plurality of subject size information filtering processes as a smoothing process (Examples 5 and 6). More preferably, when the responsiveness is the first responsiveness, the calculation means performs a filtering process using a low-pass filter of the first frequency (F) to calculate the subject size. On the other hand, when the responsiveness is the second responsiveness, the calculation means calculates the subject size by performing a filtering process using a low-pass filter having a second frequency (G) higher than the first frequency (implementation). Example 6).

Preferably, the control device has a setting means (zoom responsiveness setting unit 131) for setting the responsiveness of the zoom operation. More preferably, the setting means sets the responsiveness of the zoom operation according to the instruction of the user. Also preferably, the setting means sets the responsiveness selected from the plurality of responsiveness of the zoom operation.

(Other Examples)
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

According to each embodiment, it is possible to provide a control device, an image pickup device, a control method, and a program capable of performing an appropriate auto zoom according to the responsiveness of the zoom operation. That is, according to each embodiment, the responsiveness of the zoom operation can be switched according to the shooting conditions to enable auto-zoom shooting according to the user's shooting intention.

Although preferable examples of the present invention have been described above, the present invention is not limited to these examples, and various modifications and modifications can be made within the scope of the gist thereof.

125 AZ control unit (control device)
201 Subject size calculation unit (calculation means)
202 Target subject size determination unit (first determination means)
203 Zoom speed determination unit (second determination means)

Claims (17)

A control device that controls the zoom means for changing the imaging magnification.
A calculation means for calculating the subject size based on the captured image,
The first means of determining the target subject size and
It has a second determining means for determining the zoom speed based on the subject size and the target subject size.
The calculation means is a control device characterized in that the calculation method of the subject size is changed based on the responsiveness of the zoom operation by the zoom means. The control device according to claim 1, further comprising an acquisition means for sequentially acquiring a plurality of subject size information from the captured image. The control device according to claim 2, wherein the calculation means performs smoothing processing of a plurality of subject size information to calculate the subject size. The calculation means is
When the responsiveness is the first responsiveness, the smoothing process of the plurality of subject size information is performed to calculate the subject size.
When the responsiveness is a second responsiveness higher than the first responsiveness, the subject size is calculated without performing the smoothing process of the plurality of subject size information. The control device according to claim 3. The control device according to claim 3 or 4, wherein the calculation means performs an averaging process of a plurality of subject size information as the smoothing process. The calculation means is
When the responsiveness is the first responsiveness, the subject size is calculated by performing the averaging process of the first number of subject size information.
When the responsiveness is a second responsiveness higher than the first responsiveness, the subject size is calculated by performing the averaging process of the second number of subject size information smaller than the first number. The control device according to claim 5, wherein the control device is characterized in that it is calculated. The control device according to claim 3 or 4, wherein the calculation means performs a thinning process for removing at least one subject size information from the plurality of subject size information as the smoothing process. The calculation means is
When the absolute value of the amount of change between the first subject size information and the second subject size information continuously acquired among the plurality of subject size information exceeds a predetermined threshold value, the thinning process is performed.
The seventh aspect of claim 7, wherein the thinning process is not performed when the absolute value of the amount of change between the first subject size information and the second subject size information does not exceed the predetermined threshold value. Control device. The calculation means is
The responsiveness is the first responsiveness, and the absolute value of the amount of change between the first subject size information and the second subject size information continuously acquired among the plurality of subject size information is the first. When the threshold value of is exceeded, the thinning process is performed to calculate the subject size.
The responsiveness is a second responsiveness higher than the first responsiveness, and the absolute value of the amount of change between the first subject size information and the second subject size information is the first threshold value. The control device according to claim 7, wherein when the second threshold value larger than the second threshold value is exceeded, the thinning process is performed to calculate the subject size. The control device according to claim 3 or 4, wherein the calculation means performs a filtering process of a plurality of subject size information as the smoothing process. The calculation means is
When the responsiveness is the first responsiveness, the filtering process is performed using the low-pass filter of the first frequency to calculate the subject size.
When the responsiveness is a second responsiveness higher than the first responsiveness, the filtering process is performed using a low-pass filter having a second frequency higher than the first frequency to determine the subject size. The control device according to claim 10, wherein the control device is characterized in that it is calculated. The control device according to any one of claims 1 to 11, further comprising a setting means for setting the responsiveness of the zoom operation. The control device according to claim 12, wherein the setting means sets the responsiveness of the zoom operation according to a user's instruction. The control device according to claim 12 or 13, wherein the setting means sets the responsiveness selected from the plurality of responsiveness of the zoom operation. With the image sensor
An imaging device comprising the control device according to any one of claims 1 to 14. It is a control method for controlling the zoom means for changing the imaging magnification.
Steps to calculate the subject size based on the captured image,
Steps to determine the target subject size and
It has a step of determining a zoom speed based on the subject size and the target subject size.
A control method characterized in that, in the step of calculating the subject, the calculation method of the subject size is changed based on the responsiveness of the zoom operation by the zoom means. A program that causes a computer to execute a control method that controls a zoom means that changes the imaging magnification.
Steps to calculate the subject size based on the captured image,
Steps to determine the target subject size and
Let the computer perform the step of determining the zoom speed based on the subject size and the target subject size.
A program characterized in that, in the step of calculating the subject, the method of calculating the subject size is changed based on the responsiveness of the zoom operation by the zoom means.