JP2013214833A - Area determination device, image pickup device, and area determination program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an area determination device, an image pickup device and an area determination program which can determine a subject area framed in a picked-up image with high accuracy and thereby pick up the image of the subject properly.SOLUTION: An area determination device 141 includes: an area extraction unit 143 in which extraction information indicating a condition for determining whether extracted subject areas which were each extracted from a row of images picked up at different times of day on the basis of image feature amounts are those of the same subject and image pickup information indicating settings needed for picking up the image of a subject area are correlated to each other for each of the extracted subject areas; and an area determination unit 144 which, when a subject area framed out from a first image in the row of images and a subject area framed into a second image in the row of images are those of the same subject, selects the image pickup information correlated to the subject area framed out from the first image as image pickup information for picking up the image of the subject area framed into the second image.

Description

  The present invention relates to an area determination apparatus, an imaging apparatus, and an area determination program for determining an area.

  There has been disclosed an imaging apparatus for the purpose of optimal focus adjustment with respect to a subject even when the subject moves in the camera direction (see Patent Document 1).

JP 2009-69748 A

  However, the imaging device disclosed in Patent Document 1 has a problem that it cannot accurately determine a subject area framed in a captured image. If the subject area cannot be accurately determined, the imaging apparatus must move the focusing lens greatly in the optical axis direction every time the subject area is framed into the image, and focus on the subject area. The subject cannot be imaged properly.

  The present invention has been made in view of the above points, and is an area determination device, an imaging apparatus, and an area that can accurately determine a subject area framed in a captured image and appropriately capture the subject. An object is to provide a judgment program.

  The present invention has been made to solve the above-described problem, and extracts subject areas based on image feature amounts from image sequences captured at different times, and the extracted subject areas are the same subject. A region extracting unit that associates extraction information indicating a condition for determining whether or not the subject region is determined and imaging information indicating a setting for capturing the subject region with the extracted subject region, and the image Determining whether the subject area framed out from the first image in the column and the subject region framed in to the second image in the image sequence are subject regions of the same subject based on the extraction information; When it is determined that the subject area is the same subject, the imaging information for capturing the subject area framed in the second image is used as the imaging information. A region determining unit that selects the imaging information associated with the object regions frame out, a region judging device, characterized in that it comprises a.

  In addition, the present invention focuses on a subject determination region, a focus adjustment unit that focuses on the subject region with an optical system based on the focus position that is the imaging information selected by the region determination device, and the subject region. An imaging apparatus comprising: an imaging unit that captures an image of a state.

  The present invention also allows a computer to extract subject areas based on image feature amounts from image sequences captured at different times, and to determine whether the extracted subject areas are subject areas of the same subject. A procedure for associating the extracted information indicating the condition for determination and the imaging information indicating the setting for imaging the subject area with the extracted subject area, and the subject area framed out from the first image of the image sequence And the subject area framed in the second image in the image sequence is determined based on the extracted information, and is determined to be the subject area due to the same subject. As the imaging information for imaging the subject area framed in the second image, the subject area framed out from the first image is used. A procedure for selecting the imaging information attached response, an area determination program for execution.

  According to the present invention, the area determination device can accurately determine a subject area framed in a captured image.

It is a block diagram showing the structure of the lens barrel, imaging device, and storage medium in one Embodiment of this invention. 6 is a flowchart (first half) showing an operation procedure for executing contrast AF based on imaging information of a subject area in an embodiment of the present invention. 6 is a flowchart (second half) showing an operation procedure for executing contrast AF based on imaging information of a subject area in an embodiment of the present invention. It is a figure for demonstrating the operation | movement procedure which performs contrast AF based on the imaging information of a to-be-photographed area | region in one Embodiment of this invention. It is a figure for demonstrating the operation example which performs contrast AF based on the imaging information of a to-be-photographed area | region in one Embodiment of this invention. 6 is a flowchart showing an operation procedure for sequentially executing contrast AF based on imaging information of a subject area in an embodiment of the present invention. 6 is a flowchart (first half) illustrating an operation procedure for executing contrast AF based on action information that is imaging information of a subject area in an embodiment of the present invention. 6 is a flowchart (continuation of the first half) showing an operation procedure for executing contrast AF based on action information that is imaging information of a subject area in an embodiment of the present invention. 7 is a flowchart (second half) showing an operation procedure for executing contrast AF based on action information that is imaging information of a subject area in an embodiment of the present invention. It is a figure for demonstrating the operation | movement procedure which performs contrast AF based on the action information which is the imaging information of a to-be-photographed area | region in one Embodiment of this invention. It is a figure for demonstrating the operation | movement procedure which performs contrast AF based on the action information which is the imaging information of a to-be-photographed area | region in one Embodiment of this invention. 6 is a flowchart illustrating an operation procedure for executing contrast AF based on imaging information of a subject area when the zoom magnification is changed according to an embodiment of the present invention. 6 is a flowchart illustrating an operation procedure for executing contrast AF based on a color channel that is subject area extraction information according to an embodiment of the present invention. It is a figure which shows the relationship between AF lens position and contrast evaluation value in one Embodiment of this invention.

  An embodiment of the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram illustrating configurations of a lens barrel, an imaging device, and a storage medium. The imaging apparatus 100 captures an optical image incident from the lens barrel 111 and stores a still image format image or a moving image format image sequence in the storage medium 200.

First, the configuration of the lens barrel 111 will be described.
The lens barrel 111 includes a focus adjustment lens (hereinafter referred to as “AF (Auto Focus) lens”) 112, a lens driving unit 116, an AF encoder 117, and a lens barrel control unit 118. The lens barrel 111 may be detachably connected to the imaging device 100 or may be integrated with the imaging device 100.

  The AF lens 112 is driven by a lens driving unit 116 and guides an optical image to a light receiving surface (photoelectric conversion surface) of an imaging element 119 of an imaging unit 110 (described later).

  The AF encoder 117 detects the movement of the AF lens 112 and outputs a signal corresponding to the movement amount of the AF lens 112 to the lens barrel control unit 118. Here, the signal corresponding to the movement amount of the AF lens 112 may be a sine wave signal whose phase changes according to the movement amount of the AF lens 112, for example.

  The lens barrel control unit 118 controls the lens driving unit 116 in accordance with a drive control signal input from a CPU 190 (described later) of the imaging apparatus 100. Here, the drive control signal is a control signal for driving the AF lens 112 in the optical axis direction. The lens barrel control unit 118 changes the number of steps of the pulse voltage output to the lens driving unit 116, for example, according to the drive control signal.

  Further, the lens barrel control unit 118 outputs the position (focus position) of the AF lens 112 in the lens barrel 111 to the CPU 190 of the imaging device 100 based on a signal corresponding to the movement amount of the AF lens 112. Here, the lens barrel control unit 118 integrates, for example, signals according to the movement amount of the AF lens 112 according to the movement direction of the AF lens 112, thereby moving the AF lens 112 in the lens barrel 111 (the movement amount ( Position) may be calculated.

  The lens driving unit 116 drives the AF lens 112 in accordance with the control of the lens barrel control unit 118 and moves the AF lens 112 in the optical axis direction within the lens barrel 111.

Next, the configuration of the imaging device 100 will be described.
The imaging device 100 includes an imaging unit 110, an image processing unit 140, an area determination device 141, a display unit 150, a buffer memory unit 130, an operation unit 180, a storage unit 160, a CPU 190, and a communication unit 170. Is provided.

  The imaging unit 110 includes an imaging element 119 and an A / D (Analog / Digital) conversion unit 120. The imaging unit 110 is controlled by the CPU 190 based on the set imaging conditions (for example, aperture value, exposure amount).

  The image sensor 119 includes a photoelectric conversion surface, converts an optical image formed on the photoelectric conversion surface by the lens barrel 111 (optical system) into an electric signal, and outputs the electric signal to the A / D conversion unit 120. The image sensor 119 may be configured by, for example, a CMOS (Complementary Metal Oxide Semiconductor). Further, the image sensor 119 may convert an optical image into an electrical signal for a partial region of the photoelectric conversion surface (image clipping).

  Further, the image sensor 119 stores an image obtained when a shooting instruction from the user is received via the operation unit 180 in the storage medium 200 via the A / D conversion unit 120. On the other hand, the imaging device 119 performs A / D conversion on the buffer memory unit 130 and the display unit 150 as images that are continuously obtained in a state where a shooting instruction from the user is not received via the operation unit 180. Output via the unit 120.

  The A / D converter 120 digitizes the electrical signal converted by the image sensor 119 and outputs an image that is a digital signal to the buffer memory unit 130.

  The operation unit 180 includes, for example, a power switch, a shutter button, a multi selector (cross key), or other operation keys. The operation unit 180 receives a user operation input when operated by the user, and a signal corresponding to the received operation input. Is output to the CPU 190.

  The image processing unit 140 performs image processing on the image temporarily stored in the buffer memory unit 130 based on the image processing conditions stored in the storage unit 160. The processed image is stored in the storage medium 200 via the communication unit 170.

  The area determination device 141 extracts a subject area from the image sequence temporarily stored in the buffer memory unit 130 based on the image feature amount. In addition, the area determination device 141 has the same subject area framed out from the first image and subject area framed in the second image captured at a different time from the first image for the extracted subject area. Whether or not the subject area is a subject area is determined based on attribute information (attribute data) associated with the subject area.

  The attribute information includes extraction information indicating conditions for determining whether or not the extracted subject areas are subject areas of the same subject, and imaging information indicating settings for capturing the subject area. .

  The extracted information includes, for example, the brightness of the subject area, the color information of the subject area, the texture of the subject area, the area of the subject area, the rectangle circumscribing the subject area, the occupation ratio of the subject area with respect to the rectangle, The gravity center position of the subject area in the image frame, the distance from the center of the angle of view to the subject area, the moment of inertia of the subject area, the AF evaluation value (for example, contrast and phase difference) of the subject area, and the boundary (edge) of the subject area ), Zoom magnification, movement of the subject area in the image, information indicating whether or not the subject area is a background subject area, and the orientation of the imaging apparatus measured by the sensor (for example, the imaging apparatus 100). Information indicating the orientation of the subject with respect to. For example, whether or not the extracted subject areas are subject areas of the same subject may be determined based on the degree of approximation of the extraction information between the subject areas.

  The imaging information includes, for example, a focus position (focal distance), an aperture value, an exposure amount, and action information (action data). Here, the action information is information representing an action (for example, operation, movement, direction) of the imaging apparatus performed on the subject. For example, the action information may be referred to in order to determine a priority for sequentially focusing on a plurality of subject areas framed in the image.

  The area determination device 141 causes the storage unit 160 to store attribute information (extraction information, imaging information) associated with the subject area. In addition, the area determination device 141 sets a priority for focusing the subject area on the subject area extracted from the image temporarily stored in the buffer memory unit 130. Details of the area determination device 141 will be described later.

  The display unit 150 displays an image obtained by the imaging unit 110, an operation screen, and the like. The display unit 150 is, for example, a liquid crystal display.

  The buffer memory unit 130 temporarily stores the image captured by the imaging unit 110.

  The storage unit 160 has a determination condition referred to by the CPU 190 in scene determination, an imaging condition associated with each scene determined by the scene determination, and attribute information (extraction information and imaging information) determined for each subject area. And remember.

  The CPU 190 controls the imaging unit 110 based on the set imaging conditions (for example, aperture value, exposure amount). In addition, the CPU 190 acquires attribute information (extraction information, imaging information) from the area determination device 141.

  Based on the imaging information, the CPU 190 performs focus adjustment (AF) setting, exposure adjustment (AE) setting, white balance adjustment (AWB) setting, subject image (object) setting as pre-processing or post-processing of image processing. Controls tracking settings, night scene determination settings, color correction processing settings, subject image enlarged display settings, panning display settings, brightness optimization settings linked to zoom magnification, and the like. Further, the CPU 190 controls setting of a tone curve (gradation curve), setting whether to perform simultaneous imaging with a still image and a moving image, and the like based on the imaging information.

  Further, the CPU 190 sets whether to perform continuous shooting as pre-processing or post-processing based on the imaging information, setting for switching the subject image to be focused in continuous shooting, and setting for continuously changing the focal length. The control unit controls the setting for changing the amount of flash light emitted from the light emitting unit (not shown) of the imaging apparatus 100, the setting for changing the exposure amount, or the setting for whether or not to change the shutter speed. Further, the CPU 190 causes the image processing unit 140 to perform image processing on an image in a still image format or a moving image format based on the “signal according to the operation input” input from the operation unit 180.

  Further, the CPU 190 (focus adjustment unit) outputs a drive control signal to the lens barrel control unit 118 of the lens barrel 111 based on the imaging information, thereby adjusting the focus so as to focus on the subject area.

  For example, when the region determination device 141 determines that the subject region framed out from the first image and the subject region framed in the second image are subject regions by the same subject, the CPU 190 is as follows. Operate.

  The CPU 190 moves the AF lens 112 to the in-focus position of the subject area framed out from the first image based on the in-focus position stored in the storage unit 160 as the imaging information, so that the second image is framed. Focus on the subject area.

  In addition, when color information is stored as extraction information, the CPU 190 may execute a contrast scan based on the stored color information. Further, when the exposure amount is stored as the imaging information, the CPU 190 controls the imaging unit 110 to perform exposure with the exposure amount. Thereby, an image sequence is imaged by the imaging unit 110 according to the stored exposure amount.

  The communication unit 170 is connected to a removable storage medium 200 (such as a card memory), and writes, reads, or deletes information (image data, attribute information, etc.) on the storage medium 200.

  The storage medium 200 is a storage unit that is detachably connected to the imaging apparatus 100 and stores information (image data, attribute information, etc.). Note that the storage medium 200 may be integrated with the imaging device 100.

  The sensor 500 measures the orientation of the subject with respect to the imaging device 100, that is, the direction of the region determination device 141, and outputs information representing the measurement result to the region determination device 141.

Next, details of the area determination device 141 will be described.
The region determination device 141 includes a feature amount calculation unit 142, a region extraction unit 143, a region processing unit 144, and a priority setting unit 145.

  The feature amount calculation unit 142 acquires the image captured by the imaging unit 110 from the buffer memory unit 130. The feature amount calculation unit 142 also has a feature amount (for example, a feature amount based on luminance information) for each of a plurality of features (for example, hue, saturation, brightness, texture, edge) in the image acquired from the buffer memory unit 130. , The feature amount based on the color information) is calculated, and the calculated feature amount is output to the region extraction unit 142.

  The region extraction unit 143 acquires the feature amount calculated by the feature amount calculation unit 142. In addition, the area extraction unit 143 acquires image sequences captured by the imaging unit 110 at different times from the buffer memory unit 130. Further, the region extraction unit 143 extracts a subject region from the acquired image sequence based on the image feature amount. Here, the region extraction unit 143 may extract the subject region from all the images constituting the image sequence, or from the image that has changed only when the image constituting the image sequence has changed. A subject area may be extracted.

  The region extraction unit 143 extracts a subject region based on, for example, luminance (Y) and color information (Cr, Cb, RGB) that are image feature amounts. Further, the region extraction unit 143 extracts a subject region from an image binarized into an image feature amount within a certain range and an image feature amount outside the certain range. The area extraction unit 143 assigns a label to the subject area (labeling).

  The area extraction unit 143 also causes the storage unit 160 to store attribute information (extraction information, imaging information) associated with the subject area. Here, the region extraction unit 143 stores, for example, the in-focus position with respect to the subject region in the storage unit 160 as imaging information. In addition, the region extraction unit 143 stores, for example, at least one of the exposure amount, gain, and gradation curve for the subject region in the storage unit 160 as imaging information. In addition, the area extraction unit 143 stores, for example, action information representing at least one of the operation, movement, and orientation of the imaging device with respect to the subject in the storage unit 160 as imaging information. In addition, the area extraction unit 143 stores, for example, the orientation of the imaging device 100 measured by the sensor 500 in the storage unit 160 as extraction information. Further, the area extraction unit 143 stores, for example, the color information of the subject area in the storage unit 160 as extraction information.

  The area determination unit 144 determines whether or not the imaging apparatus 100 has moved (for example, boom up or boom down) or has changed its direction (for example, panning or tilting).

  Further, the region determination unit 144 determines whether the subject region framed out from the first image and the subject region framed in the second image captured at a different time from the first image are subject regions due to the same subject. Whether or not is determined based on the extracted information. If the area determination unit 144 determines that the subject area is the same subject, the area determination unit 144 selects the latest imaging information associated with the subject area of the same subject.

  The area determination unit 144 also includes a subject area framed out from the first image and a subject area framed in to the second image based on the zoom magnification with respect to the subject area stored in the storage unit 160 as extraction information. It is determined whether or not the subject area is the same subject.

  In addition, the region determination unit 144 determines that the subject region framed out from the first image when the time from the time when the first image is captured to the time when the second image is captured is within a predetermined time. Whether the subject area framed in the second image is the subject area of the same subject is determined based on the extracted information. In addition, the area determination unit 144 determines that the subject area framed out from the first image and the subject area framed in the second image have the same subject based on the degree to which the extracted information matches between the subject areas. It is determined whether it is an area.

  Returning to FIG. 1, the description of the area determination device 141 will be continued. The priority setting unit 145 determines “region priority” in an image for a subject region that satisfies a predetermined condition (for example, the area is equal to or greater than a threshold). The “region priority” is a region priority that a person will be more interested in. The priority setting unit 144 associates the set “region priority” with the subject region and causes the storage unit 160 to store the attribute information (extraction information, imaging information).

  The priority of the area may be determined as follows. In the following, it is assumed that the priority of a region is higher as the number indicating the priority is smaller. Moreover, the priority shown below is an example.

Priority 1.
A moving area (when there are a plurality of moving areas in the image, an area that does not protrude from the angle of view or an area that is distributed in the center of the angle of view). Also, if the moving area is widely distributed in the image due to panning, the area is not moving.
These areas are set to priority “1”.
This is because a person pays attention to a moving area and an area having a small amount of change when panning.

  Note that if it is determined that a subject area that has been set to priority 1 until it is framed out of the image has been framed in the image again, priority 1 may be set for the subject area. .

Priority 2.
Areas related to angle of view (composition), depth (perspective), and subject distance (depth, depth). An area distributed near the center of the angle of view. When there is a depth in the image, among the regions distributed on the closest side, the region does not protrude from the angle of view and has an area within a predetermined range (not extremely large).
These areas are set to priority “2”.
This is because the main subject image area may exist in a wide range from close to infinity.

Priority 3.
Regions extracted based on human face detection, animal (pet) face detection, skin color detection in the area where the face is detected, animal fur texture (pattern), or plant texture.
These areas are set to priority “3”.
This is because when a person or animal is imaged, the person pays attention to the face.

Priority 4
A region having a characteristic amount that is unique (for example, a hue that is discontinuous in the hue circle). An area divided into a background area and other areas. Highly saturated area. An area where the brightness is significantly different from the surrounding area. An area where the hue (hue) is significantly different from the surrounding area. An area where the texture (pattern) is different from the surrounding area. An area where the spatial frequency (image frequency) is higher than the surrounding area. A region in which an image has a sense of incongruity when a certain amount of discontinuous distribution has a certain amount and a region is formed as a result of labeling based on the feature amount.
These areas are set to priority “4”.

Priority 5.
An area extracted based on a specific hue or saturation.
These areas are set to the priority “5”.
This is because a person pays attention to red, yellow, and flesh-colored areas.

Priority 6.
An area extracted based on brightness (brightness).
These areas are set to priority “6”.

Priority 7.
An area extracted based on a texture (pattern).
These areas are set to priority “7”.

Priority 8.
A region extracted based on the spatial frequency (image frequency).
These areas are set to priority “8”.

Priority 9.
An area surrounded by edges extending vertically or horizontally and having a certain thickness or more.
These areas are set to the priority “9”.

Next, the processing procedure of the area determination device will be described.
<When the imaging information of the subject area is used for shooting settings (for example, focus adjustment (contrast AF))>
FIG. 2 is a flowchart (first half) showing an operation procedure for executing contrast AF based on imaging information of a subject area. FIG. 3 is a flowchart (second half) showing an operation procedure for executing contrast AF based on imaging information of the subject area. FIG. 4 shows a diagram for explaining an operation procedure for executing contrast AF based on imaging information of the subject area.

  The imaging unit 110 (see FIG. 1) transitions to the moving image mode and images a moving image format image sequence (step S1). The CPU 190 (see FIG. 1) executes focus (AF) drive (for example, wobbling) in the moving image mode as temporary focus drive (step S2). The area determination unit 144 determines whether or not the imaging apparatus 100 has moved (for example, boom-up, boom-down) or has changed its direction (for example, panning or tilting) (step S3).

  When the imaging apparatus 100 has not moved and the orientation has not been changed (step S3-No), the region extraction unit 143 selects a subject region based on the image feature amount previously calculated by the feature amount calculation unit 142. The attribute information (extraction information, imaging information) is calculated by extraction, and the calculated attribute information is stored in the storage unit 160 (step S4). The region extraction unit 143 determines whether or not a valid subject region (for example, the area is equal to or greater than a threshold value) exists in the image.

  When there are valid subject areas in the image (step S5-Yes) and there are a plurality of valid subject areas, the priority setting unit 145 (see FIG. 1) gives priority to these valid subject areas. Is determined (step S6). The CPU 190 executes contrast AF for the subject area with the highest priority. In this flowchart, the description will be continued assuming that the subject area with the highest priority is defined as the subject area 400 (see FIG. 4A). Note that an evaluation area 300 that is an area in which a contrast evaluation value is calculated may be defined in the subject area 400 (step S7).

  The region extraction unit 143 stores information indicating the in-focus position with respect to the subject region 400 in the storage unit 160 as attribute information (imaging information) determined by the subject region 400 (step S8). The feature amount calculation unit 142 captures subsequent image frames in the image sequence (step S9). The area extraction unit 143 extracts subject areas based on the image feature amount previously calculated by the feature amount calculation unit 142 to calculate attribute information (extraction information and imaging information), and stores the calculated attribute information in the storage unit 160. Store (step S10).

  The area determination unit 144 determines whether or not the subject area 400 is within the specified range of the image. Here, the specified range is, for example, a peripheral range around the position of the subject region 400 in the previously read image or a range defined slightly inside the image frame (view angle) (the specified range in FIG. 4B). (See range 600) (step S11).

  When the subject region 400 is not within the specified range of the image (step S11—No), the region determination unit 144 determines whether the imaging device 100 has moved or changed its orientation (step S12). When the imaging device 100 has not moved and the orientation has not been changed (step S12—No), the feature amount calculation unit 142 captures the subsequent image frame in the image sequence (step S13). The area extraction unit 143 extracts subject areas based on the image feature amount previously calculated by the feature amount calculation unit 142 to calculate attribute information (extraction information and imaging information), and stores the calculated attribute information in the storage unit 160. Store (step S14).

  The area determination unit 144 (see FIG. 1) determines whether or not an effective subject area exists in the image (step S15). When an effective subject area exists in the image (step S15—Yes), the area determination unit 144 determines whether or not there is a high possibility that the effective subject area in the image is the subject area 400 (existing label). To do. Here, when the imaging apparatus 100 includes the sensor 500 capable of measuring the orientation, the region determination unit 144 is present in the image based on whether the orientation of the subject with respect to the imaging apparatus 100 matches. It is determined whether there is a high possibility that the effective subject area is the subject area 400.

  The method for determining whether or not there is a high possibility that an effective subject area in the image is an existing label is not limited to the above method. For example, the area determination unit 144 determines whether or not there is a high possibility that the effective subject area in the image is the subject area 400 when the imaging apparatus 100 is not provided with the sensor 500 capable of measuring the orientation. May be determined based on the region shape (step S16).

  When there is a high possibility that an effective subject area in the image is the subject area 400 (existing label) (step S16—Yes), the area determination unit 144 stores attribute information (extraction information) determined by the subject area 400. Reading from the unit 160 (step S17). Then, the area determination unit 144 calculates the degree of approximation between the attribute information (extraction information) determined by the subject area 400 and the attribute information (extraction information) determined by the extracted area, and the calculated degree of approximation is higher than a predetermined threshold value. Is determined (step S18).

  When the degree of approximation is higher than the predetermined threshold (step S18—Yes), the CPU 190 moves the AF lens 112 (see FIG. 1) to the in-focus position with respect to the subject region 400. The focus position, which is attribute information (imaging information), is used for the movement of the AF lens 112 in this way (step S19). The feature amount calculation unit 142 captures subsequent image frames in the image sequence (step S20). The area extraction unit 143 extracts subject areas based on the image feature amounts calculated by the feature amount calculation unit 142 to calculate attribute information (extraction information, imaging information), and stores the calculated attribute information in the storage unit 160. (Step S21).

  The region determination unit 144 determines whether or not the subject region 400 that has been given the highest priority in the previous time is within the specified range of the image (step S22). When the subject area 400 that was the highest priority last time is not within the specified range of the image (step S22—No), the area determination unit 144 determines whether the imaging apparatus 100 has moved or changed its orientation (step S23). ). When the imaging device 100 has not moved and the orientation has not been changed (step S23—No), the feature amount calculation unit 142 returns the process to step 13.

  On the other hand, in step S3, when the imaging device 100 is moving or changing the direction (step S3-Yes), the region determination unit 144 repeats the determination process in step S3. In step S5, if there is no effective subject area in the image (step S5-No), the CPU 190 returns the process to step 2. In step S11, when the subject region 400 is within the specified range of the image (step S11-Yes), the feature amount calculation unit 142 returns the process to step 9.

  In Step S12, when image pick-up device 100 has moved or changed the direction (Step S12-Yes), region judgment part 144 repeats judgment processing in Step S12. In step S15, when there is no effective subject area in the image (step S15-No), and when a predetermined time has elapsed, the CPU 190 executes contrast AF (step S24). Then, the feature amount calculation unit 142 returns the process to step 13.

  In step S16, if there is no high possibility that the effective subject area in the image is the subject area 400 (existing label) (step S16-No), and there are a plurality of effective subject areas, priority is given. The degree setting unit 145 (see FIG. 1) determines priorities for these effective subject areas (step S25). Further, the CPU 190 executes contrast AF for the subject area having the highest priority. In this flowchart, the description will be continued assuming that the subject region with the highest priority is defined as the subject region 410 (see FIG. 4C). In the subject area 410, an evaluation area 310 that is an area in which a contrast evaluation value is calculated may be defined (step S26). The region extraction unit 143 stores information indicating the in-focus position with respect to the subject region 410 in the storage unit 160 as attribute information (imaging information) determined by the subject region 410 (step S27). Then, the region extraction unit 143 returns the process to step S20.

  In step S18, when the degree of approximation is not higher than the predetermined threshold (step S18-No), the CPU 190 determines the AF lens 112 (see FIG. 1) set near the focus position with respect to the subject region 400. The AF lens 112 is moved before the driving range (step S28). Then, the CPU 190 executes contrast AF in a narrow drive range based on attribute information (imaging information) determined by the subject area 400. The focus position, which is attribute information (imaging information), is used for contrast AF in this way (step S29). Then, the CPU 190 returns the process to step S20.

  In step S22, if the subject area 400 that has been given the highest priority in the previous time is within the specified range of the image (step S22-Yes), the feature amount calculation unit 142 returns the process to step S20.

  FIG. 5 shows a diagram for explaining an operation example in which contrast AF is executed based on imaging information of a subject area. Here, the horizontal axis represents time. The vertical axis represents the position of the AF lens 112.

  First, it is assumed that the CPU 190 executes a contrast scan by calculating a contrast evaluation value while driving the AF lens 112 (see FIG. 1) between time t1 and time t2. Further, it is assumed that an in-focus position A that can be focused on the subject area 400 (see FIG. 4A) is determined. The area extracting unit 143 stores the in-focus position A with respect to the subject area 400 in the storage unit 160 as attribute information (imaging information) determined by the subject area 400.

  At time t <b> 3, the CPU 190 moves the AF lens 112 to the focus position A with respect to the subject area 400. Thereafter, it is assumed that the subject region 400 is out of the frame from the image. Then, the CPU 190 executes a contrast scan, and moves the AF lens 112 to the in-focus position B with respect to the subject area 410 (see FIG. 4C). The area extraction unit 143 stores the in-focus position B with respect to the subject area 410 in the storage unit 160 as attribute information (imaging information) determined by the subject area 410.

  After that, it is assumed that the subject area 400 is again framed into the image. The area determination unit 144 has a subject area 400 of the same subject in which the subject area framed out from the image captured between time t1 and time t2 and the subject area framed in the image captured until time t4 are the same. It is determined based on attribute information (extraction information). Here, it is assumed that the region determination unit 144 determines that the subject region 400 is the same subject.

  Between time t4 and time t5, the CPU 190 focuses on the subject region 400 (see FIG. 4A) based on the attribute information (imaging information = focus position A) without executing contrast scan. The AF lens 112 is moved to the position A.

<When focusing on multiple subject areas in an image sequentially>
FIG. 6 is a flowchart showing an operation procedure for sequentially executing contrast AF based on imaging information of the subject area. The imaging unit 110 (see FIG. 1) transitions to the moving image mode and images a moving image format image sequence (step Sa1). The region extraction unit 143 extracts a subject region based on the image feature amount calculated by the feature amount calculation unit 142. Here, it is assumed that the subject areas 400, 410, and 420 (not shown in FIG. 4) are extracted (step Sa2). The area extraction unit 143 calculates attribute information (extraction information, imaging information) of each extracted subject area, and stores the calculated attribute information in the storage unit 160 for each subject area (step Sa3).

  The feature amount calculation unit 142 captures the subsequent image frame in the image sequence (step Sa4). The area extraction unit 143 extracts subject areas based on the image feature amounts calculated by the feature amount calculation unit 142 to calculate attribute information (extraction information, imaging information), and stores the calculated attribute information in the storage unit 160. (Step Sa5). The area determination unit 144 (see FIG. 1) determines whether the subject area (existing label) extracted up to the previous time is present in the image (step Sa6).

  When the subject area (existing label) extracted up to the previous time does not exist in the image (step Sa6-No), the area determination unit 144 determines whether the imaging device 100 has moved or changed its direction (step Sa7). When the imaging apparatus 100 has not moved and the orientation has not been changed (step Sa7-No), the area determination unit 144 determines whether or not the subject area focused last time is in the first central area. That is, it is determined whether or not the area is closest to the center of the angle of view (step Sa8).

  When the previously focused subject area (existing label) is in the first center area (step Sa8-Yes), the CPU 190 determines the attribute determined by the subject area (for example, the subject area 400) in the first center area. Based on the information (imaging information), the AF lens 112 is moved to the in-focus position for the subject area. The in-focus position, which is attribute information (imaging information), is used for the movement of the AF lens 112 in this way (step Sa9). Then, after waiting for a certain period of time to elapse (step Sa10), the feature amount calculation unit 142 returns the process to step Sa4.

  On the other hand, in step Sa8, when the subject area focused last time is not in the first center area (step Sa8-No), the area determination unit 144 determines whether the subject area focused last time is in the second center area. It is determined whether it is in the area immediately outside the first central area (step Sa11).

  When the previously focused subject area (existing label) is in the second center area (step Sa11-Yes), the CPU 190 determines the attribute determined by the subject area in the second center area (for example, the subject area 410). Based on the information (imaging information), the AF lens 112 is moved to the in-focus position for the subject area. The focus position, which is attribute information (imaging information), is used for the movement of the AF lens 112 in this way (step Sa12). Then, after waiting for a certain period of time to elapse (step Sa10), the feature amount calculation unit 142 returns the process to step Sa4.

  In step Sa11, if the subject area (existing label) focused last time is not in the second central area (step Sa11-No), the CPU 190 exists in the third central area just outside the second central area. The AF lens 112 is moved to the in-focus position for the subject area based on attribute information (imaging information) determined by the subject area (for example, the subject area 420 (not shown in FIG. 4)). The focus position, which is attribute information (imaging information), is used for the movement of the AF lens 112 in this way (step Sa13). Then, after waiting for a certain period of time to elapse (step Sa10), the feature amount calculation unit 142 returns the process to step Sa4.

  In step Sa6, if the subject region (existing label) extracted up to the previous time is present in the image (step Sa6-Yes), the feature amount calculation unit 142 returns the process to step Sa4. In Step Sa7, when the imaging device 100 has moved or changed its direction (Step Sa7-Yes), the region determination unit 144 repeats the determination process in Step Sa7.

<Changing the priority (attention flag) of the subject area based on the action information of the imaging device for the subject area>
FIG. 7 is a flowchart (first half) showing an operation procedure for executing contrast AF based on action information that is imaging information of a subject area. FIG. 8 is a flowchart (continuation of the first half) showing an operation procedure for executing contrast AF based on action information that is imaging information of a subject area. FIG. 9 is a flowchart (second half) showing an operation procedure for executing contrast AF based on action information that is imaging information of a subject area.

  FIG. 10 is a diagram for explaining an operation procedure for executing contrast AF based on action information that is imaging information of a subject area. FIG. 11 is a diagram for explaining an operation procedure for executing contrast AF based on action information that is imaging information of a subject area.

  The imaging unit 110 (see FIG. 1) transitions to the moving image mode and images a moving image format image sequence (step Sb1). The CPU 190 (see FIG. 1) executes focus (AF) drive (for example, wobbling) in the moving image mode as temporary focus drive (step Sb2). The area determination unit 144 determines whether or not the imaging apparatus 100 has moved (for example, boom-up, boom-down) or has changed its direction (for example, panning or tilting) (step Sb3).

  When the imaging device 100 has not moved and the orientation has not been changed (step Sb3-No), the region extraction unit 143 selects a subject region based on the image feature amount previously calculated by the feature amount calculation unit 142. The attribute information (extraction information, imaging information) is calculated by extraction, and the calculated attribute information is stored in the storage unit 160 (step Sb4). The region extraction unit 143 determines whether or not a valid subject region (for example, the area is equal to or greater than a threshold) is present in the image (step Sb5).

  When there are valid subject areas in the image (step Sb5-Yes) and there are a plurality of valid subject areas, the priority setting unit 145 (see FIG. 1) gives priority to these valid subject areas. Is determined (step Sb6). The CPU 190 executes contrast AF for the subject area with the highest priority. In this flowchart, the description will be continued assuming that the subject area with the highest priority is defined as the subject area 420 (see FIG. 10A). In the subject area 420, an evaluation area 320 that is an area in which a contrast evaluation value is calculated may be defined (step Sb7).

  The area extraction unit 143 stores information indicating the in-focus position with respect to the subject area 420 in the storage unit 160 as attribute information (imaging information) determined by the subject area 420 (step Sb8). The feature amount calculation unit 142 captures subsequent image frames in the image sequence (step Sb9). The area extraction unit 143 extracts subject areas based on the image feature amount previously calculated by the feature amount calculation unit 142 to calculate attribute information (extraction information and imaging information), and stores the calculated attribute information in the storage unit 160. Store (step Sb10). The region determination unit 144 determines whether or not the subject region 420 is within the specified range of the image (step Sb11).

  When the subject area 420 is not within the specified range of the image (step Sb11-No), the area determination unit 144 determines whether the imaging apparatus 100 has moved or changed its orientation (step Sb12). When the imaging device 100 has not moved and the orientation has not been changed (step Sb12-No), the feature amount calculation unit 142 captures the subsequent image frame in the image sequence (step Sb13). The area extraction unit 143 extracts subject areas based on the image feature amount previously calculated by the feature amount calculation unit 142 to calculate attribute information (extraction information and imaging information), and stores the calculated attribute information in the storage unit 160. Store (step Sb14).

  The area determination unit 144 (see FIG. 1) determines whether an effective subject area that is not the subject area 420 exists in the image. In this flowchart, the description is continued assuming that an effective subject area that is not the subject area 420 is defined as the subject area 430 (see FIG. 10B) (step Sb15). When an effective subject area exists in the image (step Sb15-Yes), the feature amount calculation unit 142 captures the subsequent image frame in the image sequence (step Sb16).

  The area extraction unit 143 extracts subject areas based on the image feature amount previously calculated by the feature amount calculation unit 142 to calculate attribute information (extraction information and imaging information), and stores the calculated attribute information in the storage unit 160. Store (step Sb17). The region determination unit 144 determines whether or not the subject region 430 is present in the image (step Sb18).

  When the subject region 430 (see FIG. 10C) is not present in the image (step Sb18—No), the region determination unit 144 determines whether the imaging device 100 has moved or changed its direction (step Sb19). . When the imaging apparatus 100 has not moved and the orientation has not been changed (step Sb19-No), the feature amount calculation unit 142 captures the subsequent image frame in the image sequence (step Sb20).

  The area extraction unit 143 extracts subject areas based on the image feature amounts calculated by the feature amount calculation unit 142 to calculate attribute information (extraction information, imaging information), and stores the calculated attribute information in the storage unit 160. (Step Sb21). The area determination unit 144 (see FIG. 1) determines whether or not an effective subject area exists in the image (step Sb22). When the effective subject area exists in the image (step Sb22-Yes), the area determination unit 144 determines whether the effective subject area in the image is the subject area 420 or the subject area 430 (step Sb23). ).

  When the effective subject area in the image is not the subject area 420 and not the subject area 430 (step Sb23-No), the feature amount calculation unit 142 captures the subsequent image frame in the image sequence (step Sb24).

  The area extraction unit 143 extracts subject areas based on the image feature amounts calculated by the feature amount calculation unit 142 to calculate attribute information (extraction information, imaging information), and stores the calculated attribute information in the storage unit 160. Let In this flowchart, the description will be continued assuming that the moving subject area 440 (see FIG. 11A) has been extracted. Here, when the subject area 440 is moving, the area determination unit 144 determines whether or not the imaging apparatus 100 is tracking the subject area 440 by moving or changing the direction (see FIG. 11B). (See FIG. 11C) is determined (step Sb25).

  When the imaging apparatus 100 is tracking the subject area 440 by changing the movement or orientation (step Sb25-Yes), the area determination unit 144 determines whether the subject area 440 is within the specified range of the image. Determination is made (step Sb26). When the subject area 440 is not within the specified range of the image (step Sb26-No), the area determination unit 144 determines whether the imaging device 100 has moved or changed its direction (step Sb27).

  When the imaging device 100 has not moved and the orientation has not been changed (step Sb27-No), the feature amount calculation unit 142 captures the subsequent image frame in the image sequence (step Sb28). The area extraction unit 143 extracts subject areas based on the image feature amounts calculated by the feature amount calculation unit 142 to calculate attribute information (extraction information, imaging information), and stores the calculated attribute information in the storage unit 160. (Step Sb29).

  The area determination unit 144 (see FIG. 1) determines whether or not an effective subject area exists in the image (step Sb30). When there is an effective subject area in the image (step Sb30-Yes), the area determination unit 144 determines whether the effective subject area in the image is the subject area 420, the subject area 430, or the subject area 440. (Step Sb31).

  When the effective subject area in the image is the subject area 420, the subject area 430, or the subject area 440 (step Sb31-Yes), the area determination unit 144 determines whether the imaging apparatus 100 has moved or changed its orientation. Determination is made (step Sb32). When the imaging apparatus 100 has not moved and the orientation has not been changed (step S32—No), the region extracting unit 143 determines that a certain inner region (image area) of the image frame is present if a plurality of subject regions are present in the image. For example, the subject area is extracted from the center area) (step Sb33).

  The area extraction unit 143 excludes subject areas that exist outside the fixed inner area of the image frame from the processing target (step Sb34). The area determination unit 144 (see FIG. 1) determines whether or not the effective subject area is only the subject area 430 (see FIG. 10B) (step Sb35).

  When the effective subject area is only the subject area 430 (step Sb35-Yes), the area extracting unit 143 extracts the subject area based on the image feature amount calculated by the feature amount calculating unit 142 and extracts attribute information (extraction). Information, imaging information (for example, infinite focus position, contrast evaluation value of each color) is calculated, and the calculated attribute information is stored in the storage unit 160. The CPU 190 is placed at the focus position with respect to the subject region 430 at the AF lens 112 ( The focus position, which is attribute information (imaging information), is used for the movement of the AF lens 112 in this way (step Sb36).

  The feature amount calculation unit 142 captures subsequent image frames in the image sequence (step Sb37). The area extraction unit 143 extracts subject areas based on the image feature amounts calculated by the feature amount calculation unit 142 to calculate attribute information (extraction information, imaging information), and stores the calculated attribute information in the storage unit 160. (Step Sb38).

  The area determination unit 144 determines whether or not the subject area having the highest priority last time is within the specified range of the image. Here, the prescribed area is a range around the position of the subject area in the previous image frame or a range slightly inside the image frame (view angle) (step Sb39).

  When the subject area that has been given the highest priority in the previous time is not within the specified range of the image (step Sb39-No), the area determination unit 144 determines whether the imaging device 100 has moved or changed its orientation (step Sb40). . When the imaging apparatus 100 has not moved and the orientation has not been changed (step S40—No), the region extraction unit 143 selects a subject region based on the image feature amount previously calculated by the feature amount calculation unit 142. Extract (step Sb41).

  The region extraction unit 143 determines whether or not a valid subject region (for example, the area is equal to or larger than the threshold) is present in the image (step Sb42). When an effective subject area exists in the image (step S42—Yes), the area extraction unit 143 analyzes the subject area, and attribute information (extraction information, imaging information) determined by the analyzed subject area is stored in the storage unit 160. Remember me. The CPU 190 executes contrast AF for the subject area with the highest priority. The focus position that is the attribute information (imaging information) is used for the movement of the AF lens 112 in this way (step Sb43). Then, the region extraction unit 143 proceeds with the process to step Sb3.

  On the other hand, in step Sb11, when the subject area 420 (see FIG. 10A) is within the prescribed range of the image (step Sb11-Yes), and when the subject area 420 is in the central area for a certain time or more. The priority setting unit 145 increases the priority of the subject area 420. Further, the region extraction unit 143 extracts a subject region based on the image feature amount calculated by the feature amount calculation unit 142, calculates attribute information (extraction information, imaging information), and stores the calculated attribute information in the storage unit 160. Remember me. Here, the region extracting unit 143 stores “action information indicating that the subject region is set in the central region”. Further, the region extraction unit 143 sets a value of 1 to the attention flag that is a flag representing the degree of attention of the subject region by the photographer (step Sb44).

  In step Sb18, when the subject region 430 (see FIG. 10C) is present in the image (step Sb18-Yes), and when the subject region 430 is present in the image is equal to or shorter than the predetermined time, The priority setting unit 145 lowers the priority of the subject area 430. Further, the region extraction unit 143 extracts a subject region based on the image feature amount calculated by the feature amount calculation unit 142, calculates attribute information (extraction information, imaging information), and stores the calculated attribute information in the storage unit 160. Remember me. Here, the region extracting unit 143 stores “action information indicating that the subject region is ignored”. Further, the region extraction unit 143 sets a value of 0 to the attention flag, which is a flag indicating the degree of attention of the subject region by the photographer (step Sb45).

  In step Sb26, when the subject region 440 (see FIG. 11A) is within the prescribed range of the image (step Sb26-Yes), the region determination unit 144 measures the time that the region 440 is in the image. Whether or not the subject area 440 is tracked is determined based on the measurement result.

  When the imaging apparatus 100 (see FIG. 1) is tracking the subject area 440, the priority setting unit 145 decreases the priority of the subject area 430. Further, the region extraction unit 143 extracts a subject region based on the image feature amount calculated by the feature amount calculation unit 142, calculates attribute information (extraction information, imaging information), and stores the calculated attribute information in the storage unit 160. Remember me. Here, the area extraction unit 143 stores “action information indicating that tracking is being performed”. In addition, the region extraction unit 143 sets a value of 2 to the attention flag, which is a flag indicating the degree of attention of the subject region by the photographer (step Sb46).

  In step Sb30, if an effective subject area exists in the image (step Sb30-No), the CPU 190 executes contrast AF for the subject area having the highest priority (step Sb47).

  In step Sb35, when the effective subject area is not the subject area 430 (step Sb35-No), the area determination unit 144 determines that the effective subject area is only the subject area 420 (see FIG. 10A), or It is determined whether or not the effective subject areas are the subject area 420 and the subject area 430 (step Sb48).

  When the effective subject area is only the subject area 420, or when the effective subject areas are the subject area 420 and the subject area 430 (step Sb48-Yes), the area extracting unit 143 calculates the image feature calculated by the feature amount calculating unit 142. Based on the amount, the subject area 420 is extracted to calculate attribute information (extraction information, imaging information), and the calculated attribute information is stored in the storage unit 160. The CPU 190 moves the AF lens 112 (see FIG. 1) to a focus position with respect to the subject area 420. The focus position that is the attribute information (imaging information) is used for the movement of the AF lens 112 in this way (step Sb49).

  On the other hand, in step Sb48, when the effective subject area is not only the subject area 420 and the effective subject areas are not the subject area 420 and the subject area 430 (step Sb48-No), the area determination unit 144 It is determined whether or not only the subject area 440 (see FIG. 11) or the effective subject areas are the subject area 420, the subject area 430, and the subject area 440 (step Sb50).

  When the effective subject area is only the subject area 440, or when the effective subject areas are the subject area 420, the subject area 430, and the subject area 440 (step Sb50-Yes), the area extracting unit 143 includes the feature amount calculating unit 142. Based on the calculated image feature amount, the subject area 420 is extracted to calculate attribute information (extraction information, imaging information), and the calculated attribute information is stored in the storage unit 160. The CPU 190 moves the AF lens 112 (see FIG. 1) to a focus position with respect to the subject area 440. The focus position that is the attribute information (imaging information) is used for the movement of the AF lens 112 in this way (step Sb51).

  On the other hand, when the effective subject area is not only the subject area 440 and the effective subject areas are not the subject area 420, the subject area 430, and the subject area 440 in step Sb50 (step Sb50-No), the area extracting unit 143 and the CPU 190 Advances the process to step Sb43.

  In Step Sb3, when the imaging device 100 has moved or changed its direction (Step Sb3-No), the region determination unit 144 repeats the determination process in Step Sb3. In step Sb5, if there is no effective subject area in the image (step Sb5-No), the CPU 190 (see FIG. 1) returns the process to step Sb2.

  In step Sb12, when the imaging apparatus 100 has moved or changed the direction (step Sb12-Yes), the region determination unit 144 repeats the determination process in step Sb12. In step Sb15, when there is no effective subject area in the image (step Sb15-No), the feature amount calculating unit 142 returns the process to step Sb13.

  In step Sb19, when the imaging device 100 has moved or changed the direction (step Sb19-Yes), the region determination unit 144 repeats the determination process in step Sb19. In step Sb22, when there is no effective subject area in the image (step Sb22-No), the feature amount calculating unit 142 returns the process to step Sb20.

  In step Sb23, when the effective subject area in the image is the subject area 420 or the subject area 430 (step Sb23-Yes), the area determination unit 144 advances the process to step Sb32. In step Sb32, when the imaging device 100 has moved or changed the direction (step S32—Yes), the region determination unit 144 repeats the determination process in step Sb32.

  In step Sb39, when the subject area that has been given the highest priority in the previous time is within the specified range of the image (step Sb39-Yes), the feature amount calculation unit 142 returns the process to step Sb37. In Step Sb40, when image pick-up device 100 has moved or changed the direction (Step S40-Yes), region determination unit 144 repeats the determination process in Step Sb40. In step Sb42, when there is no effective subject area in the image (step S42-No), the area extraction unit 143 returns the process to step Sb41.

<When the zoom magnification is changed after the subject area is extracted>
FIG. 12 is a flowchart illustrating an operation procedure for executing contrast AF based on imaging information of the subject area when the zoom magnification is changed. The imaging unit 110 (see FIG. 1) transitions to the moving image mode and images a moving image format image sequence (step Sc1). The region extraction unit 143 extracts a subject region based on the image feature amount calculated by the feature amount calculation unit 142. Here, it is assumed that the subject region 400 (see FIG. 4A) has been extracted (step Sc2). The area extraction unit 143 calculates the attribute information (extraction information, imaging information) of the extracted subject area 400, and stores the calculated attribute information in the storage unit 160 in association with the subject area 400 (step Sc3). Then, it is assumed that the subject area 400 is out of frame from the image.

  The feature amount calculation unit 142 takes in the subsequent image frame in the image sequence (step Sc4). The area extraction unit 143 acquires the zoom magnification from the CPU 190 and stores the acquired zoom magnification (step Sc5). The region extraction unit 143 extracts a subject region based on the image feature amount calculated by the feature amount calculation unit 142. The region extraction unit 143 calculates attribute information (extraction information, imaging information) of the extracted subject region, and stores the calculated attribute information in the storage unit 160 for each subject region (step Sc6).

  The region extraction unit 143 converts the extraction information (for example, the size and position of the subject region) according to the stored zoom magnification. For example, when the extracted information represents the size of the subject area, the area extraction unit 143 converts the size of the subject area (for example, multiplies or divides the zoom magnification) according to the stored square of the zoom magnification (step Sc7). . The region determination unit 144 determines whether an effective subject region including the subject region (existing label) extracted up to the previous time is in the prescribed region of the image. Here, the prescribed region is a region around the position converted according to the zoom magnification of the subject region in the previous image frame, or a range slightly inside the image frame (view angle) converted according to the zoom magnification (step) Sc8). When an effective subject area including the subject area extracted up to the previous time is in the prescribed area of the image (step Sc8-Yes), the area determination unit 144 determines whether the imaging apparatus 100 has moved or changed its orientation. Determination is made (step Sc9).

  If the imaging apparatus 100 has not moved and the orientation has not been changed (step Sc10-No), if the effective subject area is the subject area 400, the CPU 190 determines attribute information (imaging information) determined by the subject area 400. ), The AF lens 112 is moved to the in-focus position with respect to the subject area 400. On the other hand, if the effective subject area is other than the subject area 400, the area extracting unit 143 updates attribute information (imaging information) determined by the subject area, and stores the updated attribute information (imaging information) in the storage unit 160. Let

  In addition, the feature amount calculation unit 142 captures a subsequent image frame in the image sequence (step Sc11). The region extraction unit 143 extracts a subject region based on the image feature amount calculated by the feature amount calculation unit 142. The area extraction unit 143 calculates attribute information (extraction information, imaging information) of the extracted subject area, and stores the calculated attribute information in the storage unit 160 for each subject area (step Sc12).

  The region determination unit 144 determines whether or not the subject region 400 is within the specified range of the image (step Sc13). When the subject region 400 is within the specified range of the image (step Sc13—Yes), the region determination unit 144 determines whether the imaging device 100 has moved or changed its orientation (step Sc14). When the imaging apparatus 100 has not moved and the orientation has not changed (step Sc14-No), the region extraction unit 143 extracts a subject region based on the image feature amount calculated by the feature amount calculation unit 142. (Step Sc15).

  The region determination unit 144 determines whether or not a valid subject region exists in the image (step Sc16). When there is an effective subject area in the image (step Sc16-Yes), the area extraction unit 143 analyzes the subject area, and attribute information (extraction information, imaging information) determined by the analyzed subject area is stored in the storage unit 160. Remember. The CPU 190 executes contrast AF for the subject area with the highest priority. The in-focus position, which is attribute information (imaging information), is used for the movement of the AF lens 112 in this way (step Sc17). Then, the region extraction unit 143 returns the process to Step Sc4.

  On the other hand, in step Sc8, when there is no effective subject area including the subject area extracted so far in the prescribed area of the image (step Sc8-No), the feature amount calculation unit 142 advances the process to step Sc4. In Step Sc9, when the imaging device 100 has moved or changed its direction (Step Sc9-Yes), the region determination unit 144 repeats the determination process in Step Sc9. In step Sc13, when the subject area 400 is not within the specified range of the image (step Sc13-No), the feature amount calculation unit 142 returns the process to step Sc11.

  In Step Sc14, when the imaging apparatus 100 has moved or changed its direction (Step Sc14-Yes), the region determination unit 144 repeats the determination process in Step Sc14. In Step Sc16, when there is no effective subject area in the image (Step Sc16-No), the area extracting unit 143 returns the process to Step Sc15.

<When calculating the contrast evaluation value (adding adjacent pixel differences), storing the color channel with the maximum contrast evaluation value and utilizing the contrast evaluation value of that color channel for the next contrast AF>
FIG. 13 is a flowchart showing an operation procedure for executing contrast AF based on a color channel which is subject area extraction information. FIG. 14 shows the relationship between the AF lens position and the contrast evaluation value. Here, the horizontal axis represents the AF lens position. The vertical axis represents the contrast evaluation value. In FIG. 14, the peak value of the contrast evaluation value is assumed to be higher in the order of the red (R) channel, the green (G) channel, and the blue (B) channel.

  Returning to FIG. 13, the description of the operation procedure is continued. The imaging unit 110 (see FIG. 1) transitions to the moving image mode and images a moving image format image sequence (step Sd1). The CPU 190 (see FIG. 1) executes focus (AF) drive (for example, wobbling) in the moving image mode as temporary focus drive (step Sd2). The CPU 190 executes contrast AF on the contrast evaluation value window defined for the image. The area extraction unit 143 stores information indicating the color channel with the maximum contrast evaluation value, the rank of the contrast evaluation values, and the ratio of the contrast evaluation values for each object area as attribute information (extraction information). The area extraction unit 143 stores the in-focus position in the storage unit 160 for each subject area as attribute information (imaging information) (step Sd3).

  The area determination unit 144 determines whether or not a certain time has elapsed since the exposure amount has changed significantly (step Sd4). When a certain time has elapsed since the exposure amount has changed significantly (step Sd4-Yes), the area determination unit 144 determines whether the similarity between the stored focus position and the focus position determined by the subject area is high. Determine whether.

  Here, for example, when the order of the peak values of the contrast evaluation values is R> G> B, it is determined that the contrast evaluation values are similar. Further, for example, the contrast evaluation values may be determined to be similar based on the peak values (R / G) and (B / G) of the contrast evaluation values. Further, for example, it may be determined that the contrast evaluation values are similar based on the RGB statistics (for example, average value, variance) in the contrast evaluation value window defined for the image (step Sd5).

  When the similarity between the stored in-focus position and the in-focus position determined by the subject area is high (step Sd5-Yes), the CPU 190 determines the peak of the contrast evaluation value (in-focus) based on the stored in-focus position. Contrast AF (focus drive) limited to the vicinity of position P1) is executed. The focus position, which is attribute information (imaging information), is used for the movement of the AF lens 112 in this way (step Sd6). Then, the area determination unit 144 returns the process to step Sd4.

  On the other hand, in step Sd4, when a certain amount of time has not elapsed since the exposure amount changed significantly (step Sd4-No), the region determination unit 144 repeats the determination process in step Sd4. In step Sd5, when the similarity between the stored focus position and the focus position determined by the subject area is not high (step Sd5-No), the CPU 190 returns the process to step Sd2.

As described above, the area determination device 141 extracts subject areas from image sequences captured at different times based on the image feature amounts, and whether the extracted subject areas are subject areas based on the same subject. A region extracting unit 143 that associates the extracted information indicating the condition for determining whether or not the imaging information indicating the setting for capturing the subject region with the extracted subject region, and the first image of the image sequence. It is determined based on the extracted information whether the subject area framed out and the subject area framed in the second image of the image sequence are subject areas based on the same subject. If it is determined that there is a frame out from the first image as the imaging information for imaging the subject area framed in the second image Comprising the the area determination unit 144 for selecting the imaging information associated with the object region.
With this configuration, when the region determination unit 144 determines that the subject region is the same subject, the region determination unit 144 has framed out the first image as the imaging information for capturing the subject region framed in the second image. The imaging information associated with the subject area is selected. Thereby, the area determination device 141 can accurately determine the subject area framed in the captured image, and appropriately capture the subject based on the selected imaging information.

Further, the area extracting unit 143 associates the focus position of the subject area with the subject area as the imaging information.
Thereby, the area determination device 141 can capture an image of a subject area focused in a short time based on the selected focus position.

Further, the region extraction unit 143 associates at least one of the exposure amount, the gain, and the gradation curve with the subject region as the imaging information.
Thereby, the area determination device 141 can image a subject area that is appropriately exposed based on at least one of the selected exposure amount, gain, and gradation curve.

In addition, the area extraction unit 143 associates action information representing at least one of the operation, movement, and orientation of the imaging device with respect to the subject as the imaging information with the subject area.
Thereby, the area determination device 141 can capture an image of a subject area appropriately tracked based on the selected action information.

In addition, the region extraction unit 143 associates the orientation of the imaging device measured by the sensor with the subject region as the extraction information.
Thereby, the area determination device 141 can accurately determine whether or not the subject region framed in the captured image is the same subject, based on the orientation of the imaging device.

The area extracting unit 143 associates the color information of the captured subject area with the subject area as the extraction information.
Thereby, the area determination device 141 can accurately determine whether the subject area framed in the captured image is the same subject or not based on the color information of the subject area.

Further, the region determination unit 144 determines whether or not the subject region is the same subject based on the zoom magnification that is the extraction information.
Thereby, the area determination device 141 can accurately determine whether the subject area framed in the captured image is the same subject or not based on the zoom magnification.

In addition, the area determination unit 144 determines whether the area of the same subject is within a predetermined time from the time when the first image is captured to the time when the second image is captured. Determine whether or not.
Thereby, the area determination device 141 can accurately determine the subject area that is framed in the captured image without being based on an image whose imaging time interval is too far.

Further, the area determination unit 144 determines whether or not the extracted information is a subject area of the same subject based on the degree to which the extracted information matches between the subject areas.
Thereby, the area determination device 141 accurately determines whether or not the subject area framed in the captured image is the same subject based on the degree to which the extracted information matches between the subject areas. can do.

In addition, the imaging apparatus 100 includes an area determination device, a focus adjustment unit that focuses the subject area using an optical system based on the focus position that is the imaging information selected by the area determination apparatus, and a focus on the subject area. An imaging unit that captures an image in a state of being performed.
With this configuration, the CPU 190 uses the optical system (AF lens 112) to focus on the subject area based on the stored imaging information (focus position). Thereby, the imaging apparatus 100 can capture an image in a state where the subject area determined by the area determination apparatus 141 with high accuracy is in focus.

In addition, the imaging unit 110 captures an image according to an exposure amount that is the imaging information selected by the area determination device.
With this configuration, the CPU 190 controls the exposure amount in the image sensor 119 based on the stored imaging information (exposure amount). Thereby, the imaging device 100 can capture an image in a state where the subject region determined with high accuracy by the region determination device 141 is optimally exposed.

Further, the CPU 190 executes a contrast scan based on the color information that is the extraction information.
With this configuration, the CPU 190 executes contrast scanning (contrast AF) based on the stored color information (color channel). Thereby, the imaging apparatus 100 can capture an image in a state where the subject area determined by the area determination apparatus 141 with high accuracy is in focus.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes designs and the like that do not depart from the gist of the present invention.

  For example, the attribute information (imaging information) may be used not only for focus adjustment but also for shooting settings such as exposure adjustment and white balance adjustment.

  Further, by recording the program for realizing the area determination device and the imaging device described above on a computer-readable recording medium, causing the computer system to read and execute the program recorded on the recording medium, Execution processing may be performed. Here, the “computer system” may include an OS and hardware such as peripheral devices.

  Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used. The “computer-readable recording medium” means a flexible disk, a magneto-optical disk, a ROM, a writable nonvolatile memory such as a flash memory, a portable medium such as a CD-ROM, a hard disk built in a computer system, etc. This is a storage device.

Further, the “computer-readable recording medium” refers to a volatile memory (for example, DRAM (Dynamic) in a computer system serving as a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. Random Access Memory)) that holds a program for a certain period of time is also included.
The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line.
The program may be for realizing a part of the functions described above.
Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, and what is called a difference file (difference program) may be sufficient.

DESCRIPTION OF SYMBOLS 100 ... Imaging device, 110 ... Imaging part, 111 ... Lens barrel, 140 ... Image processing part, 141 ... Area | region determination apparatus, 142 ... Feature-value calculation part, 143 ... Area extraction part, 144 ... Area | region determination part, 145 ... Priority Degree setting unit, 160 ... storage unit, 190 ... CPU, 500 ... sensor

Claims (13)

Extraction information indicating conditions for extracting subject areas from image sequences captured at different times based on image feature amounts and determining whether the extracted subject areas are subject areas of the same subject. And an area extraction unit that associates imaging information indicating settings for imaging the subject area with each of the extracted subject areas;
Based on the extracted information, it is determined whether the subject area framed out from the first image in the image sequence and the subject area framed in the second image in the image sequence are subject areas of the same subject. When it is determined that the subject area is the same subject, the imaging information for capturing the subject area framed in the second image is associated with the subject area framed out from the first image. An area determination unit for selecting imaging information;
An area determination apparatus comprising:   The region determination apparatus according to claim 1, wherein the region extraction unit associates a focus position of the subject region with the subject region as the imaging information.   The region determination apparatus according to claim 1, wherein the region extraction unit associates at least one of an exposure amount, a gain, and a gradation curve with the subject region as the imaging information.   4. The method according to claim 1, wherein the region extraction unit associates action information representing at least one of an operation, a movement, and a direction of the imaging apparatus with respect to the subject as the imaging information with the subject region. The area determination device according to one item.   The region determination device according to claim 4, wherein the region extraction unit associates the orientation of the imaging device measured by the sensor with the subject region as the extraction information.   6. The region determination device according to claim 1, wherein the region extraction unit associates color information of the captured subject region with the subject region as the extraction information.   The said area | region determination part determines whether it is a subject area | region of the said same subject based on the zoom magnification which is the said extraction information, It is any one of Claims 1-6 characterized by the above-mentioned. Area determination device.   The region determination unit determines whether or not the region is the subject region of the same subject when the time from when the first image is captured to the time when the second image is captured is within a predetermined time. The region determination device according to any one of claims 1 to 7, wherein the region determination device includes:   9. The region determination device according to claim 8, wherein the region determination unit determines whether or not the extracted information is a subject region of the same subject based on a degree of coincidence between the subject regions. . The region determination device according to any one of claims 1 to 9,
A focus adjustment unit that focuses on a subject area by an optical system based on a focus position that is the imaging information selected by the area determination device;
An imaging unit that captures an image focused on the subject area;
An imaging apparatus comprising:   The imaging apparatus according to claim 10, wherein the imaging unit captures an image according to an exposure amount that is the imaging information selected by the area determination device.   The imaging apparatus according to claim 10, wherein the focus adjustment unit executes a contrast scan based on color information that is the extraction information. On the computer,
Extraction information indicating conditions for extracting subject areas from image sequences captured at different times based on image feature amounts and determining whether the extracted subject areas are subject areas of the same subject. And a procedure for associating imaging information indicating settings for imaging a subject area with each of the extracted subject areas;
Based on the extracted information, it is determined whether the subject area framed out from the first image in the image sequence and the subject area framed in the second image in the image sequence are subject areas of the same subject. When it is determined that the subject area is the same subject, the imaging information for capturing the subject area framed in the second image is associated with the subject area framed out from the first image. A procedure for selecting imaging information;
An area determination program for executing

Images