JP5395650B2 - Subject area extraction device and control method thereof, subject tracking device, and program - Google Patents

Description

  The present invention relates to a subject area extraction device, a control method thereof, a subject tracking device, and a program.

  An image processing method for detecting a specific subject from images sequentially supplied in time series and tracking the detected subject is very useful, for example, for identifying a human body region of a human face region in a moving image. ing. This image processing method can be used in many fields such as teleconferencing, man-machine interface, security, monitor system for tracking a specific subject, image compression and the like. Also, with digital cameras and digital video cameras, by specifying and tracking an arbitrary subject from a captured image using a touch panel or the like, a specific subject is detected from images sequentially supplied in time series, and the result is controlled. The focus and exposure are optimized as targets.

  For example, Patent Document 1 discloses a photographing apparatus that detects the position of a face in an image, focuses on the face, and photographs the face with an optimal exposure. In addition, tracking the detected face enables stable control in time series. Patent Document 2 describes a technique using a template matching technique as a technique for automatically tracking a specific subject. Template matching is a technique for registering a partial image obtained by cutting out an image region to be tracked as a reference image (template image), estimating a region having the highest correlation with the reference image in the image, and tracking a specific subject. .

JP 2005-318554 A JP 2001-060269 A

  In Patent Document 2, the subject is tracked based on the degree of correlation between the reference image (template image) and the supplied image. As a reference image registration method in the subject tracking, an arbitrary area in an image is designated using a touch panel or the like, and the reference image is registered with the designated area as a reference. By specifying an arbitrary area in the image using a touch panel or the like, an arbitrary subject can be tracked.

  Here, when the designated area is deviated from the center of gravity of the target tracking target, an image different from the target tracking target such as the background is included in the reference image. When a reference image in which an image different from a target tracking target such as a background is used is used, an erroneous region may be tracked due to the influence of an image different from the tracking target.

  The present invention has been made in view of the above problems, and realizes a technique for determining and extracting a target subject region from a designated region when an arbitrary subject in the image is designated.

  In order to solve the above-described problem, the subject area extraction apparatus of the present invention is a subject area extraction apparatus that extracts a predetermined subject area from an image, and designates an area that includes the predetermined subject from the image. Means for extracting a feature quantity of the predetermined subject from the area designated by the designation means, and distribution of feature pixels matching the feature quantity of the predetermined subject included in the designated area Detection means for detecting a deviation in the position of the predetermined subject in the designated area based on the situation, and detection of the position in the designated area so that the deviation in the position of the predetermined subject in the designated area is reduced Correction means for correcting the position, and the extraction means uses the hue component and / or the brightness component of each pixel in the designated area. Extracts the amount, by changing the weighting of the hue component and the luminance component in accordance with the color information obtained from the subject to extract the feature quantity.

  According to the present invention, it is possible to extract a target subject even when there is a difference between a region where the subject in the image is designated and the position of the target subject in the image.

1 is a block diagram illustrating a configuration of an imaging apparatus according to an embodiment of the present invention. It is a block diagram which shows the structure of the feature-value extraction part of FIG. 1, and a bias detection part. It is a flowchart which shows the to-be-photographed object area | region extraction process of this embodiment. It is a flowchart which shows the feature-value extraction process of this embodiment. It is a flowchart which shows the bias detection process of this embodiment. It is a figure explaining the object field extraction processing of this embodiment.

  EMBODIMENT OF THE INVENTION Below, the form for implementing this invention with reference to drawings is demonstrated in detail. The embodiment described below is an example as means for realizing the present invention, and should be appropriately modified or changed according to the configuration and various conditions of the apparatus to which the present invention is applied. It is not limited to the embodiment.

  <Configuration and Operation of Imaging Device> First, the configuration and operation of an imaging device according to an embodiment functioning as a subject area extraction device of the present invention will be described with reference to FIG. As shown in FIG. 1, in the imaging apparatus 100, light rays representing a subject image are collected by an imaging lens 101 and are incident on an imaging element 102 such as a CCD image sensor or a CMOS image sensor. The image sensor 102 outputs an electrical signal corresponding to the intensity of the incident light beam in units of pixels. This electrical signal is an analog video signal indicating a subject image captured by the image sensor 102.

  The video signal output from the image sensor 102 is subjected to analog signal processing such as correlated double sampling (CDS) in the analog signal processing unit 103. The video signal output from the analog signal processing unit 103 is converted into a digital data format by the A / D conversion unit 104 and input to the control unit 105 and the image processing unit 106.

  The control unit 105 is a CPU (Central Processing Unit), a microcontroller, or the like, and centrally controls the operation of the imaging apparatus 100. Specifically, the control unit 105 controls each unit of the imaging apparatus 100 by developing program codes stored in a ROM (Read Only Memory) in a work area of a RAM (Random Access Memory) and sequentially executing the program codes. .

  The image processing unit 106 performs image processing such as gamma correction and white balance processing on the input digital video signal. In addition to the normal image processing, the image processing unit 106 also has a function of performing image processing using information regarding a specific subject region in an image supplied from a subject region extraction unit 110 described later.

  The video signal output from the image processing unit 106 is sent to the display unit 107. The display unit 107 is, for example, an LCD or an organic EL display, and displays a video signal. The imaging apparatus 100 can cause the display unit 107 to function as an electronic viewfinder (EVF) by sequentially displaying images sequentially captured in time series by the imaging element 102 on the display unit 107. Further, the position of the subject area extracted by the subject area extraction unit 110 described later can be displayed.

  The video signal output from the image processing unit 106 is recorded on a recording medium 108, for example, a removable memory card. Note that the recording destination of the video signal may be the built-in memory of the imaging device 100 or an external device (none of which is shown) that is communicably connected via a communication interface.

  The subject specifying unit 109 includes an input interface such as a touch panel and buttons that can be operated by a user, for example, and specifies an arbitrary subject to be extracted by the subject region extracting unit 110.

  The subject region extraction unit 110 extracts a target subject region from the region designated by the subject designation unit 109 in the image supplied from the image processing unit 106. The subject region extraction unit 110 includes a feature amount extraction unit 111, a bias detection unit 112, and a region correction unit 113, which will be described later.

  The feature amount extraction unit 111 extracts a feature amount indicating the characteristics of the target subject from within a predetermined region in the image supplied from the image processing unit 106. The predetermined area corresponds to the area specified by the subject specifying unit 109. Details of the feature quantity extraction unit 111 will be described later.

  The bias detection unit 112 is based on the distribution state of feature pixels matching the feature amount extracted by the feature amount extraction unit 111 in the specified region in the image supplied from the image processing unit 106, and the purpose in the specified region. The deviation of the subject area is detected. Details of the bias detection unit 112 will be described later.

  The area correction unit 113 corrects the area designated by the subject designation unit 109 according to the amount and direction of the deviation detected by the deviation detection unit 112. The subject area extraction unit 110 extracts a subject using the corrected designated area as the subject area.

  The control unit 105 controls image capturing conditions such as a focus state and an exposure state when the image sensor 102 captures an image. Specifically, the control unit 105 controls a focus control mechanism and an exposure control mechanism (neither shown) of the imaging lens 101 based on the video signal output from the A / D conversion unit 104. For example, the focus control mechanism is an actuator that drives the imaging lens 101 in the optical axis direction, and the exposure control mechanism is an actuator that drives an aperture or a shutter.

  The control unit 105 can use information as the extraction result of the subject area extracted by the subject area extraction unit 110 for the focus control and the exposure control. Specifically, focus control using the contrast value of the subject area and exposure control using the brightness value of the subject area can be performed. Therefore, the imaging apparatus 100 can perform imaging processing in consideration of a specific subject area in the captured image. The control unit 105 also performs readout control of the image sensor 102 such as output timing and output pixels of the image sensor 102.

  <Configuration and Operation of Feature Quantity Extraction Unit 111> Here, the configuration and operation of the feature quantity extraction unit 111 in FIG. 1 will be described with reference to FIG. 2A, the feature amount extraction unit 111 includes a histogram generation unit 201, a feature space setting unit 202, a saturation value calculation unit 203, and a feature component weighting unit 204.

  The feature amount extraction unit 111 extracts a feature amount indicating the characteristics of the target subject from the designated area in the image supplied from the image processing unit 106. Here, the image supplied from the image processing unit 106 is an image expressed by an HSV color space composed of hue (H), saturation (S), and brightness (V), but is not limited thereto. Alternatively, for example, an HLS color space including hue (H), saturation (S), and luminance (L) may be used. First, the histogram generation unit 201 generates a histogram of hue components and lightness components from each pixel in the area specified by the subject specifying unit 109.

  The feature space setting unit 202 sets the feature space from the hue component and brightness component histograms generated by the histogram generation unit 201. Note that the maximum value of each histogram of the hue component and the brightness component is defined as a feature space. It should be noted that a space indicated by a histogram that is equal to or greater than a predetermined threshold in the hue component and the brightness component may be used as the feature space. Alternatively, a space in which the ratio of the number of pixels existing in the area to the number of pixels existing outside the area specified by the subject specifying unit 109 in the hue component and the brightness component is equal to or greater than a predetermined threshold may be used as the feature space. .

  The saturation value calculation unit 203 determines a pixel belonging to the feature space of the hue component set by the feature space setting unit 202 for each pixel in the region specified by the subject specification unit 109, and determines the saturation from the pixel group. The average value of is calculated. The method for calculating the saturation value from the pixels belonging to the feature space of the hue component may be another calculation method such as a median value.

  The feature component weighting unit 204 weights each component having the hue component and lightness component of the feature space set by the feature space setting unit 202 as feature amounts. Specifically, the saturation value calculated by the saturation value calculation unit 203 is compared with a threshold value, and if it is equal to or greater than the threshold value, the feature space of the hue component is mainly weighted, while the saturation average value is If it is less than the threshold, the feature space of the brightness component is mainly weighted.

  <Configuration and Operation of Bias Detection Unit> Next, the configuration and operation of the bias detection unit 112 in FIG. 1 will be described with reference to FIG. In FIG. 2B, the bias detection unit 112 includes an area dividing unit 205, a feature distribution determination unit 206, and a bias calculation unit 207.

  The bias detection unit 112 detects the bias of the position of the target subject region in the designated region based on the distribution state of the feature pixels that match the feature amount extracted by the feature amount extraction unit 111. The region dividing unit 205 divides the region in the region designated by the subject designating unit 109 by setting a symmetry axis passing through the center of the region in each of the vertical direction and the horizontal direction. In other words, the axis of symmetry is a straight line that bisects the designated area in the vertical and horizontal directions. The feature distribution determination unit 206 determines a pixel belonging to the feature amount extracted by the feature amount extraction unit 111 as a feature pixel in each pixel in the designated region. Then, the sum of the number of feature pixels in each area divided by the area dividing unit 205 is calculated.

  The bias calculation unit 207 calculates the amount and direction of the bias of the target subject area in the designated area according to the number of feature pixels in each area divided by the area dividing unit 205 by the feature distribution determination unit 206. The bias direction is the direction of a region having a large number of characteristic pixels in two regions which are orthogonal to the symmetry axis and divided on the boundary of the symmetry axis. The amount of correction depends on the difference in the number of feature pixels in the two divided regions so that the bias in the distribution of feature pixels around the symmetry axis is reduced when the position of the subject region is corrected. Value. That is, the amount of horizontal deviation is determined according to the difference in the number of feature pixels in the two regions divided by the vertical axis, and the amount of vertical deviation is determined by the two regions divided by the horizontal axis. It is determined according to the difference in the number of characteristic pixels. Here, an example in which the subject area is divided into two areas in the horizontal direction and the vertical direction has been described, but a plurality of symmetric areas are provided so that the distribution of the characteristic pixel distribution in the subject area can be detected. If it divides | segments into (2), you may divide | segment.

  In the present embodiment, the imaging apparatus 100 including the subject region extraction unit 110 that extracts a target subject region based on the region designated by the subject designation unit 109 is illustrated. However, the subject designation unit 109 of the imaging apparatus 100 designates a target subject area according to a predetermined area (such as an AF frame or a photometric frame) in an image used for a focus control mechanism or an exposure control mechanism, not an input interface. It may be configured to.

  <Subject Area Extraction Processing> Here, the subject area extraction processing executed by the subject area extraction unit 110 of the present embodiment will be described with reference to FIGS. As shown in FIG. 3, the subject area extraction unit 110 performs the processing from S301 to S305. S302 is processing by the feature amount extraction unit 111, S303 is processing by the bias detection unit 112, and S304 is processing by the region correction unit 113. As shown in FIG. 4, the feature amount extraction unit 111 performs the processing from S401 to S406. S401 is a histogram generation unit 201, S402 is a feature space setting unit 202, S403 is a saturation value calculation unit 203, and S404 to S406 are processing by a feature component weighting unit 204. Further, as shown in FIG. 5, the bias detection unit 112 performs the processing from S501 to S509. S501 is processing by the region dividing unit 205, S502 and S503 are processing by the feature distribution determining unit 206, and S504 to S509 are processing by the bias calculating unit 207.

  In FIG. 6, in the input image 601, the area surrounded by the dotted rectangle corresponds to the area specified by the subject specifying unit 109. An image 602 is a binarized image in which pixels belonging to the feature amount extracted by the feature amount extraction unit 111 are expressed in white and pixels not belonging to the feature amount are expressed in black. An image 603 shows an image obtained by dividing the designated area in the horizontal direction and the vertical direction. An image 604 shows a subject extraction result image in which the designated region is corrected by the region correction unit 113 and the corrected region is extracted as a subject region.

  In S301 in FIG. 3, the subject region extraction unit 110 reads captured images sequentially captured by the image sensor 102 as an input image, and the region designated by the subject designation unit 109 with respect to the input image 601 is, for example, shown in FIG. Setting is made like a dotted rectangle shown in the input image 601. Next, in S302, the feature amount extraction unit 112 extracts a feature amount indicating the target subject from the pixels in the region specified in S301.

  Here, the process of extracting the feature amount will be described with reference to FIG. First, in S401 shown in FIG. 4, a histogram of the hue component and the brightness component is generated from each pixel in the designated area. Next, in S402, the maximum value of the histogram of the hue component and the brightness component is set as the feature space of each component. Next, in S403, for each pixel in the designated area, a pixel belonging to the feature space of the hue component is determined, and an average value of saturation indicated by the pixel determined to belong is calculated.

  Next, in S404, it is determined whether the calculated average value of saturation is equal to or greater than a predetermined threshold value. If the average value of the saturation is equal to or greater than the predetermined threshold (YES in S404), the subject is a chromatic color, so in S405, the feature amount is a feature amount mainly composed of the hue component feature space. On the other hand, if the average value of the saturation is less than the predetermined threshold (NO in S404), the subject is an achromatic color. Therefore, in S406, the feature amount is a feature amount mainly including the feature space of the brightness component.

  Here, the processing (S404 to S406) of changing the weighting of the feature component used when extracting the feature amount according to the average value of the saturation by the feature component weighting unit 204 will be described. As the extracted feature quantity, a feature quantity that can separate the target subject area from other areas such as the background is desirable. The hue component is less susceptible to illumination fluctuations and is excellent as a component that separates the subject area from other areas. However, when the target subject is an achromatic color (the saturation value is small), it is often difficult to separate the subject region from other regions by the hue component. On the other hand, the brightness component is easily affected by lighting fluctuations, and the accuracy of separating the subject area from other areas is separated by the hue component when the target subject is a chromatic color (the saturation value is large). There is a tendency to be inferior to accuracy. Based on these tendencies, the feature component weighting unit 204 weights the feature components according to the average value of saturation.

  As an example of the weighting, when the average value of saturation is equal to or greater than a predetermined threshold, only the feature space of the hue component is used as a feature amount, and when the average value of saturation is less than the predetermined threshold, Only the feature space is used as a feature value. When the average value of saturation is equal to or greater than a predetermined threshold, the hue component feature space is set to a narrow range, the brightness component feature space is set to a wide range, and both the hue component and the brightness component feature space are set. Let the pixel to which it belongs be a feature pixel. This is based on the tendency that, when the target subject is a chromatic color (the saturation value is large), the hue variance is small and the brightness variance is large in the subject area. On the other hand, when the average value of the saturation is less than the predetermined threshold, the hue component feature space is set to a wide range, the brightness component feature space is set to a narrow range, and the hue component and the brightness component feature space are set. Let the pixel which belongs to be a feature pixel. This is based on the tendency that when the target subject is an achromatic color (the saturation value is small), the hue variance is large in the subject region.

  The above-described weighting by the feature component weighting unit 204 is an example, and is not limited thereto. For example, instead of or in addition to saturation, color information such as luminance and color difference obtained from a subject area other than hue and brightness. The weight may be changed according to the above. Further, in the present embodiment, the process of extracting the feature amount using at least one of the feature space of the brightness component and the hue component is exemplified, but the feature amount extraction method is not limited to the above-described process. For example, in order to improve the separation performance between the subject and other regions in the feature space, the feature space may be narrowed down by setting a maximum value and a minimum value based on saturation. Further, both the feature space of the hue component and the feature space of the lightness component are used, but the ratio of the weights is changed depending on whether the average value of saturation is equal to or greater than a predetermined threshold value or less than the threshold value. You may do it. For example, if the average value of saturation is equal to or greater than a predetermined threshold, the weights of the pixels included in the feature space of the hue component and the pixels included in the feature space of the lightness component are set to 2 to 1, and the subject in S303 described later What is necessary is just to detect the deviation of the position of the area. On the other hand, if the average value of the saturation is less than the predetermined threshold, the weight ratio between the pixels included in the feature space of the hue component and the pixels included in the feature space of the brightness component is set to 1: 2, and S303 described later is performed. The deviation of the position of the subject area may be detected at. Alternatively, the weighting ratio of the pixels included in the feature space of the hue component and the pixels included in the feature space of the brightness component may be continuously changed according to the average value of the saturation.

  Returning to the description of FIG. 3, in S303, the bias detection unit 113 determines the pixels belonging to the feature amount extracted in S302 as feature pixels, and determines the target object region within the specified region from the distribution state of the feature pixels. Detect position bias. Here, the process of detecting the bias will be described with reference to FIG. First, in S501 shown in FIG. 5, as shown in an image 603 in FIG. 6, axes are set in the vertical direction and the horizontal direction, and the designated area is divided into four parts so as to be equally divided. Next, in S502, each pixel in the designated area determines a pixel belonging to the extracted feature quantity, and a pixel belonging to the feature quantity is set as a feature pixel. The designated area divided image 603 is an image in which feature pixels are expressed in white and non-feature pixels are expressed in black.

  Next, in S503, the sum of the number of feature pixels in each divided region is calculated. Next, in step S504, the difference in the number of feature pixels between the regions divided by the horizontal axis (the difference in the number of feature pixels between the upper two regions and the lower two regions in the designated region divided image 603) is a predetermined threshold value. It is determined whether it is above. If it is equal to or greater than the threshold (YES in S504), a vertical bias is detected in S505. On the other hand, if it is less than the threshold value (NO in S504), no vertical bias is detected in S506. In the case of the designated area divided image 603, no vertical deviation is detected (S506).

  Next, in step S507, the difference in the number of feature pixels between regions divided by the vertical axis (the difference in the number of feature pixels between the left two regions and the right two regions in the designated region divided image 603) is a predetermined threshold value. It is determined whether it is above. If it is equal to or greater than the threshold (YES in S507), a horizontal deviation is detected in S508. On the other hand, if it is less than the threshold value (NO in S507), no horizontal bias is detected in S509. In the case of the designated area divided image 603, a horizontal deviation is detected (S508). The correction amount is determined based on the difference amount of the number of feature pixels.

  Returning to the description of FIG. 3, in S <b> 304, the designated area is corrected so as to reduce the bias according to the detected amount and direction of the bias. In step S <b> 305, a corrected region is extracted as a subject region, such as a region surrounded by a dotted rectangle shown in the subject extraction result image 604. The subject extraction result image 604 in FIG. 6 shows the result of correcting the designated area in the right direction.

  In the above-described processing, the subject is extracted with the corrected designated region as the subject region. However, the following processing may be performed. That is, in the corrected designated area, the deviation detecting unit 112 detects the deviation again. When the deviation is detected, the area correcting unit 113 corrects the designated area based on the detected amount and direction of the deviation. . On the other hand, if no bias is detected, the area is extracted as a subject area. That is, the processes by the bias detection unit 112 and the region correction unit 113 are repeatedly executed until the bias is stabilized.

  According to the above-described embodiment, even when the designated area in the image and the position of the target subject in the image are shifted, the feature amount is extracted from the designated area, and the distribution of the feature pixels in the designated area The deviation of the position of the target subject is detected from the situation, and the designated area is corrected. Thereby, a target subject area can be extracted.

  In the above-described embodiment, the subject area extraction apparatus that performs the process of extracting the subject area from the image has been described. However, the present invention may be applied to a subject tracking apparatus that extracts a subject region from images sequentially supplied in time series based on the result of extracting the subject region and tracks the subject in time series. For example, the subject can be tracked by registering the extracted subject region as a reference image and extracting the region having the highest degree of correlation between the partial region of the image sequentially supplied in time series and the reference image.

  Further, in the above-described embodiment, the case where the subject region extraction device that performs the subject region extraction processing is applied to the imaging device is illustrated, but the device to which the subject region extraction device is applied is not limited to the imaging device. For example, a subject region extraction device that performs subject region extraction may be applied to a display device that displays an image (reproduction data) supplied from an external device or a recording medium. In this display device, the reproduction data is used as subject area extraction processing data, and the subject area extraction processing is performed. A control unit such as a microcontroller in the display device controls display conditions for displaying an image based on subject information (position, size, etc. of the subject in the image) extracted by subject region extraction processing. . Specifically, information indicating a subject such as a frame is superimposed on the position of the subject in the image, and the brightness and color of the display image are controlled according to the brightness and color information of the subject portion.

  Further, the above-described embodiment is an example, and the present invention is not limited to this. The configuration and operation in the above-described embodiment can be changed as appropriate. For example, the area correction unit 113 may correct not only the position of the subject area but also the size and shape of the subject area. When correcting the size of the subject area, for example, the degree of feature pixels is determined on each side in the outer peripheral direction of the position-corrected area. If the degree of feature pixels is large, the area of that side is determined. And extract as a subject area. Further, the degree of presence of the feature pixel is determined at each side in the inner peripheral direction of the position-corrected area, and if the degree of existence of the feature pixel is small, the area excluding the area of the side is extracted as the subject area. . Thereby, an appropriate subject area can be extracted.

  [Other Embodiments] The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, etc.) of the system or apparatus reads and executes the program code. It is processing to do. In this case, the program and the storage medium storing the program constitute the present invention.

Claims (9)

A subject area extraction device for extracting a predetermined subject area from an image,
Designation means for designating an area including the predetermined subject from the image;
Extraction means for extracting a feature amount of the predetermined subject from the area designated by the designation means;
Detecting means for detecting a deviation of the position of the predetermined subject in the specified region based on a distribution state of feature pixels matching a feature amount of the predetermined subject included in the specified region;
Correction means for correcting the position of the designated area so that the deviation of the position of the predetermined subject in the designated area is reduced,
The extraction unit extracts the feature amount using at least one of a hue component and a brightness component of each pixel in the designated area, and the hue component and the brightness component according to color information obtained from a subject. The object region extracting apparatus is characterized in that the feature amount is extracted by changing the weight of the subject area. The extraction means generates a histogram of hue components and a histogram of brightness components from each pixel of the designated area;
Setting means for setting a feature space of a hue component and a lightness component by each histogram generated by the histogram generation means;
Calculating means for calculating the saturation of the pixels belonging to the feature space of the hue component set by the setting means in each pixel of the designated region;
Weighting means for weighting the feature space of the hue component and lightness component as the feature amount by comparing the saturation with a threshold value,
The weighting unit performs weighting mainly on a feature space of the hue component if the saturation value calculated by the calculation unit is greater than or equal to a threshold value, and if the saturation value is less than the threshold value, the lightness component The subject area extraction apparatus according to claim 1, wherein weighting is performed mainly on a feature space of the subject area.   The subject region extraction apparatus according to claim 1, wherein the detection unit determines, as a feature pixel, a pixel belonging to the feature amount extracted by the extraction unit in each pixel included in the subject region.   The detection means sets one or more straight lines that divide the specified region into at least two equal parts, calculates a difference in the number of feature pixels in two or more regions divided by the straight line, and calculates the calculated difference 4. The subject region extraction apparatus according to claim 1, wherein if there is a threshold value greater than or equal to a threshold value, it is detected that there is a bias. 5.   The correcting means is orthogonal to one or more set straight lines that divide the designated area into at least two equal parts, and the number of characteristic pixels is two or more areas divided by the straight lines. The subject area extraction apparatus according to claim 4, wherein the designated area is corrected by an amount corresponding to the bias in a direction of a large area.   For the area corrected by the correcting means, when the bias is detected by the detecting means, the area specified by the correcting means is corrected, and when the bias is not detected by the detecting means, the specified 6. The subject area extraction apparatus according to claim 1, wherein the area is extracted as a subject area. A subject area extraction device according to any one of claims 1 to 6,
Means for registering a reference image used for subject tracking based on an extraction result by the subject region extraction device;
A subject tracking device that extracts a subject region according to a degree of correlation between a partial region of a supplied image and the registered reference image. A method for controlling a subject area extracting apparatus for extracting a predetermined subject area from an image, comprising:
A designation step for designating an area including the predetermined subject from the image;
An extraction step of extracting a feature amount of the predetermined subject from the designated region;
A detection step of detecting a bias in the position of the predetermined subject in the specified region based on a distribution state of feature pixels matching a feature amount of the predetermined subject included in the specified region;
A correction step of correcting the position of the designated area so that the deviation of the position of the predetermined subject in the designated area is reduced,
In the extraction step, the feature amount is extracted using at least one of a hue component and a brightness component of each pixel in the designated region, and the hue component and the brightness component are determined according to color information obtained from a subject. The control method is characterized in that the feature amount is extracted by changing the weighting of.   A program for causing a computer to execute the control method according to claim 8.

Images