JP5786544B2 - Image processing apparatus, imaging apparatus, and program - Google Patents

Description

  The present invention relates to an image processing device, an imaging device, and a program.

  A technique for extracting an image area from a captured image based on a histogram of image feature amounts is known (see, for example, Patent Document 1).

Japanese Patent Laid-Open No. 4-10079

  However, with the technique described in Patent Document 1, it has been impossible to extract an image region that is noticed by a person from a captured image, that is, an image region of a subject.

  The present invention has been made to solve the above problems, and an object of the present invention is to provide an image processing apparatus, an imaging apparatus, and a program that can extract an image area of a subject from a captured image.

  In order to solve the above problem, an embodiment of the present invention includes a detection unit that detects a plurality of subject regions from an image, and a first region of the image among the plurality of subject regions detected by the detection unit. And an extraction unit that extracts a subject area in contact with two opposing sides.

  Further, an embodiment of the present invention includes: a detection unit that detects a plurality of subject areas from an image; and an extraction unit that extracts a specific subject area among the plurality of subject areas detected by the detection unit. And the extraction unit is an image processing apparatus that does not extract a subject region whose area in contact with one side forming the first region of the image is equal to or greater than a first threshold.

  Further, an embodiment of the present invention includes: a detection unit that detects a plurality of subject areas from an image; and an extraction unit that extracts a specific subject area among the plurality of subject areas detected by the detection unit. And the extraction unit extracts a subject region in which a ratio of a length of one side forming the first region of the image and a length of a portion of the side that is in contact with the subject region is equal to or greater than a second threshold value. It is an image processing device that does not.

  In one embodiment of the present invention, a detection unit that detects a plurality of subject areas from an image, and a first range defined in the image among the plurality of subject areas detected by the detection unit is formed. And an extraction unit that extracts a subject area that is in contact with two opposing sides.
  Further, an embodiment of the present invention includes: a detection unit that detects a plurality of subject areas from an image; and an extraction unit that extracts a specific subject area among the plurality of subject areas detected by the detection unit. And the extraction unit is an image processing apparatus that does not extract a subject area whose area in contact with one side forming a first range defined in the image is equal to or greater than a first threshold.
  Further, an embodiment of the present invention includes: a detection unit that detects a plurality of subject areas from an image; and an extraction unit that extracts a specific subject area among the plurality of subject areas detected by the detection unit. The extraction unit includes a length of one side forming a first range defined in the image, and the subject region is in contact with the side of the plurality of subject regions detected by the detection unit. This is an image processing apparatus that does not extract a subject area whose ratio to the length of the existing part is equal to or greater than the second threshold.
  In addition, an embodiment of the present invention includes a lens including the above-described image processing device, and a control unit that controls a lens that guides an optical image to an imaging device using a subject area extracted by the extraction unit of the image processing device. It is a control device.
  In one embodiment of the present invention, the lens is controlled by an imaging device that captures an optical image emitted from the lens, the image processing device described above, and a subject area extracted by the extraction unit of the image processing device. And a control unit.
  In one embodiment of the present invention, a detection step of detecting a plurality of subject regions from an image and a first region of the image among the plurality of subject regions detected in the detection step are formed in a computer. , An extraction step for extracting a subject area that is in contact with two opposite sides.
  According to one embodiment of the present invention, a detection step of detecting a plurality of subject areas from an image on a computer, and a first step defined in the image among the plurality of subject areas detected in the detection step. And an extraction step for extracting a subject area that is in contact with two opposite sides that form a range.

  According to the present invention, an image area of a subject can be extracted from a captured image.

It is a block diagram which shows the structure of the image processing apparatus and imaging device by this embodiment. It is a flowchart which shows the image processing in the image processing apparatus in this embodiment. It is a 1st image figure which shows an example of the process of the background exclusion part in this embodiment. It is a 2nd image figure which shows an example of the process of the background exclusion part in this embodiment. It is a flowchart which shows the process of the frame edge exclusion part in this embodiment. It is a 1st image figure which shows an example of a process of the frame edge exclusion part in this embodiment. It is a 2nd image figure which shows an example of the process of the frame edge exclusion part in this embodiment. It is a 3rd image figure which shows an example of the process of the frame edge exclusion part in this embodiment. It is a 4th image figure which shows an example of a process of the frame edge exclusion part in this embodiment. It is an image figure which shows an example of the to-be-photographed object extraction result in this embodiment.

Hereinafter, an image processing apparatus and an imaging apparatus according to an embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram illustrating the configuration of the image processing apparatus 1 and the imaging apparatus 100 according to the present embodiment.
The imaging device 100 captures an optical image incident from the lens barrel 111 and stores the obtained image in the storage medium 200 as a still image or a moving image. In addition, the imaging apparatus 100 includes the image processing apparatus 1 (image processing unit 140).

First, the configuration of the lens barrel 111 will be described.
The lens barrel 111 includes a focus adjustment lens (hereinafter referred to as “AF (Auto Focus) lens”) 112, a lens driving unit 116, an AF encoder 117, and a barrel control unit 118. The lens barrel 111 may be detachably connected to the imaging device 100 or may be integrated with the imaging device 100.

  The AF lens 112 is driven by a lens driving unit 116 and guides an optical image to a light receiving surface (photoelectric conversion surface) of an image sensor 119 of the image capturing unit 110 described later.

  The AF encoder 117 detects the movement of the AF lens 112 and outputs a signal corresponding to the movement amount of the AF lens 112 to the lens barrel control unit 118. Here, the signal corresponding to the movement amount of the AF lens 112 may be a sine wave signal whose phase changes according to the movement amount of the AF lens 112, for example.

  The lens barrel control unit 118 controls the lens driving unit 116 in accordance with a drive control signal input from the CPU 190 of the imaging apparatus 100. Here, the drive control signal is a control signal for driving the AF lens 112 in the optical axis direction. The lens barrel control unit 118 changes the number of steps of the pulse voltage output to the lens driving unit 116, for example, according to the drive control signal.

  Further, the lens barrel control unit 118 outputs the position (focus position) of the AF lens 112 in the lens barrel 111 to the CPU 190 based on a signal corresponding to the movement amount of the AF lens 112. Here, the lens barrel control unit 118 integrates, for example, signals according to the movement amount of the AF lens 112 according to the movement direction of the AF lens 112, thereby moving the AF lens 112 in the lens barrel 111 (the movement amount ( Position) may be calculated.

  The lens driving unit 116 drives the AF lens 112 in accordance with the control of the lens barrel control unit 118 and moves the AF lens 112 in the optical axis direction within the lens barrel 111.

Next, the configuration of the imaging device 100 will be described.
The imaging apparatus 100 includes an imaging unit 110, an image processing unit 140, a display unit 150, a buffer memory unit 130, an operation unit 180, a storage unit 160, a CPU 190, a communication unit 170, and a bus 300.

  The imaging unit 110 includes an imaging element 119 and an A / D (Analog / Digital) conversion unit 120. The imaging unit 110 is controlled by the CPU 190 in accordance with the set imaging conditions (for example, aperture value, exposure value, etc.).

  The image sensor 119 includes a photoelectric conversion surface, converts an optical image formed on the photoelectric conversion surface by the lens barrel 111 (optical system) into an electric signal, and outputs the electric signal to the A / D conversion unit 120. The image sensor 119 may be configured by, for example, a CMOS (Complementary Metal Oxide Semiconductor). Further, the image sensor 119 may convert an optical image into an electrical signal for a partial region of the photoelectric conversion surface (image clipping).

  Further, the image sensor 119 stores an image obtained when a shooting instruction from the user is received via the operation unit 180 in the storage medium 200 via the A / D conversion unit 120. On the other hand, the imaging device 119 performs A / D conversion on the buffer memory unit 130 and the display unit 150 as images that are continuously obtained in a state where a shooting instruction from the user is not received via the operation unit 180. Output via the unit 120.

  The A / D converter 120 digitizes the electrical signal converted by the image sensor 119 and outputs an image that is a digital signal to the buffer memory unit 130.

  The operation unit 180 includes, for example, a power switch, a shutter button, a multi-selector (cross key), or other operation keys. The operation unit 180 receives a user operation input when operated by the user, and outputs a signal corresponding to the operation input to the CPU 190. Output to.

The image processing unit 140 (image processing apparatus 1) performs image processing on the image temporarily stored in the buffer memory unit 130 based on the image processing conditions stored in the storage unit 160. Then, the image-processed image is stored in the storage medium 200 via the communication unit 170.
Note that the image processing unit 140 corresponds to the image processing apparatus 1.

  The image processing unit 140 (the image processing apparatus 1) includes an image region (mask) determined for each feature amount of the image feature from the image temporarily stored in the buffer memory unit 130, and “image region information” described later. (Label attribute) "is detected. Then, the image processing unit 140 associates the image region determined for each feature amount of the image feature with the region information and causes the storage unit 160 and the storage medium 200 to store the image region.

  Here, the information (label attribute) of the image area includes, for example, the distribution (position, shape (moment of inertia)), area, number, subject distance (defocus amount), motion vector (difference data) in the image, Hue, saturation, brightness, texture (pattern), brightness (Y), color difference (Cb, Cr), edge, AF contrast evaluation value, set flag, set label (identifier), calculated score, or set priority Degree. These may be used in combination for the image area information, or may be used alone. The score is a score (evaluation score) for comparing the labeled image areas.

Further, the image processing unit 140 extracts the image area of the subject from the image areas of the subject candidates using the image area (mask) as the image area of the subject candidates.
Here, the image area of the subject is an image area that a person pays attention to from the captured image. The detailed configuration of the image processing unit 140 will be described later.
Further, the image processing unit 140 generates, for example, a rectangular area for AF based on the extracted image area of the subject.

The display unit 150 is, for example, a liquid crystal display, and displays an image obtained by the imaging unit 110, an operation screen, and the like.
The buffer memory unit 130 temporarily stores the image captured by the imaging unit 110.

  The storage unit 160 stores various determination conditions referred to by the CPU 190 at the time of imaging, various set imaging conditions, image area information, and the like.

  The CPU 190 controls the imaging unit 110 according to the set imaging conditions (for example, aperture value, exposure value, etc.). In addition, the CPU 190 acquires image area information from the storage unit 160 or the storage medium 200.

  Then, based on the image area information, the CPU 190 performs focus adjustment (AF) setting, exposure adjustment (AE) setting, white balance adjustment (AWB) setting, subject image (object) as pre-processing or post-processing. Control settings such as tracking, night scene determination settings, color correction processing settings, enlarged subject image display settings, panning display settings, brightness optimization settings linked to zoom magnification, etc. .

  Further, the CPU 190 causes the image processing unit 140 to perform image processing as a still image or a moving image based on the “signal according to the operation input” input from the operation unit 180.

  The communication unit 170 is connected to a removable storage medium 200 such as a card memory, and writes, reads, or erases information (image data, area information, etc.) to the storage medium 200.

The storage medium 200 is a storage unit that is detachably connected to the imaging apparatus 100 and stores information (image data, area information, and the like). Note that the storage medium 200 may be integrated with the imaging device 100.
The bus 300 includes an imaging unit 110, a lens barrel control unit 118 of the lens barrel 111, an image processing unit 140, a display unit 150, a buffer memory unit 130, an operation unit 180, a storage unit 160, and a CPU 190. Are connected to the communication unit 170 and transfer data output from each unit.

Next, details of the image processing unit 140 will be described.
The image processing unit 140 includes a feature amount calculation unit 10, a region detection unit 15, and a subject determination unit 20.

  The feature amount calculation unit 10 calculates the feature of the captured image. That is, the feature amount calculation unit 10 calculates a feature amount for each feature of the input image. Specifically, the feature amount calculation unit 10 acquires an image captured by the imaging unit 110 from the buffer memory unit 130. The feature amount calculation unit 10 calculates the feature amount for each of a plurality of features (for example, luminance (Y), color difference (Cb, Cr), etc.) in the image acquired from the buffer memory unit 130.

  The area detection unit 15 (extraction unit) detects (extracts) an image area included in the image as an image area (first image area) of a subject candidate based on the characteristics of the captured image. That is, the area detection unit 15 detects an area included in the acquired image and determined for each feature amount from the image as an image area of a subject candidate. That is, the region detection unit 15 extracts the image region of the subject candidate determined for each feature amount calculated by the feature amount calculation unit 10. Here, the area detected by the area detection unit 15 is, for example, an image area of a subject candidate included in the image captured by the imaging unit 110.

The area detection unit 15 sets a label (identifier) that identifies the image area of the subject candidate detected for each feature amount (labeling process). Here, the label is an image area composed of pixels whose feature amounts are close to each other, and is a grouped image area (set). Labeling is to sequentially detect and group image regions adjacent to pixels having feature quantities close to each other (for example, corresponding to saturation h = 0 to 20 degrees).
The area detection unit 15 stores a label for identifying an image area in the storage unit 160 and the storage medium 200 for each image area as image area information.

  The subject determination unit 20 (determination unit) extracts (determines) the subject image region (second image region) from the subject candidate image regions detected by the region detection unit 15. In other words, the subject determination unit 20 excludes an image region that is unlikely to be a subject from the subject candidate image regions by performing a background image region exclusion process and a frame border image region exclusion process, which will be described later. The image area is narrowed down to several (for example, five or less).

  The subject determination unit 20 uses a score for comparing image regions labeled based on feature amounts of different features (for example, luminance (Y) and color difference (Cb, Cr)) for each image region (for each label). To calculate. Then, the subject determination unit 20 sets “image area priority” in the image based on the calculated score. This “priority of the image area” is the priority of the image area of the subject indicating the image area that the person will be more interested in. In other words, the subject determination unit 20 extracts (determines) an image region of the subject based on the set “image region priority”. The subject determination unit 20 stores the set “image area priority” in the storage unit 160 and the storage medium 200 for each image area as image area information.

The subject determination unit 20 includes a background exclusion unit 21 and a frame edge exclusion unit 22.
The background exclusion unit 21 determines whether each of the image areas corresponds to the positional relationship between each of the image areas of the subject candidate image areas and an extraction frame (extraction area) indicating a predetermined extraction range within the imaging range of the image. It is determined whether or not the background image area indicates the background image area. That is, the subject determination unit 20 determines the subject candidate according to the positional relationship between the subject candidate image region extracted by the region detection unit 15 and the extraction frame indicating a predetermined extraction range in the input image. It is determined whether or not to extract the image area of the subject extracted from the image area. When determining that the image area is the background image area, the background exclusion unit 21 excludes the determined background image area from the subject candidate image areas (background image area exclusion processing). That is, since the background image region is unlikely to be the subject image region, the subject determination unit 20 excludes the determined background image region from the subject candidate image regions.
The extraction frame is, for example, a frame that indicates an imaging range (view angle). This extraction frame may be set to an arbitrary size inside the imaging range (view angle). In the present embodiment, an example in which the shape of the extraction frame is a quadrangle will be described, but the shape of the extraction frame may be a polygon.

For example, the background excluding unit 21 determines whether or not the image area is the background image area (whether or not to extract the subject image area extracted from the subject candidate image area) under the following three conditions: To do.
[First condition]
When the image area touches two or more sides of the extraction frame, the background exclusion unit 21 determines that the image area is a background image area. In this case, the background excluding unit 21 excludes the image area determined to be the background image area from the image areas of the subject candidates. That is, when the subject candidate image area touches two adjacent sides (or two or more sides) forming the extraction frame (extraction area), the background exclusion unit 21 extracts the subject candidate image area as the subject image area. Judge that not.

[Second condition]
The background excluding unit 21 determines that the image area is a background image area when the line width (or the area of the image area) in contact with one side of the extraction frame is equal to or greater than a predetermined first threshold. To do. In this case, the background excluding unit 21 excludes the image area determined to be the background image area from the image areas of the subject candidates. That is, when the area (or line width) of the subject candidate image area in contact with one side forming the extraction frame is equal to or larger than the predetermined first threshold value, the background exclusion unit 21 determines that the subject candidate image area is the subject. It is determined that the image area is not extracted. In this case, a form in which the ratio of the image region contacting one side of the extraction frame (for example, the ratio of the area, the ratio of the line width, or the like) may be a predetermined first threshold value or more.

[Third condition]
The background excluding unit 21 determines that the image area is not a background image area when the image area is in contact with two sides of the extraction frame and the image area is in contact with two opposite sides of the extraction frame. In this case, the background exclusion unit 21 leaves the image area in the subject candidate image area and does not exclude it from the subject candidate image area. That is, when the subject candidate image area touches two opposite sides forming the extraction frame, the background exclusion unit 21 determines to extract the subject candidate image area as the subject image area.

  In the present embodiment, an example in which the image processing apparatus 1 executes processing by the frame edge exclusion unit 22 after performing processing by the background exclusion unit 21 will be described. For this reason, the image area (second image area) of the subject extracted by the background excluding unit 21 will be described as the image region of the subject candidate in the processing in the frame edge excluding unit 22. Furthermore, in the present embodiment, an example will be described in which the frame edge exclusion unit 22 extracts a subject image region (third image region or fourth image region) from a subject candidate image region (second image region).

  The frame edge excluding unit 22 is a predetermined range that indicates each of the image regions of the subject candidate image region (second image region) and the range inside the extraction frame (extraction region) and the center portion of the imaging range. Whether each image area is a frame edge image area is determined according to the positional relationship with the center frame (center area). That is, the frame edge excluding unit 22 determines the subject candidate according to the positional relationship between the image area of the subject candidate and the central frame indicating the range of the central portion of the input image that is inside the extraction frame. It is determined whether or not to extract the image area (third image area) of the subject extracted from the image area.

  When it is determined that the image area is a frame edge image area, the frame edge exclusion unit 22 excludes the determined frame edge image area from the image areas of the subject candidates (frame edge image area exclusion processing). Here, the frame edge image area refers to an image area having a frame shape in a captured image, and is an image area formed by a frame-like object such as a window frame or a window frame, for example. That is, since the frame edge image area is unlikely to be the image area of the subject, the frame edge exclusion unit 22 excludes the determined frame edge image area from the image areas of the subject candidates.

For example, the frame edge exclusion unit 22 determines whether the image area is a frame edge image area under the following four conditions.
[Condition A]
The frame edge exclusion unit 22 determines that the image area is a frame edge image area when the aspect ratio of the image area is equal to or larger than a predetermined second threshold and the image area is outside the central frame. In this case, the frame edge exclusion unit 22 excludes the image area determined to be the frame edge image area from the image areas of the subject candidates. That is, when the aspect ratio of the subject candidate image area is equal to or larger than the predetermined second threshold and the subject candidate image area is outside the range of the center frame, the frame edge exclusion unit 22 It is determined that the image area is not extracted as the image area (third image area) of the subject.
The aspect ratio of the image area is the ratio of the long side length to the short side length of the image area and corresponds to (long side length / short side length).

  In the following [Condition B] to [Condition D], the central frame includes a first central frame (see the central frame B in FIG. 7) and a second central frame indicating a range inside the first central frame. (Refer to the center frame A in FIG. 7). The frame edge exclusion unit 22 selects the subject candidate image according to the positional relationship among the subject candidate image region, the first center frame (first center region), and the second center frame (second center region). It is determined whether or not to extract the image area (fourth image area) of the subject extracted from the area. Here, the first center frame (center frame B) is a range inside the extraction frame and indicates the range of the central portion of the input image. The second central frame (central frame A) is a range inside the first central frame and indicates a range of the central portion of the first central frame.

[Condition B]
The frame edge excluding unit 22 determines that the image area when the aspect ratio of the image area is equal to or greater than a predetermined second threshold and the image area is in the first central frame and the second central frame. Is not a frame border image region. In this case, the frame edge exclusion unit 22 leaves the image area in the subject candidate image area and does not exclude it from the subject candidate image area. That is, the frame edge excluding unit 22 has an aspect ratio of the subject candidate image area equal to or greater than a predetermined second threshold, and the subject candidate image area is located inside the first center frame (center frame B). If it is within the range and the range inside the second center frame (center frame A), it is determined that the subject candidate image area is extracted as the subject image area (fourth image area).

[Condition C]
The frame edge exclusion unit 22 has an aspect ratio of the image area equal to or greater than a predetermined second threshold, the image area is in the first center frame, and the image area is in the second center frame. If there is no other image region in the second center frame, it is determined that the image region is a frame edge image region. In this case, the frame edge exclusion unit 22 excludes the image area determined to be the frame edge image area from the image areas of the subject candidates. That is, the frame edge excluding unit 22 has an aspect ratio of the subject candidate image area equal to or greater than a predetermined second threshold, and the subject candidate image area is located inside the first center frame (center frame B). If the area is outside the range and the second center frame (center frame A) and there is an image area of another subject candidate in the second center frame (center frame A), the subject candidate Is determined not to be extracted as the subject image region (fourth image region).

[Condition D]
The frame edge exclusion unit 22 has an aspect ratio of the image area equal to or greater than a predetermined second threshold, the image area is in the first center frame, and the image area is in the second center frame. If there is no other image area in the second center frame, it is determined that the image area is not a frame edge image area. In this case, the frame edge exclusion unit 22 leaves the image area in the subject candidate image area and does not exclude it from the subject candidate image area. That is, the frame edge excluding unit 22 has an aspect ratio of the subject candidate image area equal to or greater than a predetermined second threshold, and the subject candidate image area is located inside the first center frame (center frame B). If the area is outside the range and the second center frame (center frame A) and there is no image area of another subject candidate in the subject candidate image area, the subject candidate image area is defined as the subject. It is determined that it is extracted as an image area (fourth image area).

Next, image processing in the image processing unit 140 (image processing apparatus 1) will be described.
FIG. 2 is a flowchart illustrating an example of image processing in the image processing unit 140 (image processing apparatus 1) according to the present embodiment.
In the present embodiment, an example in which an image region is detected using luminance (Y) and color differences (Cb, Cr) as feature amounts will be described.

  First, the image processing unit 140 acquires a through image (step S101). That is, the CPU 190 stores the image captured via the imaging unit 110 in the buffer memory unit 130 as a through image. The image processing unit 140 acquires a through image stored in the buffer memory unit 130. Here, the image processing unit 140 may acquire through images (through image sequences) that are continuous in time.

  Next, the image processing unit 140 performs basic image processing on the through image (step S102). Here, basic image processing includes, for example, simple exposure adjustment (simple AE), white balance adjustment (AWB), and noise reduction (NR).

  Next, the image processing unit 140 executes a resizing YUV (wire buoy) process (step S103). That is, the image processing unit 140 converts the image on which the basic image processing has been performed into an image having an image size and YUV format suitable for extracting the image area of the subject.

  Next, the image processing unit 140 generates a binary image in nine YUV spaces (step S104). That is, the area detection unit 15 of the image processing unit 140 divides the Y image (luminance image) into three in the gradation direction, and generates a binary image. The three divisions are, for example, a range less than (M−σ), a range from (M−σ) to (M + σ), and a range from (M + σ) to (M + 1.6σ). Here, M corresponds to the average value of the luminance distribution, and σ corresponds to the standard deviation. Similar to the Y image (luminance image), the image processing unit 140 divides the Cr image (color difference image) and the Cb image (color difference image) into three in the gradation direction, and generates a binary image. As a result, binary images are generated in nine YUV spaces.

  Next, the image processing unit 140 creates an image region by labeling (step S105). That is, the feature amount calculation unit 10 calculates a feature amount for each feature (for example, luminance (Y), color difference (Cb, Cr), etc.) in the binary image. The area detection unit 15 executes a labeling process for setting a label (identifier) for identifying the image area of the subject candidate detected for each feature amount, and detects the image area as the image area of the subject candidate.

  Next, the image processing unit 140 excludes the noise image region from the subject candidate image regions (step S106). That is, the subject determination unit 20 of the image processing unit 140 excludes an isolated image region (noise) from the subject candidate image regions from the subject candidate image regions.

  Next, the image processing unit 140 executes background image region exclusion processing (step S107). In other words, the background excluding unit 21 of the subject determining unit 20 determines the image according to the positional relationship between each of the image regions of the subject candidate image regions and an extraction frame indicating a predetermined extraction range within the imaging range of the image. It is determined whether or not the area is a background image area. Specifically, the background exclusion unit 21 determines whether the image area is a background image area based on the first condition to the third condition described above. That is, the background excluding unit 21 determines whether or not to extract the subject image region extracted from the subject candidate image region based on the first to third conditions described above. When determining that the image area is the background image area, the background excluding unit 21 excludes the determined background image area from the subject candidate image areas.

Here, a specific example in which the background excluding unit 21 determines whether or not the image area based on the first condition to the third condition described above is a background image area will be described.
FIG. 3 is an image diagram showing an example of processing of the background removal unit 21 according to the first condition and the second condition.
In this figure, the image is an image having an extraction frame G1 in a range of horizontal side width a × vertical side width b. As shown in FIG. 3A, in the extraction frame G1, the area detection unit 15 detects five image areas M1 to M5 (image areas of subject candidates).

  For example, since the image region M4 is in contact with the two sides of the extraction frame G1, the background exclusion unit 21 determines that the image region M4 is a background image region based on the first condition described above. In this case, the background excluding unit 21 excludes the image area determined to be the background image area from the image areas of the subject candidates.

  Further, for example, the line width d in which the image region M5 is in contact with one side of the extraction frame G1 is a certain ratio or more with respect to one side (for example, (line width d / vertical side width b) is equal to or greater than a predetermined threshold K). Therefore, based on the second condition described above, the background excluding unit 21 determines that the image area is a background image area. That is, the background exclusion unit 21 determines that the background exclusion unit 21 has a line width d in which the image region M5 is in contact with one side of the extraction frame G1 is equal to or greater than a predetermined first threshold K1 (threshold K × vertical line width b). 21 determines that the image area is a background image area. In this case, the background excluding unit 21 excludes the image area determined to be the background image area from the image areas of the subject candidates.

  As a result, the background image areas (M4, M5) are excluded from the subject candidate image areas, and as shown in FIG. 3B, the image areas (M1, M2, M3) are subject candidates by the background exclusion unit 21. Is left as an image area.

FIG. 4 is an image diagram showing an example of processing of the background removal unit 21 under the third condition.
For example, the image region M6 shown in FIG. 4A is in contact with two opposite sides (left and right vertical sides) of the extraction frame G1. Therefore, the background exclusion unit 21 determines that the image area M6 is not a background image area based on the third condition described above. In this case, the background excluding unit 21 leaves the image region M6 in the subject candidate image region and does not exclude it from the subject candidate image region.

For example, the image region M7 shown in FIG. 4B is in contact with two opposing sides (upper and lower horizontal sides) of the extraction frame G1. Therefore, the background exclusion unit 21 determines that the image area M7 is not a background image area based on the third condition described above. In this case, the background exclusion unit 21 leaves the image region M7 in the subject candidate image region and does not exclude it from the subject candidate image region.
As described above, in step S107, the background excluding unit 21 executes the excluding process of the background image area (for example, the image area (M4, M5)).

Next, the image processing unit 140 executes a frame border image region exclusion process (step S108). That is, the frame edge exclusion unit 22 of the subject determination unit 20 determines each image region according to the positional relationship between each image region of the subject candidate image regions and the above-described central frame that is the range inside the extraction frame G1. Is determined to be a frame edge image region. Specifically, the frame edge excluding unit 22 determines whether or not the image area is a frame edge image area based on the conditions A to D described above. That is, the frame edge exclusion unit 22 determines whether or not to extract the subject image region extracted from the subject candidate image region based on the above-described conditions A to D. When it is determined that the image area is a frame edge image area, the frame edge exclusion unit 22 excludes the determined frame edge image area from the image areas of the subject candidates.
The details of the frame edge image region exclusion process performed by the frame edge exclusion unit 22 of the subject determination unit 20 will be described later with reference to FIGS.

  Next, the image processing unit 140 prioritizes image areas (image areas of subject candidates) (step S109). That is, the subject determination unit 20 gives a score for comparing image regions labeled based on feature amounts of different features (for example, luminance (Y) and color difference (Cb, Cr)) for each image region (label). Every). Then, the subject determination unit 20 sets “priority of the image area” in the image based on the calculated score, and selects an image area having a high priority based on the “priority of image area” as the image area of the subject. Extract as

  Next, the image processing unit 140 generates a rectangular area for AF (step S110). That is, the image processing unit 140 generates an AF rectangular region for the subject image regions (for example, 5 or less) extracted by the subject determination unit 20 based on the “priority of the image region”. For example, the image processing unit 140 generates the AF rectangular area as a rectangular frame circumscribing the image area of the subject.

  Next, the CPU 190 executes a contrast scan on the generated rectangular area for AF (step S111). That is, the CPU 190 executes a contrast scan and causes the storage unit 160 to store the lens position at which the AF contrast evaluation value shows a peak for each rectangular area for AF.

  Next, the CPU 190 executes AF control with close priority (step S112). That is, the CPU 190 performs AF control (focus control) so that the set AF rectangular area is focused with close priority based on the lens position at which the AF contrast evaluation value stored in the storage unit 160 shows a peak. Focus adjustment control).

  Next, the imaging device 100 captures an image (step S113). That is, when the CPU 190 receives a shooting instruction from the user via the operation unit 180, the CPU 190 causes the imaging unit 110 to capture an image. Furthermore, the image processing unit 140 performs subsequent processing (image processing) based on the region information (step S114). Then, the image processing unit 140 stores the region information in the storage unit 160 and the storage medium 200 in association with the image (region) (step S115).

Next, the frame edge image region exclusion process by the frame edge exclusion unit 22 will be described.
FIG. 5 is a flowchart showing the processing of the frame edge exclusion unit 22 in the present embodiment.
FIGS. 6 to 9 are image diagrams showing specific examples in which the frame edge excluding unit 22 determines whether or not the image area according to the above conditions A to D is a frame edge image area.
Note that the processing of the frame edge exclusion unit 22 in FIG. 2 in FIG. 5 corresponds to the processing in step S108.

  In FIG. 5, first, the frame edge excluding unit 22 determines whether or not the aspect ratio (long side length / short side length) of the image region is equal to or greater than the second threshold value K2 (step S201). That is, the frame edge excluding unit 22 calculates the aspect ratio of the image area. For example, in the image region M8 shown in FIG. 6, the aspect ratio is (long side length W / short side length h). The frame edge exclusion unit 22 determines whether the calculated aspect ratio (long side length W / short side length h) is equal to or greater than the second threshold value K2. When the calculated aspect ratio (long side length W / short side length h) is equal to or greater than the second threshold value K2, the frame edge exclusion unit 22 advances the process to step S202, and when the calculated aspect ratio is less than the second threshold value K2, The process proceeds to step S205.

Next, in step S <b> 202, the frame edge exclusion unit 22 determines whether the image area is outside the range of the central frame B. Here, as shown in FIGS. 6 to 9, the central frame B is an extraction frame indicating the range of the central portion in the imaging range. For example, the central frame B is 1/2 of the extraction frame G1 as in FIGS. It has a horizontal side width and a vertical side width (an area of 1/4 of the extraction frame G1). The frame edge exclusion unit 22 advances the process to step S206 when the image area is outside the range of the center frame B (condition A), and if the image area is within the range of the center frame B, the process proceeds to step S203. Proceed to
Specifically, since the image region M8 shown in FIG. 6 is outside the range of the central frame B, the frame edge excluding unit 22 advances the process to step S206 based on the condition A described above.

Next, in step S <b> 203, the frame edge exclusion unit 22 determines whether the image area is within the range of the central frame A. Here, the center frame A indicates a range inside the center frame B as shown in FIG. 7 to FIG. 9, for example, a lateral width of 1/3 of the extraction frame G1 similar to that in FIG. 3 and FIG. It has a vertical side width (area of 1/9 of the extraction frame G1). The frame edge exclusion unit 22 advances the process to step S205 when the image area is within the range of the center frame A (condition B), and if the image area is outside the range of the center frame A, the process proceeds to step S204. Proceed to
Specifically, since the image region M9 shown in FIG. 7 is within the range of the center frame A, the frame edge exclusion unit 22 advances the process to step S205 based on the condition B described above. Further, since the image regions M10 and M12 shown in FIGS. 8 and 9 are outside the range of the central frame A, the frame edge exclusion unit 22 advances the processing to step S204.

Next, in step S <b> 204, the frame edge exclusion unit 22 determines whether or not the image area is within the central frame A. The frame edge exclusion unit 22 advances the process to step S206 when the image area is in the central frame A (condition C), and the image area does not have any other image area in the central frame A. Then, the process proceeds to step S205 (condition D).
Specifically, since the image region M10 shown in FIG. 8 has another image region M11 in the central frame A, the frame edge exclusion unit 22 advances the processing to step S206 based on the condition C described above. . In addition, since the image region M12 shown in FIG. 9 has no other image region in the central frame A (the image region M11 is outside the central frame A), the frame edge exclusion unit 22 is based on the condition D described above. Then, the process proceeds to step S205.

Next, in step S205, the frame edge exclusion unit 22 determines that the image area (M9, M12) is not a frame edge image area. In this case, the frame edge exclusion unit 22 leaves the image regions (M9, M12) in the subject candidate image regions and does not exclude them from the subject candidate image regions.
That is, the image area M9 is determined not to be a frame edge image area according to the condition B, and is left as an image area of a subject candidate. The image area M12 is determined not to be a frame edge image area according to the condition D, and is left as an image area of a subject candidate.

Next, in step S206, the frame edge exclusion unit 22 determines that the image area (M8, M10) is a frame edge image area. In this case, the frame edge exclusion unit 22 excludes the image areas M8 and M10) determined to be the frame edge image area from the image areas of the subject candidates.
That is, the image area M8 is determined to be a frame edge image area according to the condition A, and is excluded from the subject candidate image areas. The image area M10 is determined to be a frame edge image area according to the condition C, and is excluded from the subject candidate image areas.

  As described above, in step S108 in FIG. 2, the frame edge exclusion unit 22 executes the process of excluding the frame edge image area (for example, the image area (M8, M10)).

FIG. 10 is an image diagram illustrating an example of a subject extraction result by the image processing unit 140 (image processing apparatus 1) in the present embodiment.
In this figure, the image processing unit 140 extracts, for example, an image area (M14, M15) as an image area of a subject. Further, the image processing unit 140 generates a rectangular area (AF1, AF2) for AF based on the image area (M14, M15).
For example, the background removal unit 21 of the image processing unit 140 determines that the image region (M17, M18) is a background image region based on the first condition described above, and excludes the image region from the subject candidate image region. Further, for example, the frame edge exclusion unit 22 of the image processing unit 140 determines that the image area M16 is a frame edge image area according to the above-described condition C, and excludes the image area M16 from the subject candidate image area.

As described above, in the image processing apparatus 1 according to the present embodiment, the area detection unit 15 detects the image area included in the image as the image area of the subject candidate based on the characteristics of the captured image. The subject determination unit 20 extracts the subject image region from the subject candidate image regions detected by the region detection unit 15. The subject determination unit 20 (background exclusion unit 21) determines whether each image region is a background image region according to the positional relationship between each image region of the subject candidate image regions and the extraction frame G1. To do. Then, the background exclusion unit 21 excludes the determined background image area from the subject candidate image areas. That is, in the image processing apparatus 1 according to the present embodiment, the feature amount calculation unit 10 calculates the feature amount of the feature for each feature of the input image. The area detection unit 15 extracts an image area of a subject candidate determined for each feature amount calculated by the feature amount calculation unit 10. The subject determination unit 20 (background exclusion unit 21), in accordance with the positional relationship between the subject candidate image region extracted by the region detection unit 15 and an extraction frame indicating a predetermined extraction range in the input image. It is determined whether or not to extract the subject image area extracted from the subject candidate image area.
Accordingly, the subject determination unit 20 can exclude an image region (background image region) that is unlikely to be a subject from the subject candidate image region. Therefore, the image processing apparatus 1 in the present embodiment can appropriately extract the image area of the subject from the captured image.

In the present embodiment, the subject determination unit 20 calculates a score for comparing image regions labeled based on feature amounts of different features for each image region (for each label). Then, the subject determination unit 20 sets “priority of the image area” in the image based on the calculated score, and selects an image area having a high priority based on the “priority of image area” as the image area of the subject. Extract as
Thereby, the image processing apparatus 1 in the present embodiment can appropriately extract the image area of the subject from the captured image.

Further, after narrowing down the image areas of the subject candidates by the background excluding unit 21 and the frame edge excluding unit 22, an image area having a high priority is extracted as the image area of the subject based on the above-mentioned “priority of the image area”. To do.
Thereby, the calculation amount of the score for comparison with a large amount of processing can be reduced. Therefore, the processing amount of the subject determination unit 20 can be reduced.

In the present embodiment, the background exclusion unit 21 determines that the image area is a background image area when the image area touches two or more sides of the extraction frame G1. That is, when the subject candidate image area touches two adjacent sides (or two or more sides) forming the extraction frame G1, the subject determination unit 20 (background exclusion unit 21) selects the subject candidate image area as the subject image. It is determined not to extract as a region.
As a result, the background excluding unit 21 can accurately determine a background image region that is unlikely to be a subject, and excludes the background image region from the subject candidate image region. Therefore, the image processing apparatus 1 in the present embodiment can appropriately extract the image area of the subject from the captured image.

In the present embodiment, the background excluding unit 21 determines that the image area is an image of a subject candidate when the image area is in contact with two sides of the extraction frame G1 and the image area is in contact with two opposite sides of the extraction frame G1. Leave as an area. That is, the subject determination unit 20 (background exclusion unit 21) determines to extract the subject candidate image region as the subject image region when the subject candidate image region touches two opposing sides forming the extraction frame.
As a result, the background exclusion unit 21 can determine in advance an image region that is likely to be a subject and leave it as an image region of a subject candidate. Therefore, the image processing output device 1 in the present embodiment can appropriately extract the image area of the subject from the captured image.

In the present embodiment, the background excluding unit 21 is configured such that the line width where the image region is in contact with one side of the extraction frame G1 is equal to or greater than a predetermined first threshold K1 (threshold K × vertical line width b). The image area is determined to be the background image area. That is, the subject determination unit 20 (background exclusion unit 21), when the area (or line width) of the subject candidate image area in contact with one side forming the extraction frame G1 is equal to or larger than a predetermined first threshold value, It is determined that the image area of the subject candidate is not extracted as the image area of the subject.
As a result, the background excluding unit 21 can accurately determine a background image region that is unlikely to be a subject, and excludes the background image region from the subject candidate image region. Therefore, the image processing apparatus 1 in the present embodiment can appropriately extract the image area of the subject from the captured image.

In the present embodiment, the frame border excluding unit 22 of the subject determining unit 20 includes a predetermined central frame B (or center) indicating each of the image regions of the subject candidate image regions and the range of the central portion of the imaging range. Whether or not it is a frame edge image region is determined according to the positional relationship with the frame A). Then, the frame edge excluding unit 22 excludes the determined frame edge image area from the subject candidate image areas. In other words, the subject determination unit 20 (frame edge exclusion unit 22) is a center frame B (or center frame) indicating the range of the subject image area and the inner area of the extraction frame and the center portion of the input image. It is determined whether or not to extract the subject image area (third image area) extracted from the subject candidate image area in accordance with the positional relationship with A).
Thereby, the subject determination unit 20 excludes a frame edge image region (for example, an image region by a frame-like object such as a window frame or a window frame) that is unlikely to be a subject from the subject candidate image region. be able to. Therefore, the image processing apparatus 1 in the present embodiment can appropriately extract the image area of the subject from the captured image.

Further, in the present embodiment, the frame edge exclusion unit 22 removes the image area when the aspect ratio of the image area is greater than or equal to a predetermined second threshold K2 and the image area is outside the central frame B. It determines with it being a frame edge image area | region. In other words, the subject determination unit 20 (frame edge exclusion unit 22) has an aspect ratio of the subject candidate image area equal to or greater than the predetermined second threshold K2, and the subject candidate image area is outside the range of the center frame B. If it is, the image area of the subject candidate is determined not to be extracted as the image area of the subject.
Thereby, the frame edge exclusion unit 22 can accurately determine a frame edge image area that is unlikely to be a subject, and excludes the frame edge image area from the subject candidate image area. Therefore, the image processing apparatus 1 in the present embodiment can appropriately extract the image area of the subject from the captured image.

In the present embodiment, the subject determination unit 20 (frame edge exclusion unit 22) positions the subject candidate image region, the center frame B (first center region), and the center frame A (second center region). It is determined whether or not to extract the subject image area extracted from the subject candidate image area according to the relationship. Here, the center frame B is a range inside the extraction frame and indicates the range of the center portion of the input image. The center frame A is a range inside the first center frame and indicates a range of the center portion of the first center frame.
Thereby, the subject determination unit 20 excludes a frame edge image region (for example, an image region by a frame-like object such as a window frame or a window frame) that is unlikely to be a subject from the subject candidate image region. be able to. Therefore, the image processing apparatus 1 in the present embodiment can appropriately extract the image area of the subject from the captured image.

In the present embodiment, the central frame includes a central frame B (first central frame) and a central frame A (second central frame) indicating a range inside the central frame B. The frame edge excluding unit 22 determines the image area when the aspect ratio of the image area is equal to or more than a predetermined second threshold K2 and the image area is in the central frame B and the central frame A. It is left as an image area of a subject candidate. In other words, the subject determination unit 20 (frame edge exclusion unit 22) has an aspect ratio of the subject candidate image area equal to or greater than a predetermined second threshold K2, and the subject candidate image area is located inside the center frame B. If it is within the range and the range inside the central frame A, it is determined that the image area of the subject candidate is extracted as the image area of the subject.
As a result, the frame edge exclusion unit 22 can determine in advance an image region that is likely to be a subject and leave it as an image region of a subject candidate. Therefore, the image processing apparatus 1 in the present embodiment can appropriately extract the image area of the subject from the captured image.

In the present embodiment, the frame edge exclusion unit 22 has an image area aspect ratio equal to or greater than a predetermined second threshold K2, the image area is within the central frame B, and the image area is at the center. If it is not in the frame A and there is another image area in the center frame A, it is determined that the image area is a frame edge image area. In other words, the subject determination unit 20 (frame edge exclusion unit 22) has an aspect ratio of the subject candidate image area equal to or greater than a predetermined second threshold K2, and the subject candidate image area is within the central frame B. If the image is in the range outside the central frame A and there is another subject candidate image region in the central frame A, it is determined that the subject candidate image region is not extracted as the subject image region.
Thereby, the frame edge exclusion unit 22 can accurately determine a frame edge image area that is unlikely to be a subject, and excludes the frame edge image area from the subject candidate image area. Therefore, the image processing apparatus 1 in the present embodiment can appropriately extract the image area of the subject from the captured image.

In the present embodiment, the frame edge exclusion unit 22 has an image area aspect ratio equal to or greater than a predetermined second threshold K2, the image area is within the central frame B, and the image area is at the center. If it is not in the frame A and there is no other image area in the central frame A, the image area is left as an image area of the subject candidate. In other words, the subject determination unit 20 (frame edge exclusion unit 22) has an aspect ratio of the subject candidate image area equal to or greater than a predetermined second threshold K2, and the subject candidate image area is located inside the center frame B. If it is within the range and the range outside the central frame A, and there is no image area of another subject candidate in the subject candidate image area, it is determined that the subject candidate image area is extracted as the subject image area. To do.
As a result, the frame edge exclusion unit 22 can determine in advance an image region that is likely to be a subject and leave it as an image region of a subject candidate. Therefore, the image processing apparatus 1 in the present embodiment can appropriately extract the image area of the subject from the captured image.

In the present embodiment, the imaging apparatus 100 includes the image processing apparatus 1.
Thereby, the imaging device 100 can appropriately extract the image area of the subject from the captured image. Therefore, the imaging apparatus 100 can perform AF processing with high accuracy.

In addition, this invention is not limited to said embodiment, It can change in the range which does not deviate from the meaning of this invention.
For example, in the above embodiment, the subject determination unit 20 has been described as including the background exclusion unit 21 and the frame edge exclusion unit 22, but either one of the background exclusion unit 21 and the frame edge exclusion unit 22 is used. It may be provided. For example, in the embodiment including only the frame edge exclusion unit 22, the image processing apparatus 1 calculates a feature amount for each feature of the image input by the feature amount calculation unit 10. The area detection unit 15 extracts a subject candidate image area (first image area) determined for each feature amount calculated by the feature amount calculation unit 10. The subject determination unit 20 (frame edge exclusion unit 22) corresponds to the positional relationship between the image area of the subject candidate extracted by the region detection unit 15 and the central frame (center region) indicating the range of the central portion of the input image. Thus, it is determined whether or not to extract the subject image region (second image region) extracted from the subject candidate image region.
Thereby, the subject determination unit 20 can exclude an image region (frame edge image region) that is unlikely to be a subject from the subject candidate image region. Therefore, the image processing apparatus 1 in the present embodiment can appropriately extract the image area of the subject from the captured image.

  Further, in the process shown in FIG. 2, after the background image area exclusion process (step S107) by the background exclusion unit 21 is executed, the frame edge image area exclusion process (step S108) by the frame edge exclusion part 22 is executed. However, the present invention is not limited to this. For example, the frame edge image region exclusion process (step S108) by the frame edge exclusion unit 22 may be executed prior to the background image region exclusion process (step S107) by the background exclusion unit 21.

  In the above-described embodiment, the form in which the isolated image area is excluded as a noise image area (noise mask) has been described. However, an image area smaller than a predetermined size (an extremely small image area) or a predetermined size is determined. An image area larger than the size (an extremely large image area) may be a noise image area.

  In the above embodiment, the image processing unit 140 has been described as generating a rectangular area for AF, but may be configured to generate a polygonal area. Further, the image processing unit 140 has been described as generating the AF rectangular area as a rectangular frame circumscribing the image area of the subject, but may be configured to generate an area slightly larger than the image area of the object. Other forms are also possible.

  In the above-described embodiment, the form in which the image area of the subject extracted by the image processing apparatus 1 is applied to the AF control has been described. However, the present invention is not limited to this. For example, the extracted image area of the subject may be applied to image processing such as white balance processing or image area processing. Further, the extracted image area of the subject may be applied to image processing in which an image is cut out and used, or pasted like superimpose.

  1 is recorded in a computer-readable recording medium, and the program recorded in the recording medium is read into a computer system and executed. By doing so, each process of the imaging apparatus 100 and the image processing apparatus 1 may be performed. Here, the “computer system” includes an OS and hardware such as peripheral devices.

Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used.
The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Furthermore, the “computer-readable recording medium” dynamically holds a program for a short time like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory in a computer system serving as a server or a client in that case, and a program that holds a program for a certain period of time are also included. The program may be a program for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system.

  DESCRIPTION OF SYMBOLS 1 ... Image processing apparatus, 10 ... Feature-value calculation part, 15 ... Area | region detection part, 20 ... Subject determination part, 21 ... Background exclusion part, 22 ... Frame edge exclusion part, 100 ... Imaging device

Claims (14)

  A detection unit for detecting a plurality of subject areas from the image;
  An extraction unit that extracts a subject region that is in contact with two opposite sides and forms a first region of the image among the plurality of subject regions detected by the detection unit;
  An image processing apparatus comprising:   A detection unit for detecting a plurality of subject areas from the image;
  An extraction unit for extracting a specific subject region from the plurality of subject regions detected by the detection unit;
  With
  The image processing apparatus, wherein the extraction unit does not extract a subject region whose area in contact with one side forming the first region of the image is equal to or greater than a first threshold.   A detection unit for detecting a plurality of subject areas from the image;
  An extraction unit for extracting a specific subject region from the plurality of subject regions detected by the detection unit;
  With
  The extraction unit does not extract a subject area in which a ratio of a length of one side forming the first area of the image and a length of a portion where the subject area is in contact with the side is equal to or more than a second threshold value. Processing equipment.   4. The extraction unit according to claim 1, wherein the extraction unit does not extract a subject region that is in contact with two adjacent sides forming the first region among the plurality of subject regions detected by the detection unit. The image processing apparatus according to any one of claims.   The extraction unit is included in a second region in which at least a part of the subject region is a region within the first region and includes the center of the image among the plurality of subject regions detected by the detection unit. 5. The image processing apparatus according to claim 1, wherein a subject area being extracted is extracted.   A detection unit for detecting a plurality of subject areas from the image;
  An extraction unit that extracts a subject region that is in contact with two opposing sides and forms a first range defined in the image among the plurality of subject regions detected by the detection unit;
An image processing apparatus comprising:   A detection unit for detecting a plurality of subject areas from the image;
  An extraction unit for extracting a specific subject region from the plurality of subject regions detected by the detection unit;
  With
  The image processing apparatus, wherein the extraction unit does not extract a subject region whose area in contact with one side forming a first range defined in the image is equal to or greater than a first threshold.   A detection unit for detecting a plurality of subject areas from the image;
  An extraction unit for extracting a specific subject region from the plurality of subject regions detected by the detection unit;
  With
  The extraction unit includes a portion of the plurality of subject areas detected by the detection unit that is in contact with the length of one side forming the first range defined in the image and the subject region at the side. An image processing apparatus that does not extract a subject area whose ratio is equal to or greater than a second threshold.   9. The extraction unit according to claim 6, wherein the extraction unit does not extract a subject region in contact with two adjacent sides forming the first range among the plurality of subject regions detected by the detection unit. The image processing apparatus according to any one of claims.   The extraction unit is included in a second range in which at least a part of the subject region is within the first range and includes the center of the image among the plurality of subject regions detected by the detection unit. The image processing apparatus according to claim 6, wherein a subject area that has been extracted is extracted.   The image processing apparatus according to any one of claims 1 to 10,
  A control unit that controls a lens that guides an optical image to an imaging device by the subject region extracted by the extraction unit of the image processing apparatus;
  A lens control device.   An image pickup device for picking up an optical image emitted from the lens;
  The image processing apparatus according to any one of claims 1 to 10,
  A control unit that controls the lens according to the subject region extracted by the extraction unit of the image processing apparatus;
  An imaging apparatus comprising:   On the computer,
  A detection step for detecting a plurality of subject areas from the image;
  An extraction step of extracting a subject region that is in contact with two opposing sides and forms a first region of the image among the plurality of subject regions detected in the detection step;
  A program for running   On the computer,
  A detection step for detecting a plurality of subject areas from the image;
  An extraction step of extracting a subject region that is in contact with two opposing sides and forms a first range defined in the image among the plurality of subject regions detected in the detection step;
  A program for running

Images