JP5871175B2 - Region extraction device, imaging device, and region extraction program - Google Patents

Description

  The present invention relates to a region extraction device, an imaging device, and a region extraction program that extract a region from an image.

  An image processing apparatus that extracts a region from a captured image and identifies the extracted region based on a histogram of image feature values is disclosed (see Patent Document 1).

Japanese Patent Laid-Open No. 4-10079

  However, the image processing device disclosed in Patent Document 1 cannot extract a region that is noticed by a person from a captured image, that is, a region of a main subject.

  The present invention has been made in view of the above points, and an object of the present invention is to provide a region extraction device, an imaging device, and a region extraction program that can extract a region of a main subject from a captured image. .

The present invention has been made to solve the above problem, and is an area extraction device that extracts an area for calculating a contrast value used for position adjustment of a focus adjustment lens, and includes a plurality of object areas included in an image. A first region extraction unit that extracts a first subject region based on color information or luminance information, and at least a part of the subject region in a region including a center of an angle of view of the image among the plurality of subject regions a second region extracting unit that extracts the second object region that contains a pre-SL of the first object region and the second object region determination unit for selecting a specific object region for calculating the contrast value, a realm extraction device Ru comprising a.

Further, the present invention is that an imaging device includes a realm extraction equipment.

Further, the present invention causes a computer, among a plurality of the subject region contained in the images, the procedure by the color information or luminance information extracting a first object region, among the plurality of the object area, the angle of view of the image For extracting a second subject area in which at least a part of the subject area is included in an area including the center, and for adjusting the position of the focus adjustment lens among the first subject area and the second subject area. And a procedure for selecting a specific subject area for calculating a contrast value to be used .

  According to the present invention, the region extraction device can extract the region of the main subject from the captured image.

1 is a block diagram illustrating a configuration of a lens barrel 111, an imaging device 100 including a region extraction device 140, and a storage medium 200 in an embodiment of the present invention. It is a figure for demonstrating the method to extract an area | region from the image in one Embodiment of this invention. It is a figure for demonstrating the method to extract an area | region from the center of an angle of view in one Embodiment of this invention. It is a figure for demonstrating the method to extract an area | region from the vicinity of the view angle center in one Embodiment of this invention. It is the histogram of saturation S (C) in one Embodiment of this invention, and its partial enlarged view. FIG. 6 is a diagram illustrating an example of a hue histogram near the center of an angle of view according to an embodiment of the present invention. It is a figure showing an example of the area | region extracted from the vicinity of the view angle center in one Embodiment of this invention. It is a figure showing an example of the area | region extracted from the designated position in one Embodiment of this invention. It is a figure showing the example of operation of the imaging device 100 in the case of making it focus on the area | region extracted from the designated position in one Embodiment of this invention. It is a figure showing an example of the extracted field based on the position which detected the subject in one embodiment of the present invention. It is a flowchart showing the procedure of the process which extracts an area | region from the image where the person and the animal were imaged in one Embodiment of this invention. It is a flowchart showing the procedure of the process until it focuses on an area | region in one Embodiment of this invention.

  An embodiment of the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram illustrating the configuration of the lens barrel 111, the imaging device 100 including the region extraction device 140, and the storage medium 200. The imaging device 100 captures an optical image incident from the lens barrel 111 and stores the obtained image in the storage medium 200 as a still image or a moving image.

First, the configuration of the lens barrel 111 will be described.
The lens barrel 111 includes a focus adjustment lens (hereinafter referred to as “AF (Auto Focus) lens”) 112, a lens driving unit 116, an AF encoder 117, and a lens barrel control unit 118. The lens barrel 111 may be detachably connected to the imaging device 100 or may be integrated with the imaging device 100.

  The AF lens 112 is driven by a lens driving unit 116 and guides an optical image to a light receiving surface (photoelectric conversion surface) of an image sensor 119 of the image capturing unit 110 described later.

  The AF encoder 117 detects the movement of the AF lens 112 and outputs a signal corresponding to the movement amount of the AF lens 112 to the lens barrel control unit 118. Here, the signal corresponding to the movement amount of the AF lens 112 may be a sine wave signal whose phase changes according to the movement amount of the AF lens 112, for example.

  The lens barrel control unit 118 controls the lens driving unit 116 in accordance with a drive control signal input from the CPU 190 of the imaging apparatus 100. Here, the drive control signal is a control signal for driving the AF lens 112 in the optical axis direction. The lens barrel control unit 118 changes the number of steps of the pulse voltage output to the lens driving unit 116, for example, according to the drive control signal.

  Further, the lens barrel control unit 118 outputs the position (focus position) of the AF lens 112 in the lens barrel 111 to the CPU 190 based on a signal corresponding to the movement amount of the AF lens 112. Here, the lens barrel control unit 118 integrates, for example, signals according to the movement amount of the AF lens 112 according to the movement direction of the AF lens 112, thereby moving the AF lens 112 in the lens barrel 111 (the movement amount ( Position) may be calculated.

  The lens driving unit 116 drives the AF lens 112 in accordance with the control of the lens barrel control unit 118 and moves the AF lens 112 in the optical axis direction within the lens barrel 111.

Next, the configuration of the imaging device 100 will be described.
The imaging device 100 includes an imaging unit 110, a region extraction device 140, a display unit 150, a buffer memory unit 130, an operation unit 180, a storage unit 160, a CPU 190, and a communication unit 170.

  The imaging unit 110 includes an imaging element 119 and an A / D (Analog / Digital) conversion unit 120. The imaging unit 110 is controlled by the CPU 190 in accordance with the set imaging conditions (for example, aperture value, exposure value, etc.).

  The image sensor 119 includes a photoelectric conversion surface, converts an optical image formed on the photoelectric conversion surface by the lens barrel 111 (optical system) into an electric signal, and outputs the electric signal to the A / D conversion unit 120. The image sensor 119 may be configured by, for example, a CMOS (Complementary Metal Oxide Semiconductor). Further, the image sensor 119 may convert an optical image into an electrical signal for a partial region of the photoelectric conversion surface (image clipping).

  Further, the image sensor 119 stores an image obtained when a shooting instruction from the user is received via the operation unit 180 in the storage medium 200 via the A / D conversion unit 120. On the other hand, the imaging device 119 performs A / D conversion on the buffer memory unit 130 and the display unit 150 as images that are continuously obtained in a state where a shooting instruction from the user is not received via the operation unit 180. Output via the unit 120.

  The A / D converter 120 digitizes the electrical signal converted by the image sensor 119 and outputs an image that is a digital signal to the buffer memory unit 130.

  The operation unit 180 includes, for example, a power switch, a shutter button, a multi selector (cross key), or other operation keys. The operation unit 180 receives a user operation input when operated by the user, and a signal corresponding to the received operation input. Is output to the CPU 190. In addition, the operation unit 180 receives an operation input for the user to specify a predetermined position in the image, and outputs a signal corresponding to the received operation input to the region extraction device 140. Note that the operation unit 180 may be provided in the region extraction device 140.

  The area extraction device 140 performs image processing on the image temporarily stored in the buffer memory unit 130 based on the image processing conditions stored in the storage unit 160. The processed image is stored in the storage medium 200 via the communication unit 170. The region extraction device 140 extracts a region determined by the feature amount of the image from the image temporarily stored in the buffer memory unit 130. Further, the region extraction device 140 stores information indicating the extracted region in the storage unit 160 or the like. Here, the information indicating the region includes, for example, information indicating the distribution of the region and information indicating the feature amount of the region. Further, the region extraction device 140 sets a priority for the region extracted from the image. Details of the region extraction device 140 will be described later.

  The display unit 150 displays an image obtained by the imaging unit 110, an operation screen, and the like. The display unit 150 is, for example, a liquid crystal display.

  The buffer memory unit 130 temporarily stores the image captured by the imaging unit 110.

  The storage unit 160 stores determination conditions referred to by the CPU 190 in scene determination, imaging conditions associated with each scene determined by scene determination, information indicating a region, and the like.

  The CPU 190 controls the imaging unit 110 according to the set imaging conditions (for example, aperture value, exposure value). In addition, the CPU 190 acquires information indicating a region from the region extraction device 140. In addition, the CPU 190 outputs a drive control signal to the lens barrel control unit 118 of the lens barrel 111 based on the information indicating the region, so that the priority set by the region extracting device 140 among the regions extracted from the image is set. Adjust the focus to focus on a high area.

  Also, the CPU 190 performs focus adjustment (AF) setting, exposure adjustment (AE) setting, white balance adjustment (AWB) setting, subject image as pre-processing or post-processing of image processing based on the information indicating the area. (Object) tracking setting, night scene judgment setting, color correction processing setting, subject image enlarged display setting, panning display setting, brightness optimization setting linked with zoom magnification, etc. To control.

  Further, the CPU 190 controls setting of a tone curve (gradation curve), whether or not to execute simultaneous shooting of a still image and a moving image, and the like based on information indicating a region.

  In addition, the CPU 190 sets whether to perform continuous shooting as pre-processing or post-processing based on information indicating the area, setting for switching the subject image to be focused in continuous shooting, and continuously changing the focal length. , Setting for changing the amount of flash light emitted from the light emitting unit (not shown) of the image capturing apparatus 100, setting for changing the exposure amount, setting for whether to change the shutter speed, and the like.

  In addition, the CPU 190 causes the region extraction device 140 to perform image processing on an image in a still image format or a moving image format based on the “signal according to the operation input” input from the operation unit 180.

  The communication unit 170 is connected to a removable storage medium 200 such as a card memory, and writes, reads, or deletes information (image data, information indicating an area, etc.) on the storage medium 200.

  The storage medium 200 is a storage unit that is detachably connected to the imaging apparatus 100, and stores information (image data, information indicating an area, and the like). Note that the storage medium 200 may be integrated with the imaging device 100.

Next, details of the region extraction device 140 will be described.
The region extraction device 140 includes a feature amount calculation unit 141, a region extraction unit 142, a determination unit 143, a priority setting unit 144, and a storage unit 145.

  The feature amount calculation unit 141 acquires the image captured by the imaging unit 110 from the buffer memory unit 130. The feature amount calculation unit 141 calculates the feature amount for each of a plurality of features (for example, hue, saturation, brightness, texture, and edge) in the image acquired from the buffer memory unit 130, and calculates the calculated feature amount. The data is output to the area extraction unit 142.

  The region extraction unit 142 acquires the feature amount calculated by the feature amount calculation unit 141. Further, the region extraction unit 142 acquires an image captured by the imaging unit 110 from the buffer memory unit 130. Further, the region extraction unit 142 extracts a region included in the acquired image and determined for each feature amount from the image. Here, the region extracted by the region extraction unit 142 is, for example, a region of a subject image included in an image captured by the imaging unit 110.

  The region extraction unit 142 acquires the image feature amount calculated by the feature amount calculation unit 141. Further, the region extraction unit 142 acquires an image captured by the imaging unit 110 from the buffer memory unit 130. The region extraction unit 142 extracts a region from the image captured by the imaging unit 110 based on image feature amounts (for example, color information (hue, color difference), saturation, luminance).

  The region extraction unit 142 also includes a region (hereinafter referred to as a “second image”) from a certain range (section) defined for the image feature amount and an image binarized outside the range (hereinafter referred to as “binary image”). 1 area ”) and a label is assigned to the extracted first area (labeling). The area extracting unit 142 determines an AF evaluation area at the position of the extracted first area. Here, the AF evaluation area is an area where the contrast value is evaluated by contrast autofocus (AF).

  The region extraction unit 142 extracts a region (hereinafter referred to as “second region”) from a predetermined position in the captured image (special mask). For example, the region extraction unit 142 extracts the second region from the center of the view angle or the vicinity of the view angle center based on the image feature amount at the view angle center or the vicinity of the view angle center. In addition, the area extraction unit 142 extracts the second area based on the position of the area detected by the process (for example, face detection process) for detecting the area where the unique type of subject is imaged, and extracts the extracted second area. Assign a label to the area (labeling). In addition, the region extraction unit 142 extracts a second region from the position in the image designated through the operation unit 180 by the operation input. Details of the operation of the region extraction unit 142 will be described later.

  The determination unit 143 selects one or more regions as a target for setting the priority from the plurality of first regions and the second region based on the distribution of each region (final mask).

  More specifically, the determination unit 143 determines whether or not the first region or the second region satisfies a predetermined condition (for example, a region distribution condition). For example, when there are regions that are estimated to be noise (for example, regions having an area smaller than a predetermined threshold) in the image, the determination unit 143 removes (adds) those regions from the image. Further, for example, when the region estimated as the background (for example, a region having an area larger than the predetermined threshold) is included in the image, the determination unit 143 removes the region from the image. Further, for example, the determination unit 143 selects, for example, a region having a large degree of mass, that is, a region having a small moment of inertia from the center of gravity, and adds another region (a region where the moment of inertia is not small). The determination unit 143 stores information representing the distribution of the first region and the second region that have not been removed in at least one of the storage unit 145, the storage unit 160, and the storage medium 200.

  The priority setting unit 144 determines “region priority” in the image in the first region and the second region that satisfy a predetermined condition. The “region priority” is a region priority that a person will be more interested in. The priority setting unit 144 stores the set “region priority” in the storage unit 145, the storage unit 160, and the storage medium 200 for each region as information indicating the region.

  The priority of the area may be determined as follows. In the following, it is assumed that the priority of a region is higher as the number indicating the priority is smaller. Moreover, the priority shown below is an example.

Priority 1.
A moving area (when there are a plurality of moving areas in the image, an area that does not protrude from the angle of view or an area that is distributed in the center of the angle of view). Also, if the moving area is widely distributed in the image due to panning, the area is not moving.
These areas are set to priority “1”.
This is because a person pays attention to a moving area and an area having a small amount of change when panning.

Priority 2.
Areas related to angle of view (composition), depth (perspective), and subject distance (depth, depth). An area distributed near the center of the angle of view. When there is a depth in the image, among the regions distributed on the closest side, the region does not protrude from the angle of view and has an area within a predetermined range (not extremely large).
These areas are set to priority “2”.
This is because the main subject image area may exist in a wide range from close to infinity.

Priority 3.
Regions extracted based on human face detection, animal (pet) face detection, skin color detection in the area where the face is detected, animal fur texture (pattern), or plant texture.
These areas are set to priority “3”.
This is because when a person or animal is imaged, the person pays attention to the face.

Priority 4.
A region having a characteristic amount that is unique (for example, a hue that is discontinuous in the hue circle). An area divided into a background area and other areas. Highly saturated area. An area where the brightness is significantly different from the surrounding area. An area where the hue (hue) is significantly different from the surrounding area. An area where the texture (pattern) is different from the surrounding area. An area where the spatial frequency (image frequency) is higher than the surrounding area. A region in which an image has a sense of incongruity when a certain amount of discontinuous distribution has a certain amount and a region is formed as a result of labeling based on the feature amount.
These areas are set to priority “4”.

Priority 5.
An area extracted based on a specific hue or saturation.
These areas are set to the priority “5”.
This is because a person pays attention to red, yellow, and flesh-colored areas.

Priority 6.
An area extracted based on brightness (brightness).
These areas are set to priority “6”.

Priority 7.
An area extracted based on a texture (pattern).
These areas are set to priority “7”.

Priority 8.
A region extracted based on the spatial frequency (image frequency).
These areas are set to priority “8”.

Priority 9.
An area surrounded by edges extending vertically or horizontally and having a certain thickness or more.
These areas are set to the priority “9”.

Next, details of operations of the region extraction unit 142 and the determination unit 143 will be described.
The region extraction unit 142 acquires the image feature amount calculated by the feature amount calculation unit 141. Further, the region extraction unit 142 acquires an image captured by the imaging unit 110 from the buffer memory unit 130. The region extraction unit 142 extracts a region from the image captured by the imaging unit 110 based on image feature amounts (for example, color information (hue, color difference), saturation, luminance). Here, the region extracted by the region extraction unit 142 is a region of the subject included in the image acquired from the buffer memory unit 130.

  The area extraction unit 142 extracts a first area from the binary image and assigns a label to the extracted first area (labeling). Here, the binary image is an image generated by dividing the image feature amount by a threshold (upper limit and lower limit). Further, this threshold value may be defined as, for example, “average value + coefficient × standard deviation” based on the average value (Ave) and standard deviation (σ) of the histogram of the image feature amount in the image.

  FIG. 2 shows a diagram for explaining a method of extracting a region from an image. The area extraction unit 142 extracts an image 300 </ b> Y based on luminance (hereinafter referred to as “Y image”) from the image captured by the imaging unit 110. The area extraction unit 142 corrects the luminance (input value) of the image captured by the imaging unit 110 (for example, gamma correction), and calculates the corrected luminance (output value).

  The area extraction unit 142 divides the corrected luminance (output value) into a plurality of ranges. In FIG. 2, as an example, it is assumed that the corrected luminance is divided into sections 1 to 4. The area extraction unit 142 generates a binary image for each section. When the upper limit of the category 1 is “average value + K1 × standard deviation” and the lower limit of the category 1 is “average value + K2 × standard deviation”, the binary image of the category 1 includes a high luminance area.

  The region extraction unit 142 searches the distribution of the high-intensity region included in the binary image of section 1 and extracts the high-intensity region from the region image representing the search result. Similarly, the region extracting unit 142 extracts a medium luminance region from the region image by searching for a medium luminance region included in the binary image of category 2. In addition, the region extraction unit 142 extracts a low luminance region from the region image by searching for a low luminance region included in the binary image of category 4.

  Similarly, the area extraction unit 142 divides the corrected color information Cr into a plurality of ranges. In FIG. 2, as an example, it is assumed that the corrected color information Cr is divided into sections 5 to 8. The area extraction unit 142 generates a binary image for each section. When the upper limit of the category 5 is “average value + K3 × standard deviation” and the lower limit of the category 5 is “average value + K4 × standard deviation”, the binary image of the category 5 includes a red-based region.

  The area extraction unit 142 searches for a red-type area (not shown) in the area image representing the search result by searching for the distribution of the red-type area included in the binary image of category 5. Similarly, the area extraction unit 142 searches for a pink area included in the binary image of section 6 and extracts a pink area (not shown) from the area image representing the search result. In addition, the region extraction unit 142 extracts a green region (not shown) from the region image by searching for a green region included in the binary image of category 8.

  Similarly, the area extraction unit 142 divides the corrected color information Cb into a plurality of ranges. In FIG. 2, as an example, it is assumed that the corrected color information Cb is divided into sections 9 to 12. The area extraction unit 142 generates a binary image for each section. When the upper limit of the category 9 is “average value + K5 × standard deviation” and the lower limit of the category 9 is “average value + K6 × standard deviation”, the binary image of the category 9 includes a blue region.

  The region extraction unit 142 searches for the distribution of the blue region included in the binary image of section 9 and extracts a blue region (not shown) from the region image representing the search result. Similarly, the region extraction unit 142 searches for a light blue system region included in the binary image of the category 10, and extracts a light blue system region (not shown) from the region image representing the search result. In addition, the region extraction unit 142 searches for a yellow region included in the binary image of the category 12, thereby extracting a yellow region (not shown) from the region image representing the search result.

  The determination unit 143 selects one or more regions as targets for setting priority from the plurality of first regions and second regions based on the distribution of the regions. In FIG. 2, it is assumed that, as an example, the first region 340Y and the first region 350Y are selected by the determination unit 143 as targets for setting priority among the plurality of first regions extracted in the region image. The area extraction unit 142 sets the AF evaluation area 360 at the position of the first area 340Y and sets the AF evaluation area 370 at the position of the first area 350Y.

Furthermore, the region extraction unit 142 and the determination unit 143 perform region extraction processing in one or more cases described below.
<When extracting an area from the center of the angle of view of the image>
FIG. 3 shows a diagram for explaining a method of extracting a region from the center of the angle of view. The region extraction unit 142 extracts a second region 320Y including pixels having a certain range of luminance (Y ± K · α) in the luminance distribution from the angle-of-view center 310Y of the Y (luminance) image 300Y. Further, the region extraction unit 142 extracts a second region 330Cr including pixels having a certain range of color differences (Cr ± K · β) in the color difference distribution from the field angle center 310Cr of the Cr (color difference) image 300Cr. Further, the region extraction unit 142 extracts a second region composed of pixels having a certain range of color difference (Cb ± K · γ) in the color difference distribution from the angle-of-view center 310Cb of the Cb (color difference) image 300Cb. Here, the second region based on any feature amount may not be extracted because a pixel having a feature amount such as luminance (Y) or color difference (Cr, Cb) is not included in the image.

  Note that the feature amount of the image is not limited to Y, Cr, and Cb, and may be any feature amount that can extract a region, such as RGB, texture (pattern), or spatial frequency in the image.

  The determination unit 143 selects a second region having a large degree of mass, that is, a second region having a small moment of inertia from the center of gravity among the plurality of second regions. Here, in FIG. 3, the determination unit 143 compares the degree of mass of the second area 320Y extracted from the image 300Y with the degree of mass of the second area 330Cr extracted from the image 300Cr, and the degree of mass is greater. It is assumed that the large second area 320Y is selected.

  In addition, the region extraction unit 142 further accurately extracts the second region from the vicinity of the selected second region based on the same feature amount used when the second region selected by the determination unit 143 is extracted. It may be extracted. For example, in FIG. 3, since the second region 320Y is extracted based on the luminance, the region extracting unit 142 further increases the accuracy based on the emphasized luminance from the vicinity of the second region 320Y selected by the determination unit 143. The second region may be extracted well.

  Also, the determination unit 143 selects a region having a high evaluation (for example, a region having a large area) among the first region and the second region for each type of image feature amount.

<When extracting an area from the vicinity of the center of the image angle>
The region extraction unit 142 extracts the second region from the vicinity of the center of the angle of view (for example, a range from the center of the angle of view to a predetermined distance). FIG. 4 is a diagram for explaining a method of extracting a region from the vicinity of the center of the angle of view. The region extraction unit 142 determines a plurality of positions in the region 410 defined in the vicinity of the center of the angle of view. In FIG. 4, as an example, five positions are determined in advance.

  The region extraction unit 142 extracts regions from each position based on Y (luminance) at each predetermined position. Further, the region extraction unit 142 extracts a region from each position based on Cr (color difference) at each position. Further, the region extraction unit 142 extracts a region from each position based on Cb (color difference) at each position. Here, the second region based on any image feature amount may not be extracted because there is no pixel having an image feature amount such as luminance or color difference.

  The determination unit 143 selects the second region having the highest evaluation (for example, the highest density) among the second regions extracted based on the feature amounts of the image at the position 470a in the region 410. The same applies to the positions 470b to 470e. In FIG. 4, it is assumed that the second region 420Y and the second region 430Y extracted based on the luminance are selected by the region extraction unit 142.

  Further, the region extraction unit 142 selects the second region with higher accuracy based on the same image feature amount used when the second region is extracted from the vicinity of the second region selected by the determination unit 143. It may be extracted. For example, in FIG. 4, since the second region 420Y is extracted based on the luminance, the region extracting unit 142 further performs the detection based on the emphasized luminance from the vicinity of the selected second region 420Y and the second region 430Y. The second region may be extracted with high accuracy.

  In addition, the determination unit 143 selects a region having a high evaluation (for example, a region distributed on the close side) among the first region and the second region for each type of image feature amount.

<When extracting an area based on a histogram>
The region extraction unit 142 generates a histogram for the feature amount of the image near the center of the angle of view. FIG. 5 shows a histogram of saturation S (C) and a partially enlarged view thereof. The horizontal axis represents the saturation S (C). The horizontal axis represents frequency. Further, the region extraction unit 142 extracts the second region based on the histogram of the feature amount of the image.

  The region extraction unit 142 scans the histogram of saturation S to detect a saturation peak S1 having a relatively high frequency from the histogram, and partially enlarges the detected saturation peak S1. The region extracting unit 142 further calculates the deviation σ of the saturation peak S1 by further scanning the partially enlarged saturation peak S1. In addition, the region extraction unit 142 extracts, from the image, a second region including adjacent pixels having saturation included in the 3σ range centered on the saturation peak S1.

  FIG. 6 shows an example of a hue histogram near the center of the angle of view. The horizontal axis represents the hue h. The horizontal axis represents frequency. The region extraction unit 142 scans the hue histogram to detect a hue peak h1 having a relatively high frequency (predicted as a unique hue) from the histogram. In addition, the region extraction unit 142 calculates a deviation σ of the hue peak h1, and extracts, from the image, a second region including adjacent pixels having hues included in the 3σ range centered on the hue peak h1.

  The region extraction unit 142 further scans the hue histogram to detect the hue h2 included in the discontinuous histogram region (discontinuous region). Further, the region extraction unit 142 calculates a deviation σ of the hue h2, and extracts from the image a second region composed of adjacent pixels having a hue included in the “h2 ± Δ / 2” range centered on the hue h2. .

  FIG. 7 shows an example of a region extracted from the vicinity of the center of the angle of view. The second region 520 is extracted by the region extraction unit 142 based on the histogram of the feature amount of the image in the region 510 defined near the center of the angle of view of the image 500.

<When extracting an area from a specified position>
The region extraction unit 142 extracts a second region from a region located at a predetermined position in the image designated by the operation input. For example, an operation input signal corresponding to the operation input is input to the region extraction unit 142 from the operation unit 180. Here, the predetermined position is, for example, a position in the screen where the user presses the display unit 150 or a position in the screen designated by the user's line-of-sight input.

  FIG. 8 shows an example of a region extracted from the designated position. In FIG. 8, an area 610 is defined at a position in the image 600 designated by the operation input. The region extraction unit 142 performs the second region 620 from the vicinity of the region 610 based on the image feature amount with the gradation subdivided, for example, the luminance (Y) and the color difference (Cr, Cb) with the gradation subdivided. To extract. In addition, the region extraction unit 142 determines an AF evaluation region 630 including the region 610 as an image.

  In addition, the region extraction unit 142 expands the AF evaluation region 630 so as to include all the contour lines of the second region 620. Thereby, the area extraction unit 142 can increase the contrast value (signal-to-noise ratio) used for contrast / autofocus.

  FIG. 9 illustrates an operation example of the imaging apparatus 100 when focusing on an area extracted from a designated position. In the example shown in FIG. 9A, it is assumed that the user changes the camera angle and adjusts the subject to be photographed to the center of the angle of view. The area extraction unit 142 defines an AF evaluation area 640 for the subject. When the user presses down the operation unit 180, the region extraction unit 142 determines a target region to be focused. For example, it is assumed that the region extracting unit 142 determines a subject region in the AF evaluation region 640 as a target region to be focused. Thereafter, the area extracting unit 142 focuses on the area of the subject in the AF evaluation area 640 even when the user changes the camera angle.

  In the example shown in FIG. 9B, it is assumed that an AF evaluation area 650 to an AF evaluation area 670 is defined as an area of each subject because a plurality of subjects are captured in the image. When the user performs a dial operation on the operation unit 180, the region extraction unit 142 determines a region to be focused. For example, it is assumed that the region extraction unit 142 determines a subject region in the AF evaluation region 660 as a target region to be focused. Thereafter, the area extraction unit 142 focuses on the subject area in the AF evaluation area 660 even when the user changes the camera angle.

<When extracting an area based on the position where the subject is detected>
The area extraction unit 142 extracts the second area based on the position in the image indicated by the result of the face detection process. Here, the face detection processing includes processing for detecting the position and orientation of an animal's face in addition to processing for detecting the position and orientation of a human face.

  FIG. 10 shows an example of the extracted area based on the position where the subject is detected. It is assumed that the face position 840 and the orientation of the subject's face are detected by the face detection process for the imaged screen 800. The region extraction unit 142 extracts the second region 830 from the position 840 and assigns a label to the extracted second region 830 (labeling).

  Further, the region extraction unit 142 determines a position 850 to a position 870 where the subject's body is estimated to be imaged based on the orientation of the subject's face and the position 840 of the subject's face. Here, it is assumed that information representing a correspondence relationship between the position of the subject's face, the orientation of the subject's face, and the position where the subject's body is estimated to be captured is stored in the storage unit 145 in advance. . In addition, the region extraction unit 142 extracts a second region 880 that is estimated that the subject's body is imaged from the position 850 to the position 870, and assigns a label to the extracted second region 880.

<When extracting regions from images of humans and animals>
FIG. 11 is a flowchart illustrating a procedure of processing for extracting a region from an image in which a person and an animal are captured. The area extraction unit 142 initializes variables used for the process of extracting an area (step S1). The feature amount calculation unit 141 reads a cropped image (64 × 40 pixels) obtained by resizing the captured image for each image feature amount via the buffer memory unit 130 (step S2).

  The feature amount calculation unit 141 performs face detection processing on the cropped image and outputs the processing result to the region extraction unit 142. The region extraction unit 142 determines whether a human face is detected in the cropped image based on the processing result (step S3).

  When a human face is detected in the cropped image (step S3-Yes), the region extracting unit 142 displays a frame image including the human face, and the area inside the displayed frame is a constant value (for example, the cropped image). It is determined whether or not the area is equal to or larger than 1/16 of the area (step S4).

  When the area inside the displayed frame is less than a certain value (step S <b> 4 -No), the region extraction unit 142 includes a region where a human face is imaged and a region where a torso is imaged. Are extracted based on the position where the human face is detected (step S5). Further, the region extraction unit 142 extracts the second region from, for example, the vicinity of the center of the angle of view of the cropped image based on the luminance, the color difference (YUV, YCrCb), and the like (Step S6).

  Based on a predetermined condition, the determination unit 143 performs the addition and integration on the first region and the second region extracted by the region extraction unit 142. For example, when a plurality of regions having an area smaller than the predetermined threshold are close to each other in the image, the determination unit 143 integrates these regions (step S7). In addition, the determination unit 143 selects a region (for example, the first to fourth regions) in descending order of evaluation among the first region and the second region for each type of image feature amount (step S8).

  Further, the priority setting unit 144 sets priorities for the first area and the second area selected by the determination unit 143. Further, the area extracting unit 142 sets the AF evaluation area in the area having the highest priority set by the priority setting unit 144 (step S9). The CPU 190 executes a focus adjustment (AF) process based on the contrast value of the AF evaluation area.

  It should be noted that the CPU 190 performs processing for extracting a region from the image, processing for classifying a subject, processing for adjusting a dynamic range, processing for enhancing an image, processing for enhancing an image, and exposure adjustment (exposure adjustment) AE), white balance adjustment (AWB), tracking of the subject image (object), determination of whether it is a night scene, color correction processing, enlarged display of the subject image, panning display, optimization of brightness linked to the zoom magnification, etc. Control.

  On the other hand, if no human face is detected in the cropped image in step S3 (step S3-No), the region extraction unit 142 determines whether an animal (pet) face is detected in the cropped image. (Step S10). When the animal face is detected in the cropped image (step S10-Yes), the region extracting unit 142 extracts the second region in which the torso is imaged from the periphery of the animal face (step S10). S11).

  In addition, the region extraction unit 142 displays a frame image including the animal's face, and whether the area inside the displayed frame is equal to or greater than a certain value (for example, 1/16 of the area of the crop image). It is determined whether or not (step S12). When the area inside the displayed frame is greater than or equal to a certain value (step S12—Yes), the region extraction unit 142 advances the process to step S9.

  On the other hand, if the area inside the displayed frame is greater than or equal to a certain value in step S4 (step S4-Yes), the region extraction unit 142 determines the region that is estimated that the upper body has been imaged as the periphery of the human face. (Step S13). The area extracting unit 142 proceeds with the process to step S9.

  In step S10, if no animal face is detected in the cropped image (step S10-No), the region extraction unit 142 advances the process to step S6. In step S12, when the area inside the displayed frame is less than a certain value (step S12-No), the region extraction unit 142 advances the process to step S6.

Next, the processing procedure of the region extraction apparatus will be described.
FIG. 12 is a flowchart illustrating a processing procedure until focusing on an area. The feature amount calculation unit 141 captures a captured image (through image or through image sequence) from the buffer memory unit 130 (Step Sa1), and executes basic image processing on the captured image (Step S1). Sa2).

  The area extraction unit 142 reads a cropped image (64 × 40 pixels) obtained by resizing the captured image for each image feature amount via the buffer memory unit 130 (step Sa3). The region extraction unit 142 creates a binarized image based on a plurality of image feature amounts in the crop image (step Sa4). In addition, the region extraction unit 142 extracts the first region and the second region from the binarized image, and assigns a label to the extracted region (step Sa5).

  If there are regions estimated to be noise in the image, the determination unit 143 removes (adds) those regions from the image (step Sa6). Further, when the region estimated to be the background is in the image, the determination unit 143 removes the region from the image (step Sa7).

  The priority setting unit 144 determines “region priority” in the image in the first region and the second region that satisfy a predetermined condition (step Sa8). In addition, the area extraction unit 142 shapes the area for which the priority setting unit 144 determines the priority into a rectangle so as to fit the AF evaluation area (step Sa9). The CPU 190 executes a contrast scan (focus adjustment process) based on the contrast value of the AF evaluation area (step Sa10). Further, the CPU 190 gives priority to the area where the subject in close proximity is imaged based on the result of the contrast scan, and focuses the area (step Sa11).

As described above, the region extraction device 140 uses the first region (for example, the first region 340Y, the first region) based on the image feature amount (for example, color information (hue, color difference), saturation, luminance) from the captured image. 1 region 350Y) and a second region (for example, the second region 320Y, the second region 420Y, the second region 430Y) based on the image feature amount from the position determined in the image (special mask) A region extraction unit 142 that extracts a predetermined condition (for example, a region distribution condition), and determines whether at least one of the first region and the second region satisfies the first region and the second region; Among the second regions, a determination unit 143 that selects the first region and the second region (final mask) satisfying the predetermined condition, and the first region and the second region selected by the determination unit 143 Remember the distribution of A storage unit 145, a.
With this configuration, the region extraction unit 142 extracts a first region from the captured image based on the image feature amount, and extracts a second region from the position determined in the image based on the image feature amount. The determination unit 143 selects the first region and the second region that satisfy a predetermined condition. Thereby, the area extracting apparatus can extract the area of the main subject from the captured image.

In addition, the region extracting unit 142 performs the second operation from the vicinity of the angle of view based on the image feature amount at the view angle center (for example, the view angle center 310Y) or the vicinity of the view angle center (for example, the position 470a). Extract regions.
Thereby, the area extracting apparatus can extract the area of the main subject from the image captured at the center of the angle of view or near the center of the angle of view.

Further, the region extraction unit 142 extracts the second region (for example, the second region 830 and the second region 880) based on the detected face position (for example, the position 840).
Thereby, the area extracting apparatus can extract the area of the main subject from the captured image based on the detected face position.

In addition, the region extraction device 140 includes an operation unit 180 that receives an operation input for designating a position determined in the image, and the region extraction unit 142 includes a position determined in the image specified by the operation input. Based on the above, the second region is extracted.
Thereby, the area extracting apparatus can extract the area of the main subject from the image captured at the position determined in the image specified by the operation input.

In addition, the determination unit 143 selects the first region and the second region that satisfy the distribution condition based on distribution conditions (for example, the degree of lumping) that are determined in advance in the first region and the second region. To do.
Thereby, the area extracting apparatus can extract the area of the main subject from the captured image based on a predetermined distribution condition.

Further, the region extraction unit 142 extracts the second region based on a histogram of image feature values.
Thereby, the area extracting apparatus can extract the area of the main subject from the captured image based on the histogram of the image feature amount.

In addition, the imaging apparatus 100 includes a region extraction device 140 having a priority setting unit 144 that determines priority in the first region and the second region, and the priority setting among the first region and the second region. A CPU 190 that adjusts the focus so as to focus on a region with a high priority determined by the unit, and an imaging unit 110 that captures the image in a state of focusing on the region with a high priority.
With this configuration, the imaging apparatus adjusts the focal point so as to focus on a high priority area. Thereby, the imaging device can capture an image in a state in which the region of the main subject extracted from the captured image is in focus.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes designs and the like that do not depart from the gist of the present invention.

  For example, the region extraction device 140 may cause the storage medium 200 to store information indicating the distribution of the region of the subject image included in the image stored in the storage medium 200 and information indicating the feature amount of the region. .

  Further, by recording a program for realizing the region extraction device and the imaging device described above on a computer-readable recording medium, causing the computer system to read and execute the program recorded on the recording medium, Execution processing may be performed. Here, the “computer system” may include an OS and hardware such as peripheral devices.

  Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used. The “computer-readable recording medium” means a flexible disk, a magneto-optical disk, a ROM, a writable nonvolatile memory such as a flash memory, a portable medium such as a CD-ROM, a hard disk built in a computer system, etc. This is a storage device.

Further, the “computer-readable recording medium” refers to a volatile memory (for example, DRAM (Dynamic) in a computer system serving as a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. Random Access Memory)) that holds a program for a certain period of time is also included.
The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line.
The program may be for realizing a part of the functions described above.
Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, and what is called a difference file (difference program) may be sufficient.

DESCRIPTION OF SYMBOLS 100 ... Imaging device, 110 ... Imaging part, 111 ... Lens barrel, 140 ... Area extraction device, 141 ... Feature quantity calculation part, 142 ... Area extraction part, 143 ... Determination part, 144 ... Priority setting part, 145 ... Memory | storage Part

Claims (8)

An area extraction device for extracting an area for calculating a contrast value used for position adjustment of a focus adjustment lens,
A first region extraction unit that extracts a first subject region from color information or luminance information among a plurality of subject regions included in the image;
A second region extraction unit that extracts a second subject region in which at least a part of the subject region is included in a region including the center of the angle of view of the image among the plurality of subject regions ;
Among pre Symbol first object region and the second object region, and a determination unit for selecting a specific object region for calculating the contrast value,
Ru with a realm extraction device. The region extraction device according to claim 1, wherein the determination unit selects a moving region as the specific subject region . The area extracting device according to claim 1 , wherein the determination unit selects the specific subject area based on luminance information of the first subject area and the second subject area . 4. The region extraction device according to claim 1, wherein the determination unit selects the specific subject region based on saturation information of the first subject region and the second subject region . 5. The determination unit of the first object region and the second object region, according to any one of claims 4 to subject area having a specific hue from 請 Motomeko 1 you select as the specific subject area Area extraction device. 6. The determination unit according to claim 1 , wherein the determination unit selects, as the specific subject region, a subject region having a specific saturation from the first subject region and the second subject region. Region extraction device. Provided that imaging device area extraction equipment according to any of claims 6 請 Motomeko 1. On the computer,
Among a plurality of the subject region contained in the images, the procedure for extracting a first object region by the color information or luminance information,
A procedure for extracting a second subject region in which at least a part of the subject region is included in a region including the center of the angle of view of the image among the plurality of subject regions;
A procedure for selecting a specific subject region for calculating a contrast value used for position adjustment of a focus adjustment lens from among the first subject region and the second subject region ;
Extraction program to execute.

Images