JP6350562B2 - Region extraction device, imaging device, and region extraction program - Google Patents

Description

  The present invention relates to a region extraction device, an imaging device, and a region extraction program that extract a region from an image.

  An image processing apparatus that extracts a region from a captured image and identifies the extracted region based on a histogram of image feature amounts is disclosed (see Patent Document 1).

Japanese Patent Laid-Open No. 4-10079

  However, the image processing apparatus disclosed in Patent Document 1 has a problem in that it cannot extract a region of interest from a captured image, that is, it cannot appropriately extract a region of a main subject.

  The present invention has been made in view of the above points, and an object thereof is to provide a region extraction device, an imaging device, and a region extraction program that can appropriately extract a region of a main subject from an input image. And

The present invention has been made to solve the above problems, and includes a first extraction unit that extracts a first region from an image based on color information, and a second extraction that extracts a second region from the image based on luminance information. And a setting unit that sets the second region as a specific region and adjusts the resolution of the second extraction unit when the image signal of the first region is less than a threshold value.

The present invention also provides a first extraction unit that extracts a region from an image based on color information, a second extraction unit that extracts a region from the image based on luminance information, and an image signal of the region extracted by the first extraction unit. Is less than a threshold value, the region extraction apparatus includes a setting unit that sets the region extracted by the second extraction unit as a specific region and adjusts the resolution of the second extraction unit .

In addition, the present invention extracts the first region from the image based on the color information, extracts the second region from the image based on the luminance information, and when the image signal of the first region is less than a threshold, And a setting unit configured to set the second region as a specific region and extract the second region by adjusting the resolution of the extraction unit.

  In addition, the present invention provides an imaging device that includes an imaging device that images light emitted from an optical system, the above-described region extraction device, and a control unit that controls the optical system using a signal from the specific region. It is.

The present invention also provides a computer with a first extraction procedure for extracting a first region from an image based on color information, a second extraction procedure for extracting a second region from the image based on luminance information, and an image of the first region. When the signal is less than a threshold value, an area extraction program for executing an adjustment procedure for setting the second area as a specific area and adjusting the resolution of extraction in the second extraction procedure .

The present invention also provides a computer with a first extraction procedure for extracting a region from an image based on color information, a second extraction procedure for extracting a region from the image based on luminance information, and a region extracted by the first extraction procedure. If the image signal is less than the threshold value, the region extraction for setting the region extracted by the second extraction procedure as the specific region and adjusting the resolution of the extraction in the second extraction procedure is performed. It is a program.

Further, the present invention causes a computer extracts a first region from the image by the color information, the first extraction procedure for extracting the second region from the image by the luminance information, than the image signal of the first region is the threshold If it is, it sets the second region as a specific region, for performing a second extraction procedure for extracting the second region by adjusting the resolution of the extraction of the second region in the first extraction procedure This is an area extraction program.

  According to the present invention, the region extraction device can appropriately extract the region of the main subject from the input image.

It is a block diagram showing the structure of a lens barrel, an imaging device provided with a region extraction device, and a storage medium in an embodiment of the present invention. It is a figure showing the histogram of the brightness | luminance of a backlight image in one Embodiment of this invention. It is a figure showing the histogram of a brightness | luminance in one Embodiment of this invention. It is a figure showing the example of the area | region extracted from the image based on the brightness | luminance in one Embodiment of this invention. It is a figure showing the example of the area | region extracted from the image based on the amount of color components in one Embodiment of this invention. It is a flowchart showing the procedure of the process until it focuses on an area | region in one Embodiment of this invention. It is a flowchart (first half) showing the procedure of the process until it makes it focus on the area | region contained in an achromatic image in one Embodiment of this invention. It is a flowchart (second half) showing the procedure of a process until it makes it focus on the area | region contained in an achromatic color image in one Embodiment of this invention.

  An embodiment of the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram illustrating the configuration of the lens barrel 111, the imaging device 100, and the storage medium 200. The imaging device 100 captures an optical image incident from the lens barrel 111 and stores the obtained image in the storage medium 200 in a still image format or a moving image format.

First, the configuration of the lens barrel 111 will be described.
The lens barrel 111 includes a focus adjustment lens (hereinafter referred to as “AF (Auto Focus) lens”) 112, a lens driving unit 116, an AF encoder 117, and a lens barrel control unit 118. The lens barrel 111 may be detachably connected to the imaging device 100 or may be integrated with the imaging device 100.

  The AF lens 112 is driven by a lens driving unit 116, and guides an optical image to a light receiving surface (photoelectric conversion surface) of an imaging element 119 of the imaging unit 110 described later.

  The AF encoder 117 detects the movement of the AF lens 112 and outputs a signal corresponding to the movement amount of the AF lens 112 to the lens barrel control unit 118. Here, the signal corresponding to the movement amount of the AF lens 112 is a sine wave signal whose phase changes according to the movement amount of the AF lens 112, for example.

  The lens barrel control unit 118 controls the lens driving unit 116 in accordance with a drive control signal input from the CPU 190 of the imaging apparatus 100. Here, the drive control signal is a control signal for driving the AF lens 112 in the optical axis direction. The lens barrel control unit 118 changes, for example, the number of steps of the pulse voltage output to the lens driving unit 116 based on the drive control signal.

  Further, the lens barrel control unit 118 outputs the position (focus position) of the AF lens 112 in the lens barrel 111 to the CPU 190 based on a signal corresponding to the movement amount of the AF lens 112. Here, the lens barrel control unit 118 integrates, for example, a signal corresponding to the movement amount of the AF lens 112 for each movement direction of the AF lens 112, thereby moving the movement amount (position of the AF lens 112 in the lens barrel 111). ) May be calculated.

  The lens driving unit 116 drives the AF lens 112 according to control by the lens barrel control unit 118 and moves the AF lens 112 in the optical axis direction within the lens barrel 111.

Next, the configuration of the imaging device 100 will be described.
The imaging device 100 includes an imaging unit 110, a region extraction device 140, a display unit 150, a buffer memory unit 130, an operation unit 180, a storage unit 160, a CPU 190, and a communication unit 170.

  The imaging unit 110 includes an imaging element 119 and an A / D (Analog / Digital) conversion unit 120. The imaging unit 110 is controlled by the CPU 190 in accordance with the set imaging conditions (for example, aperture value and exposure value).

  The image sensor 119 includes a photoelectric conversion surface, converts an optical image formed on the photoelectric conversion surface by the lens barrel 111 (optical system) into an electrical signal, and converts the converted electrical signal to the A / D conversion unit 120. Output. The image sensor 119 may be configured by, for example, a CMOS (Complementary Metal Oxide Semiconductor). Further, the image sensor 119 may convert an optical image into an electrical signal only for a partial region of the photoelectric conversion surface (image clipping process).

  Further, the image sensor 119 stores a storage medium via the A / D converter 120 as image data based on an electrical signal corresponding to an optical image obtained when a photographing instruction from the user is received via the operation unit 180. 200. On the other hand, the image sensor 119 is an A / D converter 120 as a through image based on an electrical signal corresponding to an optical image obtained continuously in a state where a photographing instruction from a user is not received via the operation unit 180. To the buffer memory unit 130 and the display unit 150.

  The A / D conversion unit 120 digitizes the electrical signal converted by the image sensor 119 and outputs the digitized image to the buffer memory unit 130.

  The operation unit 180 includes, for example, a power switch, a shutter button, a multi selector (cross key), or other operation keys. The operation unit 180 receives a user operation input by being operated by the user, and outputs a signal corresponding to the received operation input (hereinafter referred to as an “operation signal”) to the CPU 190. The operation signal may include an operation signal indicating whether or not the image feature amount is included in the image by a predetermined amount or more. For example, when a chromatic image is adjusted to an achromatic image due to white balance adjustment, the operation signal includes information indicating that the color component amount is not included in the image by a predetermined amount or more.

  The area extraction device 140 performs image processing on the image stored in the buffer memory unit 130 based on the image processing conditions stored in the storage unit 160. The processed image is stored in the storage medium 200 via the communication unit 170.

  Further, the region extraction device 140 extracts a region from the image stored in the buffer memory unit 130 based on the image feature amount, and determines the priority for the extracted region. Further, the region extraction device 140 stores information indicating the extracted region in the storage unit 160. Here, the information indicating the region is, for example, information indicating the distribution of the region, information indicating the image feature amount of the region, and information indicating the priority set for the region.

  Here, the region extraction device 140 calculates a first image feature amount (for example, color component amount) and a second image feature amount (for example, light amount (luminance, brightness)) included in the image for each pixel. . Further, the region extraction device 140 determines whether each image feature amount is less than a predetermined threshold value.

  In addition, when the first image feature amount is equal to or greater than the threshold, the region extraction device 140 extracts a region from the image based on the first image feature amount. On the other hand, when the first image feature amount is less than the threshold, the region extraction device 140 extracts a region from the image based on the second image feature amount.

  In addition, when the operation signal indicates that the first image feature amount is not included in the image by a predetermined amount or more, the region extraction device 140 extracts a region from the image based on the second image feature amount. Extract. On the other hand, when the operation signal indicates that the first image feature amount is included in the image by a predetermined amount or more, the region extraction device 140 uses the first image feature amount and the second image feature amount. Based on the above, an area is extracted from the image.

  In addition, when the first image feature amount is less than the threshold, the region extraction device 140 adjusts the resolution for calculating the first image feature amount and the second image feature amount. In addition, when the image is an image captured in a backlight state, the region extraction device 140 is based on an evaluation value obtained from an AF evaluation region existing at a predetermined position (for example, nine locations) in the image. Extract regions from images. Here, the AF evaluation area is an area on which a contrast scan is executed. In addition, when the image is not an image captured in the backlight state, the region extraction device 140 extracts a region determined based on the image feature amount from the image.

  In addition, when the image feature amount (for example, the color component amount) is less than the threshold value, the region extraction device 140 evaluates the evaluation value obtained from the AF evaluation region located at a predetermined position (for example, nine locations) in the image Based on the above, an area is extracted from the image. In addition, when the image feature amount (for example, the color component amount) is equal to or larger than the threshold value, the region extraction device 140 extracts a region determined based on the image feature amount from the image.

The buffer memory unit 130 stores an image captured by the imaging unit 110.
The display unit 150 displays an image obtained by the imaging unit 110, an operation screen, and the like. The display unit 150 is, for example, a liquid crystal display.

  The storage unit 160 stores determination conditions referred to by the CPU 190 in scene determination, imaging conditions associated with each scene determined by scene determination, information indicating a region, and the like.

  The CPU 190 controls the imaging unit 110 according to the set imaging conditions (for example, aperture value, exposure value). In addition, the CPU 190 acquires information indicating a region from the region extraction device 140. Further, the CPU 190 controls the lens barrel control unit 118 of the lens barrel 111 based on the information indicating the region so that the region extracted from the image is focused on the region set by the region extraction device 140 and having a high priority. The focus is adjusted by outputting a drive control signal.

  Also, the CPU 190 performs focus adjustment (AF) setting, exposure adjustment (AE) setting, white balance adjustment (AWB) setting, subject image as pre-processing or post-processing of image processing based on the information indicating the area. (Object) tracking setting, night scene judgment setting, color correction processing setting, subject image enlarged display setting, panning display setting, brightness optimization setting linked with zoom magnification, etc. To control.

  Further, the CPU 190 controls setting of a tone curve (gradation curve), whether or not to execute simultaneous shooting of a still image and a moving image, and the like based on information indicating a region.

  In addition, the CPU 190 sets whether to perform continuous shooting as pre-processing or post-processing based on information indicating the area, setting for switching the subject image to be focused in continuous shooting, and continuously changing the focal length. , Setting for changing the amount of flash light emitted from the light emitting unit (not shown) of the imaging apparatus 100, setting for changing the exposure amount, setting for whether or not to change the shutter speed, and the like.

  Further, the CPU 190 causes the region extraction device 140 to perform image processing on an image in a still image format or a moving image format based on an operation signal input from the operation unit 180.

  The communication unit 170 is connected to a removable storage medium 200 (such as a card memory), and executes writing, reading, and erasing of information (image data, information indicating an area) on the storage medium 200.

  The storage medium 200 is a storage unit that is detachably connected to the imaging apparatus 100, and stores various types of information (for example, image data and information indicating an area). Note that the storage medium 200 may be integrated with the imaging apparatus 100 instead of being detachable.

Next, details of the region extraction device 140 will be described.
The region extraction device 140 includes a feature amount calculation unit 141, a determination unit 142, a reception unit 143, a region extraction unit 144, and a priority setting unit 145.

  The feature amount calculation unit 141 acquires an image captured by the imaging unit 110 via the buffer memory unit 130. In addition, the feature amount calculation unit 141 calculates an image feature amount for each of a plurality of features (for example, color component amount, light amount (luminance, brightness)) included in the image acquired via the buffer memory unit 130, The calculated image feature amount is output to the region extraction unit 144.

  The determination unit 142 generates a histogram of the first image feature value and the second image feature value, and determines whether each image feature value is less than a threshold value. Here, the determination unit 142 determines whether the image feature amount is less than the threshold based on the standard deviation σ of the histogram waveform of each image feature amount. For example, the standard deviation σ in the histogram waveform of the color component amount is equal to or smaller than a predetermined color component amount, that is, the standard deviation σ of the color difference signal Cb ≦ the predetermined color component amount and the standard deviation σ of the color difference signal Cr. When it is ≦ predetermined color component amount, the determination unit 142 determines that the captured image is an achromatic image.

  Hereinafter, as an example, the first image feature amount is assumed to be a color component amount, and the second image feature amount is assumed to be a light amount (luminance, brightness).

  The determination unit 142 determines whether or not the captured image is an image captured in a backlight state (hereinafter referred to as “backlight image”). FIG. 2 shows a luminance histogram of the backlight image. The horizontal axis represents the luminance Y. The vertical axis represents the frequency N. When the luminance difference from the waveform peak of the histogram to another waveform peak is equal to or greater than a predetermined luminance difference, the determination unit 142 determines that the captured image is a backlight image.

  Returning to FIG. 1, the description of the configuration of the region extraction device 140 will be continued. The reception unit 143 receives the operation signal from the operation unit 180 and transfers the received operation signal to the region extraction unit 144.

  When the determination unit 142 determines that the captured image is a backlight image, the region extraction unit 144 evaluates the evaluation value obtained from the AF evaluation region located at a predetermined position (for example, nine locations) in the image. Based on the above, an area is extracted from the image. Here, the reason why the region is extracted from the image based on the AF evaluation region is that there is a possibility that the image has few or no appropriate regions with priority. In addition, when the determination unit 142 determines that the captured image is not a backlight image, the region extraction device 140 extracts a region determined based on the image feature amount calculated by the feature amount calculation unit 141 from the image.

  When the determination unit 142 determines that the image feature amount (for example, color component amount) included in the captured image is less than the threshold value, the region extraction unit 144 determines a position (for example, nine locations) determined in advance in the image. The area is extracted from the image on the basis of the evaluation value obtained from the AF evaluation area in (). In addition, when the determination unit 142 determines that the image feature amount included in the captured image is not less than the threshold value, the region extraction device 140 determines a region determined based on the image feature amount calculated by the feature amount calculation unit 141. Extract from the image.

  The region extraction unit 144 acquires an operation signal from the reception unit 143. The region extraction unit 144 acquires the image feature amount calculated by the feature amount calculation unit 141. In addition, the region extraction unit 144 acquires an image captured by the imaging unit 110 from the buffer memory unit 130. The region extraction unit 144 extracts a region from the image captured by the imaging unit 110 based on the image feature amount.

  In addition, the region extraction unit 144 extracts a region from an image binarized image (hereinafter referred to as “binary image”) outside a certain range (section) of the image feature amount and outside the range, and extracts the region as the extracted region. Assign labels (labeling). Here, the threshold value that defines a certain range (section) of the image feature amount is determined as, for example, “threshold = average value + coefficient × standard deviation” based on the average value and standard deviation of the histogram waveform of the image feature amount. It is done.

  The region extraction unit 144 adjusts the resolution for calculating the image feature amount when the image feature amount is less than the threshold value. FIG. 3 shows a luminance histogram. The horizontal axis represents the luminance Y. The vertical axis represents the frequency N. FIG. 3A shows a histogram of the luminance of the subject area imaged on the screen.

  The region extraction unit 144 determines the resolution of the image feature amount based on the average value Ave and standard deviation σ of the histogram. For example, a histogram with a lower resolution than that in FIG. 3C is shown in FIG. In FIG. 3B, since the luminance resolution is lower than in FIG. 3C, the frequency change of the histogram waveform is not clearly detected, so that it is difficult to detect the subject region. On the other hand, in FIG. 3C, since the resolution is higher than that in FIG. 3B, the frequency change of the histogram waveform is clearly detected, so that the subject region can be easily detected. As described above, the region extraction unit 144 can easily detect the subject region by adjusting the resolution of the image feature amount based on the average value and the standard deviation of the histogram.

  Note that the region extraction unit 144 may set an upper limit on the resolution of the image feature amount so that a subject region that is too small is not detected. Thereby, the area extraction unit 144 can easily distinguish between the subject area and the noise.

  FIG. 4 shows an example of a region extracted from an image based on luminance. The area extraction unit 144 extracts an image based on luminance (hereinafter referred to as “Y image”) from the image captured by the imaging unit 110. The area extraction unit 144 corrects the luminance (input value) of the image captured by the imaging unit 110 (for example, gamma correction), and calculates the corrected luminance Y (output value).

  The region extraction unit 144 divides the corrected luminance Y (output value) into a plurality of fixed ranges. In FIG. 4, as an example, it is assumed that the corrected luminance Y (output value) is divided into sections 1 to 4. Here, for example, the upper limit of the range of category 1 is “average value + K1 × standard deviation” and the lower limit of the range of category 1 is “average value + K2 × standard deviation” so that the binary image of category 1 includes a high luminance area. Suppose that The region extraction unit 144 generates a binary image for each section (gradation).

  The region extraction unit 144 extracts the high luminance region 300 from the region image representing the search result by searching for the distribution of the high luminance region included in the binary image of category 1. Similarly, the region extraction unit 144 extracts the medium luminance region 310 in the region image representing the search result by searching for the medium luminance region included in the binary image of category 2. In addition, the region extraction unit 144 extracts the low luminance region 320 from the region image representing the search result by searching for the low luminance region included in the binary image of category 4.

  In FIG. 4, it is assumed that the corrected luminance Y (output value) is divided into sections 5 to 7 as an example because the resolution of the image feature amount is adjusted by the region extraction unit 144. For example, it is assumed that the category 5 is changed to have a higher resolution than the category 1. The region extraction unit 144 extracts the high luminance region 301 in the region image representing the search result by searching for the distribution of the high luminance region included in the binary image of the category 5 having high resolution.

  Further, for example, it is assumed that the section 7 is changed so that the resolution is lower than that of the section 4. The region extraction unit 144 extracts the low luminance region 321 from the region image representing the search result by searching for the distribution of the low luminance region included in the binary image of the category 7 with low resolution. As described above, the region extraction unit 144 can easily detect the subject region by adjusting the resolution of the image feature amount.

  FIG. 5 shows an example of a region extracted from an image based on the color component amount. The region extraction unit 144 extracts an image based on the color component amount (Cr) (hereinafter referred to as “Cr image”) from the image captured by the imaging unit 110. The region extraction unit 144 corrects (for example, gamma correction) the color component amount (input value) of the image captured by the imaging unit 110, and calculates the corrected color component amount Cr (output value).

  If the captured image has a color component amount equal to or greater than a predetermined threshold (not an achromatic image), the region extraction unit 144 divides the corrected color component amount (Cr) into a plurality of fixed ranges. In FIG. 5, as an example, it is assumed that the corrected color component amount Cr (output value) is divided into sections 8 to 11. Here, for example, the upper limit of the range of the section 8 is “average value + K3 × standard deviation” and the lower limit of the range of the section 8 is “average value + K4 × standard deviation” so that the binary image of the section 8 includes the red region. Suppose that The region extraction unit 144 generates a binary image for each section (gradation).

  The area extraction unit 144 searches the distribution of the red area included in the binary image of section 8 to extract the red area 410 from the area image representing the search result. Similarly, the area extraction unit 144 searches for the pink area included in the binary image of section 9 to extract the pink area 430 from the area image representing the search result. In addition, the region extraction unit 144 searches for a green region included in the binary image of the category 11 and thereby extracts the green region 450 from the region image representing the search result.

  In FIG. 5, it is assumed that the corrected color component amount Cr (output value) is divided into divisions 12 to 16 as an example because the resolution of the image feature amount is adjusted by the region extraction unit 144. Here, for example, it is assumed that the section 12 is changed to have a higher resolution than the section 8. The region extraction unit 144 extracts the red region 410 from the region image representing the search result by searching for the distribution of the red region included in the binary image of the section 12 having high resolution.

Further, for example, it is assumed that the section 16 is changed so as to have a higher resolution than the section 11.
The region extraction unit 144 extracts the green region 451 from the region image representing the search result by searching for the distribution of the green region included in the binary image of the section 16 having high resolution. As described above, the region extraction unit 144 can easily detect the subject region by adjusting the resolution of the image feature amount.

  Returning to FIG. 1, the description of the configuration of the region extraction device 140 will be continued. The priority setting unit 145 determines a priority indicating a region that a person is likely to pay more attention to in the region extracted by the region extraction unit 144. The priority setting unit 145 determines the priority for each area according to the evaluation value calculated for each area.

  For example, the priority setting unit 145 determines an evaluation value for each region according to the distance from the center of the angle of view to the center of gravity of the region. Here, the priority setting unit 145 determines a higher evaluation value for a region distributed closer to the center of the angle of view. Further, for example, the priority setting unit 145 determines an evaluation value for a region according to the moment of inertia of the region. The priority setting unit 145 sets a high evaluation value in a region where the moment of inertia is small (concentrated at the center of gravity). For example, the priority setting unit 145 determines an evaluation value for a region according to the area (number of pixels) of the region. The priority setting unit 145 determines a higher evaluation value for a region closer to a predetermined area.

  The priority setting unit 145 causes the storage unit 160 to store the priority determined for each region as information indicating the region.

Next, the processing procedure of the region extraction apparatus will be described.
FIG. 6 is a flowchart showing a processing procedure until focusing on an area. The feature amount calculation unit 141 reads the captured image (through image or through image sequence) through the buffer memory unit 130 (step S1). The feature amount calculation unit 141 performs basic image processing (for example, demosaic, noise reduction, white balance adjustment, pixel missing correction) on the read image (step S2).

  The region extraction unit 144 reads an image obtained by resizing the captured image for each image feature amount (for example, YCrCb) via the buffer memory unit 130 (step S3). The determination unit 142 determines whether or not the read image has a color component amount equal to or greater than a predetermined threshold (not an achromatic image). If the read image is an achromatic image, the region extraction unit 144 adjusts the luminance resolution (step S4).

  The region extraction unit 144 generates a binary image based on the brightness when the read image does not have a color component amount equal to or greater than a predetermined threshold (achromatic color image). In addition, the region extracting unit 144 generates a binary image based on the luminance and the color component amount when the read image has a color component amount equal to or greater than a predetermined threshold (not an achromatic image). (Step S5). The region extraction unit 144 extracts a region from the binary image (region image), and assigns a label to the extracted region (labeling) (step S6).

  If there are regions that are estimated to be noise in the image, the region extraction unit 144 removes those regions from the image. In addition, when the region estimated to be the background is in the image, the region extraction unit 144 removes the region from the image (step S7). The priority setting unit 145 determines the priority for the region extracted by the region extraction unit 144 (step S8).

  The region extraction unit 144 shapes the region for which the priority setting unit 145 has set the priority into a rectangle that matches the AF evaluation region (step S9). Further, the region extraction unit 144 determines an AF evaluation region at a position predetermined in the image (step S10).

  The CPU 190 executes a contrast scan based on the contrast value of the AF evaluation area (step S11). Further, the CPU 190 gives priority to an area in which a close subject is imaged based on the result of the contrast scan, and focuses the area by the AF lens 112 (focus drive) (step S12).

  FIG. 7 is a flowchart (first half) showing a processing procedure until focusing on a region included in an achromatic image. FIG. 8 is a flowchart (second half) showing a processing procedure until focusing on an area included in the achromatic image.

  The CPU 190 sets variables used for exposure adjustment (AE) processing and white balance adjustment (AWB) processing to predetermined reference values (step Sa1). The imaging unit 110 continuously captures (exposures) through images and outputs the captured through images to the buffer memory unit 130 and the display unit 150 (step Sa2).

  The feature amount calculation unit 141 performs basic image processing (for example, demosaic, noise reduction, white balance adjustment, pixel missing correction) on the read image (step Sa3). Further, the feature amount calculation unit 141 performs YCrCb (YUV) conversion on the image that has undergone the basic image processing (step Sa4).

  The determination unit 142 determines whether or not white balance adjustment is performed in the imaging mode among the imaging mode and the image reproduction mode (step Sa5). When the white balance adjustment is not executed in the imaging mode (step Sa5-No), the determination unit 142 determines whether the captured image has a color component amount equal to or greater than a predetermined threshold (not an achromatic image). Determine. In other words, the determination unit 142 determines whether or not the achromatic color degree of the captured image is equal to or less than a threshold value (step Sa6).

  If the captured image has a color component amount equal to or greater than the predetermined threshold (not an achromatic image) (step Sa6-No), the determination unit 142 determines whether the captured image is an achromatic image by white balance adjustment. Determine whether or not. Further, the determination unit 142 determines whether or not the captured image is an achromatic image from the beginning (regardless of white balance adjustment) (step Sa7).

  When the captured image is an achromatic image by white balance adjustment, or when the captured image is originally an achromatic image (step Sa7-Yes), the determination unit 142 determines that the captured image is backlit. It is determined whether or not the image is an image (step Sa8).

  When the captured image is not a backlight image (step Sa8-No), the region extraction unit 144 adjusts the luminance resolution. Here, the region extracting unit 144 adjusts the luminance resolution so that a large number of luminance regions are extracted from the achromatic image such as a snow scene to the region image (step Sa9).

  The region extraction unit 144 extracts a luminance region (see FIG. 4) from a region image that represents a search result of regions included in the binary image. In addition, when the resolution of the color component amount is adjusted, the region extraction unit 144 extracts a color system region (see FIG. 5) in a region image that represents a search result of a region included in the binary image. . The priority setting unit 145 determines priorities for the luminance regions extracted by the region extraction unit 144 (step Sa10). The region extraction unit 144 shapes the region for which the priority setting unit 145 has determined the priority into a rectangle that matches the AF evaluation region (step Sa11).

  The CPU 190 executes a contrast scan based on the contrast value of the AF evaluation area (step Sa12). Further, the CPU 190 gives priority to an area in which a close subject is imaged based on the result of the contrast scan, and focuses the area by the AF lens 112 (focus drive) (step Sa13).

  On the other hand, if white balance adjustment is being performed in the imaging mode in step Sa5 (step Sa5-Yes), the determination unit 142 stores the degree of achromatic color (for example, the amount of color components) in the storage unit 160 ( Step Sa14). In step Sa6, when the captured image does not have a color component amount equal to or larger than the predetermined threshold (achromatic image) (step Sa6-No), the determination unit 142 stores the degree of achromatic color in the storage unit 160. (Step Sa15).

  In step Sa7, when the captured image is not an achromatic image by white balance adjustment and the captured image is not originally an achromatic image (step Sa7-No), the determination unit 142 captures the image. It is determined whether or not the color component amount of the obtained image is equal to or less than a predetermined threshold (step Sa16). Further, the region extraction unit 144 adjusts the resolution of the color component amount and the resolution of the luminance (Step Sa17). And the determination part 142 and the area | region extraction part 144 advance a process to step Sa10.

  In step Sa8, when the captured image is a backlight image (step Sa8-Yes), the region extracting unit 144 determines an AF evaluation region at a position predetermined in the image (step Sa18).

As described above, the region extraction device 140 calculates the first image feature amount (for example, color component amount) and the second image feature amount (for example, luminance) included in the input image. 141, a determination unit 142 that determines whether or not the first image feature amount is less than a threshold value, and when the first image feature amount is less than a threshold value, based on the second image feature amount When a fixed region (for example, the high luminance region 300) is extracted from the binary image and the first image feature amount is equal to or greater than a threshold value, a region (for example, a red region) determined based on the first image feature amount 410) from the binary image.
With this configuration, the region extraction unit 144 extracts a region determined based on the second image feature amount from the image when the first image feature amount is less than a threshold value. Thereby, the area extracting apparatus can appropriately extract the area of the main subject from the input image.

In addition, the determination unit 142 determines whether the color component amount that is the first image feature amount is less than the threshold value, and the region extraction unit 144 determines that the color component amount is less than the threshold value, A region determined based on the luminance that is the second image feature amount is extracted from the image. If the color component amount is equal to or greater than the threshold value, a region determined based on the color component amount is extracted from the image.
With this configuration, the region extracting unit 144 extracts a region determined based on luminance from the image when the color component amount is less than the threshold value. Thereby, the area extracting apparatus can appropriately extract the area of the main subject from the input image.

In addition, when the first image feature amount (for example, color component amount) is less than the threshold value, the region extraction unit 144 has the first image feature amount and the second image feature amount (for example, luminance). The resolution for calculating is adjusted (for example, increased) (see FIGS. 3 to 5).
With this configuration, the region extraction unit 144 adjusts the resolution for calculating the first image feature amount and the second image feature amount. Thereby, the area extracting apparatus can appropriately extract the area of the main subject from the input image.

In addition, the region extraction device 140 includes a receiving unit 143 that receives an operation signal indicating whether or not the first image feature amount (for example, color component amount) is included in the image by a predetermined amount or more. When the operation signal indicates that the first image feature amount is not included in the image by a predetermined amount or more (for example, the color image is adjusted by white balance adjustment). Is adjusted to an achromatic image), a region (for example, high luminance region 300) determined based on the second image feature amount (for example, luminance) is extracted from the image, and the first image feature amount is extracted. When the operation signal indicates that the image is included in the image by a predetermined amount or more, a region (for example, a red region 410) determined based on the first image feature amount is extracted from the image. To do.
With this configuration, when the operation signal indicates that the first image feature amount is not included in the image by a predetermined amount or more, the region extraction unit 144 sets the second image feature amount as the second image feature amount. A region determined based on the image is extracted from the image. Thereby, the area extracting apparatus can appropriately extract the area of the main subject from the input image.

Further, the determination unit 142 determines whether or not the image is an image captured in a backlight state, and the region extraction unit 144 determines in advance that the image is an image captured in a backlight state. A region is extracted from the image.
With this configuration, the region extraction unit 144 extracts a predetermined region from the image when the image is an image captured in a backlight state or when the image is an image captured in a backlight state. Thereby, the area extracting apparatus can appropriately extract the area of the main subject from the input image.

In addition, the region extraction device 140 includes a feature amount calculation unit 141 that calculates a first image feature amount (for example, color component amount) and a second image feature amount (for example, luminance) included in the input image, A receiving unit 143 that receives an operation signal indicating whether or not the first image feature amount is included in the image by a predetermined amount or more, and the first image feature amount is a predetermined amount or more. When the operation signal indicates that it is not included in the image (for example, when a chromatic image is adjusted to an achromatic image due to white balance adjustment), it is determined based on the second image feature amount. When an area (for example, a high brightness area 300) is extracted from the image and the operation signal indicates that the first image feature amount is included in the image by a predetermined amount or more, the first Based on one image feature There are defined region (for example, red areas 410) comprises an area extracting unit 144 is extracted from the image.
With this configuration, when the operation signal indicates that the first image feature amount is not included in the image by a predetermined amount or more, the region extraction unit 144 sets the second image feature amount as the second image feature amount. A region determined based on the image is extracted from the image. Thereby, the area extracting apparatus can appropriately extract the area of the main subject from the captured image.

Further, the region extraction device 140 includes a feature amount calculation unit 141 that calculates an image feature amount (for example, a color component amount) included in the input image, and whether or not the image is an image captured in a backlight state. When the determination unit 142 determines whether the image is an image captured in a backlight state, an area determined based on an AF evaluation region is extracted from the image, and the image is not an image captured in a backlight state, A region extracting unit 144 that extracts a region determined based on the image feature amount calculated by the feature amount calculating unit from the image.
With this configuration, when the image is an image captured in the backlight state, the region extraction unit 144 extracts an area determined based on the AF evaluation region from the image, and the image is not an image captured in the backlight state. In this case, a region determined based on the image feature amount calculated by the feature amount calculation unit is extracted from the image. Thereby, the area extracting apparatus can appropriately extract the area of the main subject from the captured image.

In addition, the region extraction device 140 determines a feature amount calculation unit 141 that calculates an image feature amount (for example, a color component amount) included in the input image, and determines whether the image feature amount is less than a threshold value. When the image feature amount is less than the threshold value, the determination unit 142 extracts an area determined based on the AF evaluation region from the image, and when the image feature amount is equal to or greater than the threshold value, the feature amount calculation unit 141 calculates the image feature amount. A region extracting unit 144 that extracts a region determined based on the image feature amount from the image.
With this configuration, the region extracting unit 144 extracts a region determined based on the AF evaluation region from the image when the image feature amount is less than the threshold value, and calculates the feature amount when the image feature amount is equal to or greater than the threshold value. A region determined based on the image feature amount calculated by the unit 141 is extracted from the image. Thereby, the area extracting apparatus can appropriately extract the area of the main subject from the input image.

The feature amount calculation unit 141 calculates a color component amount that is the image feature amount.
Thereby, the area extracting apparatus can appropriately extract the area of the main subject from the input image based on the color component amount.

In addition, the imaging apparatus 100 includes an area extraction device 140, a CPU 190 that adjusts the focus by the AF lens 112 so as to focus on the area extracted by the area extraction device 140, an imaging unit 110 that captures an image including the area, Is provided.
With this configuration, the region extraction device 140 appropriately extracts the region of the main subject from the captured image. Thereby, the imaging apparatus 100 can image the area of the main subject.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes designs and the like that do not depart from the gist of the present invention.

  For example, the color component may be expressed by a color difference signal (Cr, Cb) or may be expressed by an RGB signal.

  Further, by recording a program for realizing the region extraction device and the imaging device described above on a computer-readable recording medium, causing the computer system to read and execute the program recorded on the recording medium, Execution processing may be performed. Here, the “computer system” may include an OS and hardware such as peripheral devices.

  Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used. The “computer-readable recording medium” means a flexible disk, a magneto-optical disk, a ROM, a writable nonvolatile memory such as a flash memory, a portable medium such as a CD-ROM, a hard disk built in a computer system, etc. This is a storage device.

Further, the “computer-readable recording medium” refers to a volatile memory (for example, DRAM (Dynamic) in a computer system serving as a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. Random Access Memory)) that holds a program for a certain period of time is also included.
The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line.
The program may be for realizing a part of the functions described above.
Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, and what is called a difference file (difference program) may be sufficient.

DESCRIPTION OF SYMBOLS 100 ... Imaging device, 110 ... Imaging part, 111 ... Lens barrel, 140 ... Area extraction apparatus, 141 ... Feature-value calculation part, 142 ... Judgment part, 143 ... Reception part, 144 ... Area extraction part, 145 ... Priority setting Part

Claims (28)

A first extraction unit that extracts a first region from an image based on color information;
A second extraction unit that extracts a second region from the image based on luminance information;
A setting unit that sets the second region as a specific region and adjusts the resolution of the second extraction unit when the image signal of the first region is less than a threshold;
An area extraction apparatus comprising: The second extraction unit extracts a third region different from the second region from the image based on luminance information,
The area extracting apparatus according to claim 1, wherein the setting unit sets an area close to a center position of the image among the second area and the third area as the specific area.   The setting unit selects a region having a short distance among a distance from the center of gravity of the second region to the center of the image and a distance from the center of gravity of the third region to the center of the image. The region extracting apparatus according to claim 2, wherein the region extracting device is set as a specific region. The second extraction unit extracts a third region different from the second region from the image based on luminance information,
The region extraction device according to claim 1, wherein the setting unit sets a region having a small moment of inertia among the second region and the third region as the specific region. The second extraction unit extracts a third region different from the second region from the image based on luminance information,
The region extraction apparatus according to claim 1, wherein the setting unit sets a region close to a predetermined area as the specific region among the second region and the third region.   The region extraction device according to any one of claims 1 to 5, wherein the setting unit sets the first region as the specific region when the image signal of the first region is not less than a threshold value.   The said setting part calculates a color component amount from the image signal of the said 1st area | region, and determines whether the said color component amount is less than a threshold value. Region extraction device.   The region extraction device according to any one of claims 1 to 7, wherein the setting unit sets the specific region from a plurality of predetermined regions included in an image when imaged in a backlight state. A first extraction unit for extracting an area from an image based on color information;
A second extraction unit for extracting an area from the image based on luminance information;
A setting unit configured to set the region extracted by the second extraction unit as a specific region and adjust the resolution of the second extraction unit when the image signal of the region extracted by the first extraction unit is less than a threshold; ,
An area extraction apparatus comprising: The second extraction unit extracts a plurality of regions from the image based on luminance information,
The region extraction device according to claim 9, wherein the setting unit sets a region close to a center position of the image among the plurality of regions as the specific region.   The area extraction device according to claim 10, wherein the setting unit sets, as the specific area, an area having a short distance from a gravity center position of the area to a center position of the image among the plurality of areas. The second extraction unit extracts a plurality of regions from the image based on luminance information,
The region extraction device according to claim 9, wherein the setting unit sets a region having a small moment of inertia among the plurality of regions as the specific region. The second extraction unit extracts a plurality of regions from the image based on luminance information,
The region extraction device according to claim 9, wherein the setting unit sets a region close to a predetermined area as the specific region among the plurality of regions.   The said setting part sets the area | region extracted by the said 1st extraction part as a specific area | region, when the image signal of the area | region extracted by the said 1st extraction part is not less than a threshold value. The region extraction device according to one item.   The said setting part calculates the color component amount from the image signal of the area | region extracted by the said 1st extraction part, and determines whether the said color component amount is less than a threshold value. The region extraction device according to claim 1.   The region extraction device according to any one of claims 9 to 15, wherein the setting unit sets the specific region from a plurality of predetermined regions included in an image when the image is captured in a backlight state. An extraction unit that extracts a first region from an image based on color information and extracts a second region from the image based on luminance information;
A setting unit configured to extract the second region by setting the second region as a specific region when the image signal of the first region is less than a threshold, and adjusting the resolution of the extraction unit;
An area extraction apparatus comprising: The extraction unit extracts a third region different from the second region from the image based on luminance information;
The area extraction device according to claim 17, wherein the setting unit sets, as the specific area, an area close to a center position of the image among the second area and the third area.   The setting unit selects an area having a short distance among the distance from the centroid position of the second area to the center position of the image and the distance from the centroid position of the third area to the center position of the image. The region extraction device according to claim 18, which is set as: The extraction unit extracts a third region different from the second region from the image based on luminance information;
The region extraction device according to claim 17, wherein the setting unit sets a region having a small moment of inertia among the second region and the third region as the specific region. The extraction unit extracts a third region different from the second region from the image based on luminance information;
The region extraction device according to claim 17, wherein the setting unit sets a region close to a predetermined area as the specific region among the second region and the third region.   The region extraction device according to any one of claims 17 to 21, wherein the setting unit sets the first region as a specific region when an image signal of the first region is not less than a threshold value.   The said setting part calculates a color component amount from the image signal of the said 1st area | region, and determines whether the said color component amount is less than a threshold value. Region extraction device.   The area extraction device according to any one of claims 17 to 23, wherein the setting unit sets the specific area from a plurality of predetermined areas included in an image when the image is captured in a backlight state. An image sensor that images light emitted from the optical system;
The region extraction device according to any one of claims 1 to 24,
A control unit for controlling the optical system using a signal from the specific region;
An imaging apparatus comprising: On the computer,
A first extraction procedure for extracting a first region from an image based on color information;
A second extraction procedure for extracting a second region from the image according to luminance information;
An adjustment procedure for setting the second area as a specific area when the image signal of the first area is less than a threshold, and adjusting the resolution of extraction in the second extraction procedure ;
Extraction program to execute. On the computer,
A first extraction procedure for extracting an area from an image based on color information;
A second extraction procedure for extracting an area from the image based on luminance information;
Setting that adjusts the resolution of extraction in the second extraction procedure by setting the region extracted by the second extraction procedure as a specific region when the image signal of the region extracted by the first extraction procedure is less than a threshold value Procedure and
Extraction program to execute. On the computer,
A first extraction procedure for extracting a first region from an image based on color information and extracting a second region from the image based on luminance information;
When the image signal of the first area is less than a threshold, the second area is set as a specific area, and the second area is extracted by adjusting the resolution of the extraction of the second area in the first extraction procedure. A second extraction procedure;
Extraction program to execute.

Images