JP2013034154A - Region extraction device, imaging device, and region extraction program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a region extraction device, an imaging device, and a region extraction program that are capable of appropriately extracting a region of a main subject from an input image.SOLUTION: A region extraction device 140 comprises: a region extraction unit 142 that, based on color information included in image data of an input image, extracts a predetermined region from the image; and a priority setting unit 143 that determines priority for the region extracted by the region extraction unit 142 in accordance with predetermined color information.

Description

  The present invention relates to a region extraction device, an imaging device, and a region extraction program that extract a region from an image.

  An image processing apparatus that extracts a region from a captured image and identifies the extracted region based on a histogram of image feature values is disclosed (see Patent Document 1).

Japanese Patent Laid-Open No. 4-10079

  However, the image processing apparatus disclosed in Patent Document 1 has a problem in that it cannot extract an area that is noticed by a person from an input image, that is, cannot appropriately extract an area of a main subject.

  The present invention has been made in view of the above points, and an object thereof is to provide a region extraction device, an imaging device, and a region extraction program that can appropriately extract a region of a main subject from an input image. And

  The present invention has been made to solve the above-described problem, and an area extraction unit that extracts a predetermined area from the image based on color information included in image data of the input image, and the area extraction A region extracting apparatus comprising: a priority setting unit that determines priority according to predetermined color information in a region extracted by the unit.

  Further, the present invention is preliminarily determined for a region extraction unit that extracts a predetermined region from the image and a region extracted by the region extraction unit based on saturation information included in image data of the input image. And a priority setting unit that determines priority according to saturation.

  Further, according to the present invention, a region extraction unit that extracts a predetermined region from the image and a region extracted by the region extraction unit based on luminance information included in the image data of the input image are predetermined. And a priority setting unit that determines priority according to luminance.

  Further, the present invention provides a region extraction unit that extracts a predetermined region from the image based on a distance from the origin in a coordinate system of a uniform color space that represents color information included in image data of the input image, A region extraction apparatus comprising: a priority setting unit that determines priority according to the distance in a region extracted by a region extraction unit.

  The present invention also provides a region extraction device, a focus adjustment unit that adjusts a focus by an optical system so as to focus on a region having a high priority among regions extracted by the region extraction device, and a region having a high priority. And an imaging unit that captures an image including an area having a high priority in a focused state.

  The present invention also provides a computer according to a procedure for extracting a predetermined area from the image based on color information included in image data of an image input to the computer, and color information predetermined for the predetermined area. An area extraction program for executing a procedure for determining priority.

  The present invention also provides a computer according to a procedure for extracting a predetermined area from the image based on the saturation information included in the image data of the input image, and a predetermined saturation for the predetermined area. And a procedure for determining the priority.

  Further, the present invention provides a procedure for extracting a predetermined area from the image based on the luminance information included in the image data of the image input to the computer, and gives priority to the predetermined area in accordance with a predetermined luminance. A region extracting program for executing the procedure for determining the degree.

  Further, the present invention provides a computer for extracting a predetermined area from the image based on the distance from the origin in the coordinate system of the uniform color space representing the color information included in the image data of the input image; And a procedure for determining a priority according to the distance in the predetermined area.

  According to the present invention, the region extraction device can appropriately extract the region of the main subject from the input image.

It is a block diagram showing the structure of a lens barrel, an imaging device provided with a region extraction device, and a storage medium in an embodiment of the present invention. It is a figure showing the example of the area | region extracted from the image in one Embodiment of this invention. It is a figure showing the example of the area | region extracted from the image based on the changed threshold value in one Embodiment of this invention. It is a figure showing the coordinate system of uniform color space. It is a flowchart showing the procedure of the process until it focuses on an area | region in one Embodiment of this invention. It is a flowchart showing the procedure of the process which determines a high priority in a red type | system | group area | region in one Embodiment of this invention. It is a flowchart showing the procedure of the process which determines a high priority in a red area | region, a yellow area | region, and a blue area | region in one Embodiment of this invention. It is a flowchart showing the procedure of the process which determines a high priority in order of a red-type area | region, a yellow-type area | region, and a blue-type area | region in one Embodiment of this invention. It is a flowchart showing the procedure of the process which determines a high priority in a red area | region, a yellow area | region, a blue area | region, and a high-intensity area | region in one Embodiment of this invention. It is a flowchart showing the procedure of the process which determines a low priority compared with the other extracted area | region in one Embodiment of this invention in a low-intensity area | region.

  An embodiment of the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram illustrating the configuration of the lens barrel 111, the imaging device 100 including the region extraction device 140, and the storage medium 200. The imaging device 100 captures an optical image incident from the lens barrel 111 and stores the obtained image in the storage medium 200 as a still image or a moving image.

First, the configuration of the lens barrel 111 will be described.
The lens barrel 111 includes a focus adjustment lens (hereinafter referred to as “AF (Auto Focus) lens”) 112, a lens driving unit 116, an AF encoder 117, and a lens barrel control unit 118. The lens barrel 111 may be detachably connected to the imaging device 100 or may be integrated with the imaging device 100.

  The AF lens 112 is driven by a lens driving unit 116 and guides an optical image to a light receiving surface (photoelectric conversion surface) of an image sensor 119 of the image capturing unit 110 described later.

  The AF encoder 117 detects the movement of the AF lens 112 and outputs a signal corresponding to the movement amount of the AF lens 112 to the lens barrel control unit 118. Here, the signal corresponding to the movement amount of the AF lens 112 may be a sine wave signal whose phase changes according to the movement amount of the AF lens 112, for example.

  The lens barrel control unit 118 controls the lens driving unit 116 in accordance with a drive control signal input from the CPU 190 of the imaging apparatus 100. Here, the drive control signal is a control signal for driving the AF lens 112 in the optical axis direction. The lens barrel control unit 118 changes the number of steps of the pulse voltage output to the lens driving unit 116, for example, according to the drive control signal.

  Further, the lens barrel control unit 118 outputs the position (focus position) of the AF lens 112 in the lens barrel 111 to the CPU 190 based on a signal corresponding to the movement amount of the AF lens 112. Here, the lens barrel control unit 118 integrates, for example, signals according to the movement amount of the AF lens 112 according to the movement direction of the AF lens 112, thereby moving the AF lens 112 in the lens barrel 111 (the movement amount ( Position) may be calculated.

  The lens driving unit 116 drives the AF lens 112 in accordance with the control of the lens barrel control unit 118 and moves the AF lens 112 in the optical axis direction within the lens barrel 111.

Next, the configuration of the imaging device 100 will be described.
The imaging device 100 includes an imaging unit 110, a region extraction device 140, a display unit 150, a buffer memory unit 130, an operation unit 180, a storage unit 160, a CPU 190, and a communication unit 170.

  The imaging unit 110 includes an imaging element 119 and an A / D (Analog / Digital) conversion unit 120. The imaging unit 110 is controlled by the CPU 190 in accordance with the set imaging conditions (for example, aperture value, exposure value, etc.).

  The image sensor 119 includes a photoelectric conversion surface, converts an optical image formed on the photoelectric conversion surface by the lens barrel 111 (optical system) into an electric signal, and outputs the electric signal to the A / D conversion unit 120. The image sensor 119 may be configured by, for example, a CMOS (Complementary Metal Oxide Semiconductor). Further, the image sensor 119 may convert an optical image into an electrical signal for a partial region of the photoelectric conversion surface (image clipping).

  Further, the image sensor 119 stores an image obtained when a shooting instruction from the user is received via the operation unit 180 in the storage medium 200 via the A / D conversion unit 120. On the other hand, the imaging device 119 performs A / D conversion on the buffer memory unit 130 and the display unit 150 as images that are continuously obtained in a state where a shooting instruction from the user is not received via the operation unit 180. Output via the unit 120.

  The A / D converter 120 digitizes the electrical signal converted by the image sensor 119 and outputs an image that is a digital signal to the buffer memory unit 130.

  The operation unit 180 includes, for example, a power switch, a shutter button, a multi selector (cross key), or other operation keys. The operation unit 180 receives a user operation input when operated by the user, and a signal corresponding to the received operation input. Is output to the CPU 190.

  The area extraction device 140 performs image processing on the image stored in the buffer memory unit 130 based on the image processing conditions stored in the storage unit 160. The processed image is stored in the storage medium 200 via the communication unit 170.

  Further, the region extraction device 140 extracts a region from the image stored in the buffer memory unit 130 based on the image feature amount, and determines the priority for the extracted region. Further, the region extraction device 140 stores information indicating the extracted region in the storage unit 160. Here, the information indicating the region is, for example, information indicating the distribution of the region, information indicating the image feature amount of the region, and information indicating the priority set for the region.

  The display unit 150 displays an image obtained by the imaging unit 110, an operation screen, and the like. The display unit 150 is, for example, a liquid crystal display.

  The buffer memory unit 130 stores an image captured by the imaging unit 110.

  The storage unit 160 stores determination conditions referred to by the CPU 190 in scene determination, imaging conditions associated with each scene determined by scene determination, information indicating a region, and the like.

  The CPU 190 controls the imaging unit 110 according to the set imaging conditions (for example, aperture value, exposure value). In addition, the CPU 190 acquires information indicating a region from the region extraction device 140. In addition, the CPU 190 outputs a drive control signal to the lens barrel control unit 118 of the lens barrel 111 based on the information indicating the region, so that the priority set by the region extracting device 140 among the regions extracted from the image is set. Adjust the focus to focus on a high area. Thereby, an image including a high priority area is captured by the imaging unit 110 in a state where the optical system is focused on the high priority area.

  Also, the CPU 190 performs focus adjustment (AF) setting, exposure adjustment (AE) setting, white balance adjustment (AWB) setting, subject image as pre-processing or post-processing of image processing based on the information indicating the area. (Object) tracking setting, night scene judgment setting, color correction processing setting, subject image enlarged display setting, panning display setting, brightness optimization setting linked with zoom magnification, etc. To control.

  Further, the CPU 190 controls setting of a tone curve (gradation curve), whether or not to execute simultaneous shooting of a still image and a moving image, and the like based on information indicating a region.

  In addition, the CPU 190 sets whether to perform continuous shooting as pre-processing or post-processing based on information indicating the area, setting for switching the subject image to be focused in continuous shooting, and continuously changing the focal length. , Setting for changing the amount of flash light emitted from the light emitting unit (not shown) of the image capturing apparatus 100, setting for changing the exposure amount, setting for whether to change the shutter speed, and the like.

  In addition, the CPU 190 causes the region extraction device 140 to perform image processing on an image in a still image format or a moving image format based on the “signal according to the operation input” input from the operation unit 180.

  The communication unit 170 is connected to a removable storage medium 200 such as a card memory, and writes, reads, or deletes information (image data, information indicating an area, etc.) on the storage medium 200.

  The storage medium 200 is a storage unit that is detachably connected to the imaging apparatus 100, and stores information (image data, information indicating an area, and the like). Note that the storage medium 200 may be integrated with the imaging device 100.

Next, details of the region extraction device 140 will be described.
The region extraction device 140 includes a feature amount calculation unit 141, a region extraction unit 142, and a priority setting unit 143.

  The feature amount calculation unit 141 acquires the image captured by the imaging unit 110 from the buffer memory unit 130. The feature amount calculation unit 141 calculates the image feature amount for each of a plurality of features (for example, color information, saturation information, luminance (brightness) information, texture, edge) in the image acquired from the buffer memory unit 130. The calculated image feature amount is output to the region extraction unit 142.

  The region extraction unit 142 acquires the image feature amount calculated by the feature amount calculation unit 141. Further, the region extraction unit 142 acquires an image captured by the imaging unit 110 from the buffer memory unit 130. The region extraction unit 142 extracts a region from the image captured by the imaging unit 110 based on image feature amounts (for example, color information, saturation information, and luminance information). Here, the region extracted by the region extraction unit 142 is a region of the subject included in the image acquired from the buffer memory unit 130.

  In addition, the region extraction unit 142 extracts a region from an image binarized outside a certain range determined by the image feature amount and outside the range (hereinafter referred to as “binary image”), and extracts the region as the extracted region. Assign labels (labeling). Here, the certain range is determined by dividing the image feature amount by an upper limit and a lower limit of a predetermined threshold. Further, the threshold value may be defined as, for example, “average value + coefficient × standard deviation” based on the average value (Ave) and standard deviation (σ) of the image feature amount in the image.

  FIG. 2 shows an example of a region extracted from the image. The area extraction unit 142 extracts an image based on luminance (hereinafter referred to as “Y image”) from the image captured by the imaging unit 110. The area extraction unit 142 corrects the luminance (input value) of the image captured by the imaging unit 110 (for example, gamma correction), and calculates the corrected luminance (output value).

  The area extraction unit 142 divides the corrected luminance (output value) into a plurality of ranges. In FIG. 2, as an example, it is assumed that the corrected luminance is divided into sections 1 to 4. The area extraction unit 142 generates a binary image for each section. Accordingly, when the upper limit of the range of the category 1 is “average value + K1 × standard deviation” and the lower limit of the range of the category 1 is “average value + K2 × standard deviation”, the binary image of the category 1 has a high luminance area. Is included.

  The region extracting unit 142 extracts the high luminance region 300 from the binary image (region image) representing the search result by searching for the distribution of the high luminance region included in the binary image of category 1. Similarly, the region extraction unit 142 extracts the medium luminance region 310 from the region image by searching for the medium luminance region included in the binary image of category 2. Further, the region extraction unit 142 extracts the low luminance region 320 from the region image by searching for the low luminance region included in the binary image of category 4.

  Similarly, the area extraction unit 142 divides the corrected color information Cr into a plurality of ranges. In FIG. 2, as an example, it is assumed that the corrected color information Cr is divided into sections 5 to 8. The area extraction unit 142 generates a binary image for each section. As a result, when the upper limit of the range of the category 5 is “average value + K3 × standard deviation” and the lower limit of the range of the category 5 is “average value + K4 × standard deviation”, the binary image of the category 5 Is included.

  The region extraction unit 142 extracts the red-based region 400 from the region image by searching for the distribution of the red-based region included in the binary image of category 5. Similarly, the area extraction unit 142 searches for the pink area included in the binary image of Section 6 to extract the pink area 410 from the area image. In addition, the region extraction unit 142 extracts a green region 420 from the region image by searching for a green region included in the binary image of category 8.

  Similarly, the area extraction unit 142 divides the corrected color information Cb into a plurality of ranges. In FIG. 2, as an example, it is assumed that the corrected color information Cb is divided into sections 9 to 12. The area extraction unit 142 generates a binary image for each section. Accordingly, when the upper limit of the range of the category 9 is “average value + K5 × standard deviation” and the lower limit of the range of the category 9 is “average value + K6 × standard deviation”, the binary image of the category 9 includes the blue region Is included.

  The area extraction unit 142 extracts the blue area 500 from the area image by searching for the distribution of the blue area included in the binary image of section 9. Similarly, the region extraction unit 142 extracts a light blue system region 510 from the region image by searching for a light blue system region included in the binary image of the section 10. In addition, the region extraction unit 142 extracts a yellow region 520 from the region image by searching for a yellow region included in the binary image of the section 12.

  As for the luminance region, the lowest luminance region having a lower luminance may be extracted. Here, the region extraction unit 142 changes the threshold value to a lower value, and extracts a region having a luminance lower than the changed threshold value. In this way, for example, the region extraction unit 142 can extract the lowest luminance region where the black subject is imaged, not the low luminance region 320 including the region where the shadow is imaged. .

  FIG. 3 shows an example of a region extracted from the image based on the changed threshold value. In FIG. 3, it is assumed that the threshold value that defines the boundary between the category 3 and the category 4 is changed to a lower value. Thereby, the area where the shadow is imaged is excluded from the low luminance area 320, and the lowest luminance area 321 where only the black subject is imaged is extracted as the area image.

  Returning to FIG. 1, the description of the configuration of the region extraction device 140 will be continued. The priority setting unit 143 determines a priority indicating a region that a person will be more interested in in the region extracted by the region extraction unit 142. The priority setting unit 143 determines the priority for each area according to the evaluation value calculated for each area.

  For example, the priority setting unit 143 determines an evaluation value for each region according to the distance from the center of the angle of view to the center of gravity of the region. Here, the score calculation unit 143 determines a higher evaluation value for the region distributed closer to the center of the angle of view. In addition, for example, the priority setting unit 143 determines an evaluation value for a region according to the moment of inertia of the region. The priority setting unit 143 determines a high evaluation value in a region where the moment of inertia is small (concentrated at the center of gravity). For example, the priority setting unit 143 determines an evaluation value for a region according to the area (number of pixels) of the region. The score calculation unit 143 determines a higher evaluation value for a region closer to a predetermined area.

Further, the priority setting unit 143 sets priorities in the regions extracted by the region extracting unit 142 according to a predetermined hue. Here, the priority setting unit 143 determines the priority for the region extracted by the region extraction unit 142 according to the hue angle in the uniform color space (L ^* a ^* b ^* color system).

FIG. 4 shows a coordinate system of the uniform color space. L ^* is the brightness index. Moreover, a ^* and b ^* are chromaticity indices (chromicness indices). Here, the positive direction of the a ^* represents the red, the negative direction of the a ^* represents green. On the other hand, the positive direction of b ^* indicates yellow, negative direction b ^* represents blue. Further, h (= tan ⁻¹ (b ^* / a ^* )) is a hue angle (metric hue angle). C ^* (= √ ((a ^* ) ² + (b ^* ) ² )) is saturation. R (= √ ((L ^* ) ² + (a ^* ) ² + (b ^* ) ² ))) is a distance (color difference) from the origin.

  The priority setting unit 143 adds other regions to the extracted regions of red (red region 400 in FIG. 2), yellow (yellow region 520 in FIG. 2), and blue (blue region 500 in FIG. 2). A higher priority is set in comparison with the extracted region of the color. Here, the priority setting unit 143 may set a higher priority in the order of the extracted red area, the extracted yellow area, and the extracted blue area for each extracted area.

Further, the priority setting unit 143 determines the priority for the region extracted by the region extraction unit 142 according to the saturation (C ^* ). For example, the priority setting unit 143 determines a high priority for the extracted region with high saturation.

Further, the priority setting unit 143 determines the priority for the region extracted based on the luminance (Y) by the region extracting unit 142 according to the lightness (L ^* ) corresponding to the luminance. For example, the priority setting unit 143 determines a high priority for the extracted area with high brightness.

  In addition, the priority setting unit 143 adds another extracted region to an extracted region whose luminance is higher than a predetermined first threshold value (a threshold value that is a boundary between the segment 1 and the segment 2 in FIG. 2). A higher priority than that. In addition, the priority setting unit 143 has a luminance that is higher than the red extracted region (the red region 400 in FIG. 2), the yellow extracted region (the yellow region 520 in FIG. 2), and the first threshold value. The extracted areas (high luminance area 300 in FIG. 2) and blue extracted areas (blue system area 500 in FIG. 2) are assigned a higher priority in the order of the extracted areas.

  In addition, the priority setting unit 143 extracts an area where the luminance is lower than a predetermined second threshold value (a threshold value that is a boundary between the category 3 and the category 4 in FIG. 2) (the lowest luminance region in FIG. 2). 320), a low priority is set in comparison with other extracted regions.

  In addition, the priority setting unit 143 compares the extracted region (the low luminance region 320 in FIG. 3) with a luminance lower than the second threshold changed by the region extracting unit 142 with other extracted regions. Low priority.

  Further, the priority setting unit 143 sets priorities for the extracted regions in accordance with the distance R from the origin in the coordinate system of the uniform color space (see FIG. 4). For example, the priority setting unit 143 sets a high priority for the extracted region located at a position where the distance (color difference) from the origin is long in the coordinate system of the uniform color space.

  The priority setting unit 143 stores the priority determined for each region in the storage unit 160 as information indicating the region.

Next, the processing procedure of the region extraction apparatus will be described.
FIG. 5 is a flowchart showing a processing procedure until focusing on an area. The feature amount calculation unit 141 captures a captured image (through image or through image sequence) from the buffer memory unit 130 (step S1), and executes basic image processing on the captured image (step S1). S2).

  The region extraction unit 142 reads the captured image for each image feature amount (for example, YCrCb) via the buffer memory unit 130 (step S3). The region extraction unit 142 creates a binarized image based on a plurality of image feature amounts in the read image (step S4). The region extraction unit 142 extracts a region from the binarized image and assigns a label to the extracted region (step S5).

  If there are regions that are estimated to be noise in the image, the region extraction unit 142 removes those regions from the image. In addition, when the region estimated to be the background is in the image, the region extraction unit 142 removes the region from the image (step S6).

  The priority setting unit 143 sets priorities for the regions extracted by the region extraction unit 142 (step S7). Further, the region extraction unit 142 shapes the region for which the priority setting unit 143 has determined the priority into a rectangle so as to fit the AF evaluation region (step S8). The CPU 190 executes a contrast scan (focus adjustment process) based on the contrast value of the AF evaluation area (step S9). Further, the CPU 190 gives priority to an area in which a close subject is imaged based on the result of contrast scanning, and focuses the area (step S10).

Next, a procedure of processing (see step S7 in FIG. 5) in which the area extraction device determines priority will be described.
<When a high priority is set for the red area>
FIG. 6 is a flowchart showing a processing procedure for determining a high priority for the red-colored region. Here, it is assumed that a maximum of three areas are selected as an example. The priority setting unit 143 determines whether or not a red area is present in the area (see FIG. 2) extracted based on the binary image (step Sa1).

  When there is no red area (step Sa1-NO), the priority setting unit 143 selects an area other than the red area in descending order of evaluation value (step Sa2). Then, the priority setting unit 143 registers the red system region (if present) and the selected region in the storage unit 160 (see FIG. 1) as the subject region (step Sa3). On the other hand, in step Sa1, when the red region is present (step Sa1-YES), priority setting unit 143 sets the priority of the red region to priority 1, and proceeds to step Sa2.

  Thereby, in the example shown in FIG. 2, the red area 400 is defined as the area of the main subject having the priority 1. Of the areas other than the red area 400, the area having the highest evaluation value is defined as the area of the main subject with priority 2. Of the areas other than the red area 400, the area having the second highest evaluation value is defined as the area of the main subject with priority 3.

<When high priority is set for the red, yellow and blue areas>
FIG. 7 is a flowchart showing a processing procedure for determining a high priority for the red area, the yellow area, and the blue area. Here, it is assumed that a maximum of three areas are selected as an example. The priority setting unit 143 determines whether or not a red region, a yellow region, or a blue region is present in the region (see FIG. 2) extracted based on the binary image (step Sb1).

  Priority setting when there is no red area, yellow area, and blue area (step Sb1-NO), and the total number of red areas, yellow areas, and blue areas is less than 3 The unit 143 selects the remaining areas in descending order of evaluation values, and sets the priority for the selected areas in the order of evaluation values (step Sb2). Then, the priority setting unit 143 registers all the selected regions in the storage unit 160 (see FIG. 1) as subject regions (step Sb3).

  On the other hand, in step Sb1, when there is a red region, a yellow region, or a blue region (step Sb1-YES), the priority setting unit 143 selects these regions in descending order of evaluation value, and selects the selected region. A higher priority is set in the order of evaluation values, and the process proceeds to step Sb2 (step Sb4). Note that the same priority may be set for the red region, the yellow region, and the blue region.

  Thus, in the example shown in FIG. 2, the region having the highest evaluation value among the red-based region 400, the yellow-based region 520, and the blue-based region 500, for example, the yellow-based region 520 is the main subject with priority 1. It is defined as an area. The second highest evaluation value among the red color region 400, the yellow color region 520, and the blue color region 500, for example, the red color region 400, is defined as a main subject region with priority 2. The third highest evaluation value among the red-based region 400, the yellow-based region 520, and the blue-based region 500, for example, the blue-based region 500 is defined as a main subject region with priority 3. For example, when there is no blue-based region 500, the region with the highest evaluation value may be determined as the region of the main subject with priority 3.

<About the case where a high priority is set in the order of a red area, a yellow area, and a blue area>
FIG. 8 is a flowchart showing a processing procedure for determining a high priority in the order of the red area, the yellow area, and the blue area. Here, it is assumed that a maximum of three areas are selected as an example. The priority setting unit 143 determines whether or not a red area is present in the area (see FIG. 2) extracted based on the binary image (step Sc1).

  When there is no red area (step Sc1-NO), the priority setting unit 143 determines whether or not there is a yellow area in the area extracted based on the binary image (step Sc2). When there is no yellow system area (step Sc2-NO), the priority setting unit 143 determines whether or not a blue system area exists in the area extracted based on the binary image (step Sc3).

  If there is no blue-based region (step Sc3-NO), and the total number of red-based regions, yellow-based regions, and blue-based regions is less than 3, the priority setting unit 143 determines the remaining regions as evaluation values. Are selected in descending order, and a high priority is set for the selected area in the order of evaluation values (step Sc4). Then, the priority setting unit 143 registers all selected areas in the storage unit 160 (see FIG. 1) as subject areas (step Sc5).

  On the other hand, if there is a red area in step Sc1 (step Sc1-YES), priority setting unit 143 sets priority 1 for the red area, and proceeds to step Sc2 (step Sb6). In Step Sc2, if there is a red area (Step Sc1-YES), priority setting unit 143 sets priority 1 for the red area, and proceeds to Step Sc2 (Step Sb6).

  In step Sc2, if a yellow area exists (step Sc2-YES), priority setting unit 143 sets priority 1 or 2 for the yellow area, and proceeds to step Sc3 (step Sb7). In step Sc3, when there is a blue region (step Sc3-YES), the priority setting unit 143 sets priority 1, 2, or 3 for the blue region, and proceeds to step Sc4 (step Sb8). ).

  Thereby, in the example shown in FIG. 2, the red area 400 is defined as the area of the main subject having the priority 1. The yellow area 520 is defined as the area of the main subject with priority 2. In addition, the blue area 500 is defined as the area of the main subject with priority 3.

<Regarding the case where high priority is set for the red area, the yellow area, the blue area, and the high luminance area>
FIG. 9 is a flowchart showing a procedure of processing for determining a high priority for the red area, the yellow area, the blue area, and the high luminance area. Here, it is assumed that a maximum of three areas are selected as an example. The priority setting unit 143 determines whether or not a red region, a yellow region, or a blue region is present in the region (see FIG. 2) extracted based on the binary image (step Sd1).

  When there is no red area, yellow area, blue area, and high brightness area (step Sd1-NO), and the total number of red areas, yellow area, blue area, and high brightness area is less than 3 In the case of, the priority setting unit 143 selects the remaining areas in descending order of evaluation values, and determines the priority in the order of evaluation values for the selected areas (step Sd2). Then, the priority setting unit 143 registers all the selected regions in the storage unit 160 (see FIG. 1) as subject regions (step Sd3).

  On the other hand, in step Sd1, when there is a red region, a yellow region, a blue region, or a high luminance region (step Sd1-YES), priority setting unit 143 selects these regions in descending order of evaluation value. The priority is set to the selected region in the order of evaluation value, and the process proceeds to step Sd2 (step Sd4). The same priority may be set for the red region, the yellow region, the blue region, and the high luminance region.

  Accordingly, in the example shown in FIG. 2, the region having the highest evaluation value among the red region 400, the yellow region 520, the blue region 500, and the high luminance region 300, for example, the red region 400 is assigned a priority. 1 main subject area. Further, the second highest evaluation value among the red color region 400, the yellow color region 520, the blue color region 500, and the high luminance region 300, for example, the yellow color region 520, is defined as a main subject region of priority 2. It is done. The third highest evaluation value among the red color region 400, the yellow color region 520, the blue color region 500, and the high luminance region 300, for example, the high luminance region 300 is defined as a main subject region with priority 3. It is done.

<Regarding the case where a low priority is set for a low-luminance area compared to other extracted areas>
FIG. 10 is a flowchart showing a processing procedure for determining a low priority as a low-luminance area as compared with other extracted areas. Here, it is assumed that a maximum of three areas are selected as an example. The priority setting unit 143 selects an area other than the low-luminance area in descending order of evaluation value, and sets a high priority in the order of evaluation value for the selected area (Step Se1).

  In addition, when the total number of areas other than the low-luminance area is less than 3, the priority setting unit 143 selects the remaining areas in descending order of evaluation values, and sets the selected areas with high priorities in the order of evaluation values ( Step Se2). Further, the priority setting unit 143 registers all the selected areas in the storage unit 160 (see FIG. 1) as subject areas (step Se3).

  Thus, in the example shown in FIG. 2, the lowest luminance area 320 in which a black subject is captured that is not a low luminance area included in the category 3 is the subject area when no other area is extracted from the image. Are registered in the storage unit 160.

As described above, the region extraction device 140 extracts the predetermined region from the image based on the color information included in the image data of the input image, and the region extraction unit 142 extracts the predetermined region. The area includes a priority setting unit 143 that determines priority according to predetermined color information.
With this configuration, the region extraction device 140 determines the priority for the region extracted from the image according to predetermined color information. Thereby, the area extracting apparatus can appropriately extract the area of the main subject from the input image.

In addition, the priority setting unit 143 assigns the priority to the region extracted by the region extraction unit 142 according to the hue angle (h) in the uniform color space (L ^* a ^* b ^* color system, CIELab color space). Determine. Thereby, the area extracting apparatus can appropriately extract the area of the main subject from the input image according to the hue angle.

  In addition, the priority setting unit 143 extracts the predetermined region when the region extraction unit 142 extracts a predetermined region from the image based on the color information of the red component, the yellow component, and the blue component included in the image data. A higher priority is determined for the region than the region extracted based on color information other than the red component, yellow component, and blue component included in the image data. Thereby, the area extracting apparatus can appropriately extract the areas of the main subjects of red, yellow and blue from the input image.

  The priority setting unit 143 includes an area extracted based on the color information of the red component, an area extracted based on the color information of the yellow component, and an area extracted based on the color information of the blue component included in the image data. High priority is set in order of the area. Thereby, the area extracting apparatus can appropriately extract the main subject area having a higher priority in the order of the red component, the yellow component, and the blue component from the input image.

Also, the priority setting unit 143 determines the priority according to the saturation (C ^* ) of the region extracted by the region extraction unit 142. Thereby, the area extracting apparatus can appropriately extract the area of the main subject having high saturation from the input image.

  In addition, the region extraction unit 142 extracts a predetermined region from the image based on the luminance information included in the image data, and the priority setting unit 143 responds to the luminance of the region extracted by the region extraction unit 142. , Define priority. Thereby, the area extracting apparatus can appropriately extract the area of the main subject having high luminance from the input image.

  In addition, the region extraction unit 142 extracts a region having a luminance higher than a predetermined first threshold value, and the priority setting unit 143 adds another region having a luminance higher than the first threshold value to the extracted region. Set a higher priority compared to the extracted area. Thereby, the area extracting apparatus can appropriately extract the area of the main subject having high luminance from the input image.

  In addition, the priority setting unit 143 may extract an area extracted based on the color information of the red component, an area extracted based on the color information of the yellow component, an area having a luminance higher than the first threshold, and the blue component. High priority is determined in the order of the areas extracted based on the color information. Thereby, the area extracting apparatus can appropriately extract the area of the main subject that is higher in the order of the red component, the yellow component, the high luminance, and the blue component from the input image.

  In addition, the region extraction unit 142 extracts a region having a luminance lower than a predetermined second threshold, and the priority setting unit 143 adds another region to the extracted region having a luminance lower than the second threshold. Set a lower priority compared to the extracted region. Thereby, the area extracting apparatus can appropriately extract the area of the main subject that is not the low luminance area (for example, the area where the shadow is imaged) from the input image.

  In addition, the region extraction unit 142 changes the second threshold value to a lower value, extracts a region having a luminance lower than the changed second threshold value, and the priority setting unit 143 uses the changed second threshold value. A lower priority is set for the extracted area having a lower luminance than the other extracted areas. Thereby, the area extracting apparatus can appropriately extract the area of the main subject from the input image.

  The priority setting unit 143 determines priority according to the distance (color difference) from the origin in the coordinate system of the uniform color space. Thereby, the area extracting apparatus can appropriately extract the area of the black main subject that is not the area where the shadow is captured from the input image.

  Further, the region extraction device 140 extracts a predetermined region from the image based on the saturation information included in the image data of the input image, and the region extracted by the region extraction unit 142. A priority setting unit 143 that determines priority according to predetermined saturation. Thereby, the area extracting apparatus can appropriately extract the area of the main subject from the input image.

  In addition, the region extracting device 140 extracts a predetermined region from the image based on the luminance information included in the image data of the input image, and the region extracted by the region extracting unit 142 in advance. A priority setting unit 143 that determines the priority according to the determined luminance. Thereby, the area extracting apparatus can appropriately extract the area of the main subject from the input image.

Further, the region extracting device 140 is based on a distance (color difference) from the origin in a coordinate system of a uniform color space (L ^* a ^* b ^* color system) representing color information included in image data of an input image. A region extracting unit 142 that extracts a predetermined region from the image; and a priority setting unit 143 that determines a priority for the region extracted by the region extracting unit 142 according to the distance.
Thereby, the area extracting apparatus can appropriately extract the area of the main subject from the captured image according to at least one of hue, saturation, and brightness.

In addition, the imaging apparatus 100 includes an area extraction device 140 and a CPU 190 that adjusts the focus by an optical system (for example, the AF lens 112) so as to focus on a region having a high priority among the regions extracted by the region extraction device 140 And an imaging unit 110 that captures an image including the high priority area in a state where the high priority area is in focus.
With this configuration, the region extraction device 140 appropriately extracts the region of the main subject from the input image. Accordingly, the imaging apparatus 100 can capture an image including the main subject area in a state where the main subject area is in focus.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes designs and the like that do not depart from the gist of the present invention.

  Further, by recording a program for realizing the region extraction device and the imaging device described above on a computer-readable recording medium, causing the computer system to read and execute the program recorded on the recording medium, Execution processing may be performed. Here, the “computer system” may include an OS and hardware such as peripheral devices.

  Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used. The “computer-readable recording medium” means a flexible disk, a magneto-optical disk, a ROM, a writable nonvolatile memory such as a flash memory, a portable medium such as a CD-ROM, a hard disk built in a computer system, etc. This is a storage device.

Further, the “computer-readable recording medium” refers to a volatile memory (for example, DRAM (Dynamic) in a computer system serving as a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. Random Access Memory)) that holds a program for a certain period of time is also included.
The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line.
The program may be for realizing a part of the functions described above.
Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, and what is called a difference file (difference program) may be sufficient.

DESCRIPTION OF SYMBOLS 100 ... Imaging device, 110 ... Imaging part, 111 ... Lens barrel, 140 ... Area extraction apparatus, 141 ... Feature-value calculation part, 142 ... Area extraction part, 143 ... Priority setting part

Claims (19)

An area extracting unit that extracts a predetermined area from the image based on color information included in the image data of the input image;
A priority setting unit that determines priority according to predetermined color information in the region extracted by the region extraction unit;
An area extracting apparatus comprising:   The region extraction apparatus according to claim 1, wherein the priority setting unit determines a priority for the region extracted by the region extraction unit according to a hue angle in a uniform color space.   The priority setting unit, when the region extraction unit extracts a predetermined region from the image based on color information of a red component, a yellow component, and a blue component included in the image data, The method according to claim 1, wherein a higher priority is determined than a region extracted based on color information other than a red component, a yellow component, and a blue component included in the image data. Region extraction device.   The priority setting unit includes an area extracted based on red component color information, an area extracted based on yellow component color information, and an area extracted based on blue component color information included in the image data. The region extraction apparatus according to any one of claims 1 to 3, wherein a high priority is determined in order.   5. The region extraction device according to claim 1, wherein the priority setting unit determines priority according to a saturation of the region extracted by the region extraction unit. . The region extraction unit extracts a predetermined region from the image based on luminance information included in the image data,
The region extraction device according to any one of claims 1 to 5, wherein the priority setting unit determines a priority according to luminance of the region extracted by the region extraction unit. The region extraction unit extracts a region having a luminance higher than a predetermined first threshold,
The said priority setting part determines a high priority compared with the said other extracted area | region to the said extracted area | region of the brightness | luminance higher than the said 1st threshold value, Region extraction device.   The priority setting unit includes an area extracted based on the color information of the red component, an area extracted based on the color information of the yellow component, an area having a luminance higher than the first threshold, and the color information of the blue component 8. The region extracting apparatus according to claim 6, wherein a high priority is determined in the order of the regions extracted based on the regions. The region extraction unit extracts a region having a luminance lower than a predetermined second threshold,
The said priority setting part determines a low priority compared with the other said extracted area | region to the said extracted area | region of the brightness | luminance lower than the said 2nd threshold value from Claim 6 characterized by the above-mentioned. The region extraction device according to claim 8. The region extraction unit changes the second threshold value to a lower value, extracts a region having a luminance lower than the changed second threshold value,
The priority setting unit sets a lower priority in the extracted area having a luminance lower than the changed second threshold value as compared with the other extracted areas. The region extraction device described.   The area extraction device according to any one of claims 1 to 10, wherein the priority setting unit determines priority according to a distance from an origin in a coordinate system of a uniform color space. An area extraction unit that extracts a predetermined area from the image based on the saturation information included in the image data of the input image;
A priority setting unit that determines priority according to a predetermined saturation in the region extracted by the region extraction unit;
An area extracting apparatus comprising: An area extracting unit that extracts a predetermined area from the image based on luminance information included in the image data of the input image;
A priority setting unit that determines priority according to a predetermined luminance in the region extracted by the region extraction unit;
An area extracting apparatus comprising: An area extraction unit that extracts a predetermined area from the image based on a distance from the origin in a coordinate system of a uniform color space representing color information included in image data of the input image;
A priority setting unit that determines priority according to the distance to the region extracted by the region extraction unit;
An area extracting apparatus comprising: The region extraction device according to any one of claims 1 to 14,
A focus adjusting unit that adjusts a focus by an optical system so as to focus on a high priority area among the areas extracted by the area extracting device;
An imaging unit that captures an image including the high priority area in a state in which the high priority area is focused;
An imaging apparatus comprising: On the computer,
A procedure for extracting a predetermined region from the image based on color information included in the image data of the input image;
A procedure for determining priority in the predetermined area according to predetermined color information;
Extraction program to execute. On the computer,
A procedure for extracting a predetermined region from the image based on the saturation information included in the image data of the input image;
In the predetermined area, a procedure for determining a priority according to a predetermined saturation;
Extraction program to execute. On the computer,
A procedure for extracting a predetermined region from the image based on luminance information included in the image data of the input image;
In the predetermined area, a procedure for setting a priority according to a predetermined luminance;
Extraction program to execute. On the computer,
A procedure for extracting a predetermined region from the image based on a distance from the origin in a coordinate system of a uniform color space representing color information included in image data of the input image;
In the predetermined area, a procedure for determining priority according to the distance;
Extraction program to execute.

Images