JP2011191860A - Imaging apparatus, imaging processing method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus, an imaging processing method, and a program, for preventing the failure of the cutout of an object area. <P>SOLUTION: This imaging apparatus 100 is provided with: an electronic imaging part 2 which picks up an object existence image in which an object exists in a background; an image area determination part 8d which determines whether there exists a closed area almost surrounded by pixels in a color which is different from the color of a background image (for example, an object non-existence image) in the object existence image and whether there exists the same color area in almost the same color as the color of the background image in the closed area; and a display part 10 which informs, when it is determined that there exists the same color area in the closed area, a user that there exists the same color area. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an imaging apparatus, an imaging processing method, and a program.

  Conventionally, an application that uses an imaging device to capture an image with a subject in the background and a background image without the subject, generate difference information from the background image and the image with the subject, and extract only the subject It is known (see, for example, Patent Document 1).

Japanese Patent Laid-Open No. 10-21408

  However, the application disclosed in Patent Document 1 has a problem that the subject area cannot be properly cut out when the color of the subject to be cut out and the color tone of the background image are substantially equal.

  Accordingly, an object of the present invention is to provide an imaging device, an imaging processing method, and a program that can prevent a failure in clipping of a subject area.

In order to solve the above problem, an imaging apparatus according to claim 1 is provided.
An acquisition unit that acquires a background image, an imaging unit that captures a subject-existing image in which a subject exists in the background, and a first region that is substantially surrounded by a region having a color different from the color of the background image. A first area determining means for determining whether or not the first area exists, and when the first area determining means determines that the first area exists, the background is included in the first area in the subject existing image. A second area determining means for determining whether or not a second area having substantially the same color as the image exists; and when the second area determining means determines that the second area exists, the fact that the second area exists. And an informing means for informing.

The invention according to claim 2 is the imaging apparatus according to claim 1,
The first area is a closed area surrounded by pixels of a different color from the background image.

The invention according to claim 3 is the imaging apparatus according to claim 1 or 2,
The background of the subject existing image captured by the imaging means is substantially the same as the corresponding region of the background image.

An imaging device according to a fourth aspect of the present invention provides:
Imaging means for capturing a subject existing image in which a subject is present in the background, nonflatness determining means for determining whether or not the nonflatness of the background portion of the subject existing image is equal to or less than a predetermined value, and the nonflatness When the non-flatness is determined to be equal to or less than a predetermined value by the determination unit, it is determined whether or not the subject existing image includes a first region substantially surrounded by a region having a color different from the color of the background image. A first area determination unit that performs the first area determination, and the first area determination unit determines that the first area exists, the first area of the subject-existing image has substantially the same color as the color of the background portion. Second area determination means for determining whether or not the second area exists, and notification means for notifying that the second area exists when it is determined by the second area determination processing that the second area exists. And features To have.

The invention according to claim 5 is the imaging apparatus according to claim 4,
The first area is a closed area surrounded by pixels of a different color from the background portion.

Invention of Claim 6 is an imaging device as described in any one of Claims 1-5,
The subject presence images are sequentially picked up by the image pickup means and displayed on a display means.

The invention according to claim 7 is the imaging apparatus according to claim 6,
The informing means displays the second region in the subject existing image displayed on the display means in a display form that can be identified.

The invention according to claim 8 is the imaging apparatus according to any one of claims 1 to 7,
The image processing device further includes image generation means for generating a subject image by cutting out a region where the subject exists from the subject existing image.

The imaging processing method of the invention according to claim 9 is:
An imaging processing method using an imaging apparatus including an imaging unit that captures a subject existing image in which a subject is present in the background, an acquisition process for acquiring a background image, and a color of the background image on the acquired subject existing image A first area determination process for determining whether or not a first area substantially surrounded by areas of different colors exists, and when the first area determination process determines that the first area exists, the subject A second area determination process for determining whether or not a second area having substantially the same color as the background image exists in the first area in the existing image, and that the second area exists by the second area determination means. And a notification process for notifying that the second area exists when it is determined.

The imaging processing method of the invention according to claim 10 is:
An imaging processing method using an imaging device including an imaging unit that captures a subject existing image in which a subject is present in a background, and determines whether or not the non-flatness of a background portion of the subject existing image is equal to or less than a predetermined value And when the non-flatness is determined to be less than or equal to a predetermined value by the non-flatness determination process, the subject-existing image is substantially surrounded by an area of a color different from the color of the background image. A first area determination process for determining whether or not the first area exists, and when it is determined by the first area determination process that the first area exists, the first area in the subject existing image A second region determination process for determining whether or not a second region having substantially the same color as the color of the background portion exists, and when the second region determination process determines that the second region exists, The fact that the second area exists It is characterized in that it comprises, a notification process of notifying.

The program of the invention described in claim 11 is
The computer of the imaging apparatus is substantially surrounded by an acquisition unit that acquires a background image, an imaging control unit that captures a subject existing image in which a subject exists in the background, and an area of a color different from the color of the background image. A first area determining means for determining whether or not the first area exists, and when the first area determining means determines that the first area exists, the first area determining means 2nd area determination means for determining whether or not the second area having substantially the same color as the background image exists, and the second area determination means determines that the second area exists when the second area determination means determines that the second area exists. It is characterized by functioning as a notification control means for notifying that there is a problem.

The program of the invention described in claim 12 is:
An imaging control unit that causes a computer of the imaging apparatus to capture a subject existing image in which a subject exists in the background, and a non-flatness determination unit that determines whether or not the non-flatness of the background portion of the subject existing image is equal to or less than a predetermined value When the non-flatness determination means determines that the non-flatness is less than or equal to a predetermined value, the subject existing image has a first area substantially surrounded by an area of a color different from the color of the background image. A first region determining means for determining whether or not the first region is present, and when the first region determining means determines that the first region exists, the color of the background portion in the first region in the subject existing image A second region determining means for determining whether or not there is a second region having substantially the same color as the first region, and notifying that the second region exists when the second region determining unit determines that the second region exists. Notification system Means, is characterized in that to function as a.

  According to the present invention, it is possible to appropriately prevent failure in clipping of the subject area.

It is a block diagram which shows schematic structure of the imaging device of Embodiment 1 to which this invention is applied. 3 is a flowchart illustrating an example of an operation related to subject clipping processing by the imaging apparatus of FIG. 1. FIG. 3 is a flowchart showing a continuation of the subject clipping process in FIG. 2. FIG. It is a figure which shows typically an example of the image for demonstrating the object clipping process of FIG. It is a figure which shows typically an example of the image for demonstrating the object clipping process of FIG. It is a block diagram which shows schematic structure of the imaging device of Embodiment 2 to which this invention is applied. 7 is a flowchart illustrating an example of an operation related to a subject clipping process performed by the imaging apparatus of FIG. 6. FIG. 8 is a flowchart illustrating an example of an operation related to an extraction background generation process in the subject clipping process of FIG. 7. FIG. FIG. 9 is a flowchart showing a continuation of the extraction background generation processing of FIG. 8. FIG. It is a flowchart which shows an example of the operation | movement which concerns on the area | region detection process in the to-be-cut object process of FIG.

  Hereinafter, specific embodiments of the present invention will be described with reference to the drawings. However, the scope of the invention is not limited to the illustrated examples.

[Embodiment 1]
The imaging apparatus 100 according to the first embodiment captures a subject-existing image P1 in which the subject S exists in the background after capturing the subject non-existing image P1 in which the subject S does not exist in the background, and the subject-existing image. When it is determined that the same color area A2 having the same color as the subject non-existing image P1 exists in the closed region A1 substantially surrounded by pixels of a different color from the subject non-existing image P1 in P2, The fact that the same color area A2 exists is notified.

FIG. 1 is a block diagram illustrating a schematic configuration of an imaging apparatus 100 according to the first embodiment to which the present invention is applied.
As illustrated in FIG. 1, the imaging apparatus 100 according to the first embodiment includes a lens unit 1, an electronic imaging unit 2, an imaging control unit 3, an image data generation unit 4, an image memory 5, and a feature amount calculation unit 6. A block matching unit 7, an image processing unit 8, a display control unit 9, a display unit 10, a recording medium control unit 11, an operation input unit 12, and a central control unit 13.
Further, the imaging control unit 3, the feature amount calculation unit 6, the block matching unit 7, the image processing unit 8, and the central control unit 13 are designed as a custom LSI 1A, for example.

The lens unit 1 includes a plurality of lenses and includes a zoom lens, a focus lens, and the like.
Although not shown, the lens unit 1 includes a zoom driving unit and a focus lens that move the zoom lens in the optical axis direction when imaging the subject S (for example, a mouse or the like; see FIG. 5C or the like). A focusing drive unit or the like that moves the lens in the optical axis direction may be provided.

  The electronic imaging unit 2 is composed of, for example, an image sensor such as a charge coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS), and converts an optical image that has passed through various lenses of the lens unit 1 into a two-dimensional image signal. To do.

Although not shown, the imaging control unit 3 includes a timing generator, a driver, and the like. Then, the imaging control unit 3 scans and drives the electronic imaging unit 2 with a timing generator and a driver, converts the optical image into a two-dimensional image signal with the electronic imaging unit 2 every predetermined period, and the electronic imaging unit 2 Image frames are read out from the imaging area for each screen and output to the image data generation unit 4.
In addition, the imaging control unit 3 performs adjustment control of imaging conditions of the subject S such as AF (automatic focusing process), AE (automatic exposure process), AWB (automatic white balance), and the like.

Further, when the subject clipping mode (described later) is set as the imaging mode, the imaging control unit 3 sets the subject S in the background that has passed through the lens unit 1 based on the imaging instruction operation of the shutter button 12a by the user. An optical image of the existing subject presence image P2 (see FIG. 5C) is converted into a two-dimensional image signal by the electronic imaging unit 2 under a predetermined imaging condition, and the subject presence image P2 is captured from the imaging region of the electronic imaging unit 2. The image frame concerning is read.
Here, the electronic image pickup unit 2 and the image pickup control unit 3 constitute an image pickup unit that picks up the subject existing image P2 in which the subject S is present in the background at a predetermined timing.

  In addition, the imaging control unit 3 includes an imaging stop unit 3a. The imaging stop unit 3a is a subject non-existing image (background) in the subject existing image P2 by an image area determination unit 8d (described later) of the image processing unit 8. When it is determined that the same color area (second area) A2 having substantially the same color as the subject non-existing image P1 exists in the closed area (first area) A1 surrounded by pixels of a color different from the image P1 Then, the imaging of the subject existing image P2 by the electronic imaging unit 2 is stopped (details will be described later).

The image data generation unit 4 appropriately adjusts the gain for each RGB color component with respect to the analog value signal of the image frame transferred from the electronic imaging unit 2, and then performs sample holding by a sample hold circuit (not shown). The digital signal is converted into digital data by a / D converter (not shown), color processing including pixel interpolation processing and γ correction processing is performed by a color process circuit (not shown), and then a digital luminance signal Y and color difference signal Cb , Cr (YUV data). In addition, the image data generation unit 4 performs reduction processing of YUV data at a predetermined magnification both horizontally and vertically to generate low-resolution reduced image data for live view display.
YUV data output from the color process circuit is DMA-transferred to an image memory 5 used as a buffer memory via a DMA controller (not shown).

  A demosaic unit (not shown) for developing the digital data after A / D conversion may be mounted on the custom LSI 1A.

  The image memory 5 is composed of, for example, a DRAM or the like, and temporarily stores YUV data of the subject existing image P2 and the subject non-existing image P1 transferred from the image data generating unit 4. The image memory 5 temporarily stores data processed by the feature amount calculation unit 6, the block matching unit 7, the image processing unit 8, the central control unit 13, and the like.

The feature amount calculation unit 6 performs feature extraction processing for extracting feature points from the subject nonexistent image P1 with reference to the subject nonexistent image P1 in which the subject S does not exist in the background.
Specifically, the feature amount calculation unit 6 selects, for example, a predetermined number (or a predetermined number or more) of high feature block areas (feature points) based on the YUV data of the subject non-existing image P1, The contents of the block are extracted as a template (for example, a square of 16 × 16 pixels).
Here, the feature extraction process is a process of selecting a feature having a high characteristic convenient for tracking from a large number of candidate blocks.

The block matching unit 7 performs block matching processing for positioning the subject non-existing image P1 and the subject existing image P2.
Specifically, the block matching unit 7 optimally matches where the template extracted by the feature extraction process corresponds to in the subject existing image P2, that is, the template pixel value in the subject existing image P2. Search for a position (corresponding region). Here, the imaging of the subject S is performed with, for example, a plain background with a small feature amount in the background, so that the background portion other than the subject S of the subject existing image P2 and the background portion of the subject nonexisting image P1 are supported. It is substantially the same as the area to be performed.
Then, the block matching unit 7 determines the difference between the subject non-existing images P1 and the subject existing images P2 having the best evaluation values (for example, the sum of squared differences (SSD) and the sum of absolute differences (SAD)). Is calculated as a motion vector of the template.

  The image processing unit 8 includes an image acquisition unit 8a, a division unit 8b, a color calculation unit 8c, an image region determination unit 8d, a registration unit 8e, a subject image extraction unit 8f, a position information generation unit 8g, A cut-out image generation unit 8h.

The image acquisition unit 8 a acquires YUV data of the subject nonexistent image (background image) P <b> 1 temporarily stored in the image memory 5. That is, the image acquisition unit 8a performs image capturing from the image data generation unit 4 after the subject nonexistent image P1 (see FIG. 4A) is captured by the electronic imaging unit 2 and the imaging control unit 3 in the subject clipping process (described later). The YUV data of the subject nonexistent image P1 transferred and temporarily stored in the image memory 5 is acquired.
Here, the image acquisition unit 8a constitutes acquisition means for acquiring the subject non-existence image (background image) P1 stored in the image memory 5.

The dividing unit 8b divides each of the subject existing image P2 and the subject non-existing image P1 into a plurality of image blocks B,.
That is, the dividing unit 8b performs a subject clipping process on the basis of the YUV data of the subject non-existing image P1 acquired by the image acquiring unit 8a. The image is divided into image blocks B,... (See FIG. 4B). In addition, the dividing unit 8b converts the entire subject existing image P2 into the subject nonexistent image based on the YUV data of the subject existing image P2 displayed in the live view acquired by the image acquiring unit 8a in the subject clipping process. Similar to P1, the image is divided into a plurality of image blocks B,.
Note that the image data of the subject presence image P2 and the subject non-existence image P1 to be processed by the dividing unit 8b may be, for example, high-resolution image data recorded on the recording medium 11a for recording. Low-resolution reduced image data for live view display generated by the data generation unit 4 may be used, or dedicated image data having a predetermined resolution adjusted for calculating the average color of each image block B described later. There may be.

The color calculation unit 8c calculates a color representing each image block B for each of the plurality of image blocks B,... Of the subject non-existing image P1 and the subject existing image P2 divided by the dividing unit 8b. Specifically, the color calculation unit 8c calculates, for example, an average value, a median value, a mode value, and the like for the colors of a plurality of pixels in each image block B as representative values.
Further, the color calculation unit 8c calculates the average color of the subject nonexistent image P1 from the average color of each image block B of the subject nonexistent image P1.

The image area determination unit 8d includes a same-color area A2 having substantially the same color as the subject non-existing image P1 in the closed area A1 that is substantially surrounded by pixels of a different color from the subject non-existing image P1 in the subject existing image P2. It is determined whether or not it exists.
That is, the image area determination unit 8d performs representative colors (for example, average color) of the plurality of image blocks B of the subject non-existing image P1 and the subject existing image P2 divided by the dividing unit 8b in the subject clipping process. ), It is determined whether or not the same color area A2 that is substantially the same color as the subject non-existing image P1 exists in the closed area A1 of the subject existing image P2. Specifically, the image area determination unit 8d determines the average color of each of the plurality of image blocks B of the subject existing image (live view image) P2 displayed in live view and the average color of the subject non-existing image P1. The average color of each image block B of the subject presence image P2 and the average color of the subject non-existence image P1 are compared and determined based on the difference between the average color of the subject non-existence image P1 in the subject presence image P2. A closed region A1 surrounded by color pixels is specified. When the closed region A1 is specified, the image region determination unit 8d converts the average color of each of the plurality of image blocks B of the live view image of the subject presence image P2 and the average color of the subject non-existence image P1. Based on the average color of each image block B in the closed region A1 of the subject presence image P2 and the average color of the subject non-existence image P1, the average of the subject non-existence image P1 in the closed region A1 is determined. It is determined whether or not there is a same color area A2 composed of pixels of substantially the same color as the color.

It should be noted that the determination of the colors of the subject non-existing image P1 and the subject existing image P2 may be made strictly, or the difference between the colors of the subject non-existing image P1 and the subject existing image P2 is a color within a predetermined range. May be within an error range. The error range may be a predetermined fixed range set in advance, manually based on a predetermined operation of the operation input unit 12 by the user, or the image area determination unit 8d and the central control unit 13. Etc. may be automatically and arbitrarily changed as appropriate.
Further, the closed region A1 is not limited to the subject existing image P2 that is completely surrounded by pixels of a different color from the subject non-existing image P1, and for example, has a color different from that of the subject non-existing image P1. Even if the pixels are separated from each other, an allowable range may be used as long as it can be replaced with an effective region by expansion processing, contraction processing, labeling processing, or the like. That is, the closed region (first region) A1 does not have to be completely surrounded by pixels of a different color from the subject non-existing image P1 in the subject existing image P2.
Further, in the determination of the closed region A1 or the same color region A2, the image region determination unit 8d compares and determines the average colors of the corresponding image blocks B of the subject existing image (live view image) P2 and the subject non-existing image P1. You may make it do.

  In this manner, the image area determination unit 8d is a closed area (first area) in which the subject presence image P2 captured by the electronic imaging unit 2 and the imaging control unit 3 is substantially surrounded by pixels of a color different from the subject non-existence image P1. (Area) First area determination means for determining whether or not A1 exists, and whether or not the same color area (second area) A2 having the same color as the subject non-existing image P1 exists in the closed area A1. The determination and second area determination means are configured.

The alignment unit 8e aligns the subject presence image P2 and the subject non-existence image P1 based on the feature points extracted from the subject non-existence image P1.
That is, the alignment unit 8e calculates a coordinate conversion formula (projection conversion matrix) of each pixel of the subject existing image P2 with respect to the subject nonexistent image P1 based on the feature points extracted from the subject nonexistent image P1, and the coordinates. The subject presence image P2 is coordinate-transformed according to the conversion formula and aligned with the subject non-existence image P1.

The subject image extraction unit 8f generates difference information (for example, a dissimilarity map or the like) of each corresponding pixel between the subject non-existence image P1 and the subject presence image P2 aligned by the alignment unit 8e. A subject image including the subject S is extracted from the subject presence image P2 based on the difference information.
Note that the subject image extraction unit 8f may use, for example, Equation (2) described later when generating the dissimilarity map as the difference information of each pixel, but is not limited thereto.

The position information generation unit 8g specifies the position of the subject image extracted in the subject presence image P2, and generates position information indicating the position of the subject image in the subject presence image P2.
Here, the position information includes, for example, an alpha map. The alpha map is an alpha value (0) for subjecting each pixel of the subject existing image P2 to alpha blending the subject image with a predetermined background. ≦ α ≦ 1).

Based on the generated alpha map, the cut-out image generation unit 8h does not transmit a pixel having an alpha value of 1 to a predetermined single color image (not shown) among the pixels of the subject existing image P2. In addition, the subject image is combined with a predetermined single color image so as to transmit pixels with an alpha value of 0, and image data of a subject cutout image (not shown) is generated.
As a result, the cut-out image generation unit 8h obtains a subject cut-out image obtained by cutting out a region including the subject from the subject existing image P2 in which the background and the subject exist.
Here, the cut-out image generation unit 8h constitutes image generation means for cutting out a subject area including the subject S from the subject existing image P2 to generate a subject cut-out image.

The display control unit 9 performs control to read display image data temporarily stored in the image memory 5 and display the data on the display unit 10.
Specifically, the display control unit 9 includes a VRAM, a VRAM controller, a digital video encoder, and the like. The digital video encoder reads the luminance signal Y and the color difference signals Cb and Cr read from the image memory 5 and stored in the VRAM (not shown) under the control of the central control unit 13 via the VRAM controller. The data is periodically read from the VRAM, a video signal is generated based on these data, and is output to the display unit 10.
The display unit 10 is, for example, a liquid crystal display device, and displays a live view image, a REC view image, and the like on a display screen based on a video signal from the display control unit 9.
That is, the display control unit 9 is captured as a live view image or a main captured image based on a plurality of image frames generated by imaging the subject S by the electronic imaging unit 2 and the imaging control unit 3 in the imaging mode. A REC view image or the like is displayed on the display screen of the display unit 10.

Further, the display control unit 9 displays the same color region A2 in the closed region A1 of the subject existing image P2 determined by the image region determination unit 8d in a distinguishable manner on the display screen of the display unit 10 (see FIG. 5B). ). Specifically, the display control unit 9 changes the color of the same color area A2 in the closed area A1 of the subject existing image P2 displayed in live view to another color (preferably a color that easily stimulates the retina). Or by lighting (flashing) the changed color at a predetermined interval to notify that the same color area A2 exists, thereby enabling the user to identify the same color area A2. In FIG. 5B, as a display mode in which the same color area A2 can be identified, a portion corresponding to the same color area A2 is schematically shown by hatching.
Here, when it is determined that the same color area A2 exists in the closed area A1 of the subject existing image P2 determined by the image area determination unit 8d, the display unit 10 and the display control unit 9 include the same color area A2. Informing means for informing the effect is configured.

In addition, although the display of the same color area A2 by the display unit 10 is illustrated as a notification mode of the same color area A2 in the closed area A1 of the subject existing image P2, it is only an example and is not limited to this, and arbitrarily arbitrary Can be changed.
That is, the mode of notification of the same color area A2 may be any mode as long as it is a method capable of grasping and recognizing the same color area A2 by human senses, in particular, visual, auditory, tactile, etc. The fact that the same color area A2 exists may be notified by sound (voice etc.) or vibration.

The recording medium control unit 11 is configured such that the recording medium 11a is detachable, and controls reading of data from the loaded recording medium 11a and writing of data to the recording medium 11a. That is, the recording medium control unit 11 records the image data for recording the captured image encoded by the JPEG compression unit (not shown) of the image processing unit 8 on the recording medium 11a.
Further, the image data of the subject image is associated with the alpha map generated by the position information generation unit 8g of the image processing unit 8, and the extension of the image data of the subject cutout image is stored as “.jpe”. Yes.

  The recording medium 11a is composed of, for example, a non-volatile memory (flash memory) or the like.

  The operation input unit 12 is for performing a predetermined operation of the imaging apparatus 100. Specifically, the operation input unit 12 includes a shutter button 12a related to an instruction to shoot the subject S, a selection determination button 12b related to an instruction to select an imaging mode or a function on the menu screen, and a zoom related to an instruction to adjust the zoom amount. Buttons (not shown) and the like are provided, and predetermined operation signals are output to the central control unit 13 in accordance with operations of these buttons.

  The central control unit 13 controls each unit of the imaging apparatus 100. Specifically, the central control unit 13 includes a CPU, a RAM, and a ROM (all not shown), and performs various control operations according to various processing programs (not shown) for the imaging apparatus 100.

Next, the subject clipping process performed by the imaging apparatus 100 will be described with reference to FIGS.
2 and 3 are flowcharts showing an example of an operation related to the subject clipping process.

The subject clipping process is executed when a subject clipping mode is selected from a plurality of imaging modes displayed on the menu screen based on a predetermined operation of the selection determination button 12b of the operation input unit 12 by the user. It is processing.
As shown in FIG. 2, first, based on a predetermined operation of the shutter button 12a of the operation input unit 12 by the user, the imaging control unit 3 sets the focus lens focus position and exposure conditions (shutter speed, aperture, amplification factor, etc.). ) And white balance and other imaging conditions are adjusted so that an optical image of a subject non-existing image P1 (see FIG. 4A) in which a predetermined subject (for example, subject S) does not exist in the background is set under the predetermined imaging condition. An image is picked up by the electronic image pickup unit 2 (step S1). The subject non-existing image P1 captured here is preferably a flat image of substantially a single color.
Then, after generating the YUV data of the subject non-existing image P1 transferred from the electronic imaging unit 2, the image data generation unit 4 temporarily stores the YUV data of the subject non-existing image P1 in the image memory 5.
Further, the imaging control unit 3 maintains a state in which imaging conditions such as a focus position, an exposure condition, and a white balance at the time of imaging the subject non-existing image P1 are fixed.

Next, after the image acquisition unit 8a of the image processing unit 8 acquires the YUV data of the subject non-existing image P1 temporarily stored in the image memory 5, the dividing unit 8b displays the entire subject non-existing image P1. Is divided into a plurality of image blocks B of a predetermined size (step S2; see FIG. 4B).
Subsequently, the color calculation unit 8c of the image processing unit 8 calculates an average color representing each image block B for each of the plurality of image blocks B of the subject nonexistent image P1 divided by the division unit 8b. Then, the average color of the subject non-existing image P1 is calculated from the average color of each image block B (step S3).

Next, the imaging control unit 3 converts the optical image of the image that has passed through the lens unit 1 into a two-dimensional image signal by the electronic imaging unit 2 under a predetermined imaging condition, and the image frame from the imaging region of the electronic imaging unit 2 Is read out to cause the electronic imaging unit 2 to capture a live view image (step S4; see FIG. 4C).
Subsequently, the image acquisition unit 8 a of the image processing unit 8 captures a plurality of image frames related to the live view image captured by the electronic imaging unit 2, transferred from the image data generation unit 4, and temporarily stored in the image memory 5. The YUV data is sequentially acquired, and for each of the acquired live view images, the dividing unit 8b converts the entire image into a plurality of image blocks B having a predetermined size like the subject non-existing image P1. Divide (step S5; refer to FIG. 5A).
Here, after the subject non-existing image P1 is captured, the subject S moves within the angle of view, or the subject S is moved within the angle of view so that the subject S exists within the angle of view. The live view image constitutes the subject presence image P2.

Next, the color calculation unit 8c of the image processing unit 8 represents each image block B with respect to each of the plurality of image blocks B,... Of the subject existing image P2 displayed in the live view divided by the dividing unit 8b. An average color is calculated (step S6).
Subsequently, the image region determination unit 8d of the image processing unit 8 determines whether or not there is a closed region A1 surrounded by pixels having a color different from the average color of the subject non-existing image P1 in the live view image (Step S1). S7). Specifically, the image area determination unit 8d determines the average color of each of the plurality of image blocks B of the subject existing image (live view image) P2 displayed in live view and the average color of the subject non-existing image P1. , The closed region A1 surrounded by pixels having a color different from the average color of the subject non-existing image P1 in the subject existing image P2 is specified. Then, the image area determination unit 8d determines whether or not the closed area A1 exists in the live view image according to the identification result.

If it is determined in step S7 that the closed region A1 exists in the live view image (subject present image P2) (step S7; YES), the image region determination unit 8d includes the subject nonexistent image in the closed region A1. It is determined whether or not there is a same color area A2 composed of pixels having a color substantially equal to the average color of P1 (step S8). Specifically, the image region determination unit 8d compares and determines the average color of each image block B in the closed region A1 of the subject existing image P2 and the average color of the subject non-existing image P1, thereby determining whether the image is within the closed region A1. It is determined whether or not there is a same color area A2 composed of pixels of approximately the same color as the average color of the subject non-existing image P1.
Here, if it is determined that the same color area A2 exists in the closed area A1 of the live view image (step S8; YES), the display control unit 9 is based on a plurality of image frames captured by the electronic imaging unit 2. By changing the color of the same color area A2 in the closed area A1 of the subject existing image P2 displayed in live view on the display screen of the display unit 10 to another color and displaying it, the fact that the same color area A2 exists is displayed. Notification is made (step S9; see FIG. 5B).

Next, the central control unit 13 determines whether or not an imaging instruction for the subject existing image P2 is input based on a predetermined operation of the shutter button 12a of the operation input unit 12 by the user (step S10).
Here, when it is determined that the imaging instruction for the subject presence image P2 is not input (step S10; NO), the imaging stop unit 3a of the imaging control unit 3 captures the subject presence image P2 by the electronic imaging unit 2. The central control unit 13 returns the processing to step S1.
In addition, the process after step S1 is the same as that of the above, The detailed description is abbreviate | omitted.

On the other hand, if it is determined in step S10 that an instruction to capture the subject existing image P2 is input (step S10; YES), the imaging control unit 3 electronically captures the optical image of the subject existing image P2 under a predetermined imaging condition. The image is taken by the unit 2 (step S11). That is, the same color area A2 in the closed area A1 is substantially the same color as the background of the subject non-existing image P1, and therefore the same color area A2 may not be properly cut out, but depending on the user's will When it is desired to cut out an image of a portion other than the same color area A2 of the closed area A1 of the subject image, the electronic imaging unit 2 and the imaging control unit 3 capture an optical image of the subject existing image P2 under a predetermined imaging condition.
Note that the predetermined imaging condition may be an imaging condition that is fixed after the imaging of the subject non-existing image P1, or may be an imaging condition that is newly set at the imaging timing of the subject existing image P2.
Then, after generating the YUV data of the subject existing image P2 transferred from the electronic imaging unit 2, the image data generating unit 4 temporarily stores the YUV data of the subject existing image P2 in the image memory 5.

On the other hand, when it is determined in step S8 that the same color area A2 does not exist in the closed area A1 of the live view image (step S8; NO), the central control unit 13 causes the shutter button 12a of the operation input unit 12 by the user to be used. Based on the predetermined operation, it is determined whether or not an imaging instruction for the subject presence image P2 has been input (step S12).
If it is determined in step S7 that the closed region A1 does not exist in the live view image (subject present image P2) (step S7; NO), the central control unit 13 moves to step S12 and the same as above. In step S12, it is determined whether or not an imaging instruction for the subject presence image P2 has been input.
When it is determined in step S12 that an instruction to capture the subject presence image P2 has not been input (step S12; NO), the central control unit 13 returns the process to step S4, and as described above, the imaging control unit. 3, the optical image of the image that has passed through the lens unit 1 is converted into a two-dimensional image signal by the electronic imaging unit 2, and the image frame related to the subject existing image P <b> 2 is read from the imaging region of the electronic imaging unit 2. Thus, a live view image is captured (step S4).
In addition, the process after step S4 is the same as that of the above, The detailed description is abbreviate | omitted.

  If it is determined in step S12 that an imaging instruction for the subject presence image P2 has been input (step S12; YES), the central control unit 13 proceeds to step S11, and the imaging control unit 3 An optical image of the subject presence image P2 is imaged by the electronic imaging unit 2 under an imaging condition fixed after the subject non-existence image P1 is imaged (step S11).

Next, the central control unit 13 uses the feature amount calculation unit 6, the block matching unit 7, and the image processing unit 8 as a reference for the subject presence image based on the YUV data of the subject non-existence image P 1 temporarily stored in the image memory 5. A projective transformation matrix for projectively transforming the P2 YUV data is calculated using a predetermined image transformation model (for example, a similarity transformation model or a joint transformation model) (step S13).
Specifically, the feature amount calculation unit 6 selects a predetermined number (or a predetermined number or more) of highly featured block regions (feature points) based on the YUV data of the subject non-existing image P1, and selects the relevant block. Is extracted as a template. Then, the block matching unit 7 searches the subject existing image P2 for a position where the pixel value of the template extracted by the feature extraction process is optimally matched, and the evaluation value of the pixel value difference is the best. An optimum offset between the subject non-existing image P1 and the subject existing image P2 is calculated as a motion vector of the template. Then, the alignment unit 8e of the image processing unit 8 statistically calculates the entire motion vector based on the motion vectors of the plurality of templates calculated by the block matching unit 7, and performs feature point correspondence related to the motion vector. The projection transformation matrix of the subject existing image P2 is calculated by using this.

  Next, the central control unit 13 causes the alignment unit 8e to perform projective transformation of the subject presence image P2 based on the calculated projective transformation example, thereby obtaining the YUV data of the subject presence image P2 and the subject non-existence image P1. A process of aligning the YUV data is performed (step S14).

Then, the central control unit 13 causes the subject image extraction unit 8f of the image processing unit 8 to perform processing for extracting a subject image including the subject S from the subject presence image P2 (step S15).
Specifically, the subject image extraction unit 8f applies a low-pass filter to each of the YUV data of the subject presence image P2 and the YUV data of the subject non-existence image P1 to remove high-frequency components of each image. Thereafter, the subject image extraction unit 8f calculates a difference for each corresponding pixel between the subject presence image P2 and the subject non-existence image P1 subjected to the low-pass filter, and generates a difference map. Subsequently, the subject image extraction unit 8f binarizes the dissimilarity map relating to each pixel with a predetermined threshold value, and then performs a contraction process in order to remove an area in which the dissimilarity is caused by fine noise or camera shake from the dissimilarity map. Do. Thereafter, the subject image extraction unit 8f performs a labeling process, removes regions below a predetermined value and regions other than the maximum region, identifies the largest island pattern as a subject image, and corrects the shrinkage. The expansion process is performed.

  Next, the central control unit 13 causes the position information generation unit 8g of the image processing unit 8 to generate an alpha map indicating the position of the extracted subject image in the subject existing image P2 (step S16).

Thereafter, the central control unit 13 causes the cut-out image generation unit 8h of the image processing unit 8 to perform processing for generating image data of the subject image obtained by combining the subject image with a predetermined single color image (step S17).
Specifically, the cut-out image generation unit 8h reads out the subject existing image P2, the single color image, and the alpha map and develops them in the image memory 5, and thereafter, for all the pixels of the subject existing image P2, pixels with an alpha value of 0. (Α = 0) for transmission, and for pixels with an alpha value of 0 <α <1, (0 <α <1), blending with a predetermined single color is performed, and for pixels with an alpha value of 1 (α = 1) Do nothing and do not transmit light to a predetermined single color.

  Thereafter, the central control unit 13 associates the alpha map generated by the position information generation unit 8g of the image processing unit 8 with the image data of the subject image in a predetermined storage area of the recording medium 9, and extracts the subject clipped image. The extension of the image data is saved as one file with “.jpe” (step S18).

As described above, according to the imaging apparatus 100 of the first embodiment, when imaging the subject presence image P2 in which the subject S is present in the background, the surrounding area is surrounded by pixels of a different color from the subject nonexistence image P1 in the subject presence image P2. When the same color area A2 having substantially the same color as the subject non-existing image P1 exists in the closed area A1, the fact that the same color area A2 exists in the closed area A1 in the subject existing image P2 is notified. It is possible to allow the user to grasp a portion of the same color area A2 in which clipping of the subject area from the subject existing image P2 may fail. Accordingly, when the user changes the background of the subject S, the subject S, the composition, or the like, the color tone of the same color area A2 of the subject existing image P2 captured by the subject clipping process is changed to the color tone of the subject non-existing image. Therefore, the same color area A2 cannot be cut out, and it is possible to appropriately prevent failure in cutting out the subject area from the subject existing image P2. As a result, the subject nonexistent image P1 and the subject present image P2 need not be taken again and again for the subject clipping process.
Specifically, when the subject presence image P2 in which the subject S exists within the same background as the background of the subject non-existence image P1, the same color region A2 of the closed region A1 in the subject presence image P2 displayed in live view is displayed. When the color tone of the same color area A2 is substantially equal to the color tone of the subject non-existing image P1, the color of the same color area A2 is changed to another color, or the changed color is lit (flashed) at a predetermined interval. Since the area A2 is notified to the user in an identifiable manner, the user can detect the portion of the same color area A2 in which clipping of the subject area from the subject existing image P2 may fail before the recording operation of the subject existing image P2 is performed. Can be visually recognized. Accordingly, it is possible to alert the user that the proper clipping of the subject area may fail if the state is unchanged, and to change the background of the subject S, the subject S, the composition, and the like.
When the same color area A2 exists in the closed area A1 in the subject existing image P2, the imaging of the subject existing image P2 by the electronic imaging unit 2 is stopped, so that the subject existing image that may fail to crop the subject area Can be prevented.

In the first embodiment, the subject non-existence image P1 is captured first, and then the subject presence image P2 is captured. However, the imaging order is an example, and the present invention is not limited to this. However, the order can be arbitrarily changed so that the order is reversed. That is, when the subject presence image P2 is captured first, it is determined whether or not the region in the closed region A1 and the subject non-existence image P1 have substantially the same color.
Moreover, although it was set as the structure which images both the to-be-photographed object nonexistence image P1 and the to-be-photographed object P2, it is an example and it is not restricted to this, It can change arbitrarily arbitrarily. For example, the subject non-existence image P1 may be an image captured by an imaging device other than the imaging device 100 and acquired via the recording medium 11a or a predetermined communication line.

[Embodiment 2]
Hereinafter, an imaging apparatus 200 according to the second embodiment to which the present invention is applied will be described with reference to FIGS.
The imaging apparatus 200 according to the second embodiment captures the subject presence image P2 in which the subject S is present in a predetermined background at a predetermined timing, and whether or not the non-flatness of the background portion of the subject presence image P2 is equal to or less than a predetermined value. Determine whether. Then, the imaging apparatus 200 includes the background portion in the closed region A1 that is substantially surrounded by pixels of a different color from the background portion in which the non-flatness is determined to be equal to or less than the predetermined value in the subject presence image P2. When it is determined that the same color area A2 having substantially the same color exists, the fact that the same color area A2 exists is notified.
Note that the imaging apparatus 200 according to the second embodiment has substantially the same configuration as the imaging apparatus 100 according to the first embodiment except for the points described below, and the description thereof is omitted.

FIG. 6 is a block diagram illustrating a schematic configuration of the imaging apparatus 200 according to the second embodiment.
As shown in FIG. 6, the image processing unit 8 of the imaging apparatus 200 according to the second embodiment includes an image acquisition unit 8a, a division unit 8b, a color calculation unit 8c, an image region determination unit 8d, and a cut-out image generation unit 8h. Are provided with a non-flatness calculation unit 8i, a non-flatness determination unit 8j, a background generation unit 8k, and a mask generation unit 8l.

なお、上記式（１）において、ｂ：各ブロック領域の画素値としては、例えば、輝度値が挙げられる。 The non-flatness calculator 8i calculates the non-flatness of the subject existing image P2 and the background portion of the subject existing image P2.
That is, the non-flatness calculation unit 8i calculates the variation amount (non-flatness) of the pixel values in the plurality of image blocks B of the subject existing image P2 divided by the dividing unit 8b of the image processing unit 8. Specifically, the non-flatness calculation unit 8i calculates a standard deviation according to the following formula (1) as a variation amount for each of the plurality of image blocks B,. .

In the above formula (1), b: a pixel value of each block area includes, for example, a luminance value.

The non-flatness determination unit 8j determines the non-flatness of each image block B calculated by the non-flatness calculation unit 8i.
That is, the non-flatness determination unit 8j determines whether or not the variation amount of the pixel value in the block is equal to or less than a predetermined value in the background generation process. Specifically, for example, the non-flatness determination unit 8j performs intra-block determination on the image block B including the subject S among the plurality of image blocks B of the subject existing image P2 (see FIG. 4C). The pixel value variation amount is large, and the pixel value variation amount is not determined to be equal to or less than a predetermined value. On the other hand, the non-flatness determination unit 8j is an image block B that does not include the subject S among the plurality of image blocks B of the subject existing image P2 that is imaged against a plain background with a small feature amount, that is, The plain background portion having a small feature amount has a pixel value variation amount in the block that is significantly smaller than that of the image block B including the subject S, and the pixel value variation amount is determined to be equal to or less than a predetermined value.
Then, the image processing unit 8 identifies an image block B whose non-flatness is a predetermined value or less based on the determination result by the non-flatness determining unit 8j. That is, the image processing unit 8 includes an image region including at least one image block B in which the non-flatness of the pixel values in the block is determined to be equal to or less than a predetermined value as a result of the determination by the non-flatness determining unit 8j. Is identified.

The background generation unit 8k generates an extraction background image (not shown) for extracting a subject area using the chroma key technique.
That is, the background generation unit 8k sets the background color to a color substantially equal to the color of the image block B of the subject existing image P2 for which the non-flatness of the pixel value is determined to be equal to or less than the predetermined value by the non-flatness determination unit 8j. A background image for extraction is generated. For example, when the subject S is photographed against a plain background with a small amount of feature, such as a business desk or a whiteboard, the background generation unit 8k generates an extraction background image having substantially the same color as the plain background. Generate.
Note that the chroma key is a method of cutting out the subject S from one image data using a specific color background. In the chroma key, a blue or green screen having a complementary color relationship with the subject S is usually used as the background, and the amount of variation in pixel values in the subject existing image P2 is almost equal to the image block B having a predetermined value or less. By generating the color extraction background image, the background portion and the subject portion can be separated based on the color information of the extraction background image and the subject presence image P2.

The image area determination unit 8d determines whether the same color area A2 having substantially the same color as that of the extraction background image exists in the closed area A1 that is substantially surrounded by pixels of a color different from that of the extraction background image in the subject existing image P2. Determine whether or not.
That is, the image area determination unit 8d performs subject extraction based on the representative colors (for example, average colors) of the plurality of image blocks B of the subject existing image P2 divided by the division unit 8b in the subject clipping process. It is determined whether or not the same color area A2 exists in the closed area A1 of the existing image P2. Specifically, the image area determination unit 8d compares and determines the average color of each of the plurality of image blocks B of the subject existing image P2 and the color of the extraction background image generated by the background generation unit 8k. Thus, it is determined whether or not there is a closed area A1 surrounded by pixels of a color different from the color of the extraction background image in the subject existing image P2. When the closed region A1 is specified, the image region determination unit 8d compares and determines the average color of each image block B in the closed region A1 of the subject existing image P2 and the color of the extraction background image. It is determined whether or not there is a same color area A2 composed of pixels of approximately the same color as the color of the background image for extraction in the closed area A1.

It should be noted that the color determination between the extraction background image and the subject presence image P2 may be performed strictly, or the difference between the colors of the extraction background image and the subject presence image P2 within a predetermined range is an error. It may be within the range. The error range may be a predetermined fixed range set in advance, manually based on a predetermined operation of the operation input unit 12 by the user, or the image area determination unit 8d and the central control unit 13. Etc. may be automatically and arbitrarily changed as appropriate.
Here, the image area determination unit 8d is a closed area (first area) in which the subject existing image P2 captured by the electronic imaging unit 2 and the imaging control unit 3 is substantially surrounded by pixels of a color different from the extraction background image. ) First area determination means for determining whether or not A1 exists, and a second area for determining whether or not the same color area (second area) A2 having the same color as the extraction background image exists in the closed area A1 The determination means is configured.

なお、上記式（２）にあっては、抽出用背景画像のＹＵＶデータを「Y」、「U」、「V」で表し、被写体存在画像Ｐ２のＹＵＶデータを「Yc」、「Uc」、「Vc」で表す。また、Ｇは、色差信号U、Vのゲインを表している。
そして、マスク生成部８ｌは、生成した相違度マップを所定の閾値で二値化（０、２５５）してマスク画像を生成する。 The mask generation unit 8l generates a mask image for extracting a subject area from the subject presence image P2.
That is, the mask generation unit 81 generates a difference map by calculating the difference D between corresponding pixels of the extraction background image and the subject presence image P2 according to the following equation (2).

In the above equation (2), the YUV data of the extraction background image is represented by “Y”, “U”, “V”, and the YUV data of the subject existing image P2 is “Yc”, “Uc”, Represented by “Vc”. G represents the gain of the color difference signals U and V.
Then, the mask generation unit 8l generates a mask image by binarizing (0, 255) the generated difference map with a predetermined threshold.

  Further, the mask generation unit 8l performs a contraction process for removing fine noise and removes a pixel set smaller than a predetermined value, and then performs a expansion process for correcting the contraction, and then the same connected component is obtained. By performing a labeling process for assigning the same numbers to the pixel sets to be configured, a hole area is also filled by replacing an area having a predetermined ratio or less in the number of constituent pixels of the effective area with the effective area. Further, the mask generation unit 8l applies an averaging filter to the region information to add a composite gradation to the edge of the region.

The cutout image generation unit 8h combines the image of the subject S with a predetermined single color background image (not shown) to generate image data of the subject cutout image (not shown).
That is, the cut-out image generation unit 8h uses the chroma key technique to cut out the subject image from the subject presence image P2 using the mask image generated by the mask generation unit 8l and combines it with the single color background image to cut out the subject. Generate image data of an image.
It should be noted that since the synthetic gradation is given to the edge portion of the mask image, the mask image can be synthesized with a natural feeling that the boundary portion between the clipped subject image and the single color background image is not clear.
Here, the cut-out image generation unit 8h constitutes an image generation unit that cuts out a subject area including the subject S from the subject presence image P2 to generate a subject image.

Next, the subject clipping process performed by the imaging apparatus 200 will be described with reference to FIGS.
FIG. 7 is a flowchart illustrating an example of an operation related to the subject clipping process.

As shown in FIG. 7, first, based on a predetermined operation of the shutter button 12a of the operation input unit 12 by the user, the imaging control unit 3 sets the focus lens focus position and exposure conditions (shutter speed, aperture, amplification factor, etc.). ) And white balance and the like, and an optical image of a subject existing image P2 (for example, see FIG. 4C) in which a predetermined subject (for example, the subject S) exists in the background is adjusted under the predetermined imaging condition. An image is picked up by the electronic image pickup unit 2 (step S51).
Then, after generating the YUV data of the subject existing image P2 transferred from the electronic imaging unit 2, the image data generating unit 4 temporarily stores the YUV data of the subject existing image P2 in the image memory 5.

  Next, the central control unit 13 causes the image processing unit 8 to perform extraction background generation processing for generating an extraction background image (not shown) for extracting a subject area from the subject existing image P2 (step S52). .

Here, the extraction background generation processing will be described in detail with reference to FIGS.
8 and 9 are flowcharts showing an example of operations related to the extraction background generation processing.
As shown in FIG. 8, first, the central control unit 13 sets the processing block counter SBC and the effective block counter YBC with initial values 0 (step S61) and writes them in the image memory 5.
Then, the dividing unit 8b obtains the YUV data of the subject existing image P2 temporarily stored in the image memory 5, and converts the subject existing image P2 into a plurality of image blocks B (see FIG. 5A). ). Then, the dividing unit 8b sets the total number of the plurality of divided image blocks B,... As a constant Bmax (step S62) and writes it in the image memory 5.

Subsequently, the non-flatness calculation unit 8i, among the plurality of image blocks B of the subject existing image P2,..., One image in the image block B that has not yet calculated the standard deviation as the pixel value variation amount. For block B, the standard deviation as the variation amount of the pixel value is calculated according to the following equation (1) (step S63).

Next, the non-flatness determination unit 8j determines whether or not the calculated variation amount of the image block B is equal to or less than a predetermined value (step S64).
Here, if it is determined that the variation amount is equal to or less than the predetermined value (step S64; YES), the color calculation unit 8c determines each of the image blocks B for which the variation amount is determined to be equal to or less than the predetermined value in step S64. An average color representing the image block B is calculated (step S65). The central control unit 13 adds 1 to the effective block counter YBC (YBC = YBC + 1) (step S66).
The central control unit 13 then adds 1 to the processing block counter SBC (SBC = SBC + 1) (step S67).

  On the other hand, when it is determined in step S64 that the variation amount is not equal to or less than the predetermined value (step S64; NO), the central control unit 13 does not perform the processes of steps S65 and S66, and shifts the process to step S67. Then, 1 is added to the processing block counter SBC (SBC = SBC + 1).

After the processing in step S67, the central control unit 13 determines whether or not the value of the processing block counter SBC is equal to the value of the total number Bmax of the image blocks B (step S68).
Here, if it is determined that the value of the processing block counter SBC is not equal to the value of the total number Bmax of the image blocks B (step S68; NO), the central control unit 13 returns the processing to step S63, and the subsequent processing I do. That is, the non-flatness calculation unit 8i calculates the standard deviation as the variation amount of the pixel value for all the image blocks B, and the non-flatness determination unit 8j determines whether the variation amount of the pixel value is equal to or less than a predetermined value. Determine whether. Then, the central control unit 13 counts the number of image blocks B for which the variation amount is determined to be less than or equal to a predetermined value based on the value of the effective block counter YBC.

In step S68, it is determined that the value of the processing block counter SBC is equal to the value of the total number Bmax of the image blocks B (step S68; YES) means that the standard deviation as the variation amount of the pixel values for all the pixel blocks. And the determination of whether or not the amount of variation is equal to or less than a predetermined value is completed.
When it is determined that the value of the processing block counter SBC is equal to the value of the total number Bmax of the image blocks B (step S68; YES), as shown in FIG. 9, the image processing unit 8 sets the value of the effective block counter YBC. It is determined whether or not is equal to or greater than the required number of blocks (step S69).

If it is determined in step S69 that the value of the effective block counter YBC is equal to or greater than the required number of blocks (step S69; YES), the image processing unit 8 determines that the variation amount is equal to or less than a predetermined value. Of B, the number of blocks of a color that can be regarded as the same color is calculated as the same number of blocks (step S70).
Then, the image processing unit 8 specifies the maximum number of identical blocks that takes the maximum value among the same number of blocks, and determines whether or not the maximum number of identical blocks is equal to or greater than the required number of blocks (step S71).

If it is determined in step S71 that the number of the most identical blocks is equal to or greater than the required number of blocks (step S71; YES), the background generation unit 8k uses the average color of the image block B counted as the number of the most identical blocks as the background. YUV data of an extraction background image to be a color is generated (step S72).
Next, the image region determination unit 8d of the image processing unit 8 includes a closed region A1 surrounded by pixels of a color different from the color of the extraction background image generated by the background generation unit 8k in the subject non-existing image P1. It is determined whether or not it exists (step S73). Specifically, the image area determination unit 8d compares the average color of each of the plurality of image blocks B,... Of the subject existing image P2 with the color of the extraction background image, thereby extracting the subject existing image P2. A closed region A1 surrounded by pixels of a color different from the color of the background image for use is specified. Then, the image area determination unit 8d determines whether or not the closed area A1 exists in the subject existing image P2 according to the identification result.

If it is determined in step S73 that the closed region A1 exists in the subject existing image P2 (step S73; YES), the image region determination unit 8d is substantially equal to the color of the extraction background image in the closed region A1. It is determined whether or not the same color area A2 composed of color pixels exists (step S74). Specifically, the image area determination unit 8d extracts the image in the closed area A1 by comparing and determining the average color of each image block B in the closed area A1 of the subject existing image P2 and the color of the background image for extraction. It is determined whether or not there is a same color area A2 made up of pixels having substantially the same color as the color of the background image for use.
If it is determined that the same color area A2 exists in the closed area A1 (step S74; YES), the display control unit 9 closes the closed area A1 of the subject existing image P2 displayed on the display screen of the display unit 10. By changing the color of the same color area A2 to another color and displaying it, the fact that the same color area A2 exists is notified (step S75; see FIG. 5B), and the generation of the extraction background image is performed. Failure occurs (step S76).
Thus, the extraction background generation process is terminated.

If it is determined in step S73 that the closed region A1 does not exist in the subject presence image P2 (step S73; NO), for example, a subject whose color is not suitable for clipping with respect to the background color. S, and it is necessary to image the subject S against a background of a different color, so that the generation of the extraction background image fails (step S76).
If it is determined in step S71 that the maximum number of identical blocks is not equal to or greater than the required number of blocks (step S71; NO), for example, even if there is a stable image block B with no variation, blocks of substantially the same color are present. When the number of necessary blocks is not exceeded, for example, the background cannot be specified, and therefore generation of the extraction background image fails (step S76).
If it is determined in step S69 that the value of the effective block counter YBC is not equal to or greater than the required number of blocks (step S69; NO), the background cannot be specified because there is no stable image block B without variation. The generation of the background image for extraction fails (step S76).

  On the other hand, if it is determined in step S74 that the same color area A2 does not exist within the closed area A1 of the subject existing image P2 (step S74; NO), the extraction background image is successfully generated (step S77). The background generation process is terminated.

As shown in FIG. 7, next, the central control unit 13 causes the image processing unit 8 to determine whether or not the generation of the extraction background image has succeeded (step S53).
Here, if it is determined that the extraction background generation has not succeeded (step S53; NO), the central control unit 13 determines a predetermined message (for example, “the subject has failed to be clipped”) regarding the cropping failure of the subject S. ”And the like” are displayed on the display screen of the display unit 10 (step S54), and the subject clipping process is terminated.

  On the other hand, if it is determined in step S53 that the extraction background generation has succeeded (step S53; YES), the central control unit 13 causes the image processing unit 8 to select a subject area including the subject S from the subject existing image P2. An area detection process to be detected is performed (step S55).

Here, the region detection processing will be described in detail with reference to FIG.
FIG. 10 is a flowchart illustrating an example of an operation related to the area detection process.
As shown in FIG. 10, the mask generation unit 8l of the image processing unit 8 calculates the difference D of each corresponding pixel between the YUV data of the extraction background image and the YUV data of the subject existing image P2 by the following equation (2). ) To generate a dissimilarity map (step S81).

Next, the mask generation unit 8l binarizes the generated dissimilarity map with a predetermined threshold value to generate mask image data (step S82). Then, the mask generation unit 8l determines whether or not the binarization is successful by determining whether or not the background (binarization; 0) region is extremely small, for example (step S83).
Here, if it is determined that the binarization is successful (step S83; YES), the mask generation unit 8l corrects the remaining part of the periphery of the subject region and removes fine noise, so that the mask image data is removed. After performing a contraction process to remove a pixel set smaller than a predetermined value (step S84), an expansion process for correcting the contraction is performed (step S85).

Subsequently, when the subject S has a color similar to the background color, the subject region of the mask image is lost, and thus the mask generation unit 8l performs a labeling process for assigning the same number to the pixel sets constituting the same connected component. Thus, a hole is filled by replacing an area below a predetermined ratio in the number of constituent pixels of the effective area of the mask image data with the effective area (step S86).
After that, the mask generation unit 8l applies an averaging filter to the mask image data, adds a synthetic gradation to the edge of the subject area (step S87), and sets the area detection as successful (step S88).

On the other hand, if it is determined in step S83 that the binarization is not successful (step S83; NO), for example, if the background (binarization; 0) region is extremely small, the mask generation unit 8l. Is assumed to have failed in binarization and is determined to be area detection failure (step S89).
Thereby, the region detection process is terminated.

As shown in FIG. 7, next, the central control unit 13 causes the mask generation unit 8l of the image processing unit 8 to determine whether or not the detection of the subject region has succeeded (step S56).
Here, if it is determined that the detection of the subject area has succeeded (step S56; YES), the central control unit 13 uses the chroma key technique to perform the area detection process in the cut-out image generation unit 8h of the image processing unit 8. A subject region is cut out from the subject presence image P2 using the mask image generated in step S2, and is combined with a predetermined single-color background image (not shown) to generate image data of the subject cut-out image (not shown) (step) S57).
Specifically, the cut-out image generation unit 8h reads the subject existing image P2, the single color background image, and the mask image data, develops them in the image memory 5, and then fills the mask image data in the subject existing image P2. The pixels covered by (the part other than the subject S) are filled with a predetermined single color of the single color background image, while the pixels of the subject part are not transmitted to the predetermined single color without doing anything. To do. Note that since the synthetic gradation is given to the edge portion of the mask image, the boundary portion between the clipped subject area and the single-color background image has a natural feeling.
Thereafter, the display control unit 9 displays a subject cut-out image in which the subject S is superimposed on a predetermined single-color background image based on the image data of the subject cut-out image generated by the cut-out image generation unit 8h. To display.

Next, the recording medium control unit 11 associates the mask image data generated by the mask generation unit 8l of the image processing unit 8 with the image data of the subject image in a predetermined storage area of the recording medium 11a, and cuts out the subject The extension of the image data of the image is saved as one file with “.jpe” (step S58).
Thereby, the subject clipping process is completed.

  On the other hand, if it is determined in step S56 that the detection of the subject area has not been successful (step S56; NO), the central control unit 13 shifts the processing to step S54, and in the same manner as described above, A predetermined message relating to the clipping failure is displayed on the display screen of the display unit 10 (step S54), and the subject clipping process is terminated.

  As described above, according to the imaging apparatus 200 of the second embodiment, it is needless to say that the same effect as that of the imaging apparatus 100 described in the first embodiment can be obtained, and in particular, the presence of the subject with the subject S in the background. The same-color region of the same color as the background portion in the closed region A1 that is captured by the image P2 and surrounded by pixels of a different color from the background portion whose non-flatness is a predetermined value or less in the subject presence image P2. When A2 exists, the fact that the same color area A2 exists in the closed area A1 in the subject existence image P2 is notified, so that the portion of the same color area A2 where the clipping of the subject area from the subject existence image P2 may fail may be detected. The user can be made aware. Accordingly, when the user changes the background of the subject S, the subject S, the composition, or the like, the color tone of the same color area A2 of the captured subject existing image P2 is almost the same as the color tone of the background portion in the subject clipping process. Since they are equal, the same color area A2 cannot be cut out, and it is possible to appropriately prevent a subject area from being cut out of the subject existing image P2. As a result, the subject existence image P2 must not be taken again and again for the subject clipping process.

The present invention is not limited to the above-described embodiment, and various improvements and design changes may be made without departing from the spirit of the present invention.
For example, in the first and second embodiments, based on the color of the subject presence image P2 (including images displayed in live view) and the subject non-existence image P1 (for example, the color of each image block B), Although the closed area A1 and the same color area A2 are specified, the method for specifying these areas is merely an example, and is not limited thereto. For example, the contour of the subject image of the subject existing image P2 is detected by edge detection. You may make it do.

  Furthermore, in the first and second embodiments, the number of subject presence images P2 that are imaged and recorded based on one imaging instruction operation of the shutter button 12a of the operation input unit 12 by the user is one. Alternatively, a plurality of sheets may be used. That is, in the case where the subject existence image P2 is continuously captured as a plurality of still images or as a moving image, the subject image is generated by cutting out the area where the subject S exists for each subject presence image P2. May be.

  In the first embodiment, the alpha map and the image data of the subject image are associated with each other. In the second embodiment, the mask image data and the image data of the subject image are associated with each other. Although it is stored as one file, the alpha map or mask image data and the image data of the subject existing image P2 may be associated with each other and stored in the recording medium (recording means) 11a as one file. In this case, for reproduction of the file, a mode for reproducing the subject existing image P2 and two modes for combining and displaying the subject image (subject cutout image) by applying an alpha map or mask image data at the time of reproduction are prepared. It is good to keep.

  Furthermore, the methods exemplified in the first and second embodiments as a method for cutting out the subject image from the subject non-existing image P1 are examples, and the present invention is not limited to these. For example, in the first embodiment, mask image data may be generated and used instead of an alpha map for clipping a subject image from the subject non-existing image P1, and in the second embodiment, An alpha map may be generated and used instead of the mask image data.

  In the second embodiment, the imaging control unit 3 includes the imaging stop unit 3a. However, whether or not the imaging stop unit 3a is provided can be arbitrarily changed as appropriate.

  In the second embodiment, it is determined whether or not the closed area A1 and the same color area A2 exist for the captured subject existing image P2. However, before the subject existing image P2 is captured, the sequential determination is performed. You may make it determine with the live view image imaged. That is, the average color of the most identical block is obtained for the live view image, and it is determined whether or not there is a closed region A1 having a color different from the average color. If the closed region A1 exists, the average color of the most identical block is determined. It may be determined whether or not the same color area A2 that is an area of substantially the same color exists, and if the same color area A2 exists, the fact may be notified.

  Furthermore, the configurations of the imaging apparatuses 100 and 200 are merely examples as illustrated in the above-described embodiment, and are not limited thereto.

In addition, in the first and second embodiments, the functions of the acquisition unit, the non-flatness determination unit, the first region determination unit, the second region determination unit, and the notification unit function as the image acquisition unit of the image processing unit 8. 8a, the non-flatness determination unit 8j, the image region determination unit 8d, and the display control unit 9 are driven. However, the present invention is not limited to this, and a predetermined program is executed by the CPU of the central control unit 13. It is good also as a structure implement | achieved by performing etc.
That is, a program memory (not shown) that stores the program of the imaging apparatus 100 stores a program including an acquisition processing routine, an imaging control processing routine, an area determination processing routine, and a notification control processing routine. Then, the CPU of the central control unit 13 may function as an acquisition unit that acquires a background image stored in the image memory 5 by an acquisition process routine. Further, the CPU of the central control unit 13 may function as an imaging control unit that captures a subject existing image in which a subject exists in the background by an imaging control processing routine. In addition, the CPU of the central control unit 13 determines whether or not the image captured by the image capturing unit includes a first region that is substantially surrounded by pixels of a color different from the background image acquired by the acquiring unit. The first area determination means for determining whether or not and the second area determination means for determining whether or not the second area having a color substantially equal to the background image exists in the first area may be used. . Further, when the notification control processing routine determines that the CPU of the central control unit 13 has the second area by the second area determination means, the notification control means notifies the fact that the second area exists from the notification means. You may make it function.
Further, a program memory (not shown) that stores the program of the imaging apparatus 200 stores programs including an imaging control processing routine, a non-flatness determination processing routine, an area determination processing routine, and a notification control processing routine. Then, the CPU of the central control unit 13 may function as an imaging control unit that captures a subject existing image in which a subject is present in the background by an imaging control processing routine. Further, the CPU of the central control unit 13 functions as a non-flatness determination unit that determines whether or not the non-flatness of the background portion of the image captured by the imaging unit is equal to or less than a predetermined value by the non-flatness determination processing routine. You may make it let it. In addition, the CPU of the central control unit 13 causes the image of the image captured by the imaging unit to be different from the background portion in which the non-flatness is determined to be less than or equal to the predetermined value by the non-flatness determination unit. First region determining means for determining whether or not there is a first region substantially surrounded by the first region, and a first region for determining whether or not a second region having a color substantially equal to the background portion exists in the first region. You may make it function as a 2 area | region determination means. Further, when the notification control processing routine determines that the CPU of the central control unit 13 has the second area by the second area determination means, the notification control means notifies the fact that the second area exists from the notification means. You may make it function.

  Similarly, the image generation means may be realized by a predetermined program or the like being executed by the CPU of the central control unit 13.

  Furthermore, as a computer-readable medium storing a program for executing each of the above processes, a non-volatile memory such as a flash memory or a portable recording medium such as a CD-ROM is applied in addition to a ROM or a hard disk. Is also possible. A carrier wave is also used as a medium for providing program data via a predetermined communication line.

DESCRIPTION OF SYMBOLS 100 Imaging device 1 Lens part 2 Electronic imaging part 3 Imaging control part 8 Image processing part 8a Image acquisition part 8d Image area determination part 8h Cutout image generation part 8j Non-flatness determination part 9 Display control part 10 Display part 13 Central control part

Claims (12)

Acquisition means for acquiring a background image;
Imaging means for imaging a subject existing image in which a subject is present in the background;
First area determination means for determining whether or not the subject existing image includes a first area substantially surrounded by an area of a color different from the color of the background image;
When it is determined by the first area determination means that the first area exists, it is determined whether or not a second area having substantially the same color as the background image exists in the first area in the subject existing image. Second area determination means for performing,
A notification means for notifying that the second area exists when the second area determination means determines that the second area exists;
An imaging apparatus comprising:   The imaging apparatus according to claim 1, wherein the first area is a closed area surrounded by pixels of a different color from the background image.   The imaging apparatus according to claim 1, wherein a background of the subject existing image captured by the imaging unit is substantially the same as a corresponding region of the background image. Imaging means for imaging a subject existing image in which a subject is present in the background;
Non-flatness determining means for determining whether or not the non-flatness of the background portion of the subject existing image is equal to or less than a predetermined value;
If the non-flatness determination means determines that the non-flatness is less than or equal to a predetermined value, does the subject existing image have a first area substantially surrounded by an area of a color different from the color of the background image? First area determination means for determining whether or not,
If the first region determining means determines that the first region exists, whether or not a second region having substantially the same color as the background portion color exists in the first region in the subject existing image. Second region determining means for determining
A notification means for notifying that the second area exists when the second area determination means determines that the second area exists;
An imaging apparatus comprising:   The imaging apparatus according to claim 4, wherein the first area is a closed area surrounded by pixels having a different color from the background portion.   The imaging apparatus according to claim 1, wherein the subject presence image is sequentially captured by the imaging unit and displayed on a display unit.   The imaging apparatus according to claim 6, wherein the notification unit displays the second region in the subject existing image displayed on the display unit in a display form that can be identified.   The imaging apparatus according to claim 1, further comprising an image generation unit configured to generate a subject image by cutting out a region where the subject exists from the subject existing image. An imaging processing method using an imaging device including an imaging means for imaging a subject existing image in which a subject is present in a background,
An acquisition process to acquire a background image;
A first region determination process for determining whether or not the acquired subject presence image includes a first region substantially surrounded by a region of a color different from the color of the background image;
When it is determined by the first area determination processing that the first area exists, it is determined whether or not a second area having substantially the same color as the background image exists in the first area in the subject existing image. A second area determination process to perform,
A notification process for notifying that the second area exists when it is determined that the second area exists by the second area determination process;
An imaging processing method comprising: An imaging processing method using an imaging device including an imaging means for imaging a subject existing image in which a subject is present in a background,
Non-flatness determination processing for determining whether or not the non-flatness of the background portion of the subject existing image is a predetermined value or less;
If it is determined by this non-flatness determination processing that the non-flatness is less than or equal to a predetermined value, does the subject-existing image have a first area that is substantially surrounded by an area of a color different from the color of the background image? A first area determination process for determining whether or not;
If it is determined by the first region determination processing that the first region exists, whether or not a second region having substantially the same color as the color of the background portion exists in the first region in the subject existing image. A second region determination process for determining
A notification process for notifying that the second area exists when it is determined that the second area exists by the second area determination process;
An imaging processing method comprising: The computer of the imaging device,
Acquisition means for acquiring a background image;
Imaging control means for imaging a subject existing image in which a subject is present in the background;
First area determining means for determining whether or not the subject existing image includes a first area substantially surrounded by an area of a color different from the color of the background image;
When it is determined by the first area determination means that the first area exists, it is determined whether or not a second area having substantially the same color as the background image exists in the first area in the subject existing image. Second area determination means for
Notification control means for notifying that the second area exists when the second area determination means determines that the second area exists;
A program characterized by functioning as The computer of the imaging device,
Imaging control means for imaging a subject existing image in which a subject is present in the background;
Non-flatness determining means for determining whether or not the non-flatness of the background portion of the subject existing image is equal to or less than a predetermined value;
If the non-flatness determination means determines that the non-flatness is less than or equal to a predetermined value, does the subject existing image have a first area substantially surrounded by an area of a color different from the color of the background image? First area determination means for determining whether or not,
If the first region determining means determines that the first region exists, whether or not a second region having substantially the same color as the background portion color exists in the first region in the subject existing image. Second region determining means for determining
Notification control means for notifying that the second area exists when the second area determination means determines that the second area exists;
A program characterized by functioning as

Images