JP2011003057A - Image composition device, image specifying method, image composition method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image composition device which can simply generate a composite image, an image specifying method, an image composition method and a program.SOLUTION: An imaging apparatus 100 includes: a feature area detecting part 8b for detecting a feature area associated with a subject image of a subject extraction image from a background image; and a foreground specifying part 8d for specifying a subject extraction image including the subject image associated with the detected feature area.

Description

  The present invention relates to an image composition device, an image specifying method, an image composition method, and a program for composing a plurality of images.

  Conventionally, a technique for generating a composite image by combining a subject image with a background image or a frame image is known (see, for example, Patent Document 1).

JP 2004-159158 A

  By the way, when combining a subject image and a background image, it is necessary to specify an image and a position to be combined, and the work is complicated.

  Therefore, an object of the present invention is to provide an image composition device, an image specifying method, an image composition method, and a program that can easily generate a composite image.

In order to solve the above-described problem, an image composition device according to a first aspect of the present invention provides:
In an image synthesizing apparatus including a synthesizing unit that synthesizes a foreground image with a background image, a storage unit that stores a plurality of background images and foreground images, and one of a plurality of background images stored in the storage unit A background designating unit for designating one background image, a detecting unit for detecting a feature region from the background image designated by the background designating unit, and a foreground image associated with the feature region detected by the detecting unit. Specifying means for specifying from among a plurality of foreground images stored in the storage means.

According to a second aspect of the present invention, in the image composition device according to the first aspect,
The synthesizing unit synthesizes the foreground image specified by the specifying unit with an area of the background image in which the characteristic region is detected by the detecting unit.

According to a third aspect of the present invention, in the image composition device according to the first or second aspect,
The specifying means further includes a foreground specifying means for specifying a plurality of the foreground images and specifying any one of the foreground images specified by the specifying means, and the combining means includes the foreground image. The foreground image designated by the designation means is synthesized.

According to a fourth aspect of the present invention, in the image synthesizing device according to any one of the first to third aspects,
The feature region is a person image region in the background image, and the specifying unit specifies a foreground image including an image of the same person as the feature region.

The image synthesizing device of the invention according to claim 5 is:
Storage means for storing a plurality of background images and foreground images, foreground designating means for designating any one foreground image from the plurality of background images stored in the storage means, and stored in the storage means A background designating unit for designating any one of the plurality of background images, and a foreground image designated by the foreground designating unit from the background image designated by the background designating unit. A detecting unit for detecting a feature region; and a combining unit for combining the foreground image specified by the foreground specifying unit with the region of the background image where the feature region is detected by the detecting unit. Yes.

According to a sixth aspect of the present invention, in the image synthesizing apparatus according to the fifth aspect,
The foreground image is a person image including a face image of a person, and the feature region is a face image region of the same person as the person in the background image.

The invention described in claim 7 is the image composition apparatus according to claim 6,
The combining means combines the face image area of the foreground image and the face image area of the background image so as to overlap each other.

An image composition device according to an eighth aspect of the present invention provides:
In an image synthesizing apparatus including a synthesizing unit that synthesizes a foreground image with a background image, any one of a plurality of foreground images stored in the storage unit and a storage unit that stores a plurality of background images and foreground images Among the plurality of background images stored in the storage unit, a foreground designating unit for designating one foreground image and a background image including a feature region associated with the foreground image designated by the foreground designating unit And a specifying means for specifying from the above.

The invention according to claim 9 is the image composition apparatus according to any one of claims 1 to 8,
Further comprising changing means for changing the size of at least one of the foreground image and the background image based on the size of the foreground image synthesized by the synthesizing unit and the size of the feature region in the background image. Is characterized in that the changing means changes the size of at least one of the foreground image and the background image and then combines them.

The invention according to claim 10 is the image composition device according to any one of claims 1 to 9,
The image processing apparatus further includes reproduction means for switching and reproducing the background image and the synthesized image synthesized by the synthesis means.

The invention according to claim 11 is the image composition apparatus according to any one of claims 1 to 10,
The image processing apparatus further includes registration means for registering the foreground image and the feature region in association with each other.

The image specifying method of the invention according to claim 12 is:
An image specifying method for an image composition apparatus, comprising: storage means for storing a plurality of background images and foreground images; and composition means for synthesizing the background image and the foreground image, the image specifying method stored in the storage means. A background designating step for designating any one of the plurality of background images, a detecting step for detecting a feature region from the background image designated by the background designating step, and stored in the storage means And a specifying step of specifying a foreground image associated with the feature region detected by the detecting step from a plurality of foreground images.

The image composition method of the invention according to claim 13
An image composition method for an image composition apparatus comprising: storage means for storing a plurality of background images and foreground images; and composition means for synthesizing the background image and the foreground image, the image composition method being stored in the storage means A foreground designating step for designating any one of the plurality of background images, and a background designating step for designating any one of the background images stored in the storage means A detection step of detecting a feature region associated with the foreground image specified by the foreground image specification step from the background image specified by the background specification step; and a feature region by the detection means in the background image And a synthesizing step for synthesizing the foreground image designated by the foreground designating means in the detected area.

The image specifying method of the invention described in claim 14 is:
An image specifying method for an image composition apparatus, comprising: storage means for storing a plurality of background images and foreground images; and composition means for synthesizing the background image and the foreground image, the image specifying method stored in the storage means. A foreground designating step for designating any one of the plurality of foreground images, and a background image including a feature region associated with the foreground image designated by the foreground designating step. And a specific step of specifying from among a plurality of background images stored in.

The program of the invention according to claim 15 is:
A plurality of background images stored in the storage means are stored in the storage means. A background designating unit for designating any one of the background images, a detection unit for detecting a feature region associated with the foreground image from the background image designated by the background designating unit, and a detection unit detected by the detection unit The foreground image associated with the feature region is made to function as specifying means for specifying from among a plurality of foreground images stored in the storage means.

The program of the invention described in claim 16 is:
Foreground designating means for designating any one foreground image from among a plurality of background images stored in the storage means, a computer of an image composition device comprising storage means for storing a plurality of background images and foreground images; A background designating unit for designating any one of the plurality of background images stored in the storage unit, a foreground image designated by the foreground designating unit from a background image designated by the background designating unit; A detecting unit that detects the associated feature region; and a combining unit that combines the foreground image designated by the foreground designating unit with the region of the background image in which the feature region is detected by the detecting unit. It is characterized by.

The program of the invention described in claim 17
A plurality of foreground images stored in the storage unit are stored in the storage unit. The computer of the image composition apparatus includes a storage unit that stores a plurality of background images and foreground images, and a combination unit that combines the background image and the foreground image. A plurality of foreground designating means for designating any one of the foreground images and a background image including a feature region associated with the foreground image designated by the foreground designating means. It is characterized by functioning as a specifying means for specifying from among the background images.

  According to the present invention, it is possible to easily generate a composite image obtained by combining a background image and a subject image.

It is a block diagram which shows schematic structure of the imaging device of one Embodiment to which this invention is applied. 3 is a flowchart illustrating an example of an operation related to subject clipping processing by the imaging apparatus of FIG. 1. It is a figure which shows typically an example of the image which concerns on a subject clipping process. 3 is a flowchart illustrating an example of an operation related to background image imaging processing by the imaging apparatus of FIG. 1. 3 is a flowchart illustrating an example of an operation related to a composite image generation process by the imaging apparatus of FIG. 1. It is a flowchart which shows an example of the operation | movement which concerns on the image composition process in a composite image generation process. It is a figure which shows typically an example of the image which concerns on a synthetic image generation process. FIG. 10 is a diagram schematically illustrating an example of an image related to a subject clipping process performed by an imaging apparatus according to a first modification. It is a figure which shows typically an example of the image which concerns on the composite image generation process by the imaging device of FIG.

Hereinafter, specific embodiments of the present invention will be described with reference to the drawings. However, the scope of the invention is not limited to the illustrated examples.
FIG. 1 is a block diagram illustrating a schematic configuration of an imaging apparatus 100 according to an embodiment to which the present invention is applied.

The imaging apparatus 100 according to the present embodiment detects a feature region C associated with the subject image (foreground image) Ga of the subject cutout image P3a from the background image P4a, and the feature region C in the background image P4a is detected. A subject image Ga associated with the feature region C is combined with the region.
Specifically, as illustrated in FIG. 1, the imaging apparatus 100 includes a lens unit 1, an electronic imaging unit 2, an imaging control unit 3, an image data generation unit 4, an image memory 5, and a feature amount calculation unit. 6, a block matching unit 7, an image processing unit 8, a recording medium 9, a display control unit 10, a display unit 11, an operation input unit 12, and a central control unit 13.
Further, the imaging control unit 3, the feature amount calculation unit 6, the block matching unit 7, the image processing unit 8, and the central control unit 13 are designed as a custom LSI 1A, for example.

The lens unit 1 includes a plurality of lenses and includes a zoom lens, a focus lens, and the like.
Although not shown, the lens unit 1 includes a zoom drive unit that moves the zoom lens in the optical axis direction and a focus drive unit that moves the focus lens in the optical axis direction when imaging a subject. May be.

  The electronic imaging unit 2 is composed of an image sensor such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal-oxide Semiconductor), for example, and converts an optical image that has passed through various lenses of the lens unit 1 into a two-dimensional image signal. To do.

Although not shown, the imaging control unit 3 includes a timing generator, a driver, and the like. Then, the imaging control unit 3 scans and drives the electronic imaging unit 2 with a timing generator and a driver, converts the optical image into a two-dimensional image signal with the electronic imaging unit 2 every predetermined period, and the electronic imaging unit 2 Image frames are read out from the imaging area for each screen and output to the image data generation unit 4.
The imaging control unit 3 performs AF (automatic focusing processing), AE (automatic exposure processing), AWB (automatic white balance), and the like as adjustment control of the imaging conditions of the subject.

The lens unit 1, the electronic imaging unit 2, and the imaging control unit 3 configured as described above function as an imaging unit as a background image P4a (see FIG. 7A) related to image synthesis processing and a subject presence image P1a (see FIG. a), etc.).
In addition, the lens unit 1, the electronic imaging unit 2, and the imaging control unit 3 capture the subject clipped image P3a (FIG. 3) in a state where the imaging conditions for capturing the subject presence image P1a are fixed after the subject presence image P1a is captured. The non-existing subject image P2a (see FIG. 3B) for generating (see (c)) is captured.

The image data generation unit 4 appropriately adjusts the gain for each RGB color component with respect to the analog value signal of the image frame transferred from the electronic imaging unit 2, and then performs sample holding by a sample hold circuit (not shown). The digital signal is converted into digital data by a / D converter (not shown), color processing including pixel interpolation processing and γ correction processing is performed by a color process circuit (not shown), and then a digital luminance signal Y and color difference signal Cb , Cr (YUV data).
The luminance signal Y and the color difference signals Cb and Cr output from the color process circuit are DMA-transferred to an image memory 5 used as a buffer memory via a DMA controller (not shown).

  The image memory 5 is configured by, for example, a DRAM or the like, and temporarily stores data processed by the feature amount calculation unit 6, the block matching unit 7, the image processing unit 8, the central control unit 13, and the like.

The feature amount calculation unit 6 performs feature extraction processing for extracting feature points from the subject non-existing image P2a with reference to the subject non-existing image P2a. Specifically, the feature amount calculation unit 6 selects, for example, a predetermined number (or a predetermined number or more) of high feature block areas (feature points) based on the YUV data of the subject non-existing image P2a, The contents of the block are extracted as a template (for example, a square of 16 × 16 pixels).
Here, the feature extraction process is a process of selecting a feature having a high characteristic convenient for tracking from a large number of candidate blocks.

  When the subject cutout image P3a is generated, the block matching unit 7 performs block matching processing for aligning the subject nonexistent image P2a and the subject present image P1a. Specifically, the block matching unit 7 determines where the template extracted in the feature extraction process corresponds in the subject existing image P1a, that is, the pixel value of the template optimally matches in the subject existing image P1a. Search for a position (corresponding region). Then, the optimum offset between the subject non-existing image P2a and the subject existing image P1a having the best evaluation value (for example, the sum of squared differences (SSD), the sum of absolute differences (SAD), etc.) Calculated as a template motion vector.

The image processing unit 8 includes a cutout image generation unit 8a that generates image data of the subject cutout image P3a. Specifically, the cut-out image generation unit 8a includes a positioning unit, a subject image extraction unit, a position information generation unit, an image generation unit, and the like, although not illustrated.
As described in FIGS. 3A to 3C, the alignment unit performs coordinate conversion of each pixel of the subject presence image P1a with respect to the subject nonexistence image P2a based on the feature points extracted from the subject nonexistence image P2a. An equation (projection transformation matrix) is calculated, and the subject presence image P1a is coordinate-transformed according to the coordinate transformation equation to align with the subject non-existence image P2a.
The subject image extraction unit generates difference information of corresponding pixels between the subject presence image P1a and the subject non-existence image P2a aligned by the alignment unit, and uses the difference information as a reference from the subject presence image P1a. A subject image (foreground image) Ga including the subject is extracted.
The position information generation unit specifies the position of the subject image Ga extracted from the subject presence image P1a, and generates position information (for example, an alpha map) indicating the position of the subject image Ga in the subject presence image P1a. Here, the alpha map represents the weight when alpha blending the subject image Ga with respect to a predetermined background for each pixel of the subject existing image P1a as an alpha value (0 ≦ α ≦ 1).
Based on the generated alpha map, the image generation unit does not transmit a pixel having an alpha value of 1 among the pixels of the subject existing image P1a to a predetermined single color image (not shown), and The image of the subject is combined with a predetermined single color image so as to transmit pixels with an alpha value of 0, and image data of the subject cutout image P3a is generated.

In addition, the image processing unit 8 includes a feature region detection unit 8b that detects a feature region C in the background image P4a.
The feature region detection unit 8b is detected by detecting a human face using a predetermined face detection method from the background image P4a in FIG. 7A designated based on a predetermined operation of the operation input unit 12 by the user. A face recognition unit 8c for recognizing a person with a different face is provided.
The face recognition unit 8c detects a face image area from the background image P4a, and detects face parts (feature parts) corresponding to eyes, nose, mouth, and the like from within each detected area. Since this face detection process is a known technique, a detailed description thereof is omitted here. Then, the face recognition unit 8c determines whether or not face image information (specific information) corresponding to the face detected in the face detection process is registered in a registration table T1 of the table storage unit 8g described later, If it is determined that the face image information is registered as a result of the determination, a person registered in association with the face image information is specified.
The feature region detection unit 8b detects the person's face image region specified by the face recognition unit 8c in the background image P4a described above as the feature region C, and then extracts subject clipped image information (subject image Ga) corresponding to the feature region C. ) Is registered in the registration table T1. When it is determined that the corresponding subject cut-out image information (subject image Ga) is registered in the registration table T1, the feature region detection unit 8b uses the feature region C for subsequent processing, while If it is determined that the clipped image information (subject image Ga) is not registered in the registration table T1, the detected feature area C is discarded.
Here, the feature region detection unit 8b constitutes a detection unit that detects the feature region C from the background image P4a designated based on a predetermined operation of the operation input unit 12 by the user.

In addition, the image processing unit 8 cuts out a subject including a subject image Ga that is associated with the feature region C detected by the feature region detection unit 8b from the plurality of subject cut-out images P3a stored in the recording medium 9. A foreground specifying unit 8d for specifying the image P3a is provided.
When the feature region C is detected by the feature region detection unit 8b, the foreground specifying unit 8d refers to the registration table T1 and includes a subject clipping image that includes the subject image Ga associated with the detected feature region C. At least one P3a is specified. Then, the foreground specifying unit 8d reads out the image data of the specified subject cutout image P3a from the recording medium 9 and temporarily stores it in the image memory 5. Thereafter, the display control unit 10 reads out the image data of the subject cutout image P3a temporarily stored in the image memory 5 and causes the display unit 11 to display the image data.
Here, the foreground specifying unit 8d is a subject image Ga associated with the feature region C detected by the feature region detection unit 8b from among a plurality of subject images (foreground images) Ga stored in the recording medium 9. A specifying means for specifying is configured.

Further, the image processing unit 8 includes a combining unit 8e that combines the background image P4a and the subject cutout image P3a as a combining unit.
The synthesizing unit 8e synthesizes the subject image (foreground image) Ga associated with the feature region C in the region where the feature region C is detected by the feature region detection unit 8b in the background image P4a. Specifically, when the face image area of the person as the feature area C is detected from the background image P4a by the feature area detection section 8b, the synthesis section 8e detects the subject of the subject cutout image P3a specified by the foreground specification section 8d. The synthesis position of the image Ga is designated as a feature area (face image area) C of the background image P4a, and the face image area of the subject image Ga and the face image area of the background image P4a are synthesized.
At this time, when a plurality of subject cutout images P3a are specified by the foreground specifying unit 8d, the combining unit 8e selects any one of the subject cutout images selected based on a predetermined operation of the operation input unit 12 by the user. The subject image Ga included in P3a is combined with the region in which the feature region C is detected in the background image P4a.
Here, the synthesizing unit 8e synthesizes the subject image (foreground image) Ga associated with the feature region C in the region where the feature region C is detected by the feature region detection unit 8b in the background image P4a. Is configured.

  The synthesizing unit 8e transmits a pixel with an alpha value of 0 among each pixel of the background image P4a, overwrites a pixel with an alpha value of 1 with the pixel value of the corresponding pixel of the subject cutout image P3a, and Among the pixels of the background image P4a, pixels with an alpha value of 0 <α <1 generate an image (background image × (1-α)) obtained by clipping the subject image Ga using a complement of 1 (1-α). Then, when the subject cutout image P3a is generated using the one's complement (1-α) in the alpha map, a value blended with the single background color is calculated, and the value is subtracted from the subject cutout image P3a, Are combined with an image (background image × (1-α)) obtained by cutting out the subject image Ga.

In addition, the image processing unit 8 includes a size changing unit 8f that changes at least one of the size of the subject image Ga and the background image P4a related to image combining in the image combining process.
The size changing unit 8f is illustrated in FIG. 7B based on the size of the subject image Ga synthesized by the synthesis unit 8e (for example, the vertical and horizontal widths of the rectangular frame surrounding the subject image Ga). The size of at least one of the subject image Ga and the background image P4a is changed so as to adjust the relative size of the subject image Ga with respect to the size of the subject composite image P5a after image synthesis. Specifically, the size changing unit 8f enlarges / reduces the size of the subject image Ga, enlarges / reduces the size of the background image P4a, or trims a predetermined area, so that at least the subject image Ga is the subject image Ga. The feature region C is not included in the subject composite image P5a so as to be larger than the feature region C combined with Ga. Thereby, it is possible to prevent the same subject image Ga, that is, a plurality of the same person from being present in the subject composite image P5a.
Here, the size changing unit 8f constitutes a changing unit that changes the size of at least one of the subject image Ga and the background image P4a based on the size of the subject image (foreground image) Ga combined by the combining unit 8e. ing.

  The dimensions of the subject image Ga and the background image P4a may be changed by manually setting the dimensions based on a predetermined operation of the operation input unit 12 by the user.

The image processing unit 8 includes a table storage unit 8g that stores various tables.
The table storage unit 8g includes a registered name (for example, “person name”), identification information for identifying the image portion detected as the feature area C, and the subject image Ga extracted by the subject clipping process. A registration table T1 associated with the subject image information and the like is stored.
Here, the table storage unit 8g constitutes registration means for registering subject image information related to the subject image Ga and information for specifying the feature region C in association with each other.

The registration of various types of information in the registration table T1 may be executed, for example, when a person image is extracted as the subject image Ga in the subject clipping process, or face detection is performed from the image information already stored in the recording medium 9. It may be executed in a face registration processing mode in which a person is registered by performing processing or the like, or the user may arbitrarily register.
The identification information includes, for example, human face image information. The face image information is not limited to the face image data of the face image itself, and may be any information related to the face image for specifying a person, for example, face feature data or an image pattern.
Further, the subject image information is not limited to the image data of the subject image Ga itself, but any information such as information for specifying the subject image Ga, for example, the file name of the subject cut-out image P3a including the subject image Ga, etc. It may be.

The recording medium 9 is configured by, for example, a nonvolatile memory (flash memory) or the like, and stores image data of the subject cutout image P3a and the background image P4a encoded by the JPEG compression unit (not shown) of the image processing unit 8. .
The image data of the subject clipped image P3a is associated with the alpha map generated by the position information generation unit of the clipped image generation unit 8a, and the extension of the image data of the subject clipped image P3a is stored as “.jpe”. ing.
Here, the recording medium 9 constitutes storage means for storing a plurality of each of the background image P4a and the subject cutout image P3a (subject image Ga).

The display control unit 10 performs control to read display image data temporarily stored in the image memory 5 and display the display image data on the display unit 11.
Specifically, the display control unit 10 includes a VRAM, a VRAM controller, a digital video encoder, and the like. The digital video encoder reads the luminance signal Y and the color difference signals Cb and Cr read from the image memory 5 and stored in the VRAM (not shown) under the control of the central control unit 13 via the VRAM controller. The data is periodically read from the VRAM, a video signal is generated based on these data, and is output to the display unit 11.

  The display unit 11 is, for example, a liquid crystal display device, and displays an image captured by the electronic imaging unit 2 based on a video signal from the display control unit 10 on a display screen. Specifically, the display unit 11 displays a live view image based on a plurality of image frames generated by imaging the subject by the lens unit 1, the electronic imaging unit 2, and the imaging control unit 3 in the imaging mode. The REC view image captured as the actual captured image is displayed.

Further, after the image synthesis by the synthesis unit 8e, the display unit 11 switches and reproduces the subject synthesized image P5a and the background image P4a generated by the image synthesis at predetermined intervals under the control of the display control unit 10. That is, when the image composition is completed, the background image P4a before the image composition and the subject composition image P5a generated by the image composition are switched at a predetermined timing and alternately reproduced.
Here, the display unit 11 and the display control unit 10 constitute playback means for switching and reproducing the background image P4a and the subject composite image P5a synthesized by the synthesis unit 8e.

  The operation input unit 12 is for performing a predetermined operation of the imaging apparatus 100. Specifically, the operation input unit 12 includes a shutter button 12a related to an instruction to shoot a subject, a selection determination button 12b related to an instruction to select an imaging mode and a function, a zoom button (not shown) related to an instruction to adjust the zoom amount, and the like. And outputs a predetermined operation signal to the central control unit 13 in accordance with the operation of these buttons.

The selection determination button 12b of the operation input unit 12 is a background desired by the user among the plurality of background images P4a read from the recording medium 9 and displayed on the display unit 11 in the composite image generation process. An instruction to select the image P4a is input. Then, the selection determination button 12 b outputs a predetermined selection signal corresponding to the operation of the operation input unit 12 to the central control unit 13.
The central control unit 13 designates the background image P4a designated by the input selection signal as the background image P4a related to image synthesis.
Here, the operation input unit 12 and the central control unit 13 constitute background specifying means for specifying any one of the background images P4a stored in the recording medium 9.

Further, the selection decision button 12b of the operation input unit 12 is any one desired by the user among the plurality of subject cutout images P3a read from the recording medium 9 and displayed on the display unit 11 in the composite image generation process. An instruction to select the subject cutout image P3a is input. That is, the selection determination button 12b is selected by the user among the plurality of subject cutout images P3a including the subject image Ga specified by the foreground specifying unit 8d from the plurality of subject cutout images P3a stored in the recording medium 9. An instruction to select any one of the subject cutout images P3a is input. Then, the selection determination button 12 b outputs a predetermined selection signal corresponding to the operation of the operation input unit 12 to the central control unit 13.
The central control unit 13 designates the subject cutout image P3a designated by the input selection signal as the subject cutout image P3a related to the image composition.
Here, the operation input unit 12 and the central control unit 13 constitute foreground designating means for designating any one of the subject images Ga (foreground images) Ga specified by the foreground specifying unit 8d. ing.

  The central control unit 13 controls each unit of the imaging apparatus 100. Specifically, the central control unit 13 includes a CPU (not shown), and performs various control operations according to various processing programs (not shown) for the imaging apparatus 100.

Next, the subject clipping process performed by the imaging apparatus 100 will be described with reference to FIGS.
FIG. 2 is a flowchart illustrating an example of an operation related to the subject clipping process. 3A to 3C are diagrams schematically illustrating an example of an image related to the subject clipping process.

The subject clipping process is executed when a subject clipping mode is selected from a plurality of imaging modes displayed on the menu screen based on a predetermined operation of the selection determination button 12b of the operation input unit 12 by the user. It is.
In the subject clipping process described below, it is assumed that a person (for example, a child “Mr. A”) is imaged as a subject. Further, the subject cutout image P3a generated by the subject cutout processing may be a still image or a moving image composed of a plurality of image frames.

As shown in FIG. 2, first, the central control unit 13 performs live display on the display control unit 10 based on a plurality of image frames generated by imaging a subject by the lens unit 1, the electronic imaging unit 2, and the imaging control unit 3. The view image is displayed on the display screen of the display unit 11 and superimposed on the live view image, and the imaging instruction message of the subject existing image P1a is displayed on the display screen of the display unit 11 (step S1).
At this time, the central control unit 13 causes the feature region detection unit 8b of the image processing unit 8 to detect the feature region C from each of a plurality of image frames constituting the live view image (step S2). Specifically, the face recognition unit 8c of the feature region detection unit 8b executes a face detection process for detecting a face from each image frame. When a face is detected, the central control unit 13 causes the display control unit 10 to perform control to cause the display unit 11 to display a face detection frame (not shown) so as to be superimposed on the detected face.

Thereafter, the central control unit 13 determines whether or not an imaging instruction is input based on a predetermined operation of the shutter button 12a of the operation input unit 12 by the user (step S3). Here, if it is determined that an imaging instruction has been input (step S3; YES), the central control unit 13 instructs the imaging control unit 3 to set the focus lens in-focus position using a face detection frame (not shown) as an AF area. At the same time, the exposure condition (shutter speed, aperture, signal amplification factor, etc.), white balance, etc. are adjusted, and the optical image of the subject existing image P1a (refer to FIG. (Step S4). Then, the central control unit 13 causes the image data generation unit 4 to generate YUV data of the subject existing image P1a transferred from the electronic imaging unit 2. The YUV data of the subject existing image P1a is temporarily stored in the image memory 5.
In addition, the central control unit 13 controls the imaging control unit 3 to maintain a state in which conditions such as a focus position, an exposure condition, and a white balance at the time of imaging the subject presence image P1a are fixed.

Next, the central control unit 13 causes the display control unit 10 to display a live view image based on a plurality of image frames generated by imaging the subject by the lens unit 1, the electronic imaging unit 2, and the imaging control unit 3. Is displayed on the display screen, and is superimposed on the live view image to display a semi-transparent display mode image of the subject presence image P1a and an imaging instruction message of the subject non-existence image P2a on the display screen of the display unit 11 ( Step S5).
Thereafter, the central control unit 13 determines whether or not an imaging instruction has been input based on a predetermined operation of the shutter button 12a of the operation input unit 12 by the user (step S6). Then, the user waits for the subject to move, or moves the subject outside the angle of view, and then the camera position so that the subject non-existing image P2a overlaps the semi-transparent image of the subject existing image P1a. Is adjusted, and it is determined that the shutter button 12a of the operation input unit 12 is predeterminedly operated and an imaging instruction is input (step S6; YES), the central control unit 13 includes no subject in the imaging control unit 3. An optical image of the image P2a (see FIG. 3B) is imaged by the electronic imaging unit 2 under a fixed condition after the imaging of the subject existing image P1a (step S7). Then, the central control unit 13 causes the image data generation unit 4 to generate YUV data of the subject nonexistent image P2a transferred from the electronic imaging unit 2. The YUV data of the subject existing image P1a is temporarily stored in the image memory 5.

Next, the central control unit 13 uses the YUV data of the subject non-existing image P2a temporarily stored in the image memory 5 in the feature amount calculating unit 6, the block matching unit 7, and the image processing unit 8 as a reference. A projection transformation matrix for projective transformation of the YUV data of P1a is calculated by a predetermined image transformation model (for example, a similarity transformation model or a joint transformation model) (step S8).
Specifically, the feature amount calculation unit 6 selects a predetermined number (or a predetermined number or more) of highly featured block areas (feature points) based on the YUV data of the subject non-existing image P2a, and then selects the block. Is extracted as a template. Then, the block matching unit 7 searches the subject existing image P1a for a position where the pixel value of the template extracted by the feature extraction process is optimally matched, and the evaluation value of the difference between the pixel values is the best. An optimum offset between the subject non-existing image P2a and the subject existing image P1a is calculated as a motion vector of the template. Then, the alignment unit of the cut-out image generation unit 8a statistically calculates the entire motion vector based on the motion vectors of the plurality of templates calculated by the block matching unit 7, and performs feature point correspondence related to the motion vector. The projection transformation matrix of the subject existing image P1a is calculated by using this.

  Next, the central control unit 13 causes the alignment unit of the cut-out image generation unit 8a to perform projective transformation of the subject existing image P1a based on the calculated projective transformation example, thereby obtaining the YUV data of the subject existing image P1a and the subject. A process of aligning the YUV data of the non-existing image P2a is performed (step S9).

Then, the central control unit 13 causes the subject image extraction unit of the cutout image generation unit 8a to perform a process of extracting the subject image Ga including the subject from the subject existing image P1a (step S10).
Specifically, the subject region extraction unit applies a low-pass filter to each of the YUV data of the subject presence image P1a and the YUV data of the subject non-existence image P2a to remove high-frequency components of each image. Thereafter, the subject area extraction unit calculates a difference for each corresponding pixel between the subject existing image P1a and the subject non-existing image P2a subjected to the low-pass filter, and generates a difference map. Subsequently, the subject region extraction unit binarizes the dissimilarity map relating to each pixel with a predetermined threshold, and then performs a contraction process to remove a region in which the dissimilarity is caused by fine noise or camera shake from the dissimilarity map. . Thereafter, the subject region extraction unit performs a labeling process to remove a region below a predetermined value or a region other than the maximum region, and then specifies the largest island pattern as the subject image Ga and corrects the shrinkage. The expansion process is performed.

  Next, the central control unit 13 causes the position information generation unit of the image processing unit 8 to generate an alpha map indicating the position of the extracted subject image Ga in the subject presence image P1a (step S11).

Thereafter, the central control unit 13 causes the image generation unit of the image processing unit 8 to generate image data of a subject cut-out image P3a (see FIG. 3C) obtained by combining the subject image Ga with a predetermined single color image. (Step S12).
Specifically, the image generation unit reads out the subject existence image P1a, the single color image, and the alpha map and develops them in the image memory 5. Then, for all the pixels of the subject existence image P1a, the pixels having an alpha value of 0 are obtained. (Α = 0), transmission is performed for a pixel with an alpha value of 0 <α <1, (0 <α <1), and blending with a predetermined single color is performed, and for a pixel with an alpha value of 1 (α = 1) ) Do nothing and do not allow transmission through a given single color.

  Thereafter, the central control unit 13 stores the alpha map generated by the position information generation unit of the image processing unit 8 and the image data of the subject cutout image P3a in a predetermined storage area of the recording medium 9 in a single file. (Step S13).

Next, the central control unit 13 specifies the face (feature region C) detected in the face detection process for subject image information corresponding to the subject image Ga of the subject cutout image P3a in the registration table T1 of the table storage unit 8g. The information is stored in association with the use information (for example, face image information) (step S14).
If the person name of the face detected by the face detection process has not yet been registered in the registration table T1, for example, an input of a person name is accepted based on a predetermined operation of the operation input unit 12 by the user. Registration may be performed, or registration may be performed by inputting in the person name input mode. In addition, when the person name of the face already detected in the face detection process is registered in the registration table T1 in association with the identification information, the identification name newly detected in the face detection process is not necessarily used. It is not necessary to store information (for example, face image data) itself, but in order to improve the accuracy of face detection processing, the new identification information may be added and stored, or the oldest information may be stored. It may be overwritten and stored.

  Thereby, the subject clipping process is completed. As a result, for example, image data of the subject cut-out image P3a in which the child “A-kun” who trembled with light clothing from a predetermined background is extracted as a subject is generated.

Next, background image imaging processing by the imaging apparatus 100 will be described with reference to FIG.
FIG. 4 is a flowchart illustrating an example of an operation related to the background image capturing process.

The background image capturing process is a normal still image capturing process. Based on a predetermined operation of the selection determination button 12b of the operation input unit 12 by the user, the still image capturing process is performed from among a plurality of capturing modes displayed on the menu screen. This process is executed when a mode is instructed to be selected.
In the background image imaging process described below, it is assumed that the same person as the person imaged by the subject clipping process (for example, child “A-kun”) is captured as the subject.

  As shown in FIG. 4, first, the central control unit 13 causes the display control unit 10 to base on a plurality of image frames generated by imaging the background image P4a by the lens unit 1, the electronic imaging unit 2, and the imaging control unit 3. Then, the live view image is displayed on the display screen of the display unit 11 (step S21).

Next, the central control unit 13 determines whether or not an imaging instruction has been input based on a predetermined operation of the shutter button 12a of the operation input unit 12 by the user (step S22). Here, if it is determined that an imaging instruction has been input (step S22; YES), the central control unit 13 causes the imaging control unit 3 to adjust conditions such as the focus lens focus position, exposure conditions, and white balance. Thus, the optical image of the background image P4a (see FIG. 7A) is picked up by the electronic image pickup unit 2 under a predetermined condition. Then, the central control unit 13 causes the image data generation unit 4 to generate YUV data of the background image P4a transferred from the electronic imaging unit 2 (step S23).
Subsequently, the central control unit 13 stores the YUV data of the background image P4a as an Exif format image file in a predetermined storage area of the recording medium 9 (step S24).

  Thereby, the background image capturing process is terminated. As a result, for example, still image data of the background image P4a in which the child “Mr. A” wearing winter clothes slides on the snow with a sled is generated.

Next, composite image generation processing by the imaging apparatus 100 will be described with reference to FIGS.
FIG. 5 is a flowchart illustrating an example of an operation related to the composite image generation process. FIG. 6 is a flowchart illustrating an example of an operation related to the image composition processing in the composite image generation processing. FIGS. 7A and 7B are diagrams schematically illustrating an example of an image related to the composite image generation process.

The composite image generation processing is executed when an image combination mode is selected from a plurality of imaging modes displayed on the menu screen based on a predetermined operation of the selection determination button 12b of the operation input unit 12 by the user. It is processing.
As shown in FIG. 5, first, based on a predetermined operation of the selection determination button 12b of the operation input unit 12 by the user, a desired background image P4a (see FIG. 7 (FIG. 7)) among a plurality of images recorded on the recording medium 9. When a) is selected and designated, the central control unit 13 causes the image processing unit 8 to read out the image data of the designated background image P4a and develop it in the image memory 5 (step S31).

Next, the central control unit 13 causes the feature region detection unit 8b to detect the feature region C from the background image P4a (step S32). Specifically, the face recognition unit 8c of the feature region detection unit 8b performs a face detection process for detecting a human face using a predetermined face detection method from the background image P4a, and then detects the face detection process. It is determined whether face image information corresponding to the face that has been registered is registered in the registration table T1. If it is determined that the face image information is registered as a result of the determination, the face recognition unit 8c specifies a person registered in association with the face image information. Then, the feature region detection unit 8b detects the face image region of the person specified by the face recognition unit 8c in the background image P4a as the feature region C.
When the feature region C is detected from the background image P4a as a result of the above-described feature region C detection processing (step S32; YES), the central control unit 13 causes the feature region detection unit 8b to send the feature region C and the subject image Ga. Are associated with each other (step S33). Specifically, the feature region detection unit 8b determines whether the feature region C and the subject image Ga correspond to whether subject image information corresponding to the person specified by the face recognition unit 8c is registered in the registration table T1. It is determined whether they are associated with each other. If the characteristic area detecting unit 8b determines that the subject image information is registered in the registration table T1, the characteristic area detecting unit 8b determines that the characteristic area (face image area) C and the subject image Ga are associated with each other (Step S1). S33; YES), the feature area C is used for the subsequent processing.

  Next, the central control unit 13 causes the foreground specifying unit 8d of the image processing unit 8 to refer to the registration table T1, and at least one subject cutout image P3a including the subject image Ga associated with the feature region C is selected. The image data of the subject cutout image P3a is read from the recording medium 9 and temporarily stored in the image memory 5 (step S34). Thereafter, the display control unit 10 reads out the image data of the subject cutout image P3a temporarily stored in the image memory 5 and causes the display unit 11 to display the image data.

  Subsequently, based on a predetermined operation of the selection determination button 12b of the operation input unit 12 by the user, a desired subject cutout image P3a (see FIG. 3C) among at least one subject cutout image P3a displayed on the display unit 11. ) Reference) is selected and designated (step S35), the image processing unit 8 causes the face image region of the subject image Ga of the designated subject cutout image P3a and the feature region (face image region) C of the background image P4a. Are designated as the synthesis position of the subject image Ga (step S36). Thereafter, the central control unit 13 causes the combining unit 8e to perform image combining processing using the subject image Ga in the background image P4a and the subject cutout image P3a (step S37).

On the other hand, if the feature region C is not detected from the background image P4a in step S32 (step S32; YES), or the feature region (face image region) C and the subject image Ga are associated in step S33. If it is determined that the image is not registered (step S33; NO), the central control unit 13 accepts a manual operation by the user for specifying the subject cut-out image and for the composition position of the subject image.
Specifically, a desired subject cut-out image (not shown) is selected from a plurality of images recorded on the recording medium 9 based on a predetermined operation of the selection determination button 12b of the operation input unit 12 by the user. When designated, the central control unit 13 causes the image processing unit 8 to read out the image data of the designated subject clipped image and develop it in the image memory 5 (step S38).
Subsequently, when a user-desired subject clipping image combining position is designated based on a predetermined operation of the selection determination button 12b of the operation input unit 12 by the user (step S39), the central control unit 13 performs the process in step S38. The image composition processing is performed using the subject image in the background image P4a and the user-desired subject cutout image.

Here, the image composition processing will be described in detail with reference to FIG.
Note that the image composition processing described below is performed separately for the subject clipped image P3a that has been automatically specified and read out, and the user-subjected clipped image specified in step S39. However, since the contents of the image composition processing itself are substantially equivalent, they will be described together. Further, the image composition processing will be described by exemplifying the subject cutout image P3a, and the description of the user desired subject cutout image is omitted unless otherwise specified.

As shown in FIG. 6, the composition unit 8e reads the alpha map stored in association with the subject cutout image P3a and develops it in the image memory 5 (step S51).
Note that when the composition position of the subject cutout image P3a in the background image P4a is determined, the background image P4a and the alpha map are misaligned, and an alpha value is present with α = 0. Do not create a region that does not.
Further, the size changing unit 8f enlarges / reduces the size of the subject image Ga based on the size of the subject image Ga so that the same person does not exist in the subject composite image P5a generated by the image synthesis. Or the size of the background image P4a may be enlarged or reduced, or a predetermined area may be trimmed.

  Next, the synthesizing unit 8e designates any one pixel (for example, the pixel at the upper left corner) of the background image P4a (step S52), and branches the process for the pixel based on the alpha value of the alpha map. (Step S53). Specifically, for any pixel with an alpha value of 1 among any one of the pixels in the background image P4a (step S53; α = 1), the composition unit 8e sets the pixel value of the corresponding pixel in the subject cutout image P3a. (Step S54), and for pixels with an alpha value of 0 <α <1 (step S53; 0 <α <1), an image obtained by cutting out the subject image Ga using 1's complement (1-α) ( After generating the background image × (1−α)), when the subject cutout image P3a is generated using the one's complement (1−α) in the alpha map, a value blended with the single background color is calculated, The value is subtracted from the subject cutout image P3a, and is combined with an image (background image × (1-α)) cut out of the subject image Ga (step S55). For pixels with an alpha value of 0 (step S53; α = 0) Anything so as to pass through the background image P4a without.

Subsequently, the composition unit 8e determines whether or not all the pixels of the background image P4a have been processed (step S56).
If it is determined that all the pixels have not been processed (step S56; NO), the synthesis unit 8e designates the next pixel as the processing target and moves the processing target to the pixel (step S57). ), The process proceeds to step S53.
By repeating the above processing until it is determined that all pixels have been processed in step S56 (step S56; YES), the combining unit 8e combines the subject clipped image P3a and the background image P4a. Image data of the image P5a is generated.
Thereby, the image composition process is terminated.

As shown in FIG. 5, thereafter, the central control unit 13 causes the display control unit 10 to synthesize the subject in which the subject is superimposed on the background image P4a based on the image data of the subject synthesized image P5a generated by the synthesis unit 8e. The image P5a is displayed on the display screen of the display unit 11 (step S40).
Specifically, for example, when the image is synthesized using the subject cutout image P3a automatically specified from the background image P4a, the display unit 11 and the subject synthesized image P5a generated by the image synthesis before the image synthesis. By switching and playing back the background image P4a at a predetermined timing, for example, the background image P4a where the child “A-kun” slides on the snow with a sled and the child “A-kun” who trembles when cold on the snow are expressed. The displayed subject composite image P5a is switched and reproduced and displayed.
In addition, when the image is synthesized using a subject desired clipped image, the display unit 11 reproduces and displays a subject synthesized image (not shown) generated by the image synthesis.
Thus, the composite image generation process is finished.

As described above, according to the imaging apparatus 100 of the present embodiment, the feature region C associated with the subject image (foreground image) Ga of the subject cutout image P3a is detected from the background image P4a, and the background image P4a Since the subject image Ga associated with the feature region C is combined with the region in which the feature region C is detected, the subject image Ga suitable for combination with the background image P4b is automatically identified, and the background image P4a The composition position of the subject image Ga in can be automatically determined.
That is, in the registration table T1, subject image information for specifying the subject image Ga included in the subject cutout image P3a and specification information for specifying the feature region C are registered in association with each other. Then, the foreground specifying unit 8d refers to the registration table T1, and is associated with the feature region C of the background image P4a from the subject cutout image P3a including the plurality of subject images Ga stored in the recording medium 9. Therefore, even if a plurality of subject cutout images P3a are stored in the recording medium 9, a subject cutout including the subject image Ga associated with the feature region C of the background image P4a is specified. The image P3a can be specified automatically and reliably.
In addition, since the face image area of the person of the specified subject image Ga and the face image area of the same person as the person of the subject image Ga in the background image P4a are combined, the combined subject image generated by the image combination The composition position of the subject image Ga in the background image P4a can be determined automatically and simply so that the composite subject image P5a can be easily generated so that the same person does not exist in P5a. it can.

  Even when the foreground specifying unit 8d specifies a subject cut-out image P3a including a plurality of subject images Ga, any one of the user-desired subject images Ga can be designated, and the user-desired subject Since the image Ga can be combined with the region in which the feature region C in the background image P4a is detected, a subject composite image P5a in which the user-desired subject image Ga is combined can be generated.

  Furthermore, since at least one of the subject image Ga and the background image P4a can be automatically changed based on the size of the subject image Ga to be synthesized, the size of the subject image Ga and the feature region in the background image P4a Even when the dimensions of C are different, generation of an unnatural subject composite image P5a in which a plurality of the same persons exist can be prevented.

  In addition, since the subject composite image P5a generated by the image composition and the background image P4a before the image composition are switched at a predetermined timing and alternately reproduced and displayed, even if the subject composite image P5a is a still image, the phenomenon A motion can be given to the subject image Ga in the image to be reproduced and displayed. That is, for example, there is a background image P4a (see FIG. 7A) where a child “Mr. By alternately switching the expressed subject composite image P5a (see FIG. 7 (b)), the child “A-kun” wearing a winter clothes and wearing a sled suddenly becomes lightly dressed and trembles. Can express movement. Thereby, the interest property of the image reproduced and displayed can be improved.

The present invention is not limited to the above-described embodiment, and various improvements and design changes may be made without departing from the spirit of the present invention.
For example, in the above-described embodiment, the human face image area is detected as the feature area C. However, the detected area is not limited to the human face image area. Whether or not a face is included can be arbitrarily changed as appropriate. Similarly, the subject image Ga and the background image P4a are arranged such that the face image area of the subject image Ga associated with the face image area detected as the feature area C and the face image area of the feature area C overlap. Although they are combined, it is not always necessary to combine the face image area of the subject image Ga and the face image area of the feature area C. When the subject image Ga is a human image that does not include a face, The subject image Ga and the feature area C may be combined so as to overlap.

In the above-described embodiment, a person (for example, the child “A-kun”) is exemplified as a subject related to image composition. However, the type of subject is not limited to this, and can be arbitrarily changed as appropriate. be able to. Further, the feature region C of the background image P4a and the subject image (foreground image) Ga extracted by the subject clipping process do not have to be the same, and what is the feature region C associated with the foreground image? Alternatively, the user may arbitrarily set it.
Hereinafter, as an example of a subject, an animal (for example, “dog” or the like) is used as a feature region C associated with the animal, and an article (for example, “soccer ball” or the like) is employed. Will be explained.

<Modification 1>
An imaging apparatus 100 according to Modification 1 will be described with reference to FIGS. 8 and 9.
FIGS. 8A to 8C are diagrams schematically illustrating an example of an image related to the subject clipping process performed by the imaging apparatus 100 according to the first modification, and FIGS. 9A and 9B. () Is a figure which shows typically an example of the image which concerns on a synthetic image generation process.
Note that the imaging apparatus 100 according to the first modification has a configuration substantially similar to that of the imaging apparatus 100 according to the above-described embodiment except for the configuration of the feature region detection unit 8b and the registration table T1, and the description thereof is omitted.

The feature region detection unit 8b of the image processing unit 8 detects a contrast difference using, for example, color information of image data of the background image P4b, or extracts a contour line (edge portion) of a pixel set. A region C (for example, an image portion of “soccer ball” or “tree”) is detected. Then, the feature region detection unit 8b determines whether or not the detected feature region C is registered in the registration table T1. For example, the feature region detection unit 8b determines whether or not the same image pattern as the image of the feature region C is stored as identification information, and as a result of the determination, it is determined that the feature region C is registered. Then, a registered name (for example, “soccer ball”) registered in association with the feature region C is acquired.
Next, the feature region detection unit 8b determines whether or not subject image information corresponding to the registered name of the feature region C is registered in the registration table T1. When it is determined that the subject image information (for example, “dog” in association with “soccer ball”) is registered in the registration table T1, the feature region detection unit 8b determines that the feature region C is the subject image Gb. While determining that the feature region C is associated and using the feature region C for subsequent processing, if it is determined that the subject image information is not registered in the registration table T1, the detected feature region C is discarded. .

The table storage unit 8g includes a registered name (for example, “name” of an article such as “soccer ball”, “animal name” indicating the type of animal, “name” or “nickname” of a pet), and a feature area. A registration table T1 is stored in which identification information for identifying the image portion detected as C is associated with subject image information relating to the subject image Gb extracted by subject clipping processing.
Note that the specifying information is not limited to the image data of the target image itself such as an article or an animal, but information related to an image for specifying the feature region C, for example, feature data or an image pattern of the article or animal. Any information may be used.

Next, the subject clipping process performed by the imaging apparatus 100 will be described with reference to FIG. 2 and FIGS. 8 (a) to 8 (c).
The subject clipping process described below is substantially the same as the process in the above embodiment, except that “dog” is imaged as a subject and the subject clipped image P3b of the “dog” is generated. Detailed description thereof will be omitted.

That is, as shown in FIG. 2, first, as in the above embodiment, the central control unit 13 causes the display control unit 10 to perform a process of displaying a live view image on the display screen of the display unit 11 (step S1). .
In step S2, the central control unit 13 causes the feature region detection unit 8b of the image processing unit 8 to detect the feature region C from each of a plurality of image frames constituting the live view image. Specifically, the feature region detection unit 8b detects a contrast difference using color information of image data of each image frame or extracts a contour line (edge portion) of a pixel set, thereby detecting the feature region C. (For example, an image portion of “dog”) is detected. When the feature region C is detected, the central control unit 13 causes the display control unit 10 to display a feature region detection frame (not shown) on the display unit 11 so as to be superimposed on the detected feature region C. Let it be done.

  Thereafter, as in the above embodiment, when it is determined in step S3 that an imaging instruction has been input (step S3; YES), the central control unit 13 causes the imaging control unit 3 to display a feature region detection frame (not shown). ) As an AF area, the focus position of the focus lens is adjusted, and exposure conditions (shutter speed, aperture, signal amplification factor, etc.), white balance, etc. are adjusted, and the subject existing image P1b (see FIG. 8A) Are imaged by the electronic imaging unit 2 under a predetermined condition (step S4).

Next, the central control unit 13 causes the display control unit 10 to perform a process of displaying the live view image on the display screen of the display unit 11 as in the above embodiment (step S5).
Thereafter, as in the above-described embodiment, the user waits for the subject to move, or moves the subject outside the angle of view, and then the subject nonexistent image P2b is translucent to the subject present image P1b. When it is determined that the camera position is adjusted so as to overlap the image of the camera and the shutter button 12a of the operation input unit 12 is operated in a predetermined manner and an imaging instruction is input (step S6; YES), the central control unit 13 performs imaging. The control unit 3 causes the electronic imaging unit 2 to capture an optical image of the subject non-existing image P2b (see FIG. 8B) under conditions fixed after capturing the subject existing image P1b (step S7).

Subsequently, in the same manner as in the above-described embodiment, the processes of steps S8 to S12 are performed, and the image generation unit of the image processing unit 8 combines the subject image Gb with a predetermined single color image P3b (FIG. 8 ( c)) is generated.
Thereafter, as in the above embodiment, the central control unit 13 stores the alpha map generated by the position information generation unit of the image processing unit 8 and the image data of the subject cutout image P3b in a predetermined storage area of the recording medium 9. The file is stored in association with one file (step S13).
Next, as in the above embodiment, the central control unit 13 has detected subject image information corresponding to the subject image Gb of the subject cutout image P3b in the registration table T1 of the table storage unit 8g by the feature region detection process. The information is stored in association with the information for specifying the feature area C (step S14).

  Thereby, the subject clipping process is completed. As a result, for example, image data of a subject cutout image P3b in which “dog” is extracted as a subject from a predetermined background is generated.

Next, background image imaging processing by the imaging apparatus 100 will be described with reference to FIG.
In the background image capturing process described below, a background other than capturing a background including a “soccer ball” associated with “dog” related to the subject image Gb of the subject cut-out image P3b. In this respect, the processing is substantially the same as the processing in the above embodiment, and detailed description thereof is omitted.

  That is, as shown in FIG. 4, similarly to the above embodiment, the processing of steps S21 to S24 is performed, and the electronic imaging unit 2 causes the optical image of the background image P4b (see FIG. 9A) to be obtained under a predetermined condition. After the image is captured and the image data generation unit 4 generates the YUV data of the background image P4b, the central control unit 13 stores the YUV data of the background image P4b as an Exif format image file in a predetermined storage area of the recording medium 9. Remember.

  Thereby, the background image capturing process is terminated. As a result, for example, still image data of the background image P4b having “soccer ball” in the background is generated.

Next, composite image generation processing by the imaging apparatus 100 will be described with reference to FIGS. 5, 6, 9 (a), and 9 (b).
In the composite image generation process described below, the image portion of “soccer ball” is detected as the feature region C from the background image P4b, and the image of “dog” is superimposed on the “soccer ball”. Except for this point, the processing is substantially the same as the processing in the above embodiment, and the detailed description thereof is omitted.

  That is, as shown in FIG. 5, first, based on a predetermined operation of the selection determination button 12b of the operation input unit 12 by the user, a desired background image P4b (see FIG. 5) among a plurality of images recorded on the recording medium 9. When 9 is selected and designated, the central control unit 13 causes the image processing unit 8 to read the image data of the designated background image P4b and develop it in the image memory 5 (step S31). ).

Next, the central control unit 13 causes the feature region detection unit 8b to detect the feature region C from the background image P4b (step S32). Specifically, the feature region detection unit 8b performs a process of detecting the feature region C (for example, an image portion of “soccer ball” or “tree”) from the background image P4b using a predetermined detection method, It is determined whether or not the identification information corresponding to the feature area C detected by the detection process is registered in the registration table T1. If it is determined that the identification information is registered as a result of the determination, the feature region detection unit 8b acquires a registered name (for example, “soccer ball”) registered in association with the identification information. To do.
When the feature area C is detected from the background image P4b as a result of the detection process of the feature area C (step S32; YES), the central control unit 13 causes the feature area detection unit 8b to send the feature area C and the subject image Gb. Are associated with each other (step S33). Specifically, the feature region detection unit 8b determines whether subject image information (for example, “dog” in association with “soccer ball”) corresponding to the acquired registered name is registered in the registration table T1. Then, it is determined whether or not the feature region C and the subject image Gb are associated with each other. When the characteristic area detection unit 8b determines that the subject image information is registered in the registration table T1, the characteristic area detection unit 8b determines that the characteristic area C and the subject image Gb are associated with each other (step S33; YES). The feature region C is used for subsequent processing.

Next, in step S34, as in the above embodiment, the foreground specifying unit 8d of the image processing unit 8 refers to the registration table T1, and includes a subject including the subject image Gb associated with the feature region C. At least one clipped image P3b is specified, and the image data of the subject clipped image P3b is read from the recording medium 9 and temporarily stored in the image memory 5.
Thereafter, in step S35, a desired subject cutout image P3b (of at least one subject cutout image P3b displayed on the display unit 11 based on a predetermined operation of the selection determination button 12b of the operation input unit 12 by the user. When the image processing unit 8 is selected and designated (see FIG. 8C), the image processing unit 8 selects the subject image Gb (for example, “dog” image) of the subject cutout image P3b specified by the foreground specifying unit 8d and the background image. The feature region C of the background image P4b is designated as the synthesis position of the subject image Gb so as to overlap with the feature region C of P4b (for example, the image portion of “soccer ball”) (step S36).
In step S37, the composition unit 8e performs image composition processing (see FIG. 6) using the subject image Gb in the background image P4b and the subject cutout image P3b.
Note that the image composition processing shown in FIG. 6 is the same as that in the above embodiment, and a detailed description thereof will be omitted.

  On the other hand, if the feature region C is not detected from the background image P4b in step S32 (step S32; YES), or the feature region C and the subject image Gb are not registered in association with each other in step S33. Is determined (step S33; NO), the central control unit 13 shifts the processing to step S38, but the subsequent processing is the same as in the above embodiment, and detailed description thereof is omitted.

Next, as shown in FIG. 5, in step S40, the display control unit 10 performs subject composition in which the subject is superimposed on the background image P4b based on the image data of the subject composite image P5b generated by the composition unit 8e. The image P5b is displayed on the display screen of the display unit 11. Thereby, the display unit 11 displays the subject composite image P5b in which the image of the “dog” that is the subject image Gb is superimposed on the position of the soccer ball in the background image P4b.
Similar to the above embodiment, the display unit 11 may alternately reproduce and display the subject composite image P5b generated by image composition and the background image P4b before image composition at a predetermined timing.
Thus, the composite image generation process is finished.

  Therefore, according to the imaging apparatus 100 of the first modification, as in the above embodiment, the feature image is associated with the subject image (foreground image) Gb that is the “dog” image of the subject cutout image P3b from the background image P4b. Since the region C (image portion of “soccer ball”) is detected and the subject image Gb associated with the feature region C is combined with the region in which the feature region C is detected in the background image P4b, the background image The subject image Gb suitable for the composition to P4b can be automatically specified, the composition position of the subject image Gb in the background image P4b can be automatically determined, and the subject composition image P5b can be easily generated. .

In the above embodiment and the first modification, when the foreground specifying unit 8d specifies the subject cutout image P3a (P3b) including the plurality of subject images Ga (Gb), the operation input unit 12 by the user. One of the user-desired subject images Ga (Gb) is designated based on the predetermined operation, but taking into account various conditions such as shooting conditions, shooting date and time, and dimensions of the subject image Ga (Gb). The subject image Ga (Gb) may be automatically specified. In other words, among the plurality of subject cutout images P3a (P3b), for example, an image that is not out of focus, a bright image, an image with the latest shooting date and time, an image with a large size of the subject image Ga (Gb), and the like are given priority. The subject image Ga (Gb) may be designated automatically by selecting the target image. Thus, when a plurality of subject cutout images P3a (P3b) are specified, the subject composite image P5b can be generated more easily.
In the embodiment and the first modification, the subject image Gb associated with the feature region C is combined with the region in which the feature region C is detected. However, the present invention is not limited to this. is not. For example, when the feature region C is detected, the subject image Gb associated with the feature region C is specified by the foreground specifying unit 8d, and the synthesis position is determined based on a predetermined operation of the operation input unit 12 by the user. Alternatively, one of each of the subject cutout images P3a (P3b) and the background images P4a (P4b) stored based on a predetermined operation of the operation input unit 12 by the user may be designated, and the designated background image P4a ( When the feature region C is detected from P4b), the feature region C may be set as the synthesis position of the subject image Ga (Gb) included in the designated subject cutout image P3a (P3b). That is, based on the feature region C, only the subject image Ga (Gb) may be specified, or only the synthesis position may be specified.
In the embodiment and the first modification, the subject image Gb associated with the feature region C is specified as the region in which the feature region C is detected from the designated background image P4a (P4b). However, if one of the plurality of subject cutout images P3a (P3b) stored based on a predetermined operation of the operation input unit 12 by the user is designated, the subject image Gb included in the designated subject cutout image P3a (P3b) Alternatively, the background image P4a (P4b) including the feature region C associated with may be specified.

In addition, the configuration of the imaging apparatus 100 is merely an example of the configuration described in the above embodiment, and is not limited thereto. That is, although the imaging device 100 is illustrated as an image composition device, it is not limited to this. For example, the background image P4a (P4b) is captured and the subject cut-out image P3a (P3b) is generated by an imaging device different from the imaging device 100, and the image data transferred from the imaging device is recorded and combined. It may be an image composition device that executes only image generation processing.
In the above embodiment, the display unit 11 switches and reproduces the subject composite image P5a and the background image P4a generated by image synthesis at a predetermined interval. However, the display unit 11 does not necessarily switch and reproduce these images. It is not necessary to display at least the subject composite image P5a.
Furthermore, in the above embodiment, the dimension changing unit 8f that changes the dimensions of the subject image Ga and the background image P4a is provided. However, the present invention is not limited to this, and whether or not the dimension changing unit 8f is provided. Can be arbitrarily changed as appropriate.

In the above embodiment, the functions as the background specifying means, the detecting means, and the specifying means are realized by the image processing unit 8 being driven under the control of the central control unit 13. However, the present invention is not limited to this, and a configuration realized by executing a predetermined program or the like by the central control unit 13 may be adopted.
That is, a program including a background designation processing routine, a detection processing routine, and a specific processing routine is stored in a program memory (not shown) that stores the program. Then, the CPU of the central control unit 13 is caused to function as a background designating unit for designating any one of the background images P4a stored in the recording medium 9 by the background designating processing routine. Also good. Further, the CPU of the central control unit 13 may function as detection means for detecting the feature region C from the designated background image P4a by the detection processing routine. Further, the CPU of the central control unit 13 by the specific processing routine causes the subject image (foreground image) Ga associated with the detected feature area C to be selected from the plurality of subject images Ga stored in the recording medium 9. You may make it function as a specific means to specify.
A program memory (not shown) that stores the program stores a program including a foreground designation processing routine, a background designation processing routine, a detection processing routine, and a synthesis processing routine. Then, the CPU of the central control unit 13 functions as a foreground designation means for designating any one of the plurality of subject images (foreground images) Ga stored in the recording medium 9 by the foreground designation processing routine. You may make it let it. Further, the CPU of the central control unit 13 is caused to function as a background designating means for designating any one of the background images P4a stored in the recording medium 9 by the background designating processing routine. Also good. Further, the CPU of the central control unit 13 is made to correspond to the subject image (foreground image) Ga designated by the foreground designation processing routine from the background image P4a designated by the background designation processing routine by the detection processing routine. You may make it function as a detection means to detect. Further, the CPU of the central control unit 13 functions as a combining unit that combines the subject image (foreground image) Ga designated by the foreground designation processing routine with the region where the feature region C is detected in the background image P4a. You may make it let it.
Further, a program memory (not shown) for storing the program stores a program including a foreground designation processing routine and a synthesis processing routine. Then, the CPU of the central control unit 13 functions as a foreground designation means for designating any one of the plurality of subject images (foreground images) Ga stored in the recording medium 9 by the foreground designation processing routine. You may make it let it. Further, the CPU of the central control unit 13 performs a specific processing routine so that the background image P4a including the feature region C associated with the designated subject image (foreground image) Ga is stored in the recording medium 9. You may make it function as an identification means to identify from the background image P4a.

DESCRIPTION OF SYMBOLS 100 Imaging device 1 Lens part 2 Electronic imaging part 3 Imaging control part 8 Image processing part 8a Cutout image generation part 8b Feature area detection part 8c Face recognition part 8d Foreground specific part 8e Composition part 8f Size change part 8g Table storage part 9 Recording medium 12 Operation input part 12b Selection decision button 13 Central control part Ga, Gb Subject image P4a, P4b Background image

Claims (17)

In an image synthesizing apparatus provided with a synthesizing unit that synthesizes a foreground image with a background image,
Storage means for storing a plurality of background images and foreground images;
Background designating means for designating any one of the plurality of background images stored in the storage means;
Detecting means for detecting a feature region from the background image specified by the background specifying means;
Specifying means for specifying a foreground image associated with the feature region detected by the detecting means from among a plurality of foreground images stored in the storage means;
An image composition device comprising:   The image synthesizing apparatus according to claim 1, wherein the synthesizing unit synthesizes the foreground image specified by the specifying unit with an area of the background image in which a characteristic region is detected by the detecting unit. The specifying means specifies a plurality of the foreground images,
Foreground designating means for designating any one of the foreground images identified by the identifying means;
The synthesis means includes
3. The image composition apparatus according to claim 1, wherein the foreground image designated by the foreground designating unit is synthesized. The feature region is a person image region in the background image,
The image specifying apparatus according to claim 1, wherein the specifying unit specifies a foreground image including an image of the same person as the feature region. Storage means for storing a plurality of background images and foreground images;
Foreground designating means for designating any one foreground image from a plurality of background images stored in the storage means;
Background designating means for designating any one of the plurality of background images stored in the storage means;
Detecting means for detecting a feature region associated with the foreground image specified by the foreground specifying means from the background image specified by the background specifying means;
Synthesizing means for synthesizing the foreground image designated by the foreground designating means to the area of the background image in which the characteristic area is detected by the detecting means;
An image composition device comprising: The foreground image is a human image including a human face image;
6. The image composition apparatus according to claim 5, wherein the feature area is a face image area of the same person as the person in the background image. The synthesis means includes
The image synthesizing apparatus according to claim 6, wherein the synthesis is performed so that the face image area of the foreground image and the face image area of the background image overlap. In an image synthesizing apparatus provided with a synthesizing unit that synthesizes a foreground image with a background image,
Storage means for storing a plurality of background images and foreground images;
Foreground designating means for designating any one of the foreground images stored in the storage means;
Specifying means for specifying a background image including a feature region associated with the foreground image designated by the foreground designating means from a plurality of background images stored in the storage means;
An image composition device comprising: Further comprising changing means for changing the size of at least one of the foreground image and the background image based on the size of the foreground image synthesized by the synthesizing unit and the size of the feature region in the background image,
The image synthesizing apparatus according to claim 1, wherein the synthesizing unit synthesizes after changing at least one dimension of the foreground image and the background image by the changing unit.   The image synthesizing apparatus according to any one of claims 1 to 9, further comprising a reproducing unit that switches and reproduces the background image and the synthesized image synthesized by the synthesizing unit.   The image synthesizing apparatus according to claim 1, further comprising a registration unit that registers the foreground image and the feature region in association with each other. An image specifying method of an image composition device comprising: storage means for storing a plurality of background images and foreground images; and composition means for combining the background image and the foreground image,
A background designating step of designating any one of the plurality of background images stored in the storage means;
A detection step of detecting a feature region from the background image designated by the background designation step;
A specifying step for specifying a foreground image associated with the feature region detected by the detection step from among the plurality of foreground images stored in the storage unit;
An image specifying method comprising: An image composition method for an image composition apparatus comprising: storage means for storing a plurality of background images and foreground images; and composition means for combining the background image and the foreground image,
A foreground designating step of designating any one foreground image from a plurality of background images stored in the storage means;
A background designating step of designating any one of the plurality of background images stored in the storage means;
A detecting step for detecting a feature region associated with the foreground image specified by the foreground image specifying step from the background image specified by the background specifying step;
Synthesizing a foreground image designated by the foreground designating means with an area of the background image in which the characteristic area is detected by the detecting means;
An image composition method characterized by comprising: An image specifying method of an image composition device comprising: storage means for storing a plurality of background images and foreground images; and composition means for combining the background image and the foreground image,
A foreground designating step of designating any one of the foreground images stored in the storage means;
A specifying step of specifying a background image including a feature region associated with the foreground image specified in the foreground specifying step from a plurality of background images stored in the storage unit;
An image specifying method comprising: A computer of an image composition device comprising: storage means for storing a plurality of background images and foreground images; and composition means for combining the background image and the foreground image.
Background designating means for designating any one of the plurality of background images stored in the storage means;
Detecting means for detecting a feature region associated with the foreground image from the background image specified by the background specifying means;
A specifying means for specifying a foreground image associated with the feature region detected by the detecting means from among a plurality of foreground images stored in the storage means;
A program characterized by functioning as A computer of an image composition device comprising storage means for storing a plurality of background images and foreground images,
Foreground designating means for designating any one foreground image from a plurality of background images stored in the storage means;
Background designating means for designating any one of the plurality of background images stored in the storage means;
Detecting means for detecting a feature region associated with the foreground image specified by the foreground specifying means from the background image specified by the background specifying means;
Synthesizing means for synthesizing the foreground image designated by the foreground designating means with the area of the background image in which the characteristic area is detected by the detecting means;
A program characterized by functioning as A computer of an image composition apparatus comprising storage means for storing a plurality of background images and foreground images, and composition means for synthesizing the background image and the foreground image,
Foreground designating means for designating any one of the foreground images stored in the storage means;
Specifying means for specifying a background image including a feature region associated with the foreground image designated by the foreground designating means from a plurality of background images stored in the storage means;
A program characterized by functioning as

Images