JP6606935B2 - Image processing apparatus, image processing method, and program - Google Patents

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program.

  2. Description of the Related Art Conventionally, there is known a portrait creation device that creates a portrait using feature points of facial parts such as eyes, nose, mouth, ears, and face contour (see, for example, Patent Document 1). There is also known a game device that creates a character by combining part objects prepared in advance for each part (see, for example, Patent Document 2).

JP 2004-145625 A JP 2008-61896 A

  By the way, when a portrait image is automatically generated, a color or pattern is specified in advance for the background, or it is represented by a color or pattern selected by a user through a predetermined operation. There is a problem that the background becomes unsuitable for the content of the portrait image.

  An object of the present invention is to provide an image processing apparatus, an image processing method, and a program that can make the background of a newly generated face image appropriate.

In order to solve the above problems, an image processing apparatus according to the present invention provides:
A plurality of first component images corresponding to each of the plurality of face constituent parts, a first face image having a first background, a plurality of second part images corresponding to each of the plurality of face constituent parts, and a first one. First acquisition means for acquiring at least one of each of the first and second component images to generate a new face image based on a second face image having two backgrounds;
First background information for associating the first part image acquired by the first acquisition means with the first background, and the second background image for associating the acquired second part image with the second background. Background generating means for generating a background of the new face image based on the background information of 2;
It is characterized by having.

The image processing method according to the present invention includes:
An image processing method using an image processing apparatus,
A plurality of first component images corresponding to each of the plurality of face constituent parts, a first face image having a first background, a plurality of second part images corresponding to each of the plurality of face constituent parts, and a first one. A process of acquiring at least one of the first and second component images to generate a new face image based on a second face image having two backgrounds;
First background information that associates the acquired first part image with the first background, and second background information that associates the acquired second part image with the second background. Based on the process of generating a background of the new face image,
It is characterized by including.

The program according to the present invention is
The computer of the image processing device
A plurality of first component images corresponding to each of the plurality of face constituent parts, a first face image having a first background, a plurality of second part images corresponding to each of the plurality of face constituent parts, and a first one. First acquisition means for acquiring at least one of the first and second component images to generate a new face image based on a second face image having two backgrounds,
First background information for associating the first part image acquired by the first acquisition means with the first background, and the second background image for associating the acquired second part image with the second background. Background generating means for generating a background of the new face image based on the background information of 2;
It is characterized by making it function as.

  According to the present invention, the background of a newly generated face image can be made appropriate.

It is a block diagram which shows schematic structure of the imaging device of one Embodiment to which this invention is applied. It is a figure which shows typically an example of the portrait image memorize | stored in the imaging device of FIG. FIG. 2 is a diagram illustrating an example of a component importance level table stored in the imaging apparatus in FIG. 1. 3 is a flowchart illustrating an example of an operation related to a portrait image generation process performed by the imaging apparatus of FIG. 1. It is a figure for demonstrating the portrait image generation process of FIG. It is a block diagram which shows schematic structure of the modification of an imaging device.

  Hereinafter, specific embodiments of the present invention will be described with reference to the drawings. However, the scope of the invention is not limited to the illustrated examples.

FIG. 1 is a block diagram illustrating a schematic configuration of an imaging apparatus 100 according to an embodiment to which the present invention is applied.
As shown in FIG. 1, the imaging apparatus 100 according to the present embodiment specifically includes a central control unit 1, a memory 2, an imaging unit 3, an image data generation unit 4, an image processing unit 5, and a display. Unit 6, operation input unit 7, storage unit 8, and portrait processing unit 9.
The central control unit 1, the memory 2, the imaging unit 3, the image data generation unit 4, the image processing unit 5, the display unit 6, the operation input unit 7, the storage unit 8, and the portrait processing unit 9 are connected via a bus line 10. It is connected.

  Note that a known device can be applied to the imaging device 100, and not only a digital camera having a main function as a shooting function but also a mobile phone having a shooting function that is not a main function as in the present embodiment. And mobile terminals such as smartphones.

  The central control unit 1 controls each unit of the imaging device 100. Specifically, although not shown, the central control unit 1 includes a CPU (Central Processing Unit), a RAM (Random Access Memory), and a ROM (Read Only Memory), and includes various processing programs (illustrated) for the imaging apparatus 100. Various control operations are performed according to (omitted).

  The memory 2 is constituted by, for example, a DRAM (Dynamic Random Access Memory) or the like, and temporarily stores data processed by each unit such as the central control unit 1, the image processing unit 5, and the portrait processing unit 9.

  The imaging unit 3 captures a subject. Specifically, the imaging unit 3 includes a lens unit 3a, an electronic imaging unit 3b, and an imaging control unit 3c.

The lens unit 3a includes a plurality of lenses such as a zoom lens and a focus lens, for example.
The electronic imaging unit 3b is composed of, for example, an image sensor such as a charge coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS), and converts an optical image that has passed through various lenses of the lens unit 3a into a two-dimensional image signal. To do.
In addition, although illustration is abbreviate | omitted, the imaging part 3 may be provided with the aperture_diaphragm | restriction which adjusts the quantity of the light which passes the lens part 3a.

The imaging control unit 3 c controls the shooting of the subject by the imaging unit 3. That is, the imaging control unit 3c includes a timing generator, a driver, and the like, although not shown. Then, the imaging control unit 3c scans and drives the electronic imaging unit 3b with a timing generator and a driver, converts the optical image into a two-dimensional image signal with the electronic imaging unit 3b every predetermined period, and the electronic imaging unit 3b. The frame image is read from the imaging area for each screen and is output to the image data generation unit 4.
Note that the imaging control unit 3c may perform adjustment control of conditions when photographing a subject such as AF (automatic focusing process), AE (automatic exposure process), and AWB (automatic white balance).

  The image data generation unit 4 appropriately adjusts the gain for each color component of RGB with respect to the analog value signal of the frame image transferred from the electronic image pickup unit 3b of the image pickup unit 3, and then performs a sample with a sample hold circuit (not shown). After being held and converted into digital data by an A / D converter (not shown), color process processing including pixel interpolation processing and γ correction processing is performed by a color process circuit (not shown), and then a digital luminance signal Y And color difference signals Cb and Cr (YUV data) are generated. In addition, the image data generation unit 4 outputs the generated luminance signal Y and color difference signals Cb and Cr to the memory 2 used as a buffer memory.

When storing the image, the image processing unit 5 acquires the luminance signal Y and the color difference signals Cb and Cr from the memory 2 and uses a predetermined encoding method (for example, JPEG format, motion JPEG format, MPEG format, etc.). Encode and output to the storage unit 8.
When the image is reproduced and displayed, the image processing unit 5 decodes still image data or moving image image data read from the memory 2 or the storage unit 8 according to a predetermined encoding method. And output to the display unit 6. At this time, for example, the image processing unit 5 may reduce the image processing unit 5 to a predetermined size (for example, VGA or QVGA size) based on the display resolution of the display panel 6 b and output the reduced size to the display unit 6.

  The display unit 6 includes a display control unit 6a and a display panel 6b.

  The display control unit 6a performs control to display a predetermined image in the display area of the display panel 6b based on image data of a predetermined size read from the memory 2 or the storage unit 8 and decoded by the image processing unit 5. Specifically, the display control unit 6a includes a VRAM (Video Random Access Memory), a VRAM controller, a digital video encoder, and the like. The digital video encoder reads out the luminance signal Y and the color difference signals Cb and Cr decoded by the image processing unit 5 and stored in the VRAM (not shown) from the VRAM through the VRAM controller at a predetermined reproduction frame rate. Based on these data, a video signal is generated and output to the display panel 6b.

The display panel 6b displays an image taken by the imaging unit 3 in the display area based on the video signal from the display control unit 6a. Specifically, the display panel 6b performs live updating while sequentially updating a plurality of frame images generated by photographing an object by the imaging unit 3 in a still image shooting mode or a moving image shooting mode. Display the view image. The display panel 6b displays a still image (rec view image) of the subject stored in the storage unit 8, or displays a moving image of the subject being shot.
Examples of the display panel 6b include a liquid crystal display panel and an organic EL (Electro-Luminescence) display panel. However, the display panel 6b is an example and is not limited thereto.

The operation input unit 7 is for performing a predetermined operation of the imaging apparatus 100. Specifically, the operation input unit 7 includes a shutter button related to an imaging instruction of a subject, a selection determination button related to a selection instruction for a shooting mode and a function, a zoom button related to a zoom amount adjustment instruction, and the like (all not shown). The operation part is provided.
When various buttons are operated by the user, the operation input unit 7 outputs an operation instruction corresponding to the operated button to the central control unit 1. The central control unit 1 causes each unit to execute a predetermined operation (for example, photographing a subject) according to an operation instruction output from the operation input unit 7 and input.

  The storage unit 8 is configured by, for example, a non-volatile memory (flash memory) or the like, and stores still image and image data for storing moving images encoded by the image processing unit 5 using a predetermined encoding method.

In addition, the storage unit 8 stores a portrait image A drawn by each portrait creator.
The caricature image A is data configured by combining a plurality of component images p,... Related to main face constituent parts such as eyes, nose, mouth, eyebrows, and face contours with a background image. Specifically, for example, a first portrait image A1 drawn by a first portrait painter, a second portrait image A2 drawn by a second portrait artist, and a third portrait image drawn by a third portrait artist A3 etc. are mentioned, but it is an example and it is not restricted to this (refer to Drawing 2 (a)-Drawing 2 (c)).
Further, the background of each portrait image A represents, for example, a color or pattern designated by each portrait artist. Specifically, for example, the first portrait image A1 has a blue background, the second portrait image A2 has a yellow background, and the third portrait image A3 has a green background. It is not limited.

Note that the first portrait image A1, the second portrait image A2, and the third portrait image A3 may be drawn by one portrait creator. For example, the fourth portrait creator may draw the first to third portrait images A1 to A3 of the same subject. For example, the portrait image A when the subject is happy is used as the first portrait image. The portrait image A when the subject is laughing may be the second portrait image A2, and the portrait image A when the subject is angry may be the third portrait image A3.
In this case, when creating a new portrait image Q using a plurality of portrait images A,..., A certain style of the portrait artist is obtained by using the plurality of portrait images A,... Drawn by one portrait artist. Can be realized.
In FIG. 2A to FIG. 2C, the difference in the background color is schematically represented by the difference in the pattern (pattern).

The storage unit 8 stores a component importance level table T.
The component importance degree table T defines the importance of the face constituting unit related to the component image p used for generating the portrait image Q. Specifically, the name of the face constituent part related to the component image p, the importance of the face constituent part, and the evaluation value used for generating the background of the portrait image Q are associated (see FIG. 3).
Here, for example, the importance is higher as the number is smaller, and is expressed in four levels. In addition, for example, the evaluation value increases as the number increases.
Note that the content of the component importance degree table T described above is an example and is not limited thereto, and can be arbitrarily changed as appropriate.

  The storage unit 8 may be configured, for example, such that a storage medium (not shown) is detachable and controls reading of data from the attached storage medium and writing of data to the storage medium.

The portrait processing unit 9 includes an original image acquisition unit 9a, a component image acquisition unit 9b, a related information acquisition unit 9c, a background generation unit 9d, and a portrait image generation unit 9e.
Each unit of the portrait processing unit 9 is configured by a predetermined logic circuit, for example, but the configuration is an example and the present invention is not limited thereto.

The original image acquisition unit 9a acquires an original image (not shown) to be processed in the portrait image generation process.
That is, the original image acquisition unit 9a, for example, image data (RGB data or YUV) of a still image (for example, a photographic image) generated by the image data generation unit 4 when the imaging unit 3 captures the face of the subject. Data) is acquired from the memory 2, or a copy of still image data stored in the storage unit 8 is acquired as an original image. Then, the original image acquisition unit 9a uses the acquired still image as an original image, performs a predetermined face detection process on the image data, and detects a face area.
Here, since the face detection process is a known technique, a detailed description thereof is omitted here.

The original image acquisition unit 9a is one frame image that is automatically specified from a plurality of frame images constituting the moving image, or one specified based on a predetermined operation of the operation input unit 7 by the user. The frame image may be acquired as the original image.
Each process by the portrait processing unit 9 to be described later may be performed on the image data of the original image itself, or a predetermined size (for example, reduced by a predetermined ratio of the image data of the original image as necessary). It may be performed on reduced image data of VGA size or the like.

The component image acquisition unit 9b acquires a component image p.
That is, the component image acquisition unit 9b acquires a plurality of component images p corresponding to a plurality of face constituting units from a plurality of portrait images (processed face images) A,. Specifically, the component image acquisition unit (first acquisition unit) 9b includes a plurality of first component images (for example, the component image p1) corresponding to each of the plurality of face constituting units and a first background (for example, , A blue background, etc.) and a plurality of second component images (for example, a component image p2) corresponding to each of the plurality of face constituent parts. And a first portrait image Q to generate a new portrait image Q based on a second face image (eg, second portrait image A2) having a second background (eg, yellow background). Each of the two component images is acquired.
Here, the first face image and the second face image may be different face images. Further, a new portrait image Q may be generated from three or more different face images (for example, the first to third portrait images A1 to A3, etc.).

For example, the component image acquisition unit 9b performs a predetermined face detection process on the image data of the original image acquired by the original image acquisition unit 9a to detect a face area, and uses, for example, AAM (Active Appearance Model). The main face components such as eyes, nose, mouth, eyebrows, and face contours are detected by the processing performed. Here, AAM is a technique for modeling a visual event, and is a process for modeling an image of an arbitrary face region. For example, the component image acquisition unit 9b performs predetermined registration of statistical analysis results of positions and pixel values (for example, luminance values) of predetermined characteristic parts (for example, eyes, nasal heads, and face lines) in a plurality of sample face images. Register with the means. Then, the component image acquisition unit 9b sets a shape model representing the shape of the face and a texture model representing “Appearance” in the average shape with reference to the position of the characteristic part, and uses these models to Model the image of the region. As a result, the face components such as eyes, nose, mouth, eyebrows and face contour are modeled in the original image.
And the component image acquisition part 9b specifies the component image p corresponding to each face structure part from the several portrait image A and ... memorize | stored in the memory | storage part 8 for every detected main face structure part. . Here, as a method for specifying the component image p corresponding to the face constituting unit, the following method may be mentioned. That is, for example, the component image acquisition unit 9b sets control points for each of the component images p constituting each face component and each portrait image A, and the distance between these control points is minimized by the least square method. The parameter is calculated, and the sum of the distances between the control points with the calculated parameter is set as the evaluation value of the component image p. The component image acquisition unit 9b compares the calculated evaluation values of the plurality of component images p,... To identify the component image p that most closely matches each face component. For example, the component image acquisition unit 9b identifies the component image p1 of the face contour that constitutes the first portrait image A1 as the face contour, and identifies the component image p2 of the eyebrow that constitutes the second portrait image A2 as the eyebrow. The eye part image p3 constituting the third portrait image A3 is specified as eyes, the nose part image p1 constituting the first portrait image A1 is specified as the nose, and the second portrait image A2 is specified as the mouth. The component image p2 of the mouth to be configured is specified (see FIG. 5A).

Since the face detection process is a known technique, detailed description thereof is omitted here. In addition, detection of the face component using AAM is an example, but the present invention is not limited to this. For example, edge extraction processing, anisotropic diffusion processing, template matching, etc. can be arbitrarily changed. It is.
In addition, the component image p corresponding to each face component is automatically specified. However, this is only an example and the present invention is not limited to this. For example, a plurality of images may be selected based on a predetermined operation of the operation input unit 7 by the user. The part image p desired by the user may be designated from the part images p,.

The related information acquisition unit 9c acquires related information.
That is, the related information acquisition unit 9c acquires related information that associates the component image p acquired by the component image acquisition unit 9b with the background.
Here, each of the plurality of portrait images (first and second face images) A,... Stored in the storage unit 8 is respectively a plurality of component images (first and second component images) p. The background (first and second background) is associated with the background information (first and second background information). That is, for example, each of the plurality of component images p1 constituting the first portrait image A1 is associated with the background information relating to the background color “blue” and the like, and the plurality of component images p2 constituting the second portrait image A2 are associated with each other. Background information related to the background color “yellow” or the like is associated with each other, and background information related to the background color “green” or the like is associated with each of the plurality of component images p3 constituting the third portrait image A3.
Specifically, the related information acquisition unit 9c is a portrait image corresponding to the component image p acquired by the component image acquisition unit 9b, that is, the background color or pattern of the portrait image in which the component image p is a component part. Etc. (for example, in the case of the component image p of the face outline, the background color “blue” etc. of the first portrait image A1) is acquired as related information. Then, the related information acquisition unit 9c registers all the acquired related information together in the parts list L (see FIG. 5B).

The background generation unit 9d generates a background of the portrait image Q.
That is, the background generation unit (background generation unit) 9d acquires the first component image (for example, the component image p1) and the first background (for example, the blue background) acquired as the related information by the related information acquisition unit 9c. Etc.) and second background information for associating the acquired second component image (for example, component image p2) with the second background (for example, yellow background). Based on the above, the background of the new portrait image Q generated using the component image p acquired by the component image acquisition unit 9b is generated.
Here, the first background information and the second background information may correspond to different face images. Further, a background of a new portrait image Q may be generated based on background information corresponding to three or more different face images (for example, first to third portrait images A1 to A3).

Specifically, the background generation unit 9d reads and acquires the component importance degree table T from the storage unit 8, and acquires a plurality of pieces of related information (first and second background information) acquired by the related information acquisition unit 9c. For each of the component images p associated with each other, the importance level of the corresponding face component is specified. Further, the background generation unit 9d refers to the component importance degree table T, identifies an evaluation value corresponding to the identified importance degree, and registers it in the parts list L. Then, the background generation unit 9d specifies and generates a background color or pattern of the portrait image Q generated using the component image p based on the specified evaluation value.
For example, in the case of the face contour component image p, the background generation unit 9d refers to the component importance level table T, identifies “3” as the importance level of the face contour, and identifies “2” as the evaluation value. To do. Then, the background generation unit 9d registers the specified evaluation value “2” in the parts list. In addition, the background generation unit 9d performs the same process on each of the component images p of other face constituent units (for example, eyes, nose, mouth, eyebrows, etc.).
Then, the background generation unit 9d performs a predetermined calculation (for example, addition) on the basis of the specified evaluation value for each background information (background color, etc.) of the portrait image corresponding to each component image p. The total evaluation value of information is calculated. For example, in the case of the parts list L shown in FIG. 5B, for the background color “blue” of the first portrait image A1, the face outline evaluation value “2” and the nose evaluation value “1” are added. The total evaluation value “3”, and for the background color “yellow” of the second portrait image A2, the eyebrow evaluation value “2” and the mouth evaluation value “5” are added to obtain a total evaluation value “7”. Regarding the background color “green” of the third portrait image A3, the total evaluation value “8” is set from only the eye evaluation value “8”.
Then, the background generation unit 9d compares the calculated total evaluation values and specifies the background color “green” of the third portrait image A3 related to the background information having the highest total evaluation value as the background color of the portrait image Q. .

The portrait image generation unit 9e generates a portrait image Q.
That is, the portrait image generation unit (portrait generation unit) 9e uses the component images (first and second component images) p acquired by the component image acquisition unit 9b and the background generated by the background generation unit 9d, A portrait image Q is generated as a new face image. Specifically, the portrait image generation unit 9e deforms the component image p so that the control point of each component image p acquired by the component image acquisition unit 9b corresponds to the control point of each face constituent unit, and a predetermined image In addition to combining with the hairstyle image, a portrait image Q is generated with the face outline and the outside of the hairstyle image as the background color “green” specified by the background generation unit 9d (see FIG. 5C).
Here, the portrait image generation unit 9e may generate an image that represents a predetermined part of the portrait image Q (for example, a face component such as eyes, mouth, and eyebrows) with a predetermined color.

<Caricature image generation processing>
Next, a portrait image generation process performed by the imaging apparatus 100 will be described with reference to FIGS. 4 and 5.
FIG. 4 is a flowchart illustrating an example of an operation related to the portrait image generation process. FIG. 5A schematically shows an example of a part image p of each face component acquired by the part image acquisition unit 9b, and FIG. 5B shows an example of a parts list L. 5 (c) schematically shows an example of the portrait image Q.

  The portrait image generation processing is performed when the portrait image generation mode is selected from a plurality of operation modes displayed on the menu screen based on a predetermined operation of the selection determination button of the operation input unit 7 by the user. This process is executed by each unit of the imaging apparatus 100, particularly the portrait processing unit 9, under the control of the unit 1.

  As shown in FIG. 4, first, the original image acquisition unit 9a acquires an original image (not shown) to be processed in the portrait image generation process (step S1). Specifically, the original image acquisition unit 9a, for example, copies from the storage unit 8 the image data (RGB data or YUV data) of the still image generated by the image data generation unit 4 by photographing the subject by the imaging unit 3. get.

Next, the component image acquisition unit 9b acquires a component image p corresponding to the face constituent unit in the original image acquired by the original image acquisition unit 9a (step S2). Specifically, the component image acquisition unit 9b first detects main face components such as eyes, nose, mouth, eyebrows, and face contours, for example, by processing using AAM in the face area of the original image. To do. Next, the component image acquisition unit 9b specifies a component image p corresponding to each of the detected main face constituent units from the plurality of portrait images A,... Stored in the storage unit 8 (FIG. 5 ( a)).
Then, the related information acquisition unit 9c outputs background information of the portrait image corresponding to the component image p acquired by the component image acquisition unit 9b (for example, in the case of the component image p having a face outline, the background of the first portrait image A1). Color “blue” or the like) is acquired as related information (step S3). The related information acquisition unit 9c collectively registers all the acquired background information in the parts list L (see FIG. 5B).

Next, the background generation unit 9d reads and acquires the component importance degree table T from the storage unit 8, and for each of the component images p associated with each of the plurality of background information acquired by the related information acquisition unit 9c. After identifying the importance level of the corresponding face component, an evaluation value corresponding to the identified importance level is identified and registered in the parts list L (step S4).
Subsequently, the background generation unit 9d performs a predetermined calculation (for example, addition) on the basis of the specified evaluation value for each background information (background color, etc.) of the portrait image corresponding to each component image p. A total evaluation value of each background information is calculated (step S5). For example, in the case of the parts list L shown in FIG. 5B, the background color “blue” of the first portrait image A1 becomes the added total evaluation value “3”, and the background color “yellow” of the second portrait image A2. Is the total evaluation value “7” added, and the background color “green” of the third portrait image A3 is the total evaluation value “8” added.
Then, the background generation unit 9d compares the total evaluation values calculated for each background information, and determines the portrait image from the background information having the highest total evaluation value (for example, the background color “green” of the third portrait image A3). The background color of Q is specified (step S6).

Thereafter, the portrait image generation unit 9e combines each component image p acquired by the component image acquisition unit 9b with a predetermined hairstyle image, and the background color (outside of the face outline and the hairstyle image specified by the background generation unit 9d ( For example, a portrait image Q is generated as “green” (step S7).
Thereby, the portrait image generation process ends.

As described above, according to the imaging apparatus 100 of the present embodiment, the first component image (for example, the component image p1) acquired from the first face image (for example, the first portrait image A1) and the first image. The first background information that associates the first background, the second part image (for example, the part image p2) obtained from the second face image (for example, the second portrait image A2), and the second background. Since the background of a new face image (for example, a portrait image Q) is generated based on the second background information that associates with the image, the background of the image generated using the component image p is generated Can be generated by using background information associated with each of the component images p used in the above. That is, the background color or pattern of the portrait image (for example, the third portrait image A3) corresponding to the component image p used for generating the portrait image Q is set as the background color or pattern of the newly generated portrait image Q. And the expressive power of the portrait image Q can be improved.
Thereby, the background of the face image (for example, portrait image Q etc.) newly produced | generated using the component image p can be made appropriate.

  Further, the component image p is based on the importance of the face constituent unit related to the component image p associated by each of the plurality of background information (first and second background information) acquired by the related information acquisition unit 9c. The background of a new face image generated using can be appropriately generated. Specifically, the evaluation value according to the importance of the face constituent part related to the component image p associated by each of the plurality of background information acquired by the related information acquisition unit 9c is specified, and the specified evaluation value Since the background of a new face image generated using the component image p is generated based on the above, the portrait image Q (for example, the first to third portrait images A1 to A3) is used to generate the portrait image Q. Even when a plurality of component images p having different components are used, an evaluation value can be specified in consideration of the importance of the face constituent part related to each component image p. The background of the generated portrait image Q can be generated more appropriately.

The present invention is not limited to the above-described embodiment, and various improvements and design changes may be made without departing from the spirit of the present invention.
Below, the modification of the imaging device 100 is demonstrated with reference to FIG.

The imaging apparatus 100A of this modification generates the background of the portrait image Q based on the original information of the original image in addition to the related information such as background information.
Specifically, as illustrated in FIG. 6, the portrait processing unit 109 includes an original image acquisition unit 9a, a component image acquisition unit 9b, a related information acquisition unit 9c, a background generation unit 9d, and a portrait image generation unit 9e. In addition to the above, a personal information acquisition unit 9f is provided.
Except for the points described below, the configuration and functions are substantially the same as those of the imaging apparatus 100 of the above-described embodiment, and the detailed description thereof is omitted.

The principal information acquisition unit 9f acquires the principal information of the original image.
That is, the principal information acquisition unit (second acquisition unit) 9f generates an original image (original face image) that is a source of a new portrait image Q generated using the component image p acquired by the component image acquisition unit 9b. Obtain identity information.
Here, the identity information includes age, sex, clothes, facial expression (emotion), etc., but is not limited to this example, and can be arbitrarily changed as appropriate.
The principal information acquisition unit 9f performs various kinds of image processing on the original image acquired by the original image acquisition unit 9a, and identifies the personal information such as the person's age, sex, clothes, facial expression (emotion), and the like. . Note that identification of personal information by image processing is a known technique, and detailed description thereof is omitted here.

Then, the background generation unit 9d acquires the component image based on the plurality of background information (first and second background information) acquired by the related information acquisition unit 9c and the user information acquired by the user information acquisition unit 9f. The background of the portrait image Q generated using the component image p acquired by the unit 9b is generated.
Specifically, when the principal information acquisition unit 9f acquires, for example, the age of the original image as the principal information, the background generation unit 9d displays the background color corresponding to the acquired age and the age including the age. (For example, yellow until 19 years old, orange when 20-39 years old, green when 40-59 years old, purple when over 60 years old, etc.). Subsequently, the background generation unit 9d increases the weight of the background color specified corresponding to the age and the age among the background information (background color and the like) acquired by the related information acquisition unit 9c, and sets each background information The total evaluation value of is calculated. Then, the background generation unit 9d compares the calculated total evaluation values and specifies the background color related to the background information having the highest total evaluation value as the background color of the portrait image Q.

  In addition, although the age of the person of the original image was illustrated as person information, it is an example and is not restricted to this, It can change arbitrarily arbitrarily. For example, when the personal information is the gender of the original image, the background generation unit 9d specifies a background color (for example, blue for men and red for women) corresponding to the acquired gender, and the personal information is In the case of the original image's clothes, the background color corresponding to the acquired clothes (for example, dark color for formal, light color for casual etc.) is specified, and the personal information is the facial expression of the original image. In this case, the background color corresponding to the acquired facial expression (for example, red if angry, yellow if happy) is specified.

Therefore, according to the imaging apparatus 100A of this modification, the background of the image generated using the component image p is not limited to the background information associated with the component image p used to generate the image, It can also be generated using the identity information of the original image. That is, for example, the background color and pattern of the newly generated portrait image Q can be generated in consideration of the age, sex, clothes, facial expression (emotion), etc. of the original image. Expressive power can be improved.
Thereby, the background of the face image (for example, portrait image Q etc.) newly produced | generated using the component image p can be made more appropriate.

  In the above modification, only one of the background information acquired by the related information acquisition unit 9c and the original information of the original image acquired by the personal information acquisition unit 9f is used as a reference. The background of the portrait image Q may be generated by switching. That is, the background generation unit 9d may generate the background color or pattern of the portrait image Q based only on the original information of the original image acquired by the personal information acquisition unit 9f.

  Further, in the above embodiment, for example, the date and time when the portrait image Q is generated, the operation state of the imaging device 100 (100A) by the user, and various SNSs (Social Networking Services) managed by the user. The background color and pattern of the portrait image Q may be generated in consideration of the situation when generating the portrait image Q such as the content of the article, the shooting date and time of the original image, the shooting location, and the like.

  In the above embodiment, the portrait image Q is generated. However, it is not always necessary to generate the portrait image Q, and whether or not the portrait image generation unit 9e is provided can be arbitrarily changed as appropriate. is there.

  Furthermore, in the said embodiment, although 1st-3rd portrait image A1-A3 and portrait image Q were illustrated as a processed face image, it is an example and it is not restricted to this, For example, two It is possible to arbitrarily change the image subjected to the value processing, the image subjected to the edge extraction processing, and the like as appropriate.

In the above embodiment, the portrait image A for each portrait artist is stored in the storage unit 8, but is not limited to this example. For example, for each portrait artist, The component images p of the main face constituent parts of the face may be stored in association with each other.
Note that the background of the portrait image A may be related to the emotion (expression) of the subject of the original face image of the portrait image. For example, the background of a caricature image expressing sadness uses dark colors (colors with low saturation and lightness), or the background of a caricature image expressing joy uses light colors (colors with high saturation and lightness). May be.

  Furthermore, the configuration of the imaging apparatus 100 is merely an example illustrated in the above embodiment, and is not limited thereto. Further, although the imaging apparatus 100 has been illustrated as an image processing apparatus, the present invention is not limited to this, and any configuration may be used as long as the image processing according to the present invention can be executed.

In addition, in the above embodiment, the function as the first acquisition unit and the background generation unit is driven by the component image acquisition unit 9b and the background generation unit 9d under the control of the CPU of the central control unit 1. However, the present invention is not limited to this, and may be realized by executing a predetermined program or the like by the CPU of the central control unit 1.
That is, a program including a first acquisition process routine and a background generation process routine is stored in a program memory (not shown) that stores the program. Then, the first acquisition processing routine causes the CPU of the central control unit 1 to have a plurality of first component images (for example, component images p1 and the like) corresponding to each of the plurality of face constituting units and a first background having a first background. Face image (for example, first caricature image A1), a plurality of second component images (for example, component image p2) corresponding to each of the plurality of face components, and a second background having a second background In order to generate a new portrait image Q based on the face image (for example, the second portrait image A2 or the like), it may function as a means for acquiring at least one of the first and second component images. good. In addition, the CPU of the central control unit 1 in the background generation processing routine causes the first background information to associate the acquired first component image with the first background, the acquired second component image, and the second You may make it function as a means to produce | generate the background of a new portrait image based on 2nd background information linked | related with a background.

  Similarly, the second acquisition unit and the portrait generation unit may be realized by executing a predetermined program or the like by the CPU of the central control unit 1.

  Furthermore, as a computer-readable medium storing a program for executing each of the above processes, a non-volatile memory such as a flash memory or a portable recording medium such as a CD-ROM is applied in addition to a ROM or a hard disk. Is also possible. A carrier wave is also used as a medium for providing program data via a predetermined communication line.

Although several embodiments of the present invention have been described, the scope of the present invention is not limited to the above-described embodiments, but includes the scope of the invention described in the claims and equivalents thereof.
The invention described in the scope of claims attached to the application of this application will be added below. The item numbers of the claims described in the appendix are as set forth in the claims attached to the application of this application.
[Appendix]
<Claim 1>
A plurality of first component images corresponding to each of the plurality of face constituent parts, a first face image having a first background, a plurality of second part images corresponding to each of the plurality of face constituent parts, and a first one. First acquisition means for acquiring at least one of each of the first and second component images to generate a new face image based on a second face image having two backgrounds;
First background information for associating the first part image acquired by the first acquisition means with the first background, and the second background image for associating the acquired second part image with the second background. Background generating means for generating a background of the new face image based on the background information of 2;
An image processing apparatus comprising:
<Claim 2>
The background generation means is based on the importance of the first and second face constituent parts related to the first and second component images associated with each of the first and second background information. The image processing apparatus according to claim 1, wherein a background of the new face image is generated.
<Claim 3>
The background generation means specifies and specifies an evaluation value according to the importance of the face constituent unit related to the first and second component images associated with each of the first and second background information. The image processing apparatus according to claim 1, wherein a background of the new face image is generated based on the evaluated value.
<Claim 4>
A second acquisition means for acquiring identity information of the original face image that is the source of the new face image;
The background generation means generates the background of the new face image based on the first and second background information and the identity information acquired by the second acquisition means. 4. The image processing device according to any one of 3.
<Claim 5>
5. A portrait generating unit that generates a portrait image as the new face image using the first and second component images and the background generated by the background generating unit. The image processing apparatus according to any one of the above.
<Claim 6>
An image processing method using an image processing apparatus,
A plurality of first component images corresponding to each of the plurality of face constituent parts, a first face image having a first background, a plurality of second part images corresponding to each of the plurality of face constituent parts, and a first one. A process of acquiring at least one of the first and second component images to generate a new face image based on a second face image having two backgrounds;
First background information that associates the acquired first part image with the first background, and second background information that associates the acquired second part image with the second background. Based on the process of generating a background of the new face image,
An image processing method comprising:
<Claim 7>
The computer of the image processing device
A plurality of first component images corresponding to each of the plurality of face constituent parts, a first face image having a first background, a plurality of second part images corresponding to each of the plurality of face constituent parts, and a first one. First acquisition means for acquiring at least one of the first and second component images to generate a new face image based on a second face image having two backgrounds,
First background information for associating the first part image acquired by the first acquisition means with the first background, and the second background image for associating the acquired second part image with the second background. Background generating means for generating a background of the new face image based on the background information of 2;
A program characterized by functioning as

100, 100A Imaging device 1 Central control unit 8 Storage unit 8a Component image 9, 109 Caricature processing unit 9a Original image acquisition unit 9b Component image acquisition unit 9c Related information acquisition unit 9d Background generation unit 9e Caricature image generation unit 9f Personal information acquisition unit A1 first portrait image A2 second portrait image A3 third portrait image Q portrait image

Claims (7)

A plurality of first component images corresponding to each of the plurality of face constituent parts, a first face image having a first background, a plurality of second part images corresponding to each of the plurality of face constituent parts, and a first one. First acquisition means for acquiring at least one of each of the first and second component images to generate a new face image based on a second face image having two backgrounds;
First background information for associating the first part image acquired by the first acquisition means with the first background, and the second background image for associating the acquired second part image with the second background. Background generating means for generating a background of the new face image based on the background information of 2;
An image processing apparatus comprising:   The background generation means is based on the importance of the first and second face constituent parts related to the first and second component images associated with each of the first and second background information. The image processing apparatus according to claim 1, wherein a background of the new face image is generated.   The background generation means specifies and specifies an evaluation value according to the importance of the face constituent unit related to the first and second component images associated with each of the first and second background information. The image processing apparatus according to claim 1, wherein a background of the new face image is generated based on the evaluated value. A second acquisition means for acquiring identity information of the original face image that is the source of the new face image;
The background generation means generates the background of the new face image based on the first and second background information and the identity information acquired by the second acquisition means. 4. The image processing device according to any one of 3.   5. A portrait generating unit that generates a portrait image as the new face image using the first and second component images and the background generated by the background generating unit. The image processing apparatus according to any one of the above. An image processing method using an image processing apparatus,
A plurality of first component images corresponding to each of the plurality of face constituent parts, a first face image having a first background, a plurality of second part images corresponding to each of the plurality of face constituent parts, and a first one. A process of acquiring at least one of the first and second component images to generate a new face image based on a second face image having two backgrounds;
First background information that associates the acquired first part image with the first background, and second background information that associates the acquired second part image with the second background. Based on the process of generating a background of the new face image,
An image processing method comprising: The computer of the image processing device
A plurality of first component images corresponding to each of the plurality of face constituent parts, a first face image having a first background, a plurality of second part images corresponding to each of the plurality of face constituent parts, and a first one. First acquisition means for acquiring at least one of the first and second component images to generate a new face image based on a second face image having two backgrounds,
First background information for associating the first part image acquired by the first acquisition means with the first background, and the second background image for associating the acquired second part image with the second background. Background generating means for generating a background of the new face image based on the background information of 2;
A program characterized by functioning as

Images