JP2017054406A - Image processing device, image processing method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To simply correct a portion of a face constitution part of a processed face image.SOLUTION: An imaging device 100 includes: a face constitution part specification part 9c for specifying a face constitution part designated by a user from either one of a first portrait image subjected to image processing with a first face image as an object or the first face image; a second image acquisition part 9d for acquiring a second face image including the specified face constitution part; and a portrait image correction part 9f for correcting a face constitution part in the first portrait image on the basis of a face constitution part image corresponding to the face constitution part in the acquired second face image.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program.

  2. Description of the Related Art Conventionally, there is known a portrait creation device that creates a portrait using feature points of facial parts such as eyes, nose, mouth, ears, and face contour (see, for example, Patent Document 1). There is also known a game device that creates a character by combining part objects prepared in advance for each part (see, for example, Patent Document 2).

JP 2004-145625 A JP 2008-61896 A

By the way, if you don't like some of the facial features in the caricature image that was generated based on the photo image, you would like to correct it if you generate a caricature image based on the entire newly acquired face image by taking a photo again There is a problem that parts other than the face constituting part are also changed.
Although a method in which the user manually selects a component image corresponding to the face component to be corrected is conceivable, it is complicated to select from many component images, and the shape and size desired by the user are not necessarily required. There is also a problem that it is not always possible to select the part image.

  An object of the present invention is to provide an image processing apparatus, an image processing method, and a program capable of easily correcting a face configuration part of a part of a processed face image.

In order to solve the above problems, an image processing apparatus according to the present invention provides:
A specifying means for specifying a face component designated by a user from either one of the processed face image subjected to image processing for the first unprocessed face image and the first unprocessed face image;
Obtaining means for obtaining a second unprocessed face image including the face constituting unit identified by the identifying means;
Correcting means for correcting the face constituent part in the processed face image based on a face constituent part image corresponding to the face constituent part in the second unprocessed face image acquired by the acquiring means;
It is characterized by having.

The image processing method according to the present invention includes:
An image processing method using an image processing apparatus,
Identifying a face component designated by the user from either one of the processed face image subjected to image processing on the first unprocessed face image and the first unprocessed face image;
Obtaining a second unprocessed face image including the identified face component;
Correcting the face component in the processed face image based on a face component image corresponding to the face component in the acquired second unprocessed face image;
It is characterized by including.

The program according to the present invention is
The computer of the image processing device
A specifying means for specifying a face component designated by a user from either one of the processed face image subjected to image processing for the first unprocessed face image and the first unprocessed face image;
Obtaining means for obtaining a second unprocessed face image including the face constituting part specified by the specifying means;
Correcting means for correcting the face constituent part in the processed face image based on a face constituent part image corresponding to the face constituent part in the second unprocessed face image acquired by the acquiring means;
It is characterized by making it function as.

  According to the present invention, it is possible to easily perform correction of a face constituent part of a part of a processed face image.

It is a block diagram which shows schematic structure of the imaging device of one Embodiment to which this invention is applied. 3 is a flowchart illustrating an example of an operation related to a portrait image generation process performed by the imaging apparatus of FIG. 1. It is a figure for demonstrating the portrait image generation process of FIG. It is a figure for demonstrating the portrait image generation process of FIG.

  Hereinafter, specific embodiments of the present invention will be described with reference to the drawings. However, the scope of the invention is not limited to the illustrated examples.

FIG. 1 is a block diagram illustrating a schematic configuration of an imaging apparatus 100 according to an embodiment to which the present invention is applied.
As shown in FIG. 1, the imaging apparatus 100 according to the present embodiment specifically includes a central control unit 1, a memory 2, an imaging unit 3, an image data generation unit 4, an image processing unit 5, and a display. Unit 6, operation input unit 7, storage unit 8, and portrait processing unit 9.
The central control unit 1, the memory 2, the imaging unit 3, the image data generation unit 4, the image processing unit 5, the display unit 6, the operation input unit 7, the storage unit 8, and the portrait processing unit 9 are connected via a bus line 10. It is connected.

  Note that a known device can be applied to the imaging device 100, and not only a digital camera having a main function as a shooting function but also a mobile phone having a shooting function that is not a main function as in the present embodiment. And mobile terminals such as smartphones.

  The central control unit 1 controls each unit of the imaging device 100. Specifically, although not shown, the central control unit 1 includes a CPU (Central Processing Unit), a RAM (Random Access Memory), and a ROM (Read Only Memory), and includes various processing programs (illustrated) for the imaging apparatus 100. Various control operations are performed according to (omitted).

  The memory 2 is constituted by, for example, a DRAM (Dynamic Random Access Memory) or the like, and temporarily stores data processed by each unit such as the central control unit 1, the image processing unit 5, and the portrait processing unit 9.

  The imaging unit 3 captures a subject. Specifically, the imaging unit 3 includes a lens unit 3a, an electronic imaging unit 3b, and an imaging control unit 3c.

The lens unit 3a includes a plurality of lenses such as a zoom lens and a focus lens, for example.
The electronic imaging unit 3b is composed of an image sensor such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal-Oxide Semiconductor), and converts an optical image that has passed through various lenses of the lens unit 3a into a two-dimensional image signal. To do.
In addition, although illustration is abbreviate | omitted, the imaging part 3 may be provided with the aperture_diaphragm | restriction which adjusts the quantity of the light which passes the lens part 3a.

The imaging control unit 3 c controls the shooting of the subject by the imaging unit 3. That is, the imaging control unit 3c includes a timing generator, a driver, and the like, although not shown. Then, the imaging control unit 3c scans and drives the electronic imaging unit 3b with a timing generator and a driver, converts the optical image into a two-dimensional image signal with the electronic imaging unit 3b every predetermined period, and the electronic imaging unit 3b. The frame image is read from the imaging area for each screen and is output to the image data generation unit 4.
Note that the imaging control unit 3c may perform adjustment control of conditions when photographing a subject such as AF (automatic focusing process), AE (automatic exposure process), and AWB (automatic white balance).

  The image data generation unit 4 appropriately adjusts the gain for each color component of RGB with respect to the analog value signal of the frame image transferred from the electronic image pickup unit 3b of the image pickup unit 3, and then performs a sample with a sample hold circuit (not shown). After being held and converted into digital data by an A / D converter (not shown), color process processing including pixel interpolation processing and γ correction processing is performed by a color process circuit (not shown), and then a digital luminance signal Y And color difference signals Cb and Cr (YUV data) are generated. In addition, the image data generation unit 4 outputs the generated luminance signal Y and color difference signals Cb and Cr to the memory 2 used as a buffer memory.

When storing the image, the image processing unit 5 acquires the luminance signal Y and the color difference signals Cb and Cr from the memory 2 and uses a predetermined encoding method (for example, JPEG format, motion JPEG format, MPEG format, etc.). Encode and output to the storage unit 8.
When the image is reproduced and displayed, the image processing unit 5 decodes still image data or moving image image data read from the memory 2 or the storage unit 8 according to a predetermined encoding method. And output to the display unit 6. At this time, for example, the image processing unit 5 may reduce the image processing unit 5 to a predetermined size (for example, VGA or QVGA size) based on the display resolution of the display panel 6 b and output the reduced size to the display unit 6.

  The display unit 6 includes a display control unit 6a and a display panel 6b.

  The display control unit 6a performs control to display a predetermined image in the display area of the display panel 6b based on image data of a predetermined size read from the memory 2 or the storage unit 8 and decoded by the image processing unit 5. Specifically, the display control unit 6a includes a VRAM (Video Random Access Memory), a VRAM controller, a digital video encoder, and the like. The digital video encoder reads out the luminance signal Y and the color difference signals Cb and Cr decoded by the image processing unit 5 and stored in the VRAM (not shown) from the VRAM through the VRAM controller at a predetermined reproduction frame rate. Based on these data, a video signal is generated and output to the display panel 6b.

The display panel 6b displays an image (for example, the first face image P1 (see FIG. 3A), etc.) captured by the imaging unit 3 based on the video signal from the display control unit 6a in the display area. . Specifically, the display panel 6b performs live updating while sequentially updating a plurality of frame images generated by photographing an object by the imaging unit 3 in a still image shooting mode or a moving image shooting mode. Display the view image. Further, the display panel 6b displays a still image (rec view image) of the subject stored in the storage unit 8, or displays a moving image of the subject being shot.
Examples of the display panel 6b include a liquid crystal display panel and an organic EL (Electro-Luminescence) display panel. However, the display panel 6b is an example and is not limited thereto.

The operation input unit 7 is for performing a predetermined operation of the imaging apparatus 100. Specifically, the operation input unit 7 includes a shutter button related to an instruction to shoot a subject, a selection determination button related to an instruction to select a shooting mode and a function, a zoom button related to an instruction to adjust the zoom amount, and the like (all not shown). The operation part is provided.
When various buttons are operated by the user, the operation input unit 7 outputs an operation instruction corresponding to the operated button to the central control unit 1. The central control unit 1 causes each unit to execute a predetermined operation (for example, photographing a subject) according to an operation instruction output from the operation input unit 7 and input.

The operation input unit 7 includes a touch panel 7 a provided integrally with the display panel 6 b of the display unit 6.
The touch panel 7a detects a contact position of a touch pen or a user's finger (hand) that directly or indirectly contacts a display screen forming the display area of the display panel 6b. That is, the touch panel 7a is provided, for example, on the display screen or inside the display screen, and the touch position of the touch position on the display screen by various methods such as a resistive film method, an ultrasonic surface acoustic wave method, and a capacitance method. XY coordinates are detected. Then, the touch panel 7 a outputs a position signal related to the XY coordinates of the contact position to the central control unit 1 and the portrait processing unit 9.
Specifically, for example, the touch panel 7a outputs a position signal related to the XY coordinates of the contact position touched by the user in the first portrait image Q1 displayed on the display panel 6b to the portrait processing unit 9.

  Note that the detection accuracy of the contact position on the display screen by the touch panel 7a can be arbitrarily changed as appropriate. For example, one pixel may be strictly set as the contact position, or within a predetermined range centered on one pixel. A plurality of pixels may be used as the contact position.

  The storage unit 8 is configured by, for example, a non-volatile memory (flash memory) or the like, and stores still image and image data for storing moving images encoded by the image processing unit 5 using a predetermined encoding method.

In addition, the storage unit 8 stores a component image 8a of a main face constituent unit.
For example, the component image 8a is extracted from each of a plurality of portrait images drawn by each portrait painter, for example, a plurality of component images 8a for each major face component such as eyes, nose, mouth, eyebrows, and face contour. ,... The plurality of component images 8a may be associated with, for example, representative images that represent groups that are similar in shape and size.

  The storage unit 8 may be configured, for example, such that a storage medium (not shown) is detachable and controls reading of data from the mounted storage medium and writing of data to the storage medium.

The portrait processing unit 9 includes a first image acquisition unit 9a, a portrait image generation unit 9b, a face configuration unit specification unit 9c, a second image acquisition unit 9d, an extraction unit 9e, and a portrait image correction unit 9f. doing.
Each unit of the portrait processing unit 9 is configured by a predetermined logic circuit, for example, but the configuration is an example and the present invention is not limited thereto.

The first image acquisition unit 9a acquires a first face image (first unprocessed face image) P1 (see FIG. 3A) as an original image to be processed in the portrait image generation process.
That is, for example, the first image acquisition unit 9a captures image data (RGB data, etc.) of a still image (for example, a photographic image) generated by the image data generation unit 4 when the imaging unit 3 captures the face of the subject. (YUV data) is acquired from the memory 2, or a copy of still image data stored in the storage unit 8 is acquired as the first face image P1.

The first image acquisition unit 9a is specified based on one frame image automatically specified from a plurality of frame images constituting the moving image or a predetermined operation of the operation input unit 7 by the user. One frame image may be acquired as the first face image P1.
Further, each process by the portrait processing unit 9 to be described later may be performed on the image data itself of the first face image P1, and the image data of the first face image P1 is reduced at a predetermined ratio as necessary. It may be performed on reduced image data of a predetermined size (for example, VGA size).

The portrait image generation unit 9b generates a first portrait image Q1 (see FIG. 3B) that represents the original first face image P1 as a portrait.
That is, the portrait image generation unit 9b performs predetermined image processing on the first face image P1 to generate a first portrait image (processed face image) Q1.
Specifically, the portrait image generation unit 9b performs a predetermined face detection process on the image data of the first face image P1 acquired by the first image acquisition unit 9a to detect a face area. For example, AAM Major face components such as eyes, nose, mouth, eyebrows, and face contour are detected by processing using (Active Appearance Model). Here, AAM is a technique for modeling a visual event, and is a process for modeling an image of an arbitrary face region. For example, the caricature image generation unit 9b performs predetermined registration of statistical analysis results of positions and pixel values (for example, luminance values) of predetermined characteristic parts (for example, eyes, nasal heads, face lines, etc.) in a plurality of sample face images. Register with the means. Then, the caricature image generation unit 9b sets a shape model representing the shape of the face and a texture model representing “Appearance” in the average shape with reference to the position of the feature part, and uses these models to Model the image of the region. As a result, the face components such as eyes, nose, mouth, eyebrows, and face contour are modeled in the first face image P1.
Then, the portrait image generation unit 9b specifies a component image 8a corresponding to each of the detected main face constituting units among the plurality of component images 8a, ... stored in the storage unit 8. Here, as a method for specifying the component image 8a corresponding to the face constituting unit, the following method may be mentioned. That is, for example, the caricature image generation unit 9b sets a control point for each of the face composing unit and the component image 8a, and calculates a parameter in which the distance between the control points is minimized by the least square method. The sum of the distances between the control points in the parameters is set as the evaluation value of the component image 8a. The portrait image generation unit 9b compares the calculated evaluation values of the plurality of component images 8a,... To identify the component image 8a that most closely matches each face component.
Subsequently, the portrait image generation unit 9b deforms the component image 8a so that the control point of each identified component image 8a corresponds to the control point of each face component, and combines the first portrait image with a predetermined hairstyle image. An image Q1 is generated.
Here, the portrait image generation unit 9b may generate an image represented by adding a predetermined color to a predetermined part of the first portrait image Q1 (for example, a face component such as an eye, a mouth, or an eyebrow). .

Since the face detection process is a known technique, detailed description thereof is omitted here. In addition, detection of the face component using AAM is an example, but the present invention is not limited to this. For example, edge extraction processing, anisotropic diffusion processing, template matching, etc. can be arbitrarily changed. It is.
Further, the above-described method for generating the first portrait image Q1 is an example and is not limited to this, and can be arbitrarily changed as appropriate. For example, the component image 8a corresponding to each face component is automatically specified. However, this is only an example, and the present invention is not limited to this. A plurality of components based on a predetermined operation of the operation input unit 7 by the user. A part image 8a desired by the user may be designated from the images 8a,. Furthermore, both the process of automatically identifying the part image 8a and generating the portrait image and the process of generating the portrait image using the part image 8a selected by the user can be executed. Thus, the process to be executed may be switched based on a predetermined operation of the operation input unit 7 by the user.

The face component specifying unit (specifying unit) 9c specifies the face component specified by the user from the first portrait image Q1 that has been subjected to image processing on the first face image P1.
In other words, the face component specifying unit 9c is configured based on a predetermined operation of the operation input unit 7 by the user in the first portrait image Q1 displayed in the display area of the display panel 6b by the display controller 6a. A part (for example, the left eye El) is specified (see FIG. 3C). Specifically, for example, when the face component is specified based on the touch operation of the touch panel 7a by the user in the first portrait image Q1 displayed on the display panel 6b, the face component specifying unit 9c A position signal related to the XY coordinates of the determined position is acquired, and the face constituting unit designated by the user in the first portrait image Q1 is specified.

Here, the designation of the face component may be performed by a method that designates a point near the center of the face component, for example, or may be designated by a line that surrounds substantially the entire face component. It may be performed by various methods.
In addition, although the face component is specified using the touch panel 7a, this is merely an example and the present invention is not limited to this. Other buttons of the operation input unit 7, such as up / down / left / right movement keys, are used. Alternatively, a selection device (not shown) externally connected to the apparatus main body may be used.

The second image acquisition unit 9d acquires a second face image (second unprocessed face image) P2.
That is, the second image acquisition unit (acquisition unit) 9d includes the second face image P2 including the face configuration unit (for example, the left eye El) specified by the face configuration unit specification unit 9c (see FIG. 4A). To get. Specifically, when the face component is specified by the face component specifying unit 9c, the imaging unit 3 captures a still image including at least the face component, and the second image acquisition unit 9d A copy of still image data (RGB data or YUV data) generated by the data generation unit 4 is acquired from the memory 2 as the second face image P2.
Note that the second image acquisition unit 9d may acquire, for example, a copy of still image data different from the first face image P1 stored in the storage unit 8 as the second face image P2. The second face image P2 only needs to include the face component specified by the face component specifying unit 9c. For example, the second face image P2 may be an image of only the face component or an image of the entire face. There may be.

The extraction unit 9e extracts a face component image.
That is, the extraction unit (extraction unit) 9e extracts a face component image corresponding to the face component (for example, the left eye El) specified by the face component specifying unit 9c from the second face image P2. Specifically, the extraction unit 9e detects each face component from the second face image P2 acquired by the second image acquisition unit 9d, for example, by processing using AAM, and the face component specifying unit 9c. The face constituent image corresponding to the face constituent specified by the above is specified and extracted.

The portrait image correcting unit 9f corrects a correction portion corresponding to the face constituting unit in the first portrait image Q1.
That is, the portrait image correcting unit (correcting unit) 9f is a face composing unit (for example, the left eye El) specified by the face composing unit specifying unit 9c in the second face image P2 acquired by the second image acquiring unit 9d. Based on the face component image corresponding to, the correction portion corresponding to the face component in the first portrait image Q1 is corrected or changed.
Specifically, the portrait image correcting unit 9f corresponds to the face component image extracted by the extracting unit 9e among the plurality of component images 8a,. The component image 8a (see FIG. 4B) representing the image as a picture is specified and acquired. Then, the portrait image correcting unit 9f deletes the component image 8a included in the corrected portion of the first portrait image Q1, and also specifies a new component specified at the reference position (for example, the center position) of the component image 8a. The image 8a is changed to a new component image 8a so that the reference positions of the images 8a are overlapped. Thereby, the portrait image correcting unit 9f generates a second portrait image Q2 (see FIG. 4C) in which the shape and size of the face component image to be corrected in the first portrait image Q1 is corrected.

Note that the method for specifying the component image 8a corresponding to the face component image is substantially the same as the processing content by the portrait image generator 9b described above, and detailed description is omitted, but the component image 8a is changed to a component image 8a having a different size. In the case of doing so, the portrait image correcting unit 9f applies the face component image in the first portrait image Q1 based on the face component image extracted by the extracting unit 9e and the information indicating the size of the face component image. Correct the corresponding correction part.
Specifically, the portrait image correcting unit 9f is relatively large with respect to other face components (for example, a face contour) specified by the face component specifying unit 9c in the first portrait image Q1. Get information that represents In addition, the portrait image correcting unit 9f is relatively large with respect to other face components (for example, a face contour) related to the face component related to the face component image extracted from the second face image P2 by the extracting unit 9e. Get information that represents Then, the portrait image correcting unit 9f is based on the acquired information indicating the size of the face component in the first portrait image Q1 and the information indicating the size of the face component image in the second face image P2. Then, the size of each face component image in the first portrait image Q1 and the second face image P2 is normalized, and the component image 8a having a size corresponding to the size of the face component image extracted by the extraction unit 9e. Is identified.

<Caricature image generation processing>
Next, a portrait image generation process performed by the imaging apparatus 100 will be described with reference to FIGS.
FIG. 2 is a flowchart illustrating an example of an operation related to the portrait image generation process. 3A shows an example of the first face image P1, FIG. 3B shows an example of the first portrait image Q1 representing the first face image P1 as a portrait, and FIG. ) Shows an example of the first portrait image Q1 displayed in the display area of the display panel 6b. 4A shows an example of the second face image P2, FIG. 4B shows an example of the component image 8a corresponding to the face component image, and FIG. 4C shows the second face image P2. An example of the portrait image Q2 is shown.

  The portrait image generation processing is performed when the portrait image generation mode is selected from a plurality of operation modes displayed on the menu screen based on a predetermined operation of the selection determination button of the operation input unit 7 by the user. This process is executed by each unit of the imaging apparatus 100, particularly the portrait processing unit 9, under the control of the unit 1.

As shown in FIG. 2, first, the imaging unit 3 starts a process of capturing a live view image of a subject, and the display control unit 6 a is based on the display image data generated by the image processing unit 5. The live view image is displayed on the display panel 6b (step S1).
Thereafter, based on a predetermined operation of the shutter button of the operation input unit 7 by the user, the imaging unit 3 captures a subject, and the image data generation unit 4 generates image data of a still image of the subject and outputs it to the memory 2. (Step S2). Then, the first image acquisition unit 9a acquires a copy of the image data of the still image from the memory 2 as the first face image P1 (see FIG. 3A) (step S3).
The first face image P1 may be captured by so-called self-portrait, or the shutter button may be operated by a third party other than the user.

Subsequently, the portrait image generation unit 9b generates a first portrait image Q1 (see FIG. 3B) based on the first face image P1 acquired by the first image acquisition unit 9a (step S4). Specifically, the caricature image generation unit 9b first performs main face composition such as eyes, nose, mouth, eyebrows, and face contour in the face area of the first face image P1, for example, by processing using AAM. Part. Next, the portrait image generating unit 9b specifies a component image 8a corresponding to each of the detected main face constituting units among the plurality of component images 8a stored in the storage unit 8. Then, the portrait image generating unit 9b deforms the identified component images 8a so as to correspond to the respective face constituting units, and generates a first portrait image Q1 in combination with a predetermined hairstyle image.
Thereafter, the display control unit 6a displays the first portrait image Q1 in the display area of the display panel 6b based on the image data of the first portrait image Q1 generated by the portrait image generation unit 9b (step S5; FIG. 3). (See (c)).

Next, the operation input unit 7 determines whether or not the user has performed a touch operation on the touch panel 7a in the first portrait image Q1 displayed on the display panel 6b (step S6).
If it is determined that the touch operation on the touch panel 7a is not performed by the user (step S6; NO), the CPU of the central control unit 1 ends the portrait image generation process.

On the other hand, when it is determined in step S6 that the touch operation on the touch panel 7a by the user has been performed (step S6; YES), the face constituting unit specifying unit 9c relates to the XY coordinates of the touched position on the touch panel 7a. A position signal is acquired, and the face constituent part (for example, left eye El etc.) designated by the user in the first portrait image Q1 is specified (step S7; see FIG. 3C).
Thereafter, the CPU of the central control unit 1 sets the operation mode to the still image shooting mode, and based on a predetermined operation of the shutter button of the operation input unit 7 by the user, the image pickup unit 3 is operated by the face component specifying unit 9c. The subject is photographed again with a composition including at least the identified face component, and the image data generation unit 4 generates image data of a still image of the subject and outputs it to the memory 2 (step S8). Then, the second image acquisition unit 9d acquires a copy of the image data of the still image from the memory 2 as the second face image P2 (see FIG. 4A) (step S9).
Here, when the second face image P2 is captured, the display panel 6b displays a live view image. While confirming the facial expression of the face component to be corrected as described later, so-called self-viewing is performed. You can take a picture. Further, the display control unit 6a may display on the display panel 6b an image obtained by enlarging a part including the face component to be corrected.

Next, the extracting unit 9e detects each face constituent unit in the face area of the second face image P2 acquired by the second image acquiring unit 9d, for example, by processing using AAM, and a face constituent unit specifying unit A face constituent image corresponding to the face constituent specified by 9c is specified and extracted (step S10).
Subsequently, the portrait image correcting unit 9f selects a component image 8a corresponding to the face component image extracted by the extracting unit 9e among the plurality of component images 8a stored in the storage unit 8 (FIG. 4 ( b) See) is specified and acquired (step S11). Then, the portrait image correcting unit 9f specifies the corrected part corresponding to the face component (for example, the left eye El) specified by the face component specifying unit 9c in the first portrait image Q1 and identifies the new component image. 8a is generated, and a second portrait image Q2 in which the shape and size of the face component image to be corrected in the first portrait image Q1 is corrected is generated (step S12; see FIG. 4C).

Thereafter, the display control unit 6a displays the second portrait image Q2 in the display area of the display panel 6b based on the image data of the second portrait image Q2 generated by the portrait image correction unit 9f (step S13).
At this time, the display control unit 6a may display the first portrait image Q1 on the display panel 6b in addition to the second portrait image Q2, thereby allowing the user to correct the first portrait image Q1 before correction. The subsequent second portrait image Q2 can be compared.

Next, the operation input unit 7 determines whether or not the user has performed a touch operation on the touch panel 7a in the second portrait image Q2 displayed on the display panel 6b (step S14).
Here, if it is determined that the touch operation of the touch panel 7a by the user has been performed (step S14; YES), the portrait processing unit 9 returns the process to step S7, and executes the subsequent processes. That is, it is possible to recorrect a face component (for example, the left eye El) that has been corrected in the second portrait image Q2 or to correct another face component (for example, the mouth).
When correcting a plurality of face constituent parts (for example, the left eye El and the mouth), the portrait image correcting part 9f selects all or one of the plurality of face constituent parts when all the face constituent parts are corrected. A plurality of second portrait images Q2,... Corresponding to the combinations may be generated, and the display control unit 6a may cause the display panel 6b to display all of the plurality of second portrait images Q2,. Accordingly, the user can compare (view) a plurality of second portrait images Q2,..., And can select the second portrait image Q2 having better correction contents.

  On the other hand, when it is determined in step S14 that the user has not performed the touch operation on the touch panel 7a (step S14; NO), the CPU of the central control unit 1 ends the portrait image generation process.

As described above, according to the imaging apparatus 100 of the present embodiment, the second facial image P2 including the facial component designated by the user in the first portrait image (processed facial image) Q1 is obtained and obtained. The correction portion corresponding to the face component in the first portrait image Q1 is corrected based on the face component image corresponding to the face component in the second face image P2 thus generated, and thus generated from the first face image P1. When a part of the face composition part in the first portrait image Q1 is not liked, only the part corresponding to the face composition part to be corrected in the first portrait image Q1 can be corrected, for example, a new face image It is not necessary to acquire the entire image to generate a portrait image or to manually select a component image 8a corresponding to the face component to be corrected, and to easily correct a part of the face component of the processed face image Can be done.
In particular, the first portrait image Q1 is displayed because the first portrait image Q1 is displayed in the display area of the display panel 6b, and the face component to be corrected is specified based on a predetermined operation of the operation input unit 7 by the user. It is possible to more easily correct a part of the face constituting part.

Further, a face component image corresponding to the face component is extracted from the second face image P2 including the face component to be corrected, which is captured by the imaging unit 3, and based on the extracted face component image, the second face image P2 is extracted. The correction part corresponding to the face constituent part in the one portrait image Q1 can be corrected. Specifically, the component image 8a corresponding to the extracted face component image is acquired, the modified portion corresponding to the face component in the first portrait image Q1 is changed to the acquired component image 8a, and the first It is possible to generate a second portrait image Q2 in which the shape and size of the face component image to be corrected in the first portrait image Q1 is corrected.
At this time, since the correction portion corresponding to the face component in the first portrait image Q1 is corrected based on the extracted face component image and information indicating the size of the face component image, the second face image When the face component image corresponding to the face component to be corrected extracted from P2 is used, the corresponding component image 8a can be acquired in consideration of the size of the face component image. Then, the face component image to be corrected in the first portrait image Q1 is changed by changing the corrected portion corresponding to the face component in the first portrait image Q1 to the component image 8a acquired in consideration of the size. Can be appropriately corrected.

The present invention is not limited to the above-described embodiment, and various improvements and design changes may be made without departing from the spirit of the present invention.
For example, in the above embodiment, the second facial image obtained by photographing the same subject as the first portrait image Q1 is used. However, the second facial image is not limited to this example, and a different subject is photographed. You may make it produce | generate the face image which combined the some human face structure part using the face image.

  In the above embodiment, the face component designated by the user in the first portrait image Q1 is specified. However, the present invention is not limited to this. In the first face image P1 of the original image The face component designated by the user may be specified. When there is a face component to be corrected in the first portrait image Q1, the designation of the face component to be corrected may be in the first portrait image Q1 or in the first face image P1 of the original image. .

  Furthermore, in the above embodiment, the first portrait image Q1 and the second portrait image Q2 are generated. However, it is not always necessary to generate the first portrait image Q1 and the second portrait image Q2, and the portrait image. Whether or not to include the generation unit 9b and the portrait image correction unit 9f can be arbitrarily changed as appropriate.

  Moreover, in the said embodiment, although 1st portrait image Q1 and 2nd portrait image Q2 were illustrated as a processed face image, it is an example and it is not restricted to this, For example, binarization processing The image that has been processed and the image that has been subjected to the edge extraction processing can be arbitrarily changed.

  Furthermore, the configuration of the imaging apparatus 100 is merely an example illustrated in the above embodiment, and is not limited thereto. Further, the imaging apparatus 100 has been exemplified as the image processing apparatus, but the present invention is not limited to this, and any configuration may be used as long as the image processing according to the present invention can be executed.

In addition, in the above-described embodiment, the functions as the specifying unit, the acquiring unit, and the correcting unit are controlled under the control of the CPU of the central control unit 1 by the face configuration unit specifying unit 9c, the second image acquiring unit 9d, Although the configuration realized by driving the portrait image correcting unit 9f is not limited thereto, the configuration may be realized by executing a predetermined program or the like by the CPU of the central control unit 1. .
That is, a program including a specific process routine, an acquisition process routine, and a correction process routine is stored in a program memory (not shown) that stores the program. Then, the CPU of the central control unit 1 uses the specific processing routine to process the processed face image (first portrait image Q1) subjected to image processing on the first unprocessed face image (first face image P1) and the first. You may make it function as a means to identify the face structure part designated by the user from any one of 1 unprocessed face images. Further, the CPU of the central control unit 1 may function as a means for acquiring a second unprocessed face image (second face image P2) including the specified face component by an acquisition process routine. Further, the CPU of the central control unit 1 corrects the face component in the processed face image based on the face component image corresponding to the face component in the acquired second unprocessed face image by the correction processing routine. You may make it function as a means to do.

  Similarly, the extraction means may be configured to be realized by executing a predetermined program or the like by the CPU of the central control unit 1.

  Furthermore, as a computer-readable medium storing a program for executing each of the above processes, a non-volatile memory such as a flash memory or a portable recording medium such as a CD-ROM is applied in addition to a ROM or a hard disk. Is also possible. A carrier wave is also used as a medium for providing program data via a predetermined communication line.

Although several embodiments of the present invention have been described, the scope of the present invention is not limited to the above-described embodiments, but includes the scope of the invention described in the claims and equivalents thereof.
The invention described in the scope of claims attached to the application of this application will be added below. The item numbers of the claims described in the appendix are as set forth in the claims attached to the application of this application.
[Appendix]
<Claim 1>
A specifying means for specifying a face component designated by a user from either one of the processed face image subjected to image processing for the first unprocessed face image and the first unprocessed face image;
Obtaining means for obtaining a second unprocessed face image including the face constituting unit identified by the identifying means;
Correcting means for correcting the face constituent part in the processed face image based on a face constituent part image corresponding to the face constituent part in the second unprocessed face image acquired by the acquiring means;
An image processing apparatus comprising:
<Claim 2>
An extraction means for extracting a face component image corresponding to the face component from the second unprocessed face image;
The image processing apparatus according to claim 1, wherein the correcting unit corrects the face constituent unit in the processed face image based on the face constituent unit image extracted by the extracting unit.
<Claim 3>
The correction unit acquires a component image corresponding to the face component image extracted by the extraction unit, and changes the face component in the processed face image to the component image. 2. The image processing apparatus according to 2.
<Claim 4>
The image processing apparatus according to claim 3, wherein the component image is an image representing the face component image as a picture.
<Claim 5>
The image processing apparatus according to claim 1, wherein the processed face image is an image of the first unprocessed face image represented by a portrait.
<Claim 6>
The image processing apparatus according to claim 1, wherein the first unprocessed face image and the second unprocessed face image are face images of the same subject.
<Claim 7>
An image processing method using an image processing apparatus,
Identifying a face component designated by the user from either one of the processed face image subjected to image processing on the first unprocessed face image and the first unprocessed face image;
Obtaining a second unprocessed face image including the identified face component;
Correcting the face component in the processed face image based on a face component image corresponding to the face component in the acquired second unprocessed face image;
An image processing method comprising:
<Claim 8>
The computer of the image processing device
A specifying means for specifying a face component designated by a user from either one of the processed face image subjected to image processing for the first unprocessed face image and the first unprocessed face image;
Obtaining means for obtaining a second unprocessed face image including the face constituting part specified by the specifying means;
Correcting means for correcting the face constituent part in the processed face image based on a face constituent part image corresponding to the face constituent part in the second unprocessed face image acquired by the acquiring means;
A program characterized by functioning as

DESCRIPTION OF SYMBOLS 100 Imaging device 1 Central control part 3 Imaging part 6 Display part 6a Display control part 6b Display panel 7 Operation input part 7a Touch panel 8 Storage part 8a Component image 9 Caricature processing part 9a 1st image acquisition part 9b Caricature image generation part 9c Face structure Part specifying part 9d Second image acquisition part 9e Extraction part 9f Caricature image correction part P1 First face image P2 Second face image Q1 First caricature image Q2 Second caricature image

Claims (8)

A specifying means for specifying a face component designated by a user from either one of the processed face image subjected to image processing for the first unprocessed face image and the first unprocessed face image;
Obtaining means for obtaining a second unprocessed face image including the face constituting unit identified by the identifying means;
Correcting means for correcting the face constituent part in the processed face image based on a face constituent part image corresponding to the face constituent part in the second unprocessed face image acquired by the acquiring means;
An image processing apparatus comprising: An extraction means for extracting a face component image corresponding to the face component from the second unprocessed face image;
The image processing apparatus according to claim 1, wherein the correcting unit corrects the face constituent unit in the processed face image based on the face constituent unit image extracted by the extracting unit.   The correction unit acquires a component image corresponding to the face component image extracted by the extraction unit, and changes the face component in the processed face image to the component image. 2. The image processing apparatus according to 2.   The image processing apparatus according to claim 3, wherein the component image is an image representing the face component image as a picture.   The image processing apparatus according to claim 1, wherein the processed face image is an image of the first unprocessed face image represented by a portrait.   The image processing apparatus according to claim 1, wherein the first unprocessed face image and the second unprocessed face image are face images of the same subject. An image processing method using an image processing apparatus,
Identifying a face component designated by the user from either one of the processed face image subjected to image processing on the first unprocessed face image and the first unprocessed face image;
Obtaining a second unprocessed face image including the identified face component;
Correcting the face component in the processed face image based on a face component image corresponding to the face component in the acquired second unprocessed face image;
An image processing method comprising: The computer of the image processing device
A specifying means for specifying a face component designated by a user from either one of the processed face image subjected to image processing for the first unprocessed face image and the first unprocessed face image;
Obtaining means for obtaining a second unprocessed face image including the face constituting part specified by the specifying means;
Correcting means for correcting the face constituent part in the processed face image based on a face constituent part image corresponding to the face constituent part in the second unprocessed face image acquired by the acquiring means;
A program characterized by functioning as