JP2018045366A - Image processing device, image processing method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To adjust a person's face into a state suitable for facial treatment.SOLUTION: An image processing device 1 has a face detection unit 52, a determination unit 53, and a control unit 54. The face detection unit 52 detects a person's face included in an image. The determination unit 53 determines the state of the image portion of the detected person's face. The control unit 54 adjusts the image portion of the determined person's face into a state suitable for facial treatment, according to the state of the person's face.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program.

Conventionally, in an image processing apparatus such as a digital camera, a technique for performing processing called whitening processing as in Patent Document 1 is known.
In the whitening process, a process for beautifying the skin portion of the face of the person included in the image is performed.

JP 2006-121416 A

  However, when the person to be photographed applies makeup on the face, there is a problem that the effect is different only by applying the same whitening process depending on the darkness of the makeup, the characteristics of the person, or the photographing conditions. This can be said for all the beautiful face treatments for beautifying the skin portion of a person's face, such as a treatment for smoothing the skin or browning the skin.

  The present invention has been made in view of such circumstances, and an object of the present invention is to prevent a difference in the effect of the beautiful face processing regardless of the face state of a person.

In order to achieve the above object, an image processing apparatus according to an aspect of the present invention includes:
Detecting means for detecting a human face included in the image;
A discriminating means for discriminating the state of the image portion of the detected human face;
Adjusting means for adjusting the image portion of the person's face to an appropriate state for performing the facial treatment according to the determined state of the person's face;
It is characterized by providing.

  According to the present invention, it is possible to prevent a difference in the effect of the facial processing regardless of the state of the person's face.

1 is a block diagram illustrating a hardware configuration of an image processing apparatus according to an embodiment of the present invention. It is a functional block diagram which shows the functional structure for performing a makeup | decoration face production | generation process among the functional structures of the image processing apparatus of FIG. It is a schematic diagram which shows the data structural example of a decompression | restoration / remake-up table. It is a flowchart explaining the flow of the makeup | decoration face production | generation process which the image processing apparatus of FIG. 1 which has the functional structure of FIG. 2 performs. It is a flowchart explaining the flow of the decompression | restoration process performed in step S5 of a makeup | decoration face production | generation process. It is a flowchart explaining the flow of the remake-up process performed in step S7 of a makeup | decoration face production | generation process. It is a flowchart explaining the flow of a restoration / remake-up table generation process. It is a flowchart explaining the flow of the function generation process for a restoration process performed in step S39 of a restoration / remakeup table generation process. It is a flowchart explaining the flow of the remake function generation process performed in step S41 of the restoration / remake table generation process. It is a functional block diagram which shows the functional structure for performing a makeup | decoration face production | generation process among the functional structures of the image processing apparatus in 2nd Embodiment. It is a schematic diagram which shows the data structural example of a person characteristic standardization table. It is a schematic diagram which shows the example of a data structure of an imaging condition standardization table.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[First Embodiment]
The image processing apparatus according to the present embodiment is capable of executing a process (hereinafter referred to as “restoration process”) that puts the face of a subject that has undergone makeup into a spinned state without makeup. A facial process including a whitening process and a skin-beautifying process is performed on the face of the subject that has been put into a suppin state. In the present embodiment, the whitening process is a process for whitening the color of the skin part in the face image, and the skin beautifying process is a process for smoothing the skin part in the face image. Then, the image processing apparatus according to the present embodiment performs a process (hereinafter referred to as “remake-up process”) for applying makeup designated by the user to the face of the subject that has been subjected to the facial beauty process after being put into the suppin state by the restoration process. Is performed by image processing.
Accordingly, it is possible to prevent a difference in the effect of the facial processing regardless of the state of the person's face, and it is possible to execute image processing for applying a predetermined makeup after performing the facial processing. .

FIG. 1 is a block diagram showing a hardware configuration of an image processing apparatus 1 according to an embodiment of the present invention.
The image processing apparatus 1 is configured as a digital camera, for example.

  As illustrated in FIG. 1, the image processing apparatus 1 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a bus 14, an input / output interface 15, The imaging unit 16, the input unit 17, the output unit 18, the storage unit 19, the communication unit 20, and the drive 21 are provided.

  The CPU 11 executes various processes according to a program recorded in the ROM 12 or a program loaded from the storage unit 19 to the RAM 13.

  The RAM 13 appropriately stores data necessary for the CPU 11 to execute various processes.

  The CPU 11, ROM 12, and RAM 13 are connected to each other via a bus 14. An input / output interface 15 is also connected to the bus 14. An imaging unit 16, an input unit 17, an output unit 18, a storage unit 19, a communication unit 20, and a drive 21 are connected to the input / output interface 15.

  Although not shown, the imaging unit 16 includes an optical lens unit and an image sensor.

The optical lens unit is configured by a lens that collects light, for example, a focus lens or a zoom lens, in order to photograph a subject.
The focus lens is a lens that forms a subject image on the light receiving surface of the image sensor. The zoom lens is a lens that freely changes the focal length within a certain range.
The optical lens unit is also provided with a peripheral circuit for adjusting setting parameters such as focus, exposure, and white balance as necessary.

The image sensor includes a photoelectric conversion element, AFE (Analog Front End), and the like.
The photoelectric conversion element is composed of, for example, a CMOS (Complementary Metal Oxide Semiconductor) type photoelectric conversion element or the like. A subject image is incident on the photoelectric conversion element from the optical lens unit. Therefore, the photoelectric conversion element photoelectrically converts (captures) the subject image, accumulates the image signal for a predetermined time, and sequentially supplies the accumulated image signal as an analog signal to the AFE.
The AFE performs various signal processing such as A / D (Analog / Digital) conversion processing on the analog image signal. Through various signal processing, a digital signal is generated and output as an output signal of the imaging unit 16.
Hereinafter, the output signal of the imaging unit 16 is referred to as a “captured image”. Data of the captured image is appropriately supplied to the CPU 11 or an image processing unit (not shown).

The input unit 17 includes various buttons and the like, and inputs various types of information according to user instruction operations.
The output unit 18 includes a display, a speaker, and the like, and outputs images and sounds.
The storage unit 19 includes a hard disk or a flash memory, and stores programs executed by the CPU 11 and various image data.
The communication unit 20 controls communication performed with other devices (not shown) via a network including the Internet.

  A removable medium 31 composed of a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is appropriately attached to the drive 21. The program read from the removable medium 31 by the drive 21 is installed in the storage unit 19 as necessary. The removable medium 31 can also store various data such as image data stored in the storage unit 19 in the same manner as the storage unit 19.

FIG. 2 is a functional block diagram showing a functional configuration for executing the makeup face generation process among the functional configurations of the image processing apparatus 1 of FIG.
The makeup face generation process is a process for restoring the face of a subject to which makeup has been applied to make the face in a suppin state where makeup has not been applied. A series of processes in which makeup designated by the user is applied to the face of the subject by image processing.

When the makeup face generation process is executed, as shown in FIG. 2, in the CPU 11, the image acquisition unit 51, the face detection unit 52, the determination unit 53, the adjustment unit 54, the face processing unit 55, and the remake-up The unit 56, the table management unit 57, and the function generation unit 58 function.
In addition, a restoration / remake-up table storage unit 71 is set in one area of the storage unit 19.
The restoration / remake-up table storage unit 71 is a restoration in which a function representing an image processing method for restoring a face image and an image processing method for performing a remake-up process on the face image is table-format data. / Remake table data is stored.

FIG. 3 is a schematic diagram illustrating a data configuration example of the restoration / remake-up table.
As shown in FIG. 3, in the restoration / remake-up table, a function representing an image processing method for restoring a face image (hereinafter referred to as a “restoration processing function”) and a remake of the face image are displayed. A function representing an image processing method for performing up-processing (hereinafter referred to as “remake-up function”) is stored. In FIG. 3, the restoration processing function is represented by f and the remakeup function is represented by g, and the different functions in the restoration processing function f and the remakeup function g are subscripts n, m (n and m are natural numbers) is attached.
The restoration processing function and the remake-up function have characteristics corresponding to various skin colors (light, slightly light ..., dark, etc.) and various makeup types (natural, gal, etc.). Things are available. In addition, the color of the skin here represents the color of the background before applying makeup.
Furthermore, in the present embodiment, as the remake-up process, the intensity of the facial treatment performed by image processing can be varied in multiple stages (here, a total of 13 stages up to ± 6 with reference to standard zero). The remake function corresponding to each intensity level is stored in the restoration / remake table.
The restoration / remake-up table is generated by a restoration / remake-up table generation process described later.

The image acquisition unit 51 acquires image data to be subjected to a makeup face generation process. For example, the image acquisition unit 51 acquires data of a captured image captured by the imaging unit 16 as a target for the makeup face generation process, or acquires image data stored in the storage unit 19 or the removable media 31. Alternatively, it can be acquired from another device via the communication unit 20.
In addition, the image acquisition unit 51 acquires, as image data used for the restoration / remake-up table generation process, a set of a face image without makeup and a face image with makeup applied to the same person. For example, the image acquisition unit 51 may acquire image data stored in the removable medium 31, or may acquire it from another device via the communication unit 20.
The face detection unit 52 detects the face of the subject person in the image that is the target of the makeup face generation process.
The face detection unit 52 detects the face of the subject person in the image used for the restoration / remake-up table generation process. It should be noted that the detection of the face of the subject person can be realized by using a known technique, and thus detailed description thereof is omitted here.
Further, the face detection unit 52 acquires the size and position of the face in the image, the position of the main part of the face (eyes, nose, mouth, etc.), the skin color, and the like as the information related to the detected person's face. .
The determination unit 53 determines the state of the face detected by the face detection unit 52. Specifically, the determination unit 53 applies makeup in a state in which the detected face state corresponds to any of various skin colors and various types of makeup based on information about the face acquired by the face detection unit 52. Determine whether it has been applied. At this time, the determination unit 53 determines whether the facial skin color and the makeup type detected by the face detection unit 52 are the conditions that define the restoration function in the restoration / remake-up table stored in the storage unit 19. Determine whether they match. For example, the determination unit 53 determines whether the skin color of the face detected by the face detection unit 52 is included in a range of “light”, “slightly light”, “dark”. This is determined by comparing with a color threshold. In addition, the determination unit 53 determines whether the type of facial makeup detected by the face detection unit 52 matches “natural”, “gal”, or the like, the type of color used in makeup, The determination is made based on conditions such as the position of makeup on the face and the pattern of coloring.
Further, the determination unit 53 determines the makeup type by acquiring the makeup type designated by the user in the restoration / remake-up table generation process.

The adjustment unit 54 refers to the restoration / remake-up table, and acquires a restoration processing function corresponding to the skin color and makeup type determined by the determination unit 53. Then, the adjustment unit 54 applies the acquired restoration processing function to the face image detected by the face detection unit 52, and sets the face detected by the face detection unit 52 in a skipped state (state where makeup is not applied). ) To adjust.
That is, the restoration process is executed by the determination unit 53 and the adjustment unit 54.

  The beautiful face processing unit 55 performs a beautiful face process on the face image that has been set to the skipped state by the adjustment unit 54. In the present embodiment, the face beauty processing unit 55 performs a whitening process and a skin beautification process on the face image as the face treatment. Note that since the whitening process and the skin beautifying process can be realized by using a known technique, detailed description thereof is omitted here.

  The remake-up unit 56 executes image processing (remake-up processing) for applying the type of makeup specified by the user to the face image that has been subjected to the facial processing by the facial processing unit 55. At this time, the remake-up unit 56 refers to the restoration / remake-up table, obtains a remake-up function corresponding to the skin color of the subject's face, the intensity of the beautiful face specified by the user, and the type of makeup, Applies to facial images that have undergone facial treatment. Here, it is assumed that the remake-up function is applied to the skin color region in the face image. As a result, it is possible to apply the desired makeup by image processing after performing the facial processing on the image of the face of the subject once in the suppin state. Note that the type of makeup applied at this time may be the same as or different from the type of makeup previously applied to the face image of the subject (the type of makeup prior to the restoration process). It is possible to use different types of makeup (for example, to apply gal-type makeup to the face of a subject that has been subjected to natural-type makeup).

In addition, the remake-up unit 56 acquires the strength of the facial treatment that is specified after the restoration process is specified by the user.
The remake-up unit 56 performs a smoothing process on the face image after the remake-up process according to the type of the re-make-up process. For example, the remake-up unit 56 performs a smoothing process for smoothing the boundary between the red area of the cheek and the skin color area of the face when a makeup having a dark cheek is applied in the remake-up process.

  The table management unit 57 manages the data of the restoration / remake-up table stored in the restoration / remake-up table storage unit 71. Specifically, the table management unit 57 updates the restoration / remake-up table to a new version, or corrects the contents of the restoration / remake-up table in accordance with a user instruction.

  The function generation unit 58 generates a restoration processing function and a remake-up function. Specifically, a function used to acquire a plurality of change amounts between a face image without makeup and a face image with makeup applied to the same person, and to perform restoration processing image processing, and remake Generate a function used to perform up image processing.

Next, the operation of the image processing apparatus 1 will be described.
FIG. 4 is a flowchart for explaining the flow of the makeup face generation process executed by the image processing apparatus 1 of FIG. 1 having the functional configuration of FIG.
The makeup face generation process is started in response to the user inputting an operation for instructing the input unit 17 to start the makeup face generation process.

When the makeup face generation process is started, in step S1, the image acquisition unit 51 acquires image data to be subjected to the makeup face generation process.
In step S <b> 2, the face detection unit 52 detects the face of the subject person in the image that is the target of the makeup face generation process.

In step S <b> 3, the face detection unit 52 determines whether or not the face of the subject person has been detected in the image that is the target of the makeup face generation process.
If the face of the subject person is not detected in the image that is the target of the makeup face generation process, NO is determined in step S3, and the makeup face generation process ends.
On the other hand, when the face of the subject person is detected in the image to be subjected to the makeup face generation process, YES is determined in step S3, and the process proceeds to step S4.

In step S <b> 4, the face detection unit 52 acquires information related to the detected human face of the subject (face size, position, skin color, etc. in the image).
In step S5, the adjustment unit 54 executes a restoration process. Details of the restoration process will be described later.

In step S <b> 6, the face beauty processing unit 55 performs face beauty processing (whitening processing and skin beautification processing) on the face image that has been brought into the suppin state by the restoration processing.
In step S <b> 7, the remake-up unit 56 performs a remake-up process on the face image that has been subjected to the beautiful face process. The details of the remake process will be described later.
In step S8, the remake-up unit 56 performs a smoothing process on the face image after the re-make-up process. This smoothing process is appropriately executed according to the type of remake-up process.
After step S8, the makeup face generation process ends.

Next, the restoration process executed in step S5 of the makeup face generation process will be described.
FIG. 5 is a flowchart illustrating the flow of the restoration process executed in step S5 of the makeup face generation process.
When the restoration process is started, the determination unit 53 determines the state of the face detected by the face detection unit 52 in step S11. Specifically, the determination unit 53 applies makeup in a state in which the detected face state corresponds to any of various skin colors and various types of makeup based on information about the face acquired by the face detection unit 52. Determine whether it has been applied.

In step S <b> 12, the adjustment unit 54 refers to the restoration / remake-up table, and acquires a restoration processing function corresponding to the skin color and makeup type determined by the determination unit 53.
In step S <b> 13, the adjustment unit 54 applies the acquired restoration processing function to the face image detected by the face detection unit 52, and puts the face detected by the face detection unit 52 in a suppin state (makeup is applied). Image processing to be adjusted.
After step S13, the process returns to the makeup face generation process.

Next, the remake-up process executed in step S7 of the makeup face generation process will be described.
FIG. 6 is a flowchart for explaining the flow of the remake-up process executed in step S7 of the makeup face generation process.
When the remake-up process is started, in step S21, the remake-up unit 56 obtains the makeup type (remake-up type) designated by the user.
In step S <b> 22, the remake-up unit 56 acquires the intensity of the facial treatment designated by the user.

In step S23, the remake-up unit 56 refers to the restoration / remake-up table, and obtains a remake-up function corresponding to the skin color of the subject's face, the facial strength specified by the user, and the makeup type.
In step S24, the remake-up unit 56 applies the acquired remake-up function to the face image that has been subjected to the face processing by the face processing unit 55, and applies the type of makeup specified by the user by image processing.
After step S24, the process returns to the makeup face generation process.

Next, processing for generating a restoration / remake-up table referred to in the makeup face generation processing (hereinafter referred to as “restoration / remake-up table generation processing”) will be described.
FIG. 7 is a flowchart for explaining the flow of restoration / remake-up table generation processing.
The restoration / remake-up table generation process is executed as a pre-process for generating a restoration / remake-up table in advance prior to the makeup face generation process. Therefore, the restoration / remake-up table generation process can be executed by the image processing apparatus 1 or an information processing apparatus such as a PC (Personal Computer) other than the image processing apparatus 1. Here, a description will be given assuming that the restoration / remake-up table generation process is executed in the image processing apparatus 1.
It should be noted that for the same person used for the restoration / remake-up table generation process, a set of a face image without makeup and a face with makeup is previously photographed for a plurality of persons, and the removable media 31 and other Assume that it is stored in the device.

The restoration / remake-up table generation process is started in response to an operation that instructs the input unit 17 to start the restoration / remake-up table generation process.
When the restoration / remake-up table generation processing is started, in step S31, the table management unit 57 applies makeup and an image of a face that has not been subjected to makeup for the same person used for the restoration / remake-up table generation processing. The location and number of stored face image pairs are specified.

In step S <b> 32, the face detection unit 52 sets one set (one person) of a face image without makeup and a face image with makeup applied to the same person from the storage location specified by the table management unit 57. Min) to get.
In step S <b> 33, the face detection unit 52 detects a person's face from each of the acquired images for one person, and acquires information related to the face (face size, position, etc. in the image).
In step S34, the determination unit 53 determines the type of makeup applied to the face by acquiring the type of makeup designated by the user.
In step S35, the face detection unit 52 acquires the skin color of the face.
In step S36, the table management unit 57 classifies the face image for each skin color and makeup type.
In step S <b> 37, the table management unit 57 associates the classified face image data with the information about the face acquired in step S <b> 33, classifies the classified face image data, and registers the classified face image data.

In step S <b> 38, the table management unit 57 determines whether all the classification / registration processing of the face image data of the number specified in step S <b> 31 has been completed.
If all the face image data classification / registration processes specified in step S31 have not been completed, NO is determined in step S38, and the process proceeds to step S32.
On the other hand, if all the classification / registration processes of the face image data for the number specified in step S31 have been completed, YES is determined in step S38, and the process proceeds to step S39.
In step S39, the classified face image data registered in step S37 is referred to, and processing for generating a restoration processing function (hereinafter referred to as “restoration processing function generation processing”) is executed. Details of the restoration processing function generation processing will be described later.
In step S40, the table management unit 57 associates the skin color, the makeup type, and the generated restoration processing function and registers them in the restoration / remake-up table.

In step S41, the classified face image data registered in step S37 is referred to, and a process for generating a remake function (hereinafter referred to as “remake function generation process”) is executed. Details of the remake-up function generation process will be described later.
In step S42, the table management unit 57 associates the skin color, makeup type, and the generated remake-up function with each other and registers them in the restoration / remake-up table.
After step S42, the restoration / remake-up table generation process ends.

Next, the restoration process function generation process executed in step S39 of the restoration / remakeup table generation process will be described.
FIG. 8 is a flowchart for explaining the flow of the restoration process function generation process executed in step S39 of the restoration / remakeup table generation process.
When the function generation processing for restoration processing is started, in step S51, the table management unit 57 selects one of the combinations of skin color and makeup type classified in the classified face image data.
In step S <b> 52, the image acquisition unit 51 refers to the classified face image data selected by the table management unit 57, and sets a face image and a face image to which makeup is applied for the same person. Acquire 1 set (for 1 person).

In step S <b> 53, the face detection unit 52 acquires the position of the main part (eyes, nose, mouth, etc.) of the face in each of the face image of the same person without makeup and the face image with makeup. To do.
In step S <b> 54, the function generation unit 58 matches the position of the acquired main part of the face between the face image of the same person who has not been put on the face and the face image that has been put on the face.

In step S55, the function generation unit 58 acquires the amount of change in the face image from the face image with makeup applied to the face image without makeup applied to the same person. At this time, a difference (a value including a positive / negative sign) of each pixel in the face image without makeup is acquired as a change amount with reference to the face image with makeup.
In step S56, the table management unit 57 displays the face image without makeup and the face image with makeup for all persons belonging to the combination of the skin color and makeup type selected in step S51. It is determined whether or not processing has been completed for the pair.
When the processing is completed for a set of face images that have not been subjected to makeup and face images that have undergone makeup for all persons belonging to the combination of skin color and makeup type selected in step S51, In S56, YES is determined, and the process proceeds to step S57.
On the other hand, the processing has been completed for the set of the face image and the face image with the makeup applied to all persons belonging to the combination of the skin color and the makeup type selected at step S51. If not, NO is determined in step S56, and the process proceeds to step S52.

  In step S57, the function generation unit 58 generates a restoration processing function for the skin color and makeup type based on the change amounts of the plurality of face images acquired in step S55. At this time, with respect to the face image of the person belonging to the skin color and the type of makeup, the amount of change in the face image from the average face image with makeup to the average face image without makeup ( An average change amount) is generated, and the average change amount represents a restoration processing function. That is, for a plurality of sample persons, mask data is generated by averaging the amount of change in face image from a face image with makeup to a face image without makeup. When applying a restoration processing function to a specific person's face image, by changing the size and shape of this mask according to the face of the person to be applied, and associating the average amount of change of each pixel The face image of the person can be converted into an unfaced face image. In the present embodiment, a face without makeup is generated by α-blending the average amount of change, which is a function for restoration processing, to each pixel in the face image of a person with makeup. The

In step S58, the table management unit 57 determines whether or not the processing has been completed for all combinations of skin color and makeup type classified in the classified face image data.
If the process has not been completed for all combinations of skin color and makeup type classified in the classified face image data, NO is determined in step S58, and the process proceeds to step S51.
On the other hand, if the processing is completed for all combinations of skin color and makeup type classified in the classified face image data, YES is determined in step S58, and the processing is restored / remake-up table generation processing. Return.

Next, the remake-up function generation process executed in step S41 of the restoration / remake-up table generation process will be described.
FIG. 9 is a flowchart for explaining the flow of the remake-up function generation process executed in step S41 of the restoration / remake-up table generation process.
When the remake-up function generation process is started, in step S61, the table management unit 57 selects one of the combinations of the skin color and makeup type classified in the classified face image data.
In step S <b> 62, the image acquisition unit 51 refers to the classified face image data selected by the table management unit 57, and sets a face image and a face image to which makeup is applied for the same person. Acquire 1 set (for 1 person).

In step S <b> 63, the face detection unit 52 acquires the position of the main part (eyes, nose, mouth, etc.) of the face in each of the face image without makeup and the face image with makeup applied to the same person. To do.
In step S <b> 64, the function generation unit 58 matches the position of the acquired main part of the face between the face image of the same person who has not been put on makeup and the face image that has been put on.

In step S <b> 65, the function generation unit 58 acquires the amount of change in the face image from the face image of the same person to which the makeup has been applied to the face image to which makeup has been applied. At this time, a difference (a value including a positive / negative sign) of the pixel value for each pixel in the face image with makeup is acquired as a change amount with reference to the face image without makeup.
In step S <b> 66, the table management unit 57 determines whether or not the processing has been completed for a set of face images with makeup and face images without makeup for all persons.
When the process is completed for a set of face images with makeup and face images with no makeup for all persons, YES is determined in step S66, and the process proceeds to step S67.
On the other hand, if the process has not been completed for the set of face images with makeup and face images without makeup for all persons, NO is determined in step S66, and the process proceeds to step S62. Transition.

  In step S67, the function generator 58 generates a remake-up function for the skin color and makeup type based on the change amounts of the plurality of face images acquired in step S65. At this time, with respect to the face image of a person belonging to the skin color and the type of makeup, the amount of change in the face image from the average face image without makeup to the average face image with makeup ( Average change amount) is generated, and the average change amount represents the remake-up function. That is, for a plurality of sample persons, mask data is generated by averaging the amount of change of the face image from the face image without makeup to the face image with makeup. When applying a remake-up function to a specific person's face image, by changing the size and shape of this mask according to the face of the person to be applied, and by associating the average amount of change of each pixel The face image of the person can be converted into a face image with makeup. In the present embodiment, a face with makeup is generated by α-blending the average amount of change, which is a remake-up function, to each pixel in the face image of a person without makeup. The At this time, the strength for applying makeup can be changed by changing the value of α in multiple stages according to various conditions. For example, it is possible to change α in accordance with the size of the face (select α so that the larger the face, the weaker the intensity of applying makeup).

In step S68, the table management unit 57 determines whether or not the processing has been completed for all combinations of skin color and makeup type classified in the classified face image data.
If the processing has not been completed for all combinations of skin color and makeup type classified in the classified face image data, NO is determined in step S68, and the process proceeds to step S61.
On the other hand, when all the combinations of skin color and makeup type classified in the classified face image data are processed, YES is determined in step S68, and the process is restored / remake-up table generation process. Return.

As described above, in the image processing apparatus 1 according to the present embodiment, the restoration process for changing the face of the subject to which makeup is applied to the unpinned state can be performed and the state is set to the unpinned state. Facial treatment including whitening and skin treatment can be performed on the face of the subject. The image processing apparatus 1 according to the present embodiment can perform remake-up processing for applying makeup specified by the user on the face of the subject that has been subjected to the beauty processing by image processing.
As a result, it is possible to prevent a difference in the effect of the face treatment regardless of the state of the face of the person such as the makeup or the darkness of the makeup, and after applying the face treatment, Image processing for applying makeup can be executed.

[Second Embodiment]
Next, a second embodiment of the present invention will be described.
In the image processing apparatus 1 according to the first embodiment, the restoration process is performed using the restoration / remake-up table, the face processing in the image to be processed is performed in the pinned state, and then the restoration / remake-up table is further performed. The remake-up process was performed using On the other hand, the image processing apparatus 1 according to the present embodiment standardizes the face in the image to be processed using the person characteristic standardization table and the shooting condition standardization table, which will be described later, and performs the facial processing on the standardized face. The difference is that a predetermined remake-up process is appropriately performed after that, and a standardized inverse transformation is further performed.
In the following, differences from the first embodiment will be mainly described.

FIG. 10 is a functional block diagram illustrating a functional configuration for executing the makeup face generation process among the functional configurations of the image processing apparatus 1 according to the present embodiment.
In the present embodiment, when the makeup face generation process is executed, a standardized table storage unit 72 is set in one area of the storage unit 19 as shown in FIG.
The standardization table storage unit 72 stores a person characteristic standardization table that stores restoration processing functions corresponding to various elements representing the characteristics of a person, and a photographing that stores restoration processing functions corresponding to various elements representing photographing conditions. A condition standardization table is stored.
Here, the characteristics of the person are elements related to the person as the subject, such as age, gender, race, and the like.
The shooting conditions are elements related to shooting conditions such as shooting time, shooting position, climate, and color temperature.
The restoration process in the present embodiment refers to the person characteristic standardization table and the photographing condition standardization table, and the person's face image to be subjected to the makeup face generation process is converted into a standard face state corresponding to the person characteristic. In this process, the face state is converted under standard shooting conditions.

FIG. 11 is a schematic diagram illustrating a data configuration example of the person characteristic standardization table.
As shown in FIG. 11, the person characteristic standardization table stores a restoration processing function representing an image processing method for converting a face image into a standard state for each element representing a person characteristic. ing. In FIG. 11, the restoration processing function is represented by the symbol F, and the difference in the restoration processing function F is indicated by a suffix j (j is a natural number).
The restoration processing function in FIG. 11 includes elements (characters (10's, 20's ...), gender (male, female), race (white, yellow, black) ...) Etc.)).

FIG. 12 is a schematic diagram illustrating a data configuration example of the imaging condition standardization table.
As shown in FIG. 12, the photographing condition standardization table stores a restoration processing function representing an image processing method for converting a face image into a standard state for each element representing the photographing condition. Yes. In FIG. 12, the restoration processing function is represented by a symbol G, and the difference in the restoration processing function G is indicated by a suffix k (k is a natural number).
The restoration processing function in FIG. 12 corresponds to each combination of each element (shooting time (hour), shooting position (latitude, latitude, altitude), climate (climate classification), color temperature (K), etc.) representing the shooting conditions. It is characteristic.

  In the present embodiment, the determination unit 53 determines the state of the face detected by the face detection unit 52 from the image acquired by the image acquisition unit 51. Specifically, based on the face image detected by the face detection unit 52 from the image acquired by the image acquisition unit 51, the determination unit 53 has each face state of each element representing the characteristics of the person. Determine which is the case. Further, the determination unit 53 determines which of the elements representing the shooting conditions corresponds to the shooting condition when the image in which the face is detected by the face detection unit 52 is shot. For example, the discriminating unit 53 specifies by analyzing the face information whether the face detected by the face detecting unit 52 corresponds to the age group of “10's”, “20's”, etc. . In addition, the determination unit 53 specifies the shooting time, the shooting position, and the like from the information indicating the shooting condition of the image in which the face is detected by the face detection unit 52. In addition, the information showing the imaging conditions of the image in which the face is detected by the face detection unit 52 is acquired from the imaging conditions when the imaging unit 16 captures the image when the image is captured by the imaging unit 16. Can do. Further, when this image is acquired from the outside, it can be acquired from information representing the imaging condition added to the image.

The adjustment unit 54 refers to the person characteristic standardization table and the photographing condition standardization table, and combines a restoration processing function corresponding to each element representing the characteristic of the person determined by the determination unit 53 and each element representing the photographing condition. And a restoration processing function corresponding to. Then, the adjustment unit 54 applies each acquired restoration processing function to the face image detected by the face detection unit 52, and changes the face detected by the face detection unit 52 to a standard state (character characteristics). The corresponding standard face state, and the face state under standard shooting conditions).
That is, the restoration process is executed by the determination unit 53 and the adjustment unit 54.
In the restoration process, standardization for each element representing the characteristics of the person (application of a restoration processing function corresponding to each element representing the characteristics of the person) and standardization for each element representing the shooting conditions (each representing the shooting conditions) Application of the restoration processing function corresponding to the element) may be performed.

  The face processing unit 55 performs face processing on the face image that has been brought into a standard state by the adjustment unit 54. In the present embodiment, the face beauty processing unit 55 performs a whitening process and a skin beautification process on the face image as the face treatment.

The remake-up unit 56 performs image processing for applying the type of makeup specified by the user to the face image that has been subjected to the facial processing by the facial processing unit 55. At this time, the remake-up unit 56 can apply makeup to the face image that has been subjected to the facial treatment process using a known process for applying makeup by image processing. In addition, when aiming at beautiful face processing, image processing for applying makeup can be omitted.
In addition, the remake-up unit 56 according to the present embodiment applies an inverse function of the restoration processing function used in the restoration process in the adjustment unit 54 to the face image that has undergone the facial beauty process. That is, the remake-up unit 56 performs reverse conversion of processing for converting the face detected by the face detection unit 52 into a standard state. In this case, the difference between the image of the face that is the target of the makeup face generation process and the image of the face that is in the standard state is applied again to the face of the person who has been subjected to the facial treatment in the standard state. Therefore, after performing facial processing in a state suitable for facial processing, it is converted to a face that reflects the characteristics of the person and the situation at the time of shooting (suitable for facial processing before standard state) Can reflect the characteristics of no state).

  The remake-up unit 56 performs a smoothing process on the face image after the remake-up process according to the type of the re-make-up process. For example, the remake-up unit 56 performs a smoothing process for smoothing the boundary between the red area of the cheek and the skin color area of the face when a makeup having a dark cheek is applied in the remake-up process.

The table management unit 57 manages the data of the person characteristic standardization table and the shooting condition standardization table stored in the standardization table storage unit 72. Specifically, the table management unit 57 updates the person characteristic standardization table and the shooting condition standardization table to a new version, or corrects the contents of the person characteristic standardization table and the shooting condition standardization table according to a user instruction. Or
In the present embodiment, the method for generating the person characteristic standardization table and the shooting condition standardization table is not disclosed in particular, but can be generated by the same processing as the restoration / remake-up table in the first embodiment.

By performing the same process as the makeup face generation process (see FIG. 4) in the first embodiment under such a configuration, in the image processing apparatus 1 according to the present embodiment, the person who is the target of the makeup face generation process Is converted into a standard facial state corresponding to the characteristics of the person and converted to a facial state under standard shooting conditions (restoration process), Facial treatment including whitening treatment and skin treatment can be performed. Then, the image processing apparatus 1 according to the present embodiment performs a remake-up process for applying makeup specified by the user on the face of the subject that has undergone the facial beauty process by the image process, and further calculates an inverse function of the function for the restoration process. It can be applied (inverse conversion to standardization).
As a result, regardless of the person's face state such as the person's characteristics and imaging conditions, it is possible to prevent a difference in the effect of the facial treatment, and after applying the facial treatment, a predetermined makeup is applied, Furthermore, it is possible to execute image processing for converting into a face reflecting the characteristics of the person and the situation at the time of photographing (imaging situation). Further, such an effect can be realized by a simple process compared to the first embodiment.

[Modification 1]
In the first embodiment, when determining the face state of an image to be subjected to a makeup face generation process, the face image data and makeup of a state where makeup is not applied (suppin state) are applied to a plurality of persons. It is also possible to discriminate the face image data in the determined state with reference to the data classified by the person's skin color and makeup type (the “classified face image data” described above).
That is, in this modification, when the makeup face generation process is executed, the classified face image data is stored in the storage unit 19.
Then, when the restoration process is performed on the face in the image that is the target of the makeup face generation process, the determination unit 53 determines whether the face state detected by the face detection unit 52 is any of the classified face image data (various types). Skin color and various makeup types), that is, whether the makeup is applied in a state corresponding to any of various skin colors and various makeup types. At this time, the determination unit 53 includes image data of a face (a face on which makeup is applied) detected by the face detection unit 52 among face image data in a state where makeup is applied in the classified face image data. Can be discriminated that the face detected by the face detection unit 52 belongs to the classification to which the detected face image data belongs.

[Modification 2]
In the first embodiment, as in the second embodiment, each element representing the shooting condition of the image to be subjected to the makeup face generation process is specified, and the face detected by the face detection unit 52 is changed to the standard shooting condition. It is good also as converting into the state of the face under. In this case, it is possible to execute a restoration process on the face image that is assumed to be in the face state under the standard shooting conditions, so as to be in the suspend state.
Thereby, also in the configuration of the first embodiment, it is possible to reduce the influence of the difference in the photographing conditions and perform the facial processing in a more appropriate state for performing the facial processing.

The image processing apparatus 1 configured as described above includes a face detection unit 52, a determination unit 53, and an adjustment unit 54.
The face detection unit 52 detects the face of a person included in the image.
The determination unit 53 determines the state of the image portion of the detected human face.
The adjustment unit 54 adjusts the image portion of the person's face to a state suitable for performing the facial treatment process according to the determined face state of the person.
Thus, the face of the person can be adjusted to a state suitable for performing the facial processing, and image processing for applying a predetermined makeup can be executed after performing the facial processing.

The determination unit 53 associates information related to the face of a person in a state that is reference information and is not appropriate for performing the facial processing with information related to the face of the person in an appropriate state for performing the facial processing. The state of the image portion of the detected human face is determined with reference to the reference information.
Thereby, the state of the image part of the person's face can be appropriately determined with reference to the reference information prepared in advance.

The information relating to the face of a person who is not suitable for performing the facial treatment is information relating to the face of the person in a state where makeup has been applied. Such information is information relating to the face of a person who has not been put on makeup.
The determination unit 53 searches for information related to the face of the person who has been subjected to makeup, which is determined to match the image portion of the detected person's face, Determine the state.
The adjustment unit 54 converts the image portion of the person's face into a beauty face based on the information related to the face of the person who has not been subjected to the makeup corresponding to the information related to the face of the person who has been subjected to the determined makeup. Adjust to the appropriate state for processing.
Thereby, the face of the person in the state where makeup is applied can be easily adjusted to the state where makeup is not applied.

The information related to the face of the person in the makeup state is information acquired by photographing the face of the person in the makeup state, and the person in the state suitable for performing the facial treatment The information relating to the face is information obtained by photographing a face of a person who is not makeupd by the same person as that of the person who is makeupd.
Thus, the state of the image portion of the detected person's face using information obtained by actually photographing the face with the makeup and the face without the makeup of the same person. Can be determined.

The makeup preliminarily applied to the face of the person in the state where makeup is applied is preliminarily applied by the person himself / herself included in the image, or is applied in advance to the image portion of the person's face by image processing. Makeup.
As a result, both the makeup actually applied to the face and the makeup applied to the face by image processing can be used for the processing.

The information related to the face of the person that is not appropriate for performing the facial processing is the characteristics of the person inferred from the face of the person, and the information related to the face of the person in an appropriate state for performing the facial processing is This is the content of image processing to be performed according to the estimated characteristics of the person.
The determination unit 53 estimates the characteristics of the person from the detected image portion of the face of the person, and determines the state of the detected image portion of the face of the person.
The adjustment unit 54 performs image processing to be performed according to the determined characteristics of the person, thereby adjusting the image portion of the person's face to a state suitable for performing the face beauty process.
As a result, it is possible to adjust to a state suitable for performing the facial treatment process by the process suitable for the characteristics of the person.

The information related to the face of the person that is not appropriate for performing the facial processing is the imaging state when the image is captured, and the information related to the face of the person in an appropriate state for performing the facial processing is captured. This is image processing to be performed according to the situation.
The determination unit 53 acquires the imaging state of the image and determines the state of the image portion of the detected human face.
The adjustment unit 54 performs image processing to be performed according to the determined imaging state, thereby adjusting the image portion of the person's face to a state suitable for performing the facial treatment process.
Thereby, it is possible to adjust to a state suitable for performing the facial treatment process by the process suitable for the imaging state.

The image processing apparatus further includes a face processing unit that performs the face processing on the image portion of the face of the person adjusted to an appropriate state for performing the face processing.
This makes it possible to obtain a more appropriate face processing result.

The image processing unit further includes remake-up means for performing image processing corresponding to the makeup designated by the user on the image portion of the face of the person that has been subjected to the facial treatment.
This makes it possible to obtain a more appropriate remake process result.

Remake-up means for performing image processing for returning an image portion of the face of a person that has undergone facial processing to a state that is not appropriate for performing facial processing prior to adjustment while maintaining the state in which facial processing has been performed Further prepare.
As a result, it is possible to perform the facial treatment while more appropriately reflecting the characteristics of the person.

  In addition, this invention is not limited to the above-mentioned embodiment, The deformation | transformation in the range which can achieve the objective of this invention, improvement, etc. are included in this invention.

  For example, in each of the above-described embodiments, the facial beauty processing is performed by performing whitening processing and skin whitening processing on the face of the subject. However, the present invention is not limited thereto, and may be processing such as making the skin color brown. .

  Further, in each of the above-described embodiments, the makeup applied to the face of the image to be subjected to the makeup face generation process may be an actual makeup or a makeup applied by image processing. Also good.

  Further, in each of the above-described embodiments, the image that is the target of the makeup face generation process is a newly captured image in the image processing apparatus 1 or a recorded image (such as an image acquired from the outside) is used. Any of these can be done. When a recorded image is used and the shooting condition of the image is used for processing, it is desirable to use a recorded image to which information indicating the shooting condition is added.

In the above-described embodiment, the image processing apparatus 1 to which the present invention is applied has been described using a digital camera as an example, but is not particularly limited thereto.
For example, the present invention can be applied to general electronic devices having an image processing function. Specifically, for example, the present invention can be applied to a notebook personal computer, a printer, a television receiver, a video camera, a portable navigation device, a mobile phone, a smartphone, a portable game machine, and the like.

The series of processes described above can be executed by hardware or can be executed by software.
In other words, the functional configuration of FIG. 2 is merely an example and is not particularly limited. That is, it is sufficient that the image processing apparatus 1 has a function capable of executing the above-described series of processing as a whole, and what functional blocks are used to realize this function is not particularly limited to the example of FIG. .
In addition, one functional block may be constituted by hardware alone, software alone, or a combination thereof.
The functional configuration in the present embodiment is realized by a processor that executes arithmetic processing, and the processor that can be used in the present embodiment is configured by various processing devices such as a single processor, a multiprocessor, and a multicore processor. In addition to the above, these various processing apparatuses and a combination of processing circuits such as ASIC (Application Specific Integrated Circuit) and FPGA (Field-Programmable Gate Array) are included.

When a series of processing is executed by software, a program constituting the software is installed on a computer or the like from a network or a recording medium.
The computer may be a computer incorporated in dedicated hardware. The computer may be a computer capable of executing various functions by installing various programs, for example, a general-purpose personal computer.

  The recording medium including such a program is not only constituted by the removable medium 31 of FIG. 1 distributed separately from the apparatus main body in order to provide the program to the user, but also in a state of being incorporated in the apparatus main body in advance. The recording medium etc. provided in The removable medium 31 is composed of, for example, a magnetic disk (including a floppy disk), an optical disk, a magneto-optical disk, or the like. The optical disc is composed of, for example, a CD-ROM (Compact Disk-Read Only Memory), a DVD (Digital Versatile Disc), a Blu-ray (registered trademark) Disc (Blu-ray Disc), and the like. The magneto-optical disk is configured by an MD (Mini-Disk) or the like. In addition, the recording medium provided to the user in a state of being preliminarily incorporated in the apparatus main body includes, for example, the ROM 12 in FIG. 1 in which a program is recorded, the hard disk included in the storage unit 19 in FIG.

  In the present specification, the step of describing the program recorded on the recording medium is not limited to the processing performed in chronological order according to the order, but is not necessarily performed in chronological order, either in parallel or individually. The process to be executed is also included.

  As mentioned above, although several embodiment of this invention was described, these embodiment is only an illustration and does not limit the technical scope of this invention. The present invention can take other various embodiments, and various modifications such as omission and replacement can be made without departing from the gist of the present invention. These embodiments and modifications thereof are included in the scope and gist of the invention described in this specification and the like, and are included in the invention described in the claims and the equivalents thereof.

The invention described in the scope of claims at the beginning of the filing of the present application will be appended.
[Appendix 1]
Detecting means for detecting a human face included in the image;
A discriminating means for discriminating the state of the image portion of the detected human face;
Adjusting means for adjusting the image portion of the person's face to an appropriate state for performing the facial treatment according to the determined state of the person's face;
An image processing apparatus comprising:
[Appendix 2]
The discriminating means associates information relating to a person's face in a state that is reference information and is not suitable for performing a facial treatment, and information relating to a person's face in a state suitable for performing the facial treatment. The image processing apparatus according to claim 1, wherein the state of the image portion of the detected human face is determined with reference to the reference information.
[Appendix 3]
The information relating to the face of the person who is not suitable for performing the facial treatment is information relating to the face of the person in the state of being applied with makeup, and the information of the person in an appropriate state for performing the facial treatment The information about the face is information about the face of a person who has not been put on makeup,
The discriminating means searches for information relating to the face of the person with the makeup applied, which is determined to match the image portion of the detected person's face, and the face of the person with the makeup applied in advance Determine the state of
The adjusting means is configured to determine an image portion of the person's face based on the information related to the face of the person who is not applied with the makeup corresponding to the information related to the face of the person who is applied with the determined makeup. The image processing apparatus according to appendix 2, wherein the image processing apparatus is adjusted to a state suitable for performing the facial treatment.
[Appendix 4]
The information relating to the face of the person in the makeup state is information acquired by photographing the face of the person in the makeup state, and is appropriate for performing the face processing. The information relating to the face of the person in the state is information acquired by photographing a face in a state where the makeup of the same person as the person in the state of makeup is not applied. An image processing apparatus according to 1.
[Appendix 5]
The makeup preliminarily applied to the face of the person in a state where the makeup is applied is applied in advance to the makeup included in the image by the person himself or the image portion of the person's face in advance by image processing. The image processing apparatus according to appendix 3, wherein the makeup is makeup.
[Appendix 6]
The information related to the face of the person that is not suitable for performing the face processing is a characteristic of the person estimated from the face of the person, and the information related to the face of the person in a state suitable for performing the face processing Is the content of the image processing to be performed according to the estimated characteristics of the person,
The determining means estimates the characteristics of the person from the detected image portion of the face of the person, determines the state of the detected image portion of the face of the person,
From the supplementary note 2, the adjustment means adjusts the image portion of the person's face to an appropriate state for performing the facial treatment process by performing image processing to be performed according to the determined characteristics of the person. The image processing apparatus according to any one of 5.
[Appendix 7]
The information related to the face of the person that is not appropriate for performing the face processing is an imaging state when the image is captured, and the information related to the face of the person in an appropriate state for performing the face processing is , Image processing to be performed according to the imaging situation,
The determining means acquires the imaging state of the image, determines the state of the image portion of the detected human face,
From the supplementary note 2, the adjusting means adjusts the image portion of the person's face to an appropriate state for performing the beauty processing by performing image processing to be performed according to the determined imaging state. The image processing apparatus according to any one of 5.
[Appendix 8]
The image processing according to any one of appendices 1 to 7, further comprising a facial processing unit that performs facial processing on an image portion of a human face adjusted to an appropriate state for performing the facial processing. apparatus.
[Appendix 9]
The image processing apparatus according to appendix 8, further comprising remake-up means for performing image processing corresponding to the makeup designated by the user on the image portion of the face of the person that has been subjected to the beautiful face processing.
[Appendix 10]
Remake-up means for performing image processing for returning the image portion of the person's face subjected to the facial processing to a state not suitable for performing the facial processing before adjustment while maintaining the state where the facial processing is performed The image processing apparatus according to appendix 8, further comprising:
[Appendix 11]
An image processing method executed by an image processing apparatus,
A detection process for detecting a human face included in the image;
A discrimination process for discriminating the state of the image portion of the detected human face;
An adjustment process that adjusts the image portion of the person's face to an appropriate state for performing the beauty process according to the determined face state of the person,
An image processing method comprising:
[Appendix 12]
On the computer,
A detection function for detecting a human face included in the image;
A discriminating function for discriminating the state of the image portion of the detected human face;
An adjustment function that refers to the reference information and adjusts the image portion of the person's face to an appropriate state for performing the facial treatment process according to the determined face state of the person;
A program characterized by realizing.

  DESCRIPTION OF SYMBOLS 1 ... Image processing apparatus, 11 ... CPU, 12 ... ROM, 13 ... RAM, 14 ... Bus, 15 ... Input / output interface, 16 ... Imaging part, 17 ... Input unit 18 ... Output unit 19 ... Storage unit 20 ... Communication unit 21 ... Drive 31 ... Removable media 51 ... Image acquisition unit 52 ... Face detecting unit, 53... Discriminating unit, 54... Adjusting unit, 55 .. beauty face processing unit, 56 .. remake-up unit, 57. 71 ... Restoration / remake-up table storage unit, 72 ... Standardized table storage unit

Claims (12)

Detecting means for detecting a human face included in the image;
A discriminating means for discriminating the state of the image portion of the detected human face;
Adjusting means for adjusting the image portion of the person's face to an appropriate state for performing the facial treatment according to the determined state of the person's face;
An image processing apparatus comprising:   The discriminating means associates information relating to a person's face in a state that is reference information and is not suitable for performing a facial treatment, and information relating to a person's face in a state suitable for performing the facial treatment. The image processing apparatus according to claim 1, wherein the state of the image portion of the detected human face is determined with reference to the reference information. The information relating to the face of the person who is not suitable for performing the facial treatment is information relating to the face of the person in the state of being applied with makeup, and the information of the person in an appropriate state for performing the facial treatment The information about the face is information about the face of a person who has not been put on makeup,
The discriminating means searches for information relating to the face of the person with the makeup applied, which is determined to match the image portion of the detected person's face, and the face of the person with the makeup applied in advance Determine the state of
The adjusting means is configured to determine an image portion of the person's face based on the information related to the face of the person who is not applied with the makeup corresponding to the information related to the face of the person who is applied with the determined makeup. The image processing apparatus according to claim 2, wherein the image processing apparatus is adjusted to a state suitable for performing the facial treatment.   The information relating to the face of the person in the makeup state is information acquired by photographing the face of the person in the makeup state, and is appropriate for performing the face processing. The information related to the face of the person in the state is information obtained by photographing a face in a state where the makeup of the same person as the person in the state of being applied is not applied. The image processing apparatus according to 3.   The makeup preliminarily applied to the face of the person in a state where the makeup is applied is applied in advance to the makeup included in the image by the person himself or the image portion of the person's face in advance by image processing. The image processing apparatus according to claim 3, wherein the image processing apparatus is makeup. The information related to the face of the person that is not suitable for performing the face processing is a characteristic of the person estimated from the face of the person, and the information related to the face of the person in a state suitable for performing the face processing Is the content of the image processing to be performed according to the estimated characteristics of the person,
The determining means estimates the characteristics of the person from the detected image portion of the face of the person, determines the state of the detected image portion of the face of the person,
The adjustment means adjusts an image portion of a person's face to an appropriate state for performing a facial treatment process by performing image processing to be performed according to the determined characteristics of the person. 6. The image processing apparatus according to any one of items 1 to 5. The information related to the face of the person that is not appropriate for performing the face processing is an imaging state when the image is captured, and the information related to the face of the person in an appropriate state for performing the face processing is , Image processing to be performed according to the imaging situation,
The determining means acquires the imaging state of the image, determines the state of the image portion of the detected human face,
The adjustment means adjusts an image portion of a person's face to an appropriate state for performing a facial treatment process by performing image processing to be performed according to the determined imaging situation. 6. The image processing apparatus according to any one of items 1 to 5.   The image according to any one of claims 1 to 7, further comprising a face processing unit that performs a face processing on an image portion of a person's face adjusted to an appropriate state for performing the face processing. Processing equipment.   9. The image processing apparatus according to claim 8, further comprising remake-up means for performing image processing corresponding to the makeup designated by the user on the image portion of the face of the person that has undergone the beauty processing.   Remake-up means for performing image processing for returning the image portion of the person's face subjected to the facial processing to a state not suitable for performing the facial processing before adjustment while maintaining the state where the facial processing is performed The image processing apparatus according to claim 8, further comprising: An image processing method executed by an image processing apparatus,
A detection process for detecting a human face included in the image;
A discrimination process for discriminating the state of the image portion of the detected human face;
An adjustment process that adjusts the image portion of the person's face to an appropriate state for performing the beauty process according to the determined face state of the person,
An image processing method comprising: On the computer,
A detection function for detecting a human face included in the image;
A discriminating function for discriminating the state of the image portion of the detected human face;
An adjustment function that refers to the reference information and adjusts the image portion of the person's face to an appropriate state for performing the facial treatment process according to the determined face state of the person;
A program characterized by realizing.

Images