JP5874440B2 - Image processing apparatus and image processing program - Google Patents

Description

  The present invention relates to an image processing apparatus and an image processing program.

  Conventionally, as a method for specifying a skin area of a face included in an image, a method using color space information or a method using contour information existing in the vicinity of an assumed face shape is known. For example, in Patent Document 1, for each pixel constituting a pixel group extracted with high reliability, the hue, saturation, and brightness of a predetermined area adjacent to the pixel are examined, and an area that satisfies a predetermined condition is added as a skin area. , Grow skin area. Then, the skin region is obtained by repeating until there are no neighboring pixels to be grown or the number of repetitions of this process reaches a predetermined number.

JP 2006-338377 A

  However, in the above-described specifying method of Patent Document 1, since a condition comparison is performed for each pixel, there is a problem that a discrete region is generated in the vicinity of a threshold value that is a condition, and the shape is not stable. In addition, when comparisons are simply made in the color space, it is necessary to widen the condition range in order to cope with the shooting environment of the subject, makeup, and skin color, but it prevents erroneous extraction of areas other than the face and skin. In order to do so, the condition width must be narrowed, and it is difficult to extract a skin region with high accuracy.

  Therefore, an object of the image processing apparatus and the image processing program of the present invention is to specify a facial skin region with high accuracy.

  An image processing apparatus according to the present invention includes: a first specifying unit that specifies a first area that is a part of a face area of a subject from an image; a color of a pixel that forms the first area; and a part of the first area. The similarity of the colors of the pixels constituting the second area, the expected proportion of the face area in the image, and the brightness of the pixels constituting the second area and the brightness of the pixels adjacent to the pixels Using the similarity, a calculation unit that calculates a value indicating the state of the second region, and a value indicating the state of the second region calculated by the calculation unit is specified as the skin region And a second specifying unit.

  The second specifying unit changes the shape of the second region until the value indicating the state of the second region calculated by the calculation unit is the most preferable value, and indicates a value indicating the state of the second region. You may identify the said 2nd area | region when becomes the most preferable value as a skin area | region.

  The second specifying unit may perform at least one of a process for enlarging the second area and a process for reducing the second area.

  The image processing apparatus further includes an extraction unit that extracts at least one eye region and a mouth region from the image, and the first region is located on a line connecting any pixel in the eye region and any pixel in the mouth region. A pixel region, a triangular region connecting any pixel in each of the eye regions and any pixel in the mouth region, or any pixel in each of the eye regions as a vertex, and any pixel in the mouth region Any of the rectangular area | regions included in a base may be sufficient.

  Further, the calculation unit may calculate a value indicating the state of the second region by an optimal solution.

  The most preferable value may be a minimum value.

  The most preferable value may be a local solution.

  In addition, the second specifying unit may change the shape of the second area after excluding a color area that is not a skin color from the first area.

  Moreover, you may provide the image process part which performs a beautification process to the said 2nd area | region specified as an area | region of skin.

  The image processing program of the present invention includes: a first specifying step for specifying a first area that is a part of a face area of a subject from an image; a color of a pixel that forms the first area; and a part of the first image. The similarity of the colors of the pixels constituting the second area, the expected proportion of the face area in the image, and the brightness of the pixels constituting the second area and the brightness of the pixels adjacent to the pixels Based on the calculation step for calculating the value indicating the state of the second region using the similarity, and the value indicating the state of the second region calculated in the calculation step, the second region is defined as a skin region. The second specifying step to be specified is realized by a computer.

  According to the image processing apparatus and the image processing program of the present invention, it is possible to accurately specify the facial skin area.

It is a block diagram which shows the structure of the imaging device 10 in 1st embodiment. (A) It is a figure which shows the face area | region F, the outline area | region R, and the skin color candidate area | region S. FIG. (B) is a diagram showing a schematic region _{R 2.} (C) is a diagram showing a schematic region _{R 3.} 5 is a diagram showing a detailed area D. FIG. It is a flowchart which shows the flow of the specific process of the detailed area | region D in 1st Embodiment. (A) It is a figure which shows the area | region I. FIG. (B) It is a figure which shows area | region I '. (C) It is a figure which shows area | region I ''. It is a figure which shows the relationship between energy difference (DELTA) Ei and the probability P. FIG. It is a flowchart which shows the flow of the specific process of the detailed area | region D in 2nd Embodiment.

(First embodiment)
Hereinafter, a first embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram illustrating a configuration of an imaging apparatus 10 according to the first embodiment. As shown in FIG. 1, the imaging device 10 includes an imaging lens 11, an imaging device 12, an A / D conversion unit 13, a buffer memory 14, a display unit 15, a flash RAM 16, a recording I / F unit 17, a recording medium 18, and an operation. A unit 19, a CPU 20, and a bus 21 are provided.

  The imaging lens 11 forms a subject image on the imaging surface of the imaging element 12. The imaging element 12 photoelectrically converts the subject light that has passed through the imaging lens 11 and outputs analog image signals corresponding to R, G, and B colors. An image signal output from the image sensor 12 is input to the A / D converter 13. The A / D converter 13 converts the analog image signal output from the image sensor 12 into a digital image signal. The digital image signals are collected into one frame and recorded in the buffer memory 14 as image data. The buffer memory 14 temporarily records image data in the pre-process and post-process of image processing by the image processing unit 32 described later.

  The display unit 15 displays various images. Various images displayed on the display unit 15 are an image acquired by imaging, a menu image, an image recorded on a recording medium 18 to be described later, and the like. The flash RAM 16 stores firmware and the like. The recording I / F unit 17 includes a connector for connecting the recording medium 18. By connecting the recording I / F unit 17 and the recording medium 18, data writing / reading is executed with respect to the recording medium 18.

  The operation unit 19 includes a release button 24, a cross key 25, and the like. The release button 24 is operated by the user during imaging. The cross key 25 is operated on the above menu image or the like. The states of the release button 24 and the cross key 25 are detected by the CPU 20, and a sequence based on the detected button and key states is executed.

  The CPU 20 performs overall control of the imaging apparatus 10 according to a predetermined sequence program, and executes various calculations (AF, AE, etc.) necessary for imaging. Further, the CPU 20 includes an extraction unit 29, a schematic region specifying unit 30, a detailed region specifying unit 31, and an image processing unit 32. FIG. 2A is a diagram illustrating an example of an image (reference numeral 40) to be processed. The image 40 may be an image acquired by imaging or an image recorded on the recording medium 18.

  The extraction unit 29 performs, for example, eyebrow, eye, nose, lip end points, face contour points, head vertices, and lower end of chin from an image by feature point extraction processing described in JP-A-2001-16573. Extract points. A region surrounded by a thick line in FIG.

The approximate region specifying unit 30 specifies the approximate region R based on the end points of the eyes and lips extracted by the extracting unit 29. The approximate area R is a part of the face area F and is an area that can be reliably determined to be included in the face area F. A region surrounded by a dotted line in FIG. As shown in FIG. 2A, the approximate area specifying unit 30 specifies a rectangular area having the centers of both eyes as apexes and including the center of the mouth at the bottom as the approximate area R. Incidentally, as shown in FIG. 2 (b), schematic region identifying unit 30, and the respective centers of eyes, the area of the triangle formed by connecting the center of the mouth may be identified as the approximate region R _2. Further, as shown in FIG. 2 (c), a schematic region identifying unit 30, and the center of the eye, the pixels located on a line connecting the center of the mouth may be identified as the approximate region R _3.

  The detailed area specifying unit 31 specifies the skin color candidate area S and the detailed area D. First, the skin color candidate region S will be described. The skin color candidate area S is an area obtained by excluding pixels having color coordinates from which the skin color cannot be regarded as existing in the approximate area R. As the exclusion target, for example, an object covering the eyes, mouth, and face, glasses, and cheek painting are assumed. The detailed area specifying unit 31 converts image data in the RGB color space into image data in the HSV color space. Then, the detailed area specifying unit 31 specifies the skin color candidate area S by excluding pixels whose hue is out of the range of 20 to 50 ° in the HSV color space. When the general region R is a rectangular region, the skin color candidate region S is indicated by the hatched region in FIG.

  Next, the detailed area D will be described. The detailed area D is a skin area in the face area F specified by enlarging and reducing the skin color candidate area S. Hereinafter, the skin color candidate region S is defined as an initial region, and a region in the middle from the initial region to the detailed region D is defined as a region I. The detailed area specifying unit 31 calculates a value indicating the state of the area I, and specifies the area I when this value is the most preferable value as the detailed area D. The area painted with dots in FIG. The specification of the detailed area D will be described later.

  In the first embodiment, a simulated annealing method is used as a method for obtaining an optimal solution. Simulated annealing simulates a physical process (annealing) that creates an excellent crystal structure with few defects by gradually lowering the temperature of a metal heated at a high temperature. It is an optimization method. In this method, the state with the lowest energy (optimum solution) is approached by repeating the change from the current state to the state with lower energy. At this time, if only the low energy state is continuously selected, there is a possibility of falling into a local solution, but the feature of this method is that an increase in energy is recognized with a certain probability.

  Next, energy E indicating the current state of the region I is expressed by the following equation (1). Each pixel of the image 40 is assigned a skin color or background label.

  In the equation (1), V represents a pixel of interest, and Pa represents a group of 4 adjacent (or 8 adjacent) pixel pairs of the pixel of interest. U and v are any two pixels (hereinafter referred to as adjacent pixels) in the pixel pair group, and δ and γ are coefficients.

X is a label assigned to each pixel,
X = skin if skin background otherwise.

  First, the first term on the right side of the equation (1) will be described. The first term on the right side of the equation (1) represents skinness and energy obtained from the ratio between the skin and the background. That is, the first term on the right side of the equation (1) indicates that the ratio of the face area F to the background area increases as the area composed of pixels to which the skin color label is assigned matches the actual skin area. The closer to the assumed ratio, the lower the value.

G _V (X _V ) of the first term on the right side of the equation (1) is represented by the following equation (2).

In the expression (2), A is the color (skin color) of the pixel constituting the skin color candidate area S, B is the color of the target pixel, and AB is the color of the pixel constituting the skin color candidate area S and the color of the target pixel. Difference (color difference), X indicates the label of the pixel of interest. For example, R _A , G _A , and B _A are an average value of R, an average value of G, and an average value of B, respectively, of the pixels constituting the skin color candidate region S. Further, α is a coefficient that predetermines the ratio of the skin area to the image 40, and β is a coefficient that determines the degree of skininess obtained from the skin label pixel.

The first term on the right side of equation (2) represents that the value β || AB || ² is given only when the label of the pixel is skin. The second term on the right side of the equation (2) indicates that α is given when the pixel label is skin, and (1-α) is given when the pixel label is background.

（３）式において、詳細領域特定部３１は、隣接画素ｕ、ｖのラベルＸ_ｕ、Ｘ_ｖが異なる場合に、κの値を与える。κの値は、隣接画素間の明るさが違うほど小さな値となる。 Next, the second term on the right side of the equation (1) will be described. The second term on the right side of equation (1) represents the degree of whether the brightness between adjacent pixels is the same. H _uv (X _u , X _V ) of the second term on the right side of the equation (1) is represented by the following equation (3).

In the expression (3), the detailed area specifying unit 31 gives the value of κ when the labels X _u and X _v of the adjacent pixels u and _v are different. The value of κ becomes smaller as the brightness between adjacent pixels is different.

  The detailed area specifying unit 31 operates so as to bring the energy E closer to the minimum. When the energy E is minimized, a region composed of pixels to which a skin color label is assigned becomes the detailed region D.

  The image processing unit 32 performs image processing on the image data recorded in the buffer memory 14. Examples of the image processing include well-known white balance adjustment, color interpolation, gradation conversion processing, and contour enhancement processing. The image processing unit 32 also executes processing for compressing in JPEG (Joint Photographic Experts Group) format and the like, and processing for decompressing and restoring the compressed data.

  Further, the image processing unit 32 performs a skin beautification process on the detailed region D specified by the detailed region specifying unit 31. For example, the image processing unit 32 performs exposure correction for correcting the exposure of the detailed area D to the over side. Thereby, the skin of the subject is expressed white. The image processing unit 32 performs soft focus processing. Thereby, when the subject is a woman, femininity is emphasized, softness is generated, and wrinkles, rough skin, and the like are less noticeable. Further, the image processing unit 32 corrects the saturation and lightness parameters to express the skin white and reduce dullness and the like.

The bus 21 connects the buffer memory 14, the display unit 15, the flash RAM 16, the recording I / F unit 17, and the CPU 20 to execute data and signal input / output.
Next, the flow of the detailed area D specifying process in the first embodiment will be described with reference to the flowchart of FIG. Note that the flowchart of FIG. 4 is executed when the release button 24 is fully pressed. In addition, hereinafter, for the sake of convenience, an example in which the detailed region D is specified from the image 40 obtained by capturing one person will be described.
Step S <b> 101 is processing for specifying the approximate region R. The extraction unit 29 extracts each end point of the eyes and lips from the image 40 acquired by imaging. Then, the approximate region specifying unit 30 specifies the approximate region R based on the end points of the eyes and lips.

  Step S102 is processing for specifying the skin color candidate region S. The detailed area specifying unit 31 converts image data in the RGB color space into image data in the HSV color space. Then, the detailed area specifying unit 31 specifies the skin color candidate area S by excluding pixels whose hue is out of the range of 20 to 50 ° from the approximate area R.

Step S103 is a process of calculating the initial energy _{E 0.} First, for the pixels included in the image 40, the detailed region specifying unit 31 assigns a skin color label to the pixels constituting the skin color candidate region S, and assigns a background label to the remaining pixels. Then, the detailed area specifying unit 31 uses the skin color candidate area S as the initial area, and calculates the initial energy E ₀ using the above-described expressions (1) to (3) as the same area as the area I and the initial area. .

Step S104 is processing to enlarge the area. The detailed area setting unit 31 assumes an area in which pixels PO ₁ to PO _n adjacent to the outside of the area I are added to the current area I. Hereinafter, an example in which an area in which pixels are added in order from the pixel PO ₁ is assumed will be described. The white area in FIG. 5A indicates the current area I, and the hatched area indicates the pixels PO ₁ to PO _n adjacent to the outside of the area I.

Step S105 is a process of performing acceptance determination. First, details area specifying unit 31 calculates the energy of the state assigned the label of skin color pixels PO _1, calculated with the energy, the difference ΔEi with energy in a state before assigning the label of the skin color pixel PO ₁ To do. Then, when ΔEi <0, the detailed area specifying unit 31 assigns a skin color label to the pixel PO ₁ with a probability of P = 1. On the other hand, when ΔEi> 0, a skin color label is assigned to the pixel PO ₁ with a probability of P = exp (−ΔEi / kT _c ). Note that k is Boltzmann's constant and _Tc is temperature. By assigning labels skin color pixel PO _1, region I, the partial pixel PO ₁ is enlarged.

  CPU20 progresses to step S106, when the next state is accepted (when determination of step S105 becomes YES). On the other hand, when the next state is not accepted (when the determination in step S105 is NO), the CPU 20 proceeds to step S113 described later.

Step S106 is a process of determining whether or not the processes of steps S104 and S105 have been performed on all the pixels PO _{1 to} PO _n adjacent to the outside of the region I. If the CPU 20 has processed all the pixels (when the determination in step S106 is YES), the CPU 20 proceeds to step S107. On the other hand, if the process has not been performed on all the pixels (when the determination in step S106 is NO), the CPU 20 returns to step S104 and assumes an enlarged area. As described above, the detailed region specifying unit 31 repeats the processes of Step S104 and Step S105, and assigns a skin color label when the next state is accepted for the pixels PO _{1 to} PO _n . An example of the enlarged region I ′ is shown in FIG.

Step S107 is processing to reduce the area. The detailed area setting unit 31 assumes an area obtained by excluding the pixels PI ₁ to PI _n located on the outer periphery of the area I ′ from the enlarged area I ′. Hereinafter, an example in which an area obtained by removing pixels one by _one from the pixel PI _{1 in} order is assumed. The area painted with dots in FIG. 5B indicates the pixels PI ₁ to PI _n located on the outer periphery of the area I ′.

Step S108 is a process of performing acceptance determination. First, details area specifying unit 31 calculates the energy of the state assigned the label of background pixel PI _1, calculated with the energy, the difference ΔEj of energy in a state before the pixel PI ₁ assign a label background To do. Then, when ΔEj <0, the detailed region specifying unit 31 assigns a background label to the pixel PI ₁ with a probability of P = 1. On the other hand, if ΔEj> 0, a background label is assigned to the pixel PI ₁ with a probability of P = exp (−ΔEj / kT _c ). By assigning a background label to the pixel PI ₁ , the area I ′ is reduced by the amount of the pixel PI ₁ .

  When the next state is accepted (when the determination in step S108 is YES), the CPU 20 proceeds to step S109. On the other hand, when the next state is not accepted (when the determination in step S108 is NO), the CPU 20 proceeds to step S113 described later.

Step S109 is a process of determining whether or not the processes of Step S107 and Step S108 have been performed on the pixels PI ₁ to PI _n located on the outer periphery of the region I ′. When the CPU 20 has processed all the pixels (when the determination in step S109 is YES), the CPU 20 proceeds to step S110. On the other hand, if the process has not been performed on all the pixels (when the determination in step S109 is NO), the CPU 20 returns to step S107. Thus, detailed area specifying unit 31, step S107, repeats the processing in step S108, the pixel _PI 1 _{~PI m,} if the next state is accepted, assigns a label background. An example of the reduced area I ″ is shown in FIG.

  Step S110 is a process of changing the state. The detailed area specifying unit 31 sets the area I ″ reduced in step S107 as the area I + 1 in the next state.

  Step S111 is processing to add 1 to the state change count.

  Step S112 is processing for determining whether or not the state change count has reached a threshold value. For example, the CPU 20 determines whether or not the state change count is greater than 100. If the state change count is greater than 100 (when the determination in step S112 is YES), the CPU 20 proceeds to step S113. On the other hand, if the state change count is 100 or less (NO in step S112), the CPU 20 returns to step S104 and assumes an enlarged area.

Step S113 is a process for determining termination. The CPU 20 determines whether or not the region I + 1 (0 ≦ l ≦ 100) has changed. The state where the region I + 1 does not change is a state where the temperature _Tc is sufficiently low and the energy of the region I + 1 is lowest. In this state, since the energy is the lowest and stable, even if a skin color label is assigned to a pixel adjacent to the outside of the region I + l, ΔEi> 0 is always satisfied, and the pixel located in the outer peripheral portion of the region I + l has a background. Even if the label is assigned, ΔEj> 0 always holds.

  When the CPU 20 determines that the region I + l has not changed (when the determination in step S113 is YES), the CPU 20 sets the region I + l as the detailed region D. Thereby, the skin area in the face area F becomes the detailed area D. Then, the CPU 20 proceeds to step S114. On the other hand, if the CPU 20 determines that the region I + 1 changes (when the determination in step S113 is NO), the CPU 20 proceeds to step S115.

  Step S114 is a process for performing skin beautification processing. The image processing unit 32 performs skin beautification processing on the detailed region D. Then, the image processing unit 32 records the image 40 ′ after the skin beautification process on the recording medium 18 and ends the series of processes.

Step S115 is a cooling process when the determination in step S113 is NO. The detailed area specifying unit 31 decreases the temperature _Tc . The method for lowering the temperature Tc is defined by the following equation. For example, assume that γ = 0.9.
T _{c + 1} = γT _c

Step S116 is processing to reset the state change count. The detailed area specifying unit 31 resets the state change count, returns to step S104, and assumes an enlarged area.
FIG. 6 is a diagram showing the relationship between the energy difference ΔEi and the probability P. The horizontal axis represents ΔEi, and the vertical axis represents the probability P. As shown in FIG. 6, P = 1 when ΔEi <0, and 0 <P <1 when ΔEi> 0. Further, the probability P at ΔEi> 0 depends on the temperature T _c . That is, as the temperature T _{c + 1} and the temperature T _{c + 2} are lowered and the temperature is lowered, the probability P approaches 0 and only advances in the direction of calculating the optimum solution.

  As described above, the imaging apparatus 10 according to the first embodiment calculates a value indicating the state of the region I, and specifies the detailed region D by finding an optimal solution that has the most preferable value. . Therefore, according to the imaging device 10 of the first embodiment, the detailed area D can be specified with high accuracy. Moreover, according to the imaging device 10 of 1st Embodiment, since the detailed area | region D can be pinpointed with sufficient precision, image processing, such as a skin-beautification process, can be performed to the detailed area | region D correctly.

(Second Embodiment)
Hereinafter, a second embodiment will be described. In 1st Embodiment, the example which specifies the detailed area | region D using the simulated annealing method was shown. In the second embodiment, an example in which the detailed region D is specified by setting the temperature T _c = 0 constant and enlarging the initial region will be described. Here, the configuration of the imaging apparatus in the second embodiment is the same as that of the imaging apparatus 10 of the first embodiment shown in FIG.

  Next, the flow of the detailed area D specifying process in the second embodiment will be described with reference to the flowchart of FIG. Note that the flowchart of FIG. 7 is executed when the release button 24 is fully pressed. In addition, hereinafter, for the sake of convenience, an example in which the detailed region D is specified from the image 40 obtained by capturing one person will be described.

  Steps S201 to S204 are the same processes as steps S101 to S104 in the flowchart of FIG.

Step S205 is a process of performing acceptance determination. The detailed area specifying unit 31 calculates a difference ΔEi between the energy in the state in which the skin color label is assigned to the pixel PO ₁ and the energy in the state before the skin color label is assigned. Then, when ΔEi <0, the detailed area specifying unit 31 assigns a skin color label to the pixel PO ₁ with a probability of P = 1. On the other hand, if ΔEi> 0, the process proceeds to step S208 described later. That is, in the second embodiment, the detailed area specifying unit 31 accepts the next state only when ΔEi <0.

  CPU20 progresses to step S206, when the next state is accepted (when determination of step S205 becomes YES). On the other hand, when the next state is not accepted (when the determination in step S205 is NO), the CPU 20 proceeds to step S207 described later.

  Step S206 is a process of changing the state. The detailed area specifying unit 31 sets the area I ′ expanded in step S205 as the area I + 1 in the next state.

Step S207 is processing for determining whether or not the processing of Step S204 to Step S206 has been performed on all the pixels PO _{1 to} PO _n adjacent to the outside of the region I. If the CPU 20 has processed all the pixels (if the determination in step S207 is YES), the CPU 20 proceeds to step S208. On the other hand, when the process has not been performed on all the pixels (when the determination in step S207 is NO), the CPU 20 returns to step S204 and assumes an enlarged area.

  Step S208 is processing for determining termination. The CPU 20 performs the same process as step S113, and determines whether or not the region I + q has changed. When the CPU 20 determines that the region I + q has not changed (when the determination in step S208 is YES), the CPU 20 sets the region I + q as the detailed region D. Thereby, the skin area in the face area F becomes the detailed area D. Then, the CPU 20 proceeds to step S209. On the other hand, if the CPU 20 determines that the region I + q changes (when the determination in step S208 is NO), the CPU 20 returns to step S204 and assumes an enlarged region.

  Step S209 is the same process as step S114.

  As described above, the imaging device 10 according to the second embodiment calculates a value indicating the state of the region I, and specifies the detailed region D by finding an optimal solution that makes the value the most preferable value. . Therefore, according to the imaging device 10 of the second embodiment, the same effect as that of the first embodiment can be obtained.

In the first embodiment, at the same temperature _{T C,} an example is shown in which repeated 100 times the processing of step S109 from step S104, not limited to this. For example, when it is desired to obtain an exact solution, it is preferable to increase the number of times to repeat step S104 to step S109 as the temperature T is lowered by cooling.

In the first embodiment, when lowering the temperature T _C in step S112, an example is shown in which the constant of gamma = 0.9, the value of gamma is an example, not limited.

  In the first embodiment, an example in which the skin beautification process is performed on the detailed area D in step S114 has been described. However, the process on the detailed area D is not limited to this. The same applies to the second embodiment.

  Moreover, although the example which adds the pixel which adjoins the outer side of the area | region I one pixel at a time was shown in step S104 of 1st embodiment, you may add several pixels one by one. For example, 5% of pixels extracted at random from pixels adjacent to the outside of the region I may be added. Further, the number of added pixels may be changed according to the region I. For example, in the state close to the initial region, the number of added pixels is increased, and the number of added pixels is decreased as the region I is enlarged. Thereby, in the state where the area | region I is small, the frequency | count of calculation can be reduced. In addition, in the state approaching the optimal solution, since the change is small, the detailed region D can be specified accurately. The same applies to step S107. The same applies to the second embodiment.

In the first embodiment, the number of times of cooling is not specified, but the number of times of cooling may be specified. In this case, the CPU 20 determines whether or not the number of times of cooling has reached a predetermined number S after step S116. When the CPU 20 determines that the cooling has reached the predetermined number of times S, the CPU 20 sets the temperature T _{c + S + 1} = 0, and executes the processing from step S104 to step S113. As a result, the detailed region specifying unit 31 can obtain a local solution (local minimum) in a state where the temperature T _{c + S + 1} = 0.

  In the first embodiment and the second embodiment, the example is shown in which the center of the eye and the center of the mouth are connected when the approximate region is specified. However, the present invention is not limited to this. For example, an arbitrary end point of the eye may be connected to an arbitrary end point of the mouth.

  In the first embodiment and the second embodiment, the example in which the RGB color space is used when calculating the energy E is shown, but the present invention is not limited to this. For example, a color space such as HSV or Yuv may be used.

  In the first embodiment and the second embodiment, an example in which the detailed area D is specified from the image 40 obtained by imaging one person is shown. However, the detailed area D is specified from the image 40 obtained by imaging a plurality of persons. It is also possible.

  In the first embodiment and the second embodiment, the imaging apparatus 10 has been described as an example, but the present invention is not limited to this. For example, the present invention can be similarly applied to an imaging apparatus having a configuration other than that shown in FIG. Further, the present invention can be similarly applied to an image processing apparatus such as a personal computer or a digital photo frame. Furthermore, the functions shown in FIG. 1 and the flowcharts shown in FIGS. 4 and 7 may be recorded as a program on a computer-readable recording medium, and the program may be executed by the computer.

  DESCRIPTION OF SYMBOLS 10 ... Imaging device, 20 ... CPU, 29 ... Extraction part, 30 ... Outline area specification part, 31 ... Detailed area specification part, 32 ... Image processing part

Claims (10)

A first specifying unit that specifies a first region that is a part of the face region of the subject from the image;
The similarity between the color of the pixels constituting the first area and the color of the pixels constituting the second area including a part of the first area, the assumed proportion of the face area in the image, and the first A calculation unit that calculates a value indicating the state of the second region using the similarity between the brightness of the pixels constituting the two regions and the brightness of the pixels adjacent to the pixels;
An image processing apparatus comprising: a second specifying unit that specifies the second region as a skin region based on a value indicating the state of the second region calculated by the calculating unit. The image processing apparatus according to claim 1.
The second specifying unit changes the shape of the second region until the value indicating the state of the second region calculated by the calculation unit is the most preferable value, and the value indicating the state of the second region is the highest. An image processing apparatus that identifies the second region as a skin region when a preferable value is reached. The image processing apparatus according to claim 1 or 2,
The image processing apparatus, wherein the second specifying unit performs at least one of a process of enlarging the second area and a process of reducing the second area. The image processing apparatus according to any one of claims 1 to 3,
An extraction unit that extracts at least one eye region and a mouth region from the image,
The first area is an area of a pixel located on a line connecting an arbitrary pixel in the eye area and an arbitrary pixel in the mouth area, an arbitrary pixel in each of the eye areas, and an arbitrary pixel in the mouth area. An image processing apparatus characterized in that it is either a triangular area connecting the two or a rectangular area having an arbitrary pixel in each of the eye areas as a vertex and an arbitrary pixel in the mouth area at the bottom. The image processing apparatus according to claim 1, wherein:
The image processing apparatus, wherein the calculation unit calculates a value indicating the state of the second region by an optimal solution. The image processing apparatus according to claim 5.
The most preferable value is a minimum value. The image processing apparatus according to claim 5.
The most preferable value is a local solution. The image processing apparatus according to any one of claims 1 to 7,
The image processing apparatus, wherein the second specifying unit changes a shape of the second area after excluding a color area that is not a skin color from the first area. The image processing apparatus according to any one of claims 1 to 8,
An image processing apparatus comprising: an image processing unit that performs a skin beautification process on the second region specified as a skin region. A first specifying step of specifying a first region that is a part of the face region of the subject from the image;
The similarity between the color of the pixels constituting the first area and the color of the pixels constituting the second area including a part of the first area, the assumed proportion of the face area in the image, and the first A calculation step of calculating a value indicating the state of the second region using the similarity between the brightness of the pixels constituting the two regions and the brightness of the pixels adjacent to the pixels;
An image processing program that realizes, by a computer, a second specifying step of specifying the second region as a skin region based on a value indicating the state of the second region calculated in the calculating step.

Images