JP2009059009A - Color-corrected image creation method and color-corrected image creation device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a color-corrected image creation method and device for separating with high precision even regions having similar colors such as brown hair and skin shadow, and for correcting the skin region. <P>SOLUTION: This color-corrected image creation method for newly creating a color-corrected person image based on person image data includes: a person image input stage; a reference region specification stage; a mean pixel value arithmetic operation stage: a dispersion value calculation stage; a pseudo Mahalanobis distance calculation stage; a skin probability arithmetic operation stage; and an image color correction stage. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image processing technique using a computer, and more particularly to a method and apparatus for creating a color-corrected image of a person based on person image data.

A photo retouch technique that takes a photographic image as digital data into a computer and performs various processing on the image on the computer is used in various fields.
Recently, along with the improvement in computer performance, various retouching processes for photographic images can be performed using a personal computer, and it has become common for individual users to process digital camera photographs. ing.

Whether it is a commercial photographic image or a personal photographic image, the most typical photographic image is a person image.
When this person image is taken into a computer and subjected to various digital processes, mask data indicating the skin area of the person and mask data indicating the lip area are often required. Here, the skin area is an area recognized as human skin in a photographic image, and in the case of a person's upper body image, the area from the face to the neck (areas other than skin color such as eyes, eyebrows, lips, and nostrils) Excluding).
A lip region is a region that is literally recognized as a human lip in a photographic image.
For example, in order to perform a color correction that increases redness on the skin part in order to perform a simulation of applying makeup using a reddish foundation to a person in the photograph, only the skin area is selected from the entire photograph image. Since it is necessary to perform color correction after extraction, mask data indicating a skin region is required.
Or, when performing color correction to add pink color to the lips to simulate a pink lipstick, color correction is performed by extracting only the lip area from the entire photographic image. Therefore, mask data indicating the lip area is necessary.

Under such circumstances, a technique for recognizing a skin color region from a human image has been studied.
For example, in Patent Document 1 below, it is determined whether or not the color of each pixel is a skin color by performing a statistical analysis on a digital image including a set of pixels having pixel values of the three primary colors RGB. However, a method for creating mask data indicating a skin region is disclosed.
Similarly, methods for recognizing lip areas from human images have been studied.
For example, Patent Document 2 below discloses a technique for creating mask data by extracting a region corresponding to a person's mouth using a density histogram of person image data.
JP 2002-197457 A However, there is a problem that such a skin region and lip region recognition method is difficult to apply in common to a wide variety of human images. In general, since there are individual differences in the skin color of a person, it is very difficult to quantitatively define “common skin color” as a pixel value. That is, if the “skin color” range is set too narrow, depending on the person, the original skin part may leak from the skin area. Conversely, if the “skin color” range is set too wide, Depending on the person, there is a possibility of being included in the skin area although it is not the original skin part. The same applies to the color of the lips of a person. Moreover, such a problem is caused not only by a factor “individual difference in skin and lip color for each person” but also by a factor “environment during photography”. For example, even if the same person is photographed, the color of the skin area and lip area changes depending on factors such as the position, color characteristics, and intensity of the illumination light at the time of shooting, the orientation of the person at the time of shooting, and the exposure conditions of the camera. End up. In this way, even for the same person's picture, the color of the skin area and lip area changes depending on the environment at the time of taking the photo, so it can be applied in common to an unspecified number of person images. It is extremely difficult to numerically define “color” and “lip color”. Therefore, even if automatic extraction processing of a skin region or a lip region is performed from a person image using such a conventionally proposed algorithm, sufficiently accurate region extraction cannot be performed. For this reason, the current situation is that the operator performs an operation of manually instructing the boundary line of the skin region or the lip region with the person image displayed on the display screen.

Accordingly, the applicant of the present application has proposed a method and apparatus for creating human skin or lip region mask data that can automatically extract skin regions or lip regions from a wide variety of person images. (Japanese Patent Laid-Open No. 2005-276182 to Patent Document 3)
Here, skin color recognition is performed based on the pixel values in the reference region defined in the part of the human image that is to be processed and straddles the nose, so that the skin region is automatically extracted from a wide variety of human images. Became possible.
In addition, the lip area is recognized based on the pixel value in the reference area in the estimated area where the lip is estimated to be included. Therefore, the lip area is automatically extracted from various human images. Became possible.
However, the method described here has a problem in that it is not possible to correctly separate regions of close color such as brown hair and skin shadow.

On the other hand, the Mahalanobis distance in each pixel in the face area is calculated, and the pixel whose calculated Mahalanobis distance is equal to or smaller than a threshold value is set as a skin color pixel, and the set is set as a human skin area. (Japanese Unexamined Patent Application Publication No. 2006-204311-Patent Document 4)
And here, when the color correction is performed, the representative hue of the color to be corrected is corrected so that the representative hue becomes closer to the correction target hue as the distance in the color space is closer to the correction target hue. It is also described that the correction coefficient is set so as not to be corrected so much when the representative hue of the color to be corrected is far away from the correction target hue in the color space.
However, even in the method described herein, there is a problem in that separation of regions having close colors, such as brown hair and skin shadow, cannot be performed sufficiently correctly.

As described above, recently, in a method of creating mask data indicating a skin area of a person based on the person image data, as described in JP-A-2005-276182 (Patent Document 3), a person The method of automatically extracting the skin area of the Japanese Patent Application Laid-Open No. 2002-197457 (Patent Document 1) and Japanese Patent Laid-Open No. 9-016771 (Patent Document 2), It is no longer necessary for the operator to manually indicate the boundary line of the skin area while displaying a human image on the display screen, but areas with similar colors such as brown hair and skin shadows. There remains a problem that separation of colors cannot be performed correctly, and a method of adjusting the degree of correction when performing color correction, as in the method described in JP-A-2006-203431 (Patent Document 4). It has become easy to be taken, tint etc. brown hair and Hadakage has remained a problem that separation of the near region can not be sufficiently accurately, the corresponding has been demanded. The present invention is corresponding to this, and is a color-corrected image creation method for newly creating a color-corrected image that is a color-corrected person image based on person image data. It is an object of the present invention to provide a method for creating a color correction image that can separate a skin area with high accuracy even if there is a taste-like area and can correct the skin area.
At the same time, it is an object of the present invention to provide a color correction image creation apparatus capable of performing such a color correction image creation method.

The color-corrected image creating method of the present invention is a color-corrected image creating method for newly creating a color-corrected image, which is a color-corrected person image, based on the person image data. As a set of a large number of pixels having a numerical value expressing the pixel value as a set of a number of pixels, a human image input stage to be input to a computer, and a skin area for reference as a reference for the person image data is used as a reference area. A reference region specifying step in which the computer is defined or the operator defines on the computer to identify the reference region, and an average of pixel values of each pixel included in the identified reference region Is calculated as an average pixel value, and an average pixel value calculation stage using a color having an average pixel value as a reference color, and all pixels of the person image data on the computer On the other hand, the variances for calculating the variance values in the L * axis, a * axis, and b * axis directions for the distance from the reference color in the L * a * b * space are Lv, Av, and Bv, respectively. In the value calculation step, on the computer, for each pixel constituting the person image data, the coordinate value in the L * a * b * space is (L, A, B) and the reference color having the average pixel value is obtained. A pseudo Mahalanobis distance calculating step of calculating a coordinate value in the L * a * b * space as (Lbase, Abase, Bbase) and a color difference with respect to the reference color as a pseudo Mahalanobis distance Md shown in Expression (1);
On the computer, for each pixel constituting the person image data, the obtained pseudo Mahalanobis distance is substituted as the value of the parameter x into the Gaussian equation shown in Equation (2), and the probability of the skin region is calculated. Finding the skin probability calculation stage,
(Where μ is the average of x (Md value) for the entire image, and σ ² is the variance of x (Md value) for the entire image)
On the computer, for each pixel constituting the person image data, a mask value is set based on the probability value of the skin area determined by the equation (2), and the degree of color correction is set for each pixel. And an image color correction stage for performing color correction and generating a color corrected image according to the size of the image.
In the color correction image creation method, the reference region to be referred to as a reference in the average pixel value calculation step is such that the person image data straddles the contours of both sides of the person's nose. It is defined as an area arranged at a position that does not cover the hair, eyes, or nostrils of the person.
Further, in any one of the above color-corrected image creation methods, the predetermined algorithm by the computer recognizes the right eye region and the left eye region of the person for the person image data, and positions and sizes of both the regions. The predetermined algorithm by the computer is based on the two-eye reference line segment or the right-eye region connecting the centroid point of the right-eye region and the centroid point of the left-eye region. Defining a bilateral reference line connecting the closest point to the left eye region and the closest point of the left eye region to the right eye region. A predetermined distance determined according to the distance is positioned below the both eye reference line segment, and is determined according to the length of the both eye reference line segment. A region having a constant lateral width and vertical width, is characterized in that an algorithm for defining a reference region.
In any one of the above-described methods for creating a color-corrected image of a person, in the image color correction stage, the pixel value of each pixel constituting the person image data is represented by an I value in a YIQ space, and For each pixel constituting the human image data of the original image, the I value in the YIQ space is multiplied by the probability of the skin region to obtain a new mask value, and for each pixel, the mask value The color correction is performed with the magnitude of the color correction, and color correction is performed to generate a color correction image.
Also, in any one of the above-described methods for creating a color-corrected image of a person, among the pixels constituting the person image data, the pixel value is in the first quadrant of the a * b * plane in the L * a * b * space. Pseudo Mahalanobis only for pixels that exist and have a pixel value in the L * a * b * space, with the reference color and white (skin highlight) as axes and a predetermined threshold R. A distance calculation step, a skin probability calculation step, and a color correction image generation step are performed.

A color-corrected image creating apparatus according to the present invention is a color-corrected image creating apparatus that newly creates a color-corrected image that is a color-corrected person image based on person image data. A predetermined algorithm using a human image input means for inputting as a set of a number of pixels having a numerical value expressing the pixel value as a set, and a skin area for reference as a reference using the input human image data as a reference area A reference area specifying means for specifying a reference area, an average of pixel values of each pixel included in the reference area specified by the reference area specifying means is obtained as an average pixel value, and an average pixel value is calculated L * axis and a * axis with respect to the distance between the average pixel value calculation means having the color as a reference color and the reference color in the L * a * b * space for all the pixels of the person image data , B * Variance value calculating means for calculating the variance values in the directions of Lv, Av, and Bv, and the coordinate values in the L * a * b * space for each pixel constituting the person image data (L, A, B), where the coordinate value in the L * a * b * space of the reference color having the average pixel value is (Lbase, Abase, Bbase), and the color difference with respect to the reference color is represented by the pseudo Mahalanobis distance Md shown in Expression (1). A pseudo Mahalanobis distance calculating means for calculating,
For each pixel constituting the person image data, the probability of being a skin region is obtained by a Gaussian equation shown in Equation (2) using the pseudo Mahalanobis distance calculated by the pseudo Mahalanobis distance calculating means as a parameter x. A probability calculation means;
(Where μ is the average of x (Md value) for the entire image, and σ ² is the variance of x (Md value) for the entire image)
On the computer, for each pixel constituting the person image data, a mask value is set based on the probability value of the skin area determined by the equation (2), and the degree of color correction is set for each pixel. And an image color correction unit that performs color correction and generates a color corrected image according to the size of the image.
Alternatively, the color-corrected image creating apparatus of the present invention is a color-corrected image creating apparatus that newly creates a color-corrected image that is a color-corrected person image based on the person image data. The person image input means for inputting a numerical value expressing a color as a set of pixels, and the person image data input, the person image is displayed on the display screen, and an operator's instruction is given. Based on the displayed person image, a skin area to be referred to as a reference is defined as a reference area, a reference area specifying means for specifying the reference area, and a reference area specified by the reference area specifying means. The average pixel value of each pixel included is obtained as an average pixel value, and an average pixel value calculation means that uses a color having the average pixel value as a reference color and L * for all pixels of the person image data A dispersion value calculating means for calculating values of dispersion in the directions of the L * axis, a * axis, and b * axis with respect to the distance from the reference color in the * b * space as Lv, Av, and Bv, respectively; For each pixel constituting the person image data, the coordinate value in the L * a * b * space is (L, A, B), and the coordinate value in the L * a * b * space of the reference color having the average pixel value is A pseudo Mahalanobis distance calculating means for calculating a color difference with respect to the reference color as a pseudo Mahalanobis distance Md shown in Expression (1) as (Lbase, Abase, Bbase);
For each pixel constituting the person image data, the probability of being a skin region is obtained by a Gaussian equation shown in Equation (2) using the pseudo Mahalanobis distance calculated by the pseudo Mahalanobis distance calculating means as a parameter x. A probability calculation means;
(Where μ is the average of x (Md value) for the entire image, and σ ² is the variance of x (Md value) for the entire image)
On the computer, for each pixel constituting the person image data, a mask value is set based on the probability value of the skin area determined by the equation (2), and the degree of color correction is set for each pixel. And an image color correction unit that performs color correction and generates a color corrected image according to the size of the image.
Further, in any one of the above color-corrected image creating apparatuses, the predetermined algorithm by the computer recognizes the right eye region and the left eye region of the person with respect to the person image data, and positions and sizes of both the regions. The predetermined algorithm by the computer is based on the two-line reference line segment or the right-eye region connecting the centroid point of the right-eye region and the centroid point of the left-eye region. Defining a bilateral reference line connecting the closest point to the left eye region and the closest point of the left eye region to the right eye region. A predetermined distance determined according to the distance is positioned below the both eye reference line segment, and is determined according to the length of the both eye reference line segment. A region having a constant lateral width and vertical width, is characterized in that an algorithm for defining a reference region.
Also, in any one of the above-described human color correction image creation devices, the image color correction means represents the pixel value of each pixel constituting the human image data as an I value in the YIQ space, on the computer. For each pixel constituting the human image data of the original image, the I value in the YIQ space is multiplied by the probability of the skin region to obtain a new mask value, and for each pixel, the mask value The color correction is performed by changing the magnitude of the color correction, the color correction is performed, and a color correction image is generated.
Further, in any one of the above-described color correction image creation apparatuses for a person, the pixel value of each pixel constituting the person image data is in the first quadrant of the a * b * plane in the L * a * b * space. A pixel that exists and has a pixel value in an L * a * b * space and that is in a cylinder formed with a predetermined threshold R with the reference color and white (skin highlight) as axes is selected. Since the pixel selection means is provided, only the pixels selected by the pixel selection means are processed by the pseudo Mahalanobis distance calculation means, skin probability calculation means, and image color correction means. Is.

(Function)
The color-corrected image creating method of the present invention is a color-corrected image creating method for creating a color-corrected image that is a color-corrected person image based on the person image data by adopting such a configuration. In addition, it is possible to provide a method for creating a color correction image that can separate a skin area with high accuracy even if there is an area with similar colors such as brown hair and skin shadow, and that can perform color correction of the skin area.
Specifically, by having a reference region specifying step and an average pixel value calculation step, a skin region for reference as a reference is defined as a reference region corresponding to each person image data of a wide variety of person image data. Therefore, it is possible to obtain an average pixel value that is referred to as a reference pixel value of a skin part suitable for each person image data and obtain a reference color.
Then, on the computer, for all pixels of the human image data, the dispersion in the L * axis, a * axis, and b * axis directions with respect to the distance from the reference color in the L * a * b * space. A variance value calculation stage for calculating values as Lv, Av, and Bv, respectively, and a color difference with respect to the average pixel value for each pixel constituting the person image data on the computer is represented by the pseudo equation (1). By having a pseudo Mahalanobis distance calculation step of calculating the Mahalanobis distance, it is possible to effectively weight and represent the color difference between the reference color and the color of each pixel as the pseudo Mahalanobis distance.
Further, on the computer, for each pixel constituting the person image data, the probability of being a skin region is obtained by a Gaussian equation shown in Equation (2) using the obtained pseudo Mahalanobis distance as a parameter. By having the probability calculation stage, it is assumed that whether each pixel is a skin part or not can be expressed with high accuracy in terms of probability.
Further, on the computer, a mask value is set for each pixel constituting the person image data based on a probability value of a skin region determined by the equation (2), and the degree of color correction is set for each pixel. By determining the mask value according to the magnitude of the mask value, performing color correction, and generating a color corrected image, an image color correction stage is performed, so that color correction is performed accurately on pixels that are skin portions, and color correction is performed. It is possible to generate an image.
The L * a * b * space is also known as a psychological color space or uniform color space, and is a color space that is expressed based on human perception. Is reflected as the difference in distance in space.
By utilizing this characteristic, the L * a * b * space is also used in the color distance calculation (here, calculated as the pseudo Mahalanobis distance) in the method of creating the color-corrected image.

Then, in the average pixel value calculation stage, the reference region for reference as a reference is in a position that straddles the contours of both sides of the person's nose, and the person's hair, eyes, By defining the region arranged at a position not covering the nostril, the region convenient for reference can be reliably defined as the reference region by adopting the form of the invention of claim 2. .
Further, the predetermined algorithm by the computer is an algorithm for recognizing the right eye region and the left eye region of the person with respect to the person image data and defining a reference region based on the position and size of both the regions. More specifically, according to the third aspect of the present invention, more specifically, the predetermined algorithm by the computer is the closest point of the right-eye region to the left-eye region or both-eye reference line segment connecting the centroid point of the right-eye region and the centroid point of the left-eye region. Defining a two-eye reference line segment connecting the left eye region and the closest point of the left eye region to the right eye region, and being symmetric with respect to the vertical bisector of the two-eye reference line segment, and determined according to the length of the two-eye reference line segment It is located at a distance below the reference line segment of both eyes and has a predetermined horizontal width and vertical width determined according to the length of the reference line segment of both eyes. By adopting the form of the invention of claim 4, which is an algorithm for defining a region as a reference region, it is possible to automatically extract a reference region for reference as a reference for a wide variety of person image data. .

  In particular, the pixel value of each pixel constituting the person image data is represented by the I value in the YIQ space, and the I value in the YIQ space is set for each pixel constituting the person image data on the computer. Multiply the probability of the skin region to make it a new mask value, and apply color correction to each pixel according to the magnitude of the mask value, and perform color correction to generate a color correction image Thus, by adopting the form of the invention of claim 5, compared to the case where other pixel values of the YIQ space or pixel values of other color spaces are used between the pixels of the respective pixels constituting the human image data, Effectively, it is possible to make a difference between new mask values.

  In addition, among the pixels constituting the human image data, the pixel value is present in the first quadrant of the a * b * plane in the L * a * b * space, and the pixel value is in the L * a * b * space. The pseudo Mahalanobis distance calculation step, the skin probability calculation step, and the image color correction step are performed only for pixels in a cylinder formed with a predetermined threshold R with the reference color and white (skin highlight) as axes. According to the sixth aspect of the present invention, it is possible to create color-corrected image data by limiting the area for color correction of human image data.

The color-corrected image creating apparatus according to the present invention is a color-corrected image creating apparatus that newly creates a color-corrected image that is a color-corrected person image based on the person image data by using such a configuration. In addition, it is possible to provide an apparatus for creating a color correction image that can separate a skin area with high accuracy even if there is an area with similar colors such as brown hair and skin shadow, and can correct the skin area.
By adopting the form of the invention of claim 7, it is possible to automatically define a reference area to be referred to as a reference of the skin area, and according to the form of the invention of claim 8, the operator can It is assumed that a reference area to be referred to as a skin area standard can be defined on the display.
By providing the variance value calculating means and the pseudo Mahalanobis distance calculating means, the color difference between the reference color and the color of each pixel can be effectively weighted and expressed as the pseudo Mahalanobis distance.
Further, by having the skin probability calculation means, it is possible to represent with probability accuracy whether each pixel is a skin portion or not.
Further, by having the image color correction means, it is possible to accurately perform color correction on the pixels that are skin portions and generate a color corrected image.
Then, the average pixel value calculation means is a region in the person image data that is disposed at a position that straddles the contours of both sides of the person's nose and that does not cover the person's hair, eyes, and nostrils. By defining the reference area as an embodiment of the present invention, the area convenient for reference can be reliably defined as the reference area.
Further, as the predetermined algorithm by the computer, for the person image data, an algorithm for recognizing the right eye region and the left eye region of the person and defining a reference region based on the position and size of both regions, can be cited. More specifically, the predetermined algorithm by the computer is based on a reference line segment connecting the center of gravity of the right eye region and the center of gravity of the left eye region, or the closest point of the right eye region to the left eye region and the closest point of the left eye region to the right eye region. Is defined as a bisecting line that is bilaterally symmetric with respect to the binocular reference line segment, and is below the binocular reference line segment by a predetermined distance determined according to the length of the binocular reference line segment. An area that is located apart and has a predetermined horizontal width and vertical width determined according to the length of the reference line segment of both eyes is defined as a reference area. Gorizumu and the like.
As a result, it is possible to automatically extract a reference area for reference as a reference for a wide variety of person image data.

  In particular, the pixel value of each pixel constituting the person image data is represented by the I value in the YIQ space, and the I value in the YIQ space is set for each pixel constituting the person image data on the computer. Multiply the probability of the skin region to make it a new mask value, and apply color correction to each pixel according to the magnitude of the mask value, and perform color correction to generate a color correction image By adopting the form, a new mask can be effectively created compared to the case where other pixel values of the YIQ space or pixel values of the other color spaces are used between the pixels of the pixels constituting the human image data. It is possible to make a difference in value.

  In addition, among the pixels constituting the human image data, the pixel value is present in the first quadrant of the a * b * plane in the L * a * b * space, and the pixel value is in the L * a * b * space. The pseudo Mahalanobis distance calculation step, the skin probability calculation step, and the image color correction step are performed only for pixels within a cylinder formed with a predetermined threshold R with the reference color and white (skin highlight) as axes. By adopting the form, it is possible to create color-corrected image data by limiting the area to be subjected to color correction for human image data.

The present invention, as described above, is a color-corrected image creation method for newly creating a color-corrected image that is a color-corrected person image based on person image data. Even if there is an area close to the skin area, the skin area can be separated with high accuracy, and a method for creating a color correction image capable of correcting the skin area can be provided.
At the same time, it is possible to provide a color correction image creation apparatus capable of performing such a color correction image creation method.

Embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a process flow diagram of an example of an embodiment of a color correction image creation method of the present invention, FIG. 2 is a block diagram showing a basic configuration of a color correction image creation apparatus of the present invention, and FIG. FIG. 4 is a schematic diagram for explaining a region, FIG. 4 is a diagram for explaining an algorithm for automatically defining a reference region, FIG. 5 is a diagram for explaining another algorithm for automatically defining a reference region, FIG. 6 is a diagram for explaining the reason for using the pseudo Mahalanobis distance, and FIG. 7 shows a cylindrical region having a distance R from the reference straight line connecting the reference color and white (skin highlight) in the L * a * b * space. In the figure shown,
FIG. 8 is a gradation image of the human image forehead portion using the mask value calculated using the pseudo Mahalanobis distance of the present invention, and FIG. 9 uses the mask value calculated using the Mahalanobis distance for comparison. It is a gradation image of the person image frame part in the case.
In FIG. 1, S1 to S7 indicate processing steps (also referred to as processing steps), D1 to D4 indicate data, and arrows indicate the direction of data flow.
In FIG. 2, arrows indicate the direction of data flow.
2 to 7, 10 is human image data, 21 is human image input means, 22 is reference area specifying means, 23 is average pixel value calculating means, 24 is distributed ground calculating means, 25 is pseudo Mahalanobis distance calculating means, 26 is a skin probability calculating means, 27 is an image color correcting means, 30 is a human image, 31 and 32 are eyes, 33 is a nose, 33 a is a nostril, 34 a and 34 b are contours on both sides of the nose, 35 is a reference region, 41 , 42, 51, 52 are eyes, 61 is a skin shadow region, 61A is a color of the inspection pixel 1, 62 is a brown hair region, 62A is a color of the inspection pixel 2, 63 is a reference color, and 71 is white (high skin) Light), 72 is a reference color, 73 is a reference line, and 75 is a color to be inspected.
4 and 5, G1 and G2 indicate the center of gravity. In FIG. 6, Ed1 is the Euclidean distance between the reference color and the color of the inspection pixel 1, and Ed2 is the difference between the reference color and the color of the inspection pixel 2. Euclidean distance is shown.

An example of an embodiment of a color correction image creation method of the present invention will be described with reference to the drawings.
The color-corrected image creating method of this example is a color-corrected image creating method for newly creating a color-corrected image that is a color-corrected person image based on the person image data, and here, the configuration shown in FIG. Perform in the device.
The method for creating a color-corrected image of this example is simply to calculate the probability of being a skin area for each pixel of human image data, multiplying the pixel value by the probability of being the skin area, A mask value that is a new pixel value is used, and color correction is performed for each pixel according to the magnitude of the mask value, and color correction is performed to generate a color correction image. Here, the calculation of the probability that each pixel of the human image data is a skin area is described above so that the skin area can be separated with high accuracy even if there are areas with similar colors such as skin shadows. The pseudo Mahalanobis distance Md between the color of each pixel and the reference color of the skin in the L * a * b * space represented by the equation (1) is represented by the Gaussian function G (x) shown in the equation (2) described above. As the value of x.
Note that a color defined by a predetermined algorithm and having an average pixel value of the skin region reference area is used as the skin region reference color.

The processing flow of this example will be described with reference to FIG.
First, human image data to be processed is input to a computer. (S1)
The person image input means 21 shown in FIG. 2 inputs person image data.
The person image data 10 is a set of a large number of pixels having numerical values representing colors as their pixel values.
Next, the obtained human image data D1 (10) is defined by the computer based on a predetermined algorithm with the skin region referred to as a reference as a reference region, and the reference region is specified (S2). The average of the pixel values of each pixel included in the reference area is obtained as an average pixel value. (S3)
(A color having an average pixel value is set as a reference color.)
The average value of the pixel values in the reference area is set as the reference color.
The reference area is specified by the reference area specifying means 22 shown in FIG. 2, and the average pixel value calculating means 23 calculates the average pixel value of the specific area.
On the other hand, for the obtained human image data D1, on the computer, the L * axis, a * for the distance from the reference color in the L * a * b * space with respect to all the pixels of the human image data. The dispersion values in the direction of the axis and b * axis are calculated as Lv, Av, and Bv, respectively. (S4)
It is calculated by the variance value calculation means 24 shown in FIG.
Next, using the average pixel value D2 obtained in processing step S3 and the variance value D3 obtained in step S4, L * a * b for each pixel constituting the person image data on the computer. * The coordinate value in the space is (L, A, B), the coordinate value in the L * a * b * space of the reference color having the average pixel value is (Lbase, Abase, Bbase), and the color difference with respect to the reference color is expressed by the formula ( It is calculated as the pseudo Mahalanobis distance Md shown in 1). (S5)
It is calculated by the pseudo Mahalanobis distance calculating means 25 shown in FIG.
In this case, the Mahalanobis distance Md0 is the L * axis, a * axis, b * axis for the distance from the reference color in the L * a * b * space with respect to all the pixels of the human image data. The values of standard deviations in the directions are expressed by Equation (3) as σ _L , σ _A , and σ _B , respectively.
Here, σ _L ² = Lv, σ _A ² = Av, and σ _B ² = Bv.

Next, on the computer, for each pixel constituting the person image data, the obtained pseudo Mahalanobis distance is substituted as the value of the parameter x into the Gaussian function G (x) shown in Equation (2), and the skin Find the probability of being a region. (S6)
2. Obtain the probability of being a skin region by the skin probability calculating means 26 shown in FIG. 2. Next, on the computer, for each pixel constituting the person image data, the pixel value is multiplied by the probability of the skin region. Then, this is used as a mask value which is a new pixel value, and color correction is performed for each pixel according to the magnitude of the mask value, and color correction is performed to generate a color correction image. (S7)
A color corrected image is generated by the image color correcting means 27 shown in FIG.
Note that the value of I when displayed in the YIQ space well represents the distribution of the human skin region, and in the process of generating the color correction image, the pixel value of each pixel constituting the person image data is expressed as follows. , Represented by the I value in the YIQ space, on the computer, for each pixel constituting the human image data of the original image, the I value in the YIQ space is multiplied by the probability of the skin region, It is preferable to newly set a mask value, and to perform color correction for each pixel by applying color correction depending on the magnitude of the mask value to generate a color correction image.

When color correction is performed for each pixel constituting human image data, calculation is usually performed for each component of the color to be combined with the current color.
For example, in the case of the RGB system, each of R, G, and B components is calculated as in equation (4).

Next, the reference area specifying step (processing step S2) will be specifically described.
As shown in FIG. 3, the reference region is located at a position straddling the outlines 34 a and 34 b on both sides of the person's nose 33, and on the hair, eyes 32, and nostrils 33 a of the person. In this case, the area is defined by a predetermined algorithm by a computer as an area arranged at a position that does not cover.
Examples of the predetermined algorithm include an algorithm for recognizing the right eye region and the left eye region of the person in the person image data and defining a reference region based on the position and size of both the regions.
For example, as shown in FIG. 4, a binocular reference line segment F1 connecting the centroid point of the right eye region and the centroid point of the left eye region is defined, and is symmetrical with respect to the vertical bisector F2 about the binocular reference line segment F1. A predetermined horizontal width Lx and a vertical width that are positioned below the binocular reference line segment F1 by a predetermined distance d1 that is determined according to the length of the binocular reference line segment and that are determined according to the length of the binocular reference line segment. A first algorithm that defines a region having Ly as a reference region is given.
Further, as shown in FIG. 5, a both-eye reference line segment F11 connecting the closest point Q1 with respect to the left eye region of the right eye region and the closest point Q2 with respect to the right eye region of the left eye region is defined. It is symmetrical with respect to the equipartition line F21, and is located below the binocular reference line segment by a predetermined distance d2 determined according to the length of the binocular reference line segment F11, and according to the length of the binocular reference line segment F11. An algorithm that defines a region having a predetermined horizontal width Lx1 and vertical width Ly1 determined as the reference region is given.
Here, based on the person image data, the reference area is specified by using the reference area specifying means (22 in FIG. 2) for specifying and specifying the reference area on the computer, but the person image data is used. The person image is displayed on the display screen, and on the basis of the operator's instruction, the reference area is defined on the displayed person image, and the operator defines the reference area using the reference area specifying means for specifying the reference area. You may do it.

Next, the reason for using the pseudo Mahalanobis distance will be briefly described.
In FIG. 6, the inspection pixel 61 is a skin shadow, and the inspection pixel 62 is brown hair.
When the pixel colors are close to each other in the L * a * b * space like the inspection pixel 61 and the inspection pixel 62 shown in FIG. 6, the L * a * b * between the inspection pixel 61 and the reference color. The Euclidean distance Ed1, which is a geometric distance in space, and the Euclidean distance Ed2, which is the geometric distance in the L * a * b * space between the inspection pixel 62 and the reference color, are almost the same value. In Expression (1) using the variance in each axial direction in the L * a * b * space described above, the pseudo Mahalanobis distance Md1 between the inspection pixel 61 and the reference color and the pseudo Mahalanobis distance Md2 between the inspection pixel 62 and the reference color are expressed as follows. If it calculates | requires, the difference of Md1 and Md2 will become large compared with the difference in the above-mentioned Euclidean distance.
Thus, it can be seen that the use of the pseudo Mahalanobis distance makes it easier to discriminate between the skin shadow and the brown hair.
Similarly, it is easier to discriminate the color difference using the pseudo Mahalanobis distance than when using the Mahalanobis distance Md0 shown in the equation (3) described above.
In particular, since the curvature of the probability distribution curve is smoother when the pseudo Mahalanobis distance Md is used than when the Mahalanobis distance Md0 is used, the color correction image becomes smooth and color synthesis without any sense of incongruity is performed. be able to.
Actually, by using the pseudo Mahalanobis distance Md and calculating the mask value for each pixel for the person image by the method of this example and displaying the mask value in 256 gradations, the forehead portion of the person image is shown in FIG. It became like this.
Here, for each pixel, the mask value is a value obtained by multiplying the probability of the skin region expressed by the equation (2) by the I value in the YIQ space.
Further, when the mask value is calculated for each pixel for the human image using the Mahalanobis distance Md0 instead of the pseudo Mahalanobis distance Md and the mask value is displayed in 256 gradations, the forehead portion of the human image is shown in FIG. It became like 9.
Compared with the image of FIG. 9, the hairline of the forehead is clearer in the image shown in FIG. 8 of the method of this example, and the division between the skin region and the brown hair is accurately divided. I understand that.
It is a sufficient division at a practical level.

It is not efficient in terms of processing time to perform the pseudo Mahalanobis distance calculation step, the skin probability calculation step, and the image color correction step for all the pixels of the input human image data.
Therefore, although not shown in FIG. 1, immediately before the pseudo Mahalanobis distance calculation step, among the pixels constituting the person image data, the pixel value is the first in the a * b * plane in the L * a * b * space. Pixels that exist in one quadrant and whose pixel values are outside the cylinder formed by the predetermined threshold R with the reference color and white (skin highlight) as axes in the L * a * b * space Excluded from the processing of the pseudo Mahalanobis distance calculation step, skin probability calculation step, and image color correction step, among the pixels constituting the person image data, the pixel value is the a * b * plane in the L * a * b * space In the first quadrant, and the pixel value is in a cylinder formed with a predetermined threshold R with the reference color and white (skin highlight) as axes in the L * a * b * space Only about, pseudo Mahalanobis distance calculation stage, skin probability Calculation step, is preferable to perform the image color correction step.
For example, the excluded pixels are generated as color-corrected images with the original pixel values without color correction.

As the pixel value of the input image data, a value expressed in the three primary colors RGB system or a value expressed in the three primary colors CMY system is usually used, and converted to the L * a * b * system or the YIQ system as necessary. Used.
In addition, although the said process flow is not illustrated in FIG. 1, it performs as needed.
In addition, the processing unit is not particularly shown in FIG.
This conversion is well-known and can be uniquely performed using a predetermined arithmetic expression.

In addition, the L * axis of the L * a * b * space indicates the luminance, the a * axis indicates the color tone from red to green (a positive value indicates the degree of red, and a negative value indicates the degree of green) ), B * axis indicates the color tone from yellow to blue. (Positive values indicate yellowness, negative values indicate blueness)
Y in the YIQ space takes only a positive value and is a parameter indicating luminance. I indicates a color tone from red / yellow to blue / cyan (a positive value indicates the degree of red / yellow, and a negative value) The value indicates the degree of blue / cyan), and Q indicates the color tone from magenta to green. (A positive value indicates the magenta degree, and a negative value indicates the green degree.)

Next, an example of a color correction image creation apparatus that implements the color correction image creation method will be described with reference to the block diagram shown in FIG.
The color-corrected image creation apparatus shown in FIG. 2 performs the processing flow shown in FIG. 1 and performs color correction that newly creates a color-corrected image that is a color-corrected person image based on the person image data 10. An image creation device.
The person image input means 21 is a component having a function capable of inputting person image data to be processed as a set of a large number of pixels each having a predetermined pixel value, and is a color expressed in the three primary colors RGB or the three primary colors CMY. As image data, it has a function of taking a human image.
Of course, the person image input means 21 can also be constituted by an MO drive device or a CD drive device if it is read in advance recorded on an information recording medium such as MO or CD-R.
If a person image is taken in via a communication line, the person image input means 21 can be configured by various communication devices.
The reference area specifying unit 22 uses the person image data input by the person image input unit 21 to define a skin area for reference as a reference, based on a predetermined algorithm, and specify the reference area It is a component with the function to do.
The average pixel value calculation means 23 is defined by the reference area specifying means 22 and obtains an average of the pixel values of each pixel included in the specified reference area as an average pixel value, and a color having the average pixel value is set as a reference color. , A functional component.
The variance value calculating unit 24 is configured to calculate the L * axis and the a * axis for the distance from the reference color in the L * a * b * space with respect to all the pixels of the human image data input by the human image input unit 21. , B * -axis direction dispersion values are components having a function of calculating as Lv, Av, and Bv, respectively.
The pseudo Mahalanobis distance calculating unit 25 uses, for each pixel constituting the human image data input by the human image input unit 21, a color composed of the average pixel value obtained by the average pixel value calculating unit 23 as a reference color and a pseudo Mahalanobis. Using the dispersion value obtained by the distance calculation means 25, the coordinate value in the L * a * b * space is (L, A, B), and the reference color L * a * b * space having the average pixel value Is a component having a function of calculating the color difference with respect to the reference color as the pseudo Mahalanobis distance Md shown in Expression (1), with the coordinate value at (Lbase, Abase, Bbase).
The skin probability calculation means 26 uses the pseudo Mahalanobis distance calculated by the pseudo Mahalanobis distance as a parameter for each pixel constituting the person image data input by the person image input means 21, and the probability of being a skin area. It is a component having a function obtained by the Gaussian equation shown in Equation (2).
(Where μ is the average of x (Md value) for the entire image, and σ ² is the variance of x (Md value) for the entire image)
The image color correction unit 27 multiplies the pixel value of each pixel constituting the person image data input by the person image input unit 21 by the probability of the skin region calculated by the skin probability calculation unit 26. A component having a function of setting a mask value as a new pixel value, performing color correction on each pixel according to the magnitude of the mask value, performing color correction, and generating a color correction image It is.
Specifically, the color correction image is generated while displaying a person image on the display screen.
Here, the intensity of color correction is determined in advance according to the mask value depending on the magnitude of the mask value. However, the method of applying the intensity of color correction is not particularly limited.
The processing flow of the color-corrected image creation apparatus of this example has been described in the description of the processing flow in FIG. 1, and will not be described here.

It is a processing flow figure of an example of an embodiment of a color correction image creation method of the present invention. 1 is a block diagram showing a basic configuration of a color correction image creation apparatus of the present invention. It is the schematic for demonstrating a reference | standard area | region. It is a figure for demonstrating the algorithm which defines a reference | standard area | region automatically. It is a figure for demonstrating another algorithm which defines a reference | standard area | region automatically. It is a figure for demonstrating the reason for using a pseudo Mahalanobis distance. It is the figure which showed the cylindrical area | region of distance R from the reference | standard straight line which connects a reference | standard color and white (skin highlight) in L * a * b * space. It is a gradation image of a person image frame part at the time of using the mask value computed using the pseudo Mahalanobis distance of the present invention. For comparison, there is a gradation image of the person image forehead portion using a mask value calculated using the Mahalanobis distance.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Person image data 21 Person image input means 22 Reference | standard area | region identification means 23 Average pixel value calculation means 24 Dispersion place calculation means 25 Pseudo Mahalanobis distance calculation means 26 Skin probability calculation means 27 Image color correction means 30 Person image 31, 32 Eye 33 Nose 33a Nostrils 34a, 34b Contours 35 on both sides of the nose Reference area 41, 42, 51, 52 Eye 61 Skin shadow area 61A Color 62 of inspection pixel 1 Brown hair area 62A Color of inspection pixel 2 63 Reference color 71 White (skin Highlights)
72 Reference color 73 Reference line 75 Colors G1 and G2 to be inspected Center of gravity Ed1 Euclidean distance Ed2 between the reference color and the color of inspection pixel 1 Euclidean distance between the reference color and the color of inspection pixel 2

Claims (13)

A color-corrected image creation method for newly creating a color-corrected image, which is a color-corrected person image, based on person image data, wherein the person image data has a pixel value as a pixel value. As a set of pixels, a human image input stage to be input to a computer, and the human image data is defined by the computer based on a predetermined algorithm, with a skin area to be referred as a reference as a reference area, or An operator defines on the computer and specifies the reference region, a reference region specifying step, an average of pixel values of each pixel included in the specified reference region is obtained as an average pixel value, and an average pixel value is determined. An average pixel value calculation step using the color of the image as a reference color, and all pixels of the person image data on the computer in the L * a * b * space A dispersion value calculation step of calculating dispersion values in the directions of the L * axis, a * axis, and b * axis for the distance from the reference color as Lv, Av, and Bv, respectively, on the computer, For each pixel constituting the person image data, the coordinate value in the L * a * b * space is (L, A, B), and the coordinate value in the L * a * b * space of the reference color having the average pixel value is A pseudo Mahalanobis distance calculating step of calculating a color difference with respect to the reference color as a pseudo Mahalanobis distance Md shown in Equation (1) as (Lbase, Abase, Bbase);
On the computer, for each pixel constituting the person image data, the obtained pseudo Mahalanobis distance is substituted as the value of the parameter x into the Gaussian equation shown in Equation (2), and the probability of the skin region is calculated. Finding the skin probability calculation stage,
(Where μ is the average of x (Md value) for the entire image, and σ ² is the variance of x (Md value) for the entire image)
On the computer, for each pixel constituting the person image data, a mask value is set based on the probability value of the skin area determined by the equation (2), and the degree of color correction is set for each pixel. A color-corrected image creation method comprising: an image color correction stage that performs color correction and generates a color-corrected image according to the size of the image.   The color correction image creation method according to claim 1, wherein a reference region to be referred to as a reference in the average pixel value calculation step straddles the contours of both sides of the person's nose with respect to the person image data. A color-corrected image creating method, characterized in that it is defined as an area arranged at such a position and at a position that does not cover the person's hair, eyes, or nostrils.   3. The color correction image creation method according to claim 1, wherein the predetermined algorithm by the computer recognizes the right eye region and the left eye region of the person for the person image data, and A color-corrected image creation method, which is an algorithm for defining a reference area based on the position and size of an area.   4. The color-corrected image creation method according to claim 3, wherein the predetermined algorithm by the computer is a closest point of a right-eye region to a left-eye region or both-eye reference line segment connecting a centroid point of the right-eye region and a centroid point of the left-eye region. Defining a two-eye reference line segment connecting the left eye region and the closest point of the left eye region to the right eye region, and being symmetric with respect to the vertical bisector of the two-eye reference line segment, and determined according to the length of the two-eye reference line segment The algorithm is an algorithm that defines a region having a predetermined horizontal width and vertical width that are located below the both eye reference line segments by a distance and have a predetermined horizontal width and vertical width determined according to the length of the both eye reference line segments. Color correction image creation method.   5. The method of creating a color correction image of a person according to claim 1, wherein the pixel value of each pixel constituting the person image data is an I value in a YIQ space in the image color correction step. On the computer, for each pixel constituting the human image data of the original image, the I value in the YIQ space is multiplied by the probability of the skin region, and this is used as a new mask value. A color-corrected image creating method, wherein color correction is performed for each pixel by performing color correction with the magnitude of the mask value depending on the magnitude of the mask value.   6. The color correction image creation method for a person according to claim 1, wherein among the pixels constituting the person image data, a pixel value is a * b in an L * a * b * space. * In the first quadrant of the plane, and the pixel value is in the L * a * b * space, with the reference color and white (skin highlight) as axes, and a cylinder formed with a predetermined threshold R A color-corrected image creating method characterized by performing a pseudo Mahalanobis distance calculating step, a skin probability calculating step, and a color-corrected image generating step only for a certain pixel. A color-corrected image creating apparatus that newly creates a color-corrected image, which is a color-corrected person image, based on person image data, wherein the person image data has a pixel value that is a numerical value representing a color. Using the human image input means to input as a set of pixels and the input human image data, the skin area to be referred to as a reference is defined as a reference area based on a predetermined algorithm, and the reference area is specified. An average pixel value obtained by calculating an average of pixel values of each pixel included in the reference area specified by the reference area specifying means as an average pixel value and using a color having the average pixel value as a reference color The dispersion value in the L * axis, a * axis, and b * axis directions for the distance between the calculation means and all the pixels of the human image data with respect to the reference color in the L * a * b * space. ,Respectively, The variance value calculation means for calculating as v, Av, Bv, the coordinate value in the L * a * b * space (L, A, B) and the average pixel value for each pixel constituting the person image data Pseudo Mahalanobis distance calculating means for calculating a color difference with respect to the reference color as a pseudo Mahalanobis distance Md shown in Formula (1), where the coordinate value of the reference color in the L * a * b * space is (Lbase, Abase, Bbase). When,
For each pixel constituting the person image data, the probability of being a skin region is obtained by a Gaussian equation shown in Equation (2) using the pseudo Mahalanobis distance calculated by the pseudo Mahalanobis distance calculating means as a parameter x. A probability calculation means;
(Where μ is the average of x (Md value) for the entire image, and σ ² is the variance of x (Md value) for the entire image)
On the computer, for each pixel constituting the person image data, a mask value is set based on the probability value of the skin area determined by the equation (2), and the degree of color correction is set for each pixel. And a color correction image creating device that performs color correction and generates a color corrected image according to the size of the image. A color-corrected image creating apparatus that newly creates a color-corrected image, which is a color-corrected person image, based on person image data, wherein the person image data has a pixel value that is a numerical value representing a color. The person image input means for inputting as a set of pixels and the input person image data are used to display a person image on the display screen, and on the displayed person image as a reference based on the operator's instruction Define a skin area for reference as a reference area, a reference area specifying unit for specifying the reference area, and an average pixel value of pixel values of each pixel included in the reference area specified by the reference area specifying unit The average pixel value calculation means using the color having the average pixel value as the reference color and the distance between the reference color in the L * a * b * space for all the pixels of the person image data. The variance value calculating means for calculating the variance values in the L * axis, a * axis, and b * axis directions as Lv, Av, and Bv, respectively, and for each pixel constituting the person image data, The coordinate value in the L * a * b * space is (L, A, B), the coordinate value in the L * a * b * space of the reference color having the average pixel value is (Lbase, Abase, Bbase), and the reference color Pseudo Mahalanobis distance calculating means for calculating a color difference with respect to as a pseudo Mahalanobis distance Md shown in Formula (1);
For each pixel constituting the person image data, the probability of being a skin region is obtained by a Gaussian equation shown in Equation (2) using the pseudo Mahalanobis distance calculated by the pseudo Mahalanobis distance calculating means as a parameter x. A probability calculation means;
(Where μ is the average of x (Md value) for the entire image, and σ ² is the variance of x (Md value) for the entire image)
On the computer, for each pixel constituting the person image data, a mask value is set based on the probability value of the skin area determined by the equation (2), and the degree of color correction is set for each pixel. And a color correction image creating device that performs color correction and generates a color corrected image according to the size of the image.   9. The color correction image creating apparatus according to claim 7, wherein the average pixel value calculation unit positions the person image data so as to straddle the contours of both sides of the person's nose. In addition, a color correction image creating apparatus is characterized in that a reference area is defined as an area arranged at a position that does not cover the hair, eyes, or nostrils of the person.   10. The color correction image creation apparatus according to claim 7, wherein the predetermined algorithm by the computer recognizes a right eye region and a left eye region of the person for the person image data, and A color-corrected image creating apparatus, which is an algorithm that defines a reference area based on the position and size of an area.   11. The color-corrected image creating apparatus according to claim 10, wherein the predetermined algorithm by the computer is a closest point of a right-eye region to a left-eye region or a both-eye reference line segment connecting a centroid point of the right-eye region and a centroid point of the left-eye region. Defining a two-eye reference line segment connecting the left eye region and the closest point of the left eye region to the right eye region, and being symmetric with respect to the vertical bisector of the two-eye reference line segment, and determined according to the length of the two-eye reference line segment The algorithm is an algorithm that defines a region having a predetermined horizontal width and vertical width that are located below the both eye reference line segments by a distance and have a predetermined horizontal width and vertical width determined according to the length of the both eye reference line segments. Color correction image creation device.   12. The human color-corrected image creating apparatus according to claim 7, wherein the image color correcting means sets a pixel value of each pixel constituting the human image data as an I value in a YIQ space. On the computer, for each pixel constituting the human image data of the original image, the I value in the YIQ space is multiplied by the probability of the skin region, and this is used as a new mask value. A color-corrected image creating apparatus characterized in that for each pixel, color correction is performed by applying color correction strength depending on the magnitude of the mask value to generate a color-corrected image.   13. The human color-corrected image creation device according to claim 7, wherein among the pixels constituting the human image data, the pixel value is a * b in an L * a * b * space. * In the first quadrant of the plane, and the pixel value is in the L * a * b * space, with the reference color and white (skin highlight) as axes, and a cylinder formed with a predetermined threshold R Since there is a pixel selection means for selecting a certain pixel, only the pixels selected by the pixel selection means are processed by each means of the pseudo Mahalanobis distance calculation means, skin probability calculation means, and image color correction means. A color-corrected image creating apparatus characterized by