JP6058722B2 - Skin image analysis apparatus, image processing apparatus, and computer program - Google Patents

Description

  The present invention relates to an image processing technique, and more particularly to a technique for performing skin analysis by masking facial parts from a human face image.

  2. Description of the Related Art Conventionally, an apparatus that analyzes a skin state using an image obtained by photographing a human face is known (for example, Patent Document 1). Various methods other than this patent document have been proposed as skin analysis methods.

JP 2011-130808 A

  By the way, in general, so-called facial parts such as eyebrows, eyes, nostrils, and lips are not subject to skin analysis. Therefore, when automatically analyzing skin from a human face image, it is necessary to automatically exclude facial parts.

  At this time, a method of preparing a mask for excluding face parts as a template and applying it to the face image is also conceivable. However, the position and size of the facial parts in the image may vary depending on the shooting conditions and may vary from person to person. Therefore, the facial parts may not always be accurately excluded from the template.

  Accordingly, an object of the present invention is to automatically and accurately exclude facial parts from a human face image regardless of shooting conditions and the like.

  Another object of the present invention is to automatically specify a target area for skin analysis from a human face image in order to perform skin image analysis.

  A skin image analysis apparatus according to one embodiment of the present invention includes a storage unit that stores image data of an original image having an RGB component obtained by photographing a face of a subject, and pixels having brightness and hue in a predetermined range in the original image. Means for extracting and specifying the facial region of the subject, and extracting a pixel whose G (green) component or hue normalized by the original image is within a predetermined range and including lips in the facial region. A second region including an eyebrow in the facial region in accordance with means for specifying one region and image data of a differential image obtained by differentiating the original image or image data of a grayscale image obtained by converting the original image to gray scale Means for extracting a pixel having a lightness within a predetermined range in the original image, specifying a third region including a closed eye in the facial region, and in the original image, from the facial region, 1st to 3rd region Means for extracting a pixel in which a normalized R (red) component in a region from which at least one region has been removed is within a predetermined range and identifying a fourth region including a nostril in the facial region; Means for extracting a skin region from which at least one of the first region to the fourth region is removed from the region, and analysis means for analyzing the skin condition of the subject according to the original image data of the skin region And comprising.

  In a preferred embodiment, the means for specifying the facial region defines a first range according to a distribution of pixel values of the grayscale image, and a hue image that is an image obtained by converting the pixel values of the original image into a hue. A second range is determined according to the distribution of the pixel values of the pixels, and pixels whose pixel values of the grayscale image are in the first range and whose pixel values of the hue image are in the second range are extracted. You may make it do.

  In a preferred embodiment, the means for specifying the first region including the lips in the face region has a pixel value distribution of a G ratio image which is an image obtained by converting the pixel value of the original image into G / (R + G + B). Accordingly, a third range is defined, and a region including pixels whose pixel values of the G ratio image are within the third range is defined as a first lip candidate region, and the pixel values of the G ratio image are within the third range. A wrinkle elimination process is applied to a binarized image by specifying a certain pixel, and a region including the remaining pixels is set as a second lip candidate region, and one of the first lip candidate region and the second lip candidate region May be the first region.

  In a preferred embodiment, the means for specifying the second region including the eyebrows in the face region is a differential that is an image obtained by binarizing the differential image according to a threshold value determined in accordance with a distribution of pixel values of the differential image. A binarized image may be generated, a smoothed image that is an image obtained by smoothing the differential binarized image may be generated, and the second region may be determined according to the smoothed image.

  In a preferred embodiment, the means for specifying the second region including the eyebrows in the face region is a differential that is an image obtained by binarizing the differential image according to a threshold value determined in accordance with a distribution of pixel values of the differential image. A binarized image is generated, a smoothed image that is a smoothed image of the differential binarized image is generated, a third eyebrow candidate region is determined according to the smoothed image, and a pixel value of the grayscale image Generating a grayscale binarized image that is an image obtained by binarizing the grayscale image according to a threshold value determined according to the distribution of the image; determining a fourth eyebrow candidate region according to the grayscale binarized image; One of the 3 eyebrow candidate areas and the 4th eyebrow candidate area may be set as the second area.

  In a preferred embodiment, the means for specifying the fourth region including the nostril in the face region defines a fourth range according to the brightness of the original image, and sets the pixel value of the original image to R / (R + G + B). The fifth range is determined according to the distribution of the pixel value of the R ratio image that is the image converted to, the brightness is within the fourth range, and the pixel value of the R ratio image is within the fifth range. A region including a certain pixel may be set as the fourth region.

  In a preferred embodiment, the means for identifying the fourth region including the nostril in the facial region has a lightness within a fourth range and a pixel value of the R ratio image within a fifth range. Of the plurality of regions formed by connecting the pixels, the fourth region may be specified in accordance with the mutual positional relationship of the four regions having the upper area.

  An image processing apparatus according to an embodiment of the present invention includes a storage unit that stores image data of an original image having an RGB component obtained by photographing a face of a subject, and pixels having brightness and hue within a predetermined range from the original image Then, a means for specifying the facial area of the subject, and a G (green) component or hue normalized by the original image is extracted within a predetermined range, and includes a lip in the facial area. A second region including an eyebrow in the facial region is specified according to means for specifying a region and image data of a differential image obtained by differentiating the original image or image data of a grayscale image obtained by converting the original image to gray scale Means for extracting a pixel having a lightness within a predetermined range in the original image and identifying a third region including a closed eye in the facial region; and in the original image, the first region from the facial region Of the first to third areas Means for extracting a pixel in which a normalized R (red) component in a region from which at least one region has been removed is within a predetermined range and identifying a fourth region including a nostril in the facial region; and the facial region Means for extracting a skin region from which at least one of the first to fourth regions is removed.

  A skin image analysis apparatus according to one embodiment of the present invention includes a storage unit that stores image data of an original image having RGB components obtained by photographing a face of a subject, and the lips of the subject according to the color components of the original image. Means for specifying the lip region including the eyebrow region including the eyebrow of the subject and the eye region including the closed eye of the subject according to the lightness or color component of the region above the lip region And a means for specifying a nostril region including the nostril of the subject according to color components of the region above the lip region and below the eyebrow region and the eye region, Means for identifying a skin area obtained by removing the lip, eyebrow, eye and nostril areas from the face area; and analyzing means for analyzing the skin condition of the subject according to the original image data of the skin area. Prepare.

1 is an overall configuration diagram of an image processing apparatus 1 according to an embodiment of the present invention. An overall processing procedure of the image processing apparatus 1 according to the present embodiment will be described. It is a flowchart which shows the detailed process sequence of the face area extraction process of step S230. An example of a histogram of a gray scale image and an example of a histogram of a hue image are shown. It is a flowchart which shows the detailed process sequence of the lip extraction process of step S240. It is a flowchart which shows the whole process which extracts an eye area | region and an eyebrow area | region. It is a flowchart which shows the detailed process sequence of eye extraction processing, such as step S611. It is a flowchart which shows the detailed procedure of eyebrow area | region extraction processing, such as step S613. It is a flowchart which shows the detailed procedure of the eyebrow and differential image process of step S811. It is a flowchart which shows the detailed procedure of an eyebrow and differential image process of step S813. It is a flowchart which shows the detailed procedure of the nostril extraction process of step S260.

  Hereinafter, an image processing apparatus according to an embodiment of the present invention will be described with reference to the drawings. The image processing apparatus according to the present embodiment is a skin image analysis apparatus that analyzes an image of the skin condition of a face.

  FIG. 1 is an overall configuration diagram of an image processing apparatus 1 according to the present embodiment.

  The image processing apparatus 1 includes an image processing apparatus main body 10 and a camera 5.

  The image processing apparatus main body 10 is configured by a general-purpose computer system having, for example, a processor, a memory, and the like, and each component or function described below executes, for example, a predetermined computer program on the computer system. It is realized by.

  As shown in the figure, the image processing apparatus body 10 includes a preprocessing unit 11, a face extraction unit 13, a face part extraction unit 15, a mask generation processing unit 17, a skin analysis processing unit 19, and original image data. Storage unit 300, gray image data storage unit 311, hue image data storage unit 312, G ratio image data storage unit 313, R ratio image data storage unit 314, differential image data storage unit 315, and facial area data A storage unit 350, a lip region data storage unit 360, an eye / brow region data storage unit 370, a nostril region data storage unit 380, and a mask region data storage unit 390 are provided.

  The original image data storage unit 300 stores an image including the face of the subject (person). The image data of the original image (hereinafter referred to as original image data) stored in the original image data storage unit 300 may be, for example, image data captured by the camera 5 or an image captured by a camera other than the camera 5. May be the data.

  The original image is, for example, a still image obtained by photographing the upper part of the subject including the face from the front with a black background. An image taken with a black apron may be used so that the subject's clothes are not reflected. The original image mainly shows the face and hair of the subject. The original image may be an image of the subject taken from the left or diagonally right. The file name may be determined so that the orientation of the face of the subject in the original image can be identified by the file name of the original image data stored in the original image data storage unit 300. The original image may be an image taken with the bangs raised so that facial skin exposure is increased.

  The number of pixels of the original image may be, for example, a two-dimensional image having about 2000 to 5000 pixels in both the horizontal direction (X direction) and the vertical direction (Y direction). The original image is a color image having RGB (red, green, blue) components for each pixel, for example. The original image may be one frame of a moving image.

  The preprocessing unit 11 processes the original image data stored in the original image data storage unit 300 by various methods. The preprocessing unit 11 includes a gray image generation unit 111, a hue image generation unit 113, a G ratio image generation unit 115, an R ratio image generation unit 117, and a differential image generation unit 119.

  The gray image generation unit 111 generates a grayscale image from the original image. The gray image generation unit 111 may read the original image data from the original image data storage unit 300 and generate a gray scale image from the original image using a known gray scale conversion formula. Image data of a grayscale image generated by the gray image generation unit 111 (hereinafter referred to as grayscale image data) is stored in the gray image data storage unit 311.

  The hue image generation unit 113 generates a hue image in which each pixel indicates a hue from the original image. The hue image generation unit 113 may read the original image data from the original image data storage unit 300 and generate a hue image from the original image according to the following conversion formula from RGB to hue (H: Hue). Image data of the hue image generated by the hue image generation unit 113 (hereinafter referred to as hue image data) is stored in the hue image data storage unit 312.

Ｈ：色相
Ｒ，Ｇ，Ｂはそれぞれ赤、緑及び青の画素値

H: Hue R, G, and B are pixel values of red, green, and blue, respectively

  The G ratio image generation unit 115 normalizes each pixel of the original image with a G (green) component, and generates a G ratio image indicating the ratio of the G (green) component. The G ratio image generation unit 115 reads the original image data from the original image data storage unit 300, applies a conversion formula from RGB to G component ratio (G ratio) shown in the following formula (2) to each pixel, and A G ratio image may be generated from the image. Image data of the G ratio image generated by the G ratio image generation unit 115 (hereinafter referred to as G ratio image data) is stored in the G ratio image data storage unit 313.

  G ratio = G / (R + G + B) (2)

  The R ratio image generation unit 117 normalizes each pixel of the original image with the R (red) component, and generates an R ratio image indicating the ratio of the R (red) component. The R ratio image generation unit 117 reads the original image data from the original image data storage unit 300, applies the conversion formula from RGB to R component ratio (R ratio) shown in the following formula (3) to each pixel, and An R ratio image may be generated from the image. Image data of the R ratio image generated by the R ratio image generation unit 117 (hereinafter referred to as R ratio image data) is stored in the R ratio image data storage unit 314.

  R ratio = R / (R + G + B) (3)

  The differential image generation unit 119 generates a differential image obtained by differentiating the original image. The differential image generation unit 119 reads the original image data from the original image data storage unit 300, and applies a Sobel filter or a Prewitt filter in the horizontal direction or the vertical direction, for example, to differentiate the original image from the original image. May be generated. Image data of the differential image generated by the differential image generation unit 119 (hereinafter referred to as differential image data) is stored in the differential image data storage unit 315.

  The face extraction unit 13 extracts a face area included in the original image. The face extraction unit 13 may extract a pixel whose brightness and hue are within a predetermined range from the original image and specify it as the face area of the subject. For example, the face extraction unit 13 specifies a face region according to the grayscale image and the hue image data stored in the gray image data storage unit 311 and the hue image data storage unit 312, and uses the facial region data indicating the face region. Generate. The face area data is stored in the face area data storage unit 350. Details of the processing by the face extraction unit 13 will be described later.

  The face part extracting unit 15 extracts a face part area included in the face area. In the present embodiment, the face part extraction unit 15 extracts eyebrows, closed eyes (hereinafter sometimes simply referred to as eyes), nostrils, and lips as face parts. Therefore, the face part extraction unit 15 includes a lip extraction unit 151, an eye / brow extraction unit 153, and a nostril extraction unit 155.

  The lip extraction unit 151 extracts the lip region of the subject from the face region according to the color component. For example, the lip extraction unit 151 may specify the lip region according to the G (green) component of the original image, or may specify the lip region according to the hue. The lip extraction unit 151 may generate lip region data indicating the range of the lip region by extracting pixels in which the ratio of the G (green) component of the original image is within a predetermined range as a lip region. The lip area data is stored in the lip area data storage unit 360. Details of the processing by the lip extraction unit 151 will be described later.

  The eye / brow extraction unit 153 extracts the eye region and the eyebrow region of the subject from the face region according to the brightness and the color component. For example, the eye / brow extraction unit 153 identifies an eyebrow region including the eyebrow in the face region according to image data of a differential image obtained by differentiating the original image or a grayscale image obtained by converting the original image to grayscale. Eyebrow area data indicating the area range is generated. In addition, the eye / brow extraction unit 153, for example, extracts pixels whose brightness is in a predetermined range from the original image, specifies an eye region including a closed eye in the face region, and shows an eye region range. Generate data. The eye area data and the eyebrow area data are stored in the eye / brow area data storage unit 370. Details of the processing by the eye / brow extraction unit 153 will be described later.

  The nostril extraction unit 155 extracts the nostril region of the subject from the face region according to the lightness and / or color component. For example, the nostril extraction unit 155 newly generates a region corresponding to the image data of the grayscale image within a region obtained by removing all or part of the lip, eye, and eyebrow regions from the face region, and R (red) A nostril region including the nostril is identified according to the component. The nostril extraction unit 155 may generate nostril region data indicating the nostril region as a nostril region by extracting pixels in which the ratio of the R (red) component of the original image is within a predetermined range. Alternatively, the nostril extraction unit 155 may specify the nostril region from the region extracted from the grayscale image, or may specify the nostril region from the region extracted from the R ratio image. The nostril region data is stored in the nostril region data storage unit 380. Details of the processing by the nostril extraction unit 155 will be described later.

  The mask generation processing unit 17 generates a mask for extracting a skin region to be analyzed by the skin analysis processing unit 19. The mask generated by the mask generation processing unit 17 masks at least one area of each part of the eyebrows, eyes, nostrils, and lips from the face area. That is, the skin region from which at least a part of the facial parts is removed is specified by the mask generated by the mask generation processing unit 17. The mask generation processing unit 17 generates mask data obtained by removing the area indicated by the eyebrow area data, the eye area data, the nostril area data, and the lip area data from the area indicated by the face area. The mask data is stored in the mask area data storage unit 390. Note that a part specified by a method different from the present embodiment may be used as a part of the face part area masked by the mask generation processing unit 17.

  In this embodiment, the face part extraction unit 15 extracts each region of the eyebrows, eyes, nostrils, and lips, and the mask generation processing unit 17 generates mask data for masking each region. It is not necessary to mask the region at the same time. For example, the mask generation processing unit 17 may generate mask data that masks one or more of the eyebrow, eye, nostril, and lip areas. In that case, the face part extraction unit 15 may extract only a necessary region and omit unnecessary region extraction.

  The skin analysis processing unit 19 analyzes the skin condition of the subject according to the original image data of the skin region. For example, the skin analysis processing unit 19 specifies a skin region pixel based on the mask data stored in the mask region data storage unit 390, and performs skin analysis using the original image data of the skin region pixel. The result of skin analysis is output in a predetermined format from an output device (display, printer, etc.) not shown.

  FIG. 2 shows an overall processing procedure of the image processing apparatus 1 of the present embodiment.

  First, the image processing apparatus body 10 captures original image data and stores it in the original image data storage unit 300 (S210).

  The preprocessing unit 11 generates in advance an image necessary for the subsequent processing (S220). Specifically, the preprocessing unit 11 generates a gray scale image, a hue image, a G ratio image, an R ratio image, and a differential image, respectively. Note that these images need not all be generated before starting the process of step S230. Each image may be generated by the time it becomes necessary after step S230.

  The face extraction unit 13 extracts a face area from which an area such as a background area and hair is removed from the original image, and generates face area data (S230).

  The lip extraction unit 151 extracts lip areas from the face area and generates lip area data (S240).

  The eye / brow extraction unit 153 extracts the eye region and the eyebrow region from the face region, and generates eye region data and eyebrow region data (S250).

  The nostril extraction unit 155 extracts the nostril area from the face area and generates nostril area data (S260).

  The mask generation processing unit 17 generates mask data for extracting a skin region from the original image according to the processing results of steps S230 to S260 (S270).

  The skin analysis processing unit 19 performs skin analysis processing on the skin area specified by the mask data based on the original image data (S280).

  FIG. 3 is a flowchart showing a detailed processing procedure of the facial area extraction processing in step S230.

  The face extraction unit 13 determines a threshold value for binarizing the gray scale image according to the gray scale image (S31). For example, the first range is determined according to the distribution of pixel values of the grayscale image. More specifically, for example, the face extraction unit 13 reads grayscale image data from the gray image data storage unit 311 and generates a histogram. The face extraction unit 13 may determine a threshold for binarization based on this histogram.

  FIG. 4A shows an example of a histogram of a grayscale image. The histogram shown in the figure is one typical example and has two peaks, but there may be three or more peaks. Each peak mainly corresponds to a background area (black) and a human face area from the dark side. When three peaks appear, for example, the subject's head may be a bright color (such as when wearing a crown, such as white hair, blond hair, and a hood). At this time, a peak brighter than the background appears. Therefore, the face extraction unit 13 may set the pixel value a of the first valley that exceeds the darkest peak as a binarization threshold. Then, the grayscale image is binarized with this threshold as a boundary. A range of pixel values larger than the threshold value a is the first range.

  Returning to FIG. 3, the face extraction unit 13 determines a threshold value for binarizing the hue image according to the hue image (S33). For example, the face extraction unit 13 determines the second range according to the distribution of pixel values of the hue image. More specifically, for example, the face extraction unit 13 reads the image data of the hue image from the hue image data storage unit 312 and generates a histogram. Based on this histogram, a threshold for binarization is determined. The hue value is 0 to 359, but may be converted to 0 to 255.

  FIG. 4B shows an example of a histogram of the hue image. The histogram shown in the figure is one typical example and has two peaks, but may be three or more. The case where three peaks appear is the same case as the histogram of the gray scale image. For example, the face extraction unit 13 determines a range of predetermined pixel values including the maximum peak (the lower limit threshold is b and the upper limit threshold is c). The range of the pixel values b to c may be set as the second range.

  Note that the histogram of the hue image may be generated for all the pixels, or may be generated for a pixel having a pixel value equal to or higher than the threshold value a in the grayscale image.

  The face extraction unit 13 selects a pixel whose pixel value of the grayscale image is a or more (within the first range) and whose pixel value of the hue image is between b and c (within the second range) as a face candidate. The area is specified (S35).

  The face extraction unit 13 performs a contour correction process such as a snake process on the face candidate area specified in step S35 to determine a face area (S37).

  FIG. 5 is a flowchart showing a detailed processing procedure of the lip extraction processing in step S240.

  The lip extraction unit 151 determines a threshold value for binarizing the G ratio image according to the G ratio image (S511). For example, the lip extraction unit 151 determines the third range according to the distribution of pixel values of the G ratio image. More specifically, for example, the lip extraction unit 151 reads the image data of the G ratio image from the G ratio image data storage unit 313 and generates a histogram. Based on this histogram, a threshold value for binarization may be determined. Further, the threshold value may be determined based on the pixel value d at the maximum peak. For example, pixel values such as 1/2, 1/3, 1/5, and 1/10 of the pixel value d may be used as the threshold value.

  The lip extraction unit 151 binarizes the G ratio image based on the threshold specified in step S511 (S513). Here, the region of interest (hereinafter referred to as a feature region) is binarized to white, and the other regions are binarized to black. The lip extraction unit 151 may perform noise removal processing on the binarized image. As the noise removal processing, for example, the lip extraction unit 151 may use a feature area that is isolated in a binarized image and has a small area as noise, and may apply a filter for removing the noise. Note that the binarization process may be performed in other processes described below. In the binarization process, the noise removal process described above may be executed even when not specifically mentioned.

  Although a G ratio image is used here, a hue image may be used instead of the G ratio image. When using a hue image, first, the lip extraction unit 151 generates a hue image for a pixel having a predetermined brightness or more in a grayscale image. The lip extraction unit 151 then extracts the lips in the hue image based on the spatial distribution in the original image of the pixels belonging to the predetermined hue range (for example, the region belonging to the predetermined area range located below the face). It may be specified as a region.

  Next, the lip extraction unit 151 performs two types of processing on the binarized image.

  First, the lip extraction unit 151 may set a region including pixels whose pixel values of the G ratio image or the hue image are within the third range as the first lip candidate region. Specifically, the lip extraction unit 151 may specify the first lip candidate region by performing steps S515 and S517 described below.

  The lip extraction unit 151 performs a process for removing noise at the edge portion of the face area (hereinafter referred to as edge process) (S515).

  At the edge of the face area, erroneous extraction is likely to occur due to the influence of noise such as shadows along the hair and the outline of the face. Therefore, in this embodiment, the annular region between the closed curve forming the contour of the facial region and the closed curve obtained by slightly reducing the contour, and the feature region connected thereto are removed. However, the feature region to be removed here is not necessarily noise. Therefore, in this embodiment, edge processing is performed or not performed as necessary in other processing described below.

  The lip extraction unit 151 sets a region defined as a feature region in the binarized image after the edge processing is performed as a first lip candidate region (S517).

  In response to this, the lip extraction unit 151 further applies marionette line removal processing (wrinkle removal processing) to the image binarized in step S513, and sets the region including the remaining pixels as the second lip candidate. This is an area. Specifically, the lip extraction unit 151 performs steps S519 to S523 described below to specify the second lip candidate region.

  The lip extraction unit 151 performs marionette line removal processing on the binarized image (S519). A marionette line is two lines that extend from both sides of the mouth to the chin. In order to erase the marionette line, the lip extraction unit 151 may execute a filter process for removing thin lines such as hair and wrinkles on the entire area of the binarized image, for example. It should be noted that this process may be executed as a noise removal process for other binarized images described below even when not specifically mentioned.

  The lip extraction unit 151 applies the same edge process as in step S515 to the binarized image after the marionette line removal process (S521).

  The lip extraction unit 151 sets a region defined as a feature region in the binarized image after step S521 is executed as a second lip candidate region (S523).

  The lip extraction unit 151 may execute either the determination of the first lip candidate region or the determination of the second lip candidate region first.

  The lip extraction unit 151 identifies one of the first lip candidate region and the second lip candidate region as a lip region (S525). Here, the position of the lower end of the first lip candidate area may be compared with the position of the lower end of the second lip candidate area, and the lower lip candidate area may be selected as the lip area.

  The lip extraction unit 151 determines the orientation of the face based on the position of the lip area in the face area (S527). For example, the lip extraction unit 151 divides the width of the face area (length in the X direction) into three equal parts, and the orientation of the face depends on whether the lip area belongs most to the left side, right side, or center area. May be determined. For example, when the lip area is present most frequently in the center of the face, it is determined that the face is facing forward. Note that the file name of the original image data stored in the original image data storage unit 300 may be determined according to the determination result. For example, the file name of the original image data may be a file name whose face orientation can be identified.

  6 to 10 are flowcharts of the eye / eyebrow processing for extracting the eye region and the eyebrow region performed by the eye / brow extraction unit 153. When the lip region has already been extracted, the eye / brow extraction unit 153 excludes the region below the upper end of the lip region from the processing target, and the face region above the upper end of the lip region as the target for the eye / brow processing. You may perform the process which extracts eyes and an eyebrow area | region as an area | region.

  FIG. 6 is a flowchart illustrating the entire process of extracting the eye area and the eyebrow area.

  The eye / eyebrow extraction unit 153 first executes an eye extraction process for extracting a closed eye in the face region (S611). Details of the eye extraction process will be described later.

  The eye / eyebrow extraction unit 153 executes the eyebrow extraction process after the eye extraction process (S613). Details of the eyebrow extraction process will be described later.

  The eye / brow extraction unit 153 determines whether or not the eye region and the eyebrow region are extracted in the eye extraction process and the eyebrow extraction process (S615).

  Here, if both the eye area and the eyebrow area have been extracted, it is determined whether or not the relative positional relationship between the eyebrow area and the eye area is normal (S621). Here, when the eyebrow area is located above the eye area, the relative positional relationship is correct, and the process is terminated. On the other hand, when the eyebrow region is located above the eye region, the relative positional relationship is not correct. Therefore, the eye region and eyebrow region extracted at this time are discarded, and the process returns to step S611 to execute reprocessing. .

  As in this case, in the process shown in FIG. 6, the eye extraction process may be executed a plurality of times. A detailed processing flow of the eye extraction processing is as shown in FIG. However, the eye / eyebrow extraction unit 153 changes one or more parameters for execution according to the number of times the eye extraction process is executed. For example, the eye / eyebrow extraction unit 153 may change the threshold value for binarization, the size of the feature region to be removed as noise, whether or not to apply edge processing, and the like depending on the number of times the processing is performed. good. Thereby, a different eye region is extracted every time the eye extraction process is executed.

  Further, in the process shown in FIG. 6, the eyebrow extraction process may be executed a plurality of times in the same manner as the eye extraction process. The detailed processing flow of the eyebrow extraction process is as shown in FIGS. However, the eye / eyebrow extraction unit 153 also executes the eyebrow extraction process while changing one or more parameters according to the number of times of execution similar to the eye extraction process. For example, the eye / brow extraction unit 153 changes the threshold for binarization, the size of the feature region to be removed as noise, whether to apply edge processing, and the like according to the number of times the processing is executed. Thereby, a different eyebrow region is extracted every time the eyebrow extraction process is executed.

  When it is determined in step S615 that only the eyebrow region has been extracted, the eye / eyebrow extraction unit 153 executes the eye extraction process again (S631). Thereafter, the eye / eyebrow extraction unit 153 determines whether an eye region has been extracted (S633). Here, the process of step S631 is repeated until the eye region is extracted.

  If it is determined in step S615 that neither the eye area nor the eyebrow area has been extracted, the eye / eyebrow extraction unit 153 executes the eye extraction process after executing the eyebrow extraction process (S641, S643). Thereafter, the eye / brow extraction unit 153 determines whether or not the eyebrow region and the eye region are extracted in steps S641 and S643 (S645). Here, when only one of the eyebrow region and the eye region is extracted, the processes in steps S641 and S643 are repeated until both the eyebrow region and the eye region are extracted. At this time, if the eyebrow region has been extracted, step S641 may be skipped, and if the eye region has been extracted, step S643 may be skipped. If neither the eye area nor the eyebrow area is extracted in step S645, the process returns to step S611 to execute reprocessing.

  When it is determined in step S615 that only the eye area has been extracted, the eye / eyebrow extraction unit 153 discards the extracted eye area and executes step S641 and subsequent steps.

  After steps S633 and S645, the relative positional relationship between the eyes and the eyebrows is determined in step S621 as described above.

  FIG. 7 is a flowchart showing a detailed processing procedure of the eye extraction processing in step S611 and the like. When the eye / brow extraction unit 153 executes the eye processing, if the eyebrow region has already been extracted, the following processing is performed using the region excluding the eyebrow region or the region below the lower end of the eyebrow region as a target region. May be performed.

  The eye / brow extraction unit 153 determines a threshold value for binarizing the gray scale image according to the gray scale image (S711). For example, the eye / eyebrow extraction unit 153 determines the range of pixel values according to the distribution of pixel values of the grayscale image. More specifically, for example, the eye / eyebrow extraction unit 153 reads the image data of the grayscale image from the gray image data storage unit 311 and generates a histogram for all the pixels in the face area or the pixels in the target area. The eye / brow extraction unit 153 may determine a threshold for binarization based on this histogram. The eye / eyebrow extraction unit 153 may include a counter that counts the number of times the eye extraction process illustrated in FIG. 7 is executed. The eye / eyebrow extraction unit 153 may change the threshold determination method in step S711 according to the count value of the counter. This is because whether or not the feature region can be detected correctly depends on the threshold value.

  The eye / brow extraction unit 153 binarizes the grayscale image based on the threshold value determined in step S711 (S713).

  The eye / brow extraction unit 153 performs edge processing on the binarized image generated in step S713 (S715). The eye / eyebrow extraction unit 153 may omit the edge processing in step S715 in accordance with the above-described eye extraction processing execution count counter, or may change the content of the processing (for example, along the contour). Widen or narrow the width of the annular region, or erase only the annular region and leave the region connected there).

  The eye / brow extraction unit 153 extracts a closed eye region from the binarized image (S717). For example, the eye / brow extraction unit 153 may extract the feature area having the largest area and the second largest feature area in the binarized image as the eye area.

  The eye / eyebrow extraction unit 153 determines whether the eye region extracted here is correct (S719). For example, the eye / eyebrow extraction unit 153 may make a correct / incorrect determination based on the relative positional relationship between the two extracted regions. For example, the eye / eyebrow extraction unit 153 determines whether there is no relative positional shift between the two eye regions in the vertical direction and / or whether the two eye regions are divided into left and right regions of the face. You may make a right / wrong judgment.

  The eye / eyebrow extraction unit 153 may determine whether or not the eye area has the same height in the face area in the process of step S719. The eye / eyebrow extraction unit 153 may determine, for example, whether one of the regions is a reference region and the upper end or lower end of the reference region is between the upper and lower ends of the other region. . The eye / eyebrow extraction unit 153 has a vertical misalignment when the upper or lower end of the eye / eyebrow extraction unit 153 is not between the upper and lower ends of the other region. The relative positional relationship may be determined to be incorrect.

  Further, when the original image is an image obtained by photographing the subject from the front, the eye / brow extraction unit 153 determines whether or not the eye region is divided into the left half surface and the right half surface of the face region in the process of step S719. May be. For example, the eye / eyebrow extraction unit 153 identifies the center line of the face area, and determines that the relative positional relationship is correct if the two eye areas exist separately on the left and right sides of the line.

  In addition to this, for example, it may be determined whether the area of the eye region or the aspect ratio is within a predetermined range. Further, when the eyebrow region has already been extracted, the eye / brow extraction unit 153 may perform the correctness determination in consideration of the relative positional relationship between the eye region and the eyebrow region. In step S719, the eye / eyebrow extraction unit 153 may perform right / wrong determination using one or more of the above-described criteria.

  When it is determined that the positional relationship between the two eye regions is correct (S719: correct), the eye / eyebrow extraction unit 153 determines the two regions as the eye regions (S721).

  On the other hand, when it is determined that the positional relationship between the two eye regions is not correct by the correctness determination (S719: incorrect), the eye / brow extraction unit 153 discards the extracted eye region and the eye region is not yet stored. It is determined (S723).

  FIG. 8 is a flowchart showing a detailed procedure of eyebrow area extraction processing in step S613 and the like. When performing eyebrow processing, the eye / eyebrow extraction unit 153 sets the area excluding the eye area or the area above the upper end of the eye area as the target area if the eye area has already been determined. The following processing may be performed.

  The eye / eyebrow extraction unit 153 performs eyebrow / differential image processing using the differential image as eyebrow extraction processing (S811). Details of the eyebrow / differential image processing will be described later.

  The eye / eyebrow extraction unit 153 determines whether the eyebrow area extracted in step S811 is correct (S812). This right / wrong determination is, for example, whether or not the two eyebrow areas are vertically displaced and / or whether the two eyebrow areas are divided into left and right areas of the face, as in the right / wrong determination performed for the eye area in step S719. This determination may be made on the eyebrows.

  When it is determined that the positional relationship between the two eyebrow areas is not correct (S812: incorrect), the eye / brow extraction unit 153 discards the extracted eyebrow area (S814).

  When it is determined that the positional relationship between the two eyebrow areas is correct by the right / wrong determination, these eyebrow areas remain valid as eyebrow candidate areas (S812: correct), and the eye / brow extraction unit 153 performs the eyebrow extraction process. Then, eyebrow / gray image processing is executed using the grayscale image (S816). Details of the eyebrow / gray image processing will be described later.

  The eye / brow extraction unit 153 determines whether or not the eyebrow area extracted in step S816 is correct (S818). The criterion for correctness determination may be the same as that in step S812, for example.

  If it is determined that the positional relationship between the two eyebrow areas is not correct (S818: incorrect), the eye / brow extraction unit 153 discards the extracted eyebrow area (S820).

  When it is determined that the positional relationship between the two eyebrow areas is correct by the right / wrong determination, the eyebrow extraction unit 153 determines that these eyebrow areas are effective as eyebrow candidate areas (S818: correct), and the eyebrow / differential image processing is performed. The eyebrow area is determined according to the processing result of the eyebrow / gray image processing (S822). For example, the eye / eyebrow extraction unit 153 may select an eyebrow candidate region as an eyebrow candidate when the eyebrow candidate regions are extracted and valid in the eyebrow / differential image processing and the eyebrow / gray image processing, respectively. good. For example, the eye / eyebrow extraction unit 153 may use one or more of the width, height, area, aspect ratio, and the like of the eyebrow candidate region as a reference. When the eyebrow areas extracted by the eyebrow / differential image processing and the eyebrow / gray image processing are all discarded, the process may be terminated without determining the eyebrow area. When only one of the eyebrow candidate areas of the eyebrow / differential image processing and the eyebrow / gray image processing is valid, it may be determined as the eyebrow area.

  Here, the eyebrow / differential image processing is performed first and the eyebrow / gray image processing is performed later, but the order of these is arbitrary. That is, the processing order may be reversed, or both may be executed simultaneously. In addition, the eyebrow candidate area extracted by the eyebrow / differential image processing may be used as it is as the eyebrow area without performing the eyebrow / gray image processing, or extracted by the eyebrow / gray image processing without performing the eyebrow / differential image processing. The eyebrow candidate area may be directly used as the eyebrow area.

  FIG. 9 is a flowchart showing a detailed procedure of the eyebrow / differential image processing in step S811.

  The eye / brow extraction unit 153 determines a threshold value for binarizing the pixel value distribution of the differential image according to the distribution of the pixel values of the differential image (S911). More specifically, the eye / eyebrow extraction unit 153 reads the image data of the differential image from the differential image data storage unit 315, and generates a histogram for all the pixels in the face area or the pixels in the target area. Based on this histogram, the eye / eyebrow extraction unit 153 determines a threshold value for binarization.

  The eye / eyebrow extraction unit 153 may include a counter that counts the number of times the eyebrow extraction process illustrated in FIGS. The eye / eyebrow extraction unit 153 may change the threshold determination method in step S911 according to the count value of the counter. This is because whether or not the feature region can be detected correctly depends on the threshold value.

  The eye / eyebrow extraction unit 153 binarizes the differential image according to the threshold determined in step S911 to generate a differential binary image (S913).

  The eye / eyebrow extraction unit 153 performs edge processing on the differentiated binarized image (S915). The specific processing content of the edge processing is as described above.

  Note that the eye / eyebrow extraction unit 153 may omit the edge processing in step S915 according to the above-described execution number counter of the eyebrow extraction processing, or may change the content of the processing (for example, contour) Widen or narrow the width of the annular region along the line or erase only the annular region, leaving a region connected there).

  The eye / brow extraction unit 153 smoothes the differentiated binarized image (S917). For example, the eye / brow extraction unit 153 may apply a Gaussian filter to the differential binarized image.

  The eye / brow extraction unit 153 extracts an eyebrow region in the smoothed image (S919). For example, in the smoothed image, the eye / brow extraction unit 153 may use the feature area having the largest area and the feature area having the second largest area as the eyebrow area.

  By using the differential image, the edge of the eyebrows can be extracted, and further, the spread as the eyebrow region can be extracted by further smoothing the differential image. Furthermore, eyebrows of colors other than black can be extracted by using the differential image.

  FIG. 10 is a flowchart showing a detailed procedure of eyebrow / gray image processing in step S816.

  The eye / brow extraction unit 153 determines a threshold value for binarizing the original image according to the lightness distribution of the original image (S1011). For example, the eye / eyebrow extraction unit 153 reads image data of a grayscale image from the gray image data storage unit 311 and generates a histogram for all pixels in the face area or pixels in the target area. A threshold for binarization is determined according to this histogram.

  The eye / eyebrow extraction unit 153 may change the threshold value determination method in step S1011 according to the count value of the execution number counter of the eyebrow extraction process.

  The eye / eyebrow extraction unit 153 binarizes the grayscale image according to the threshold value determined in step S1011 to generate a gray binarized image (S1013).

  The eye / eyebrow extraction unit 153 performs edge processing on the gray binarized image (S1015). The specific processing content of the edge processing is as described above.

  Note that the eye / eyebrow extraction unit 153 may omit the edge processing in step S1015 in accordance with the above-described eyebrow extraction processing execution number counter, or may change the content of the processing (for example, contour) Widen or narrow the width of the annular region along the line or erase only the annular region, leaving a region connected there).

  The eye / brow extraction unit 153 extracts an eyebrow region from the gray binarized image (S1017). The eye / brow extraction unit 153 may use, for example, the feature area having the largest area and the feature area having the second largest area as the eyebrow area.

  FIG. 11 is a flowchart showing a detailed procedure of the nostril extraction process in step S260.

  The nostril extraction unit 155 identifies the target region for the nostril extraction process by excluding the eyebrow, eye, and lip regions determined in the processing up to step S250 from the facial region (S1111).

  The nostril extraction unit 155 determines a threshold for nostril extraction in accordance with the brightness distribution of the original image of the processing target region (S1113). For example, the nostril extraction unit 155 determines the fourth range according to the brightness distribution of the original image. More specifically, for example, the nostril extraction unit 155 reads the image data of the grayscale image of the processing target region from the gray image data storage unit 311 and generates a histogram. The nostril extraction unit 155 may determine the threshold value according to this histogram. A range of pixel values smaller than this threshold is the fourth range.

  The nostril extraction unit 155 determines a threshold for nostril extraction according to the R ratio image of the processing target region (S1115). For example, the nostril extraction unit 155 determines the fifth range according to the distribution of the pixel values of the R ratio image. More specifically, for example, the nostril extraction unit 155 reads the image data of the R ratio image of the processing target region from the R ratio image data storage unit 314, and generates a histogram. The nostril extraction unit 155 may determine a threshold value according to this histogram. A range of pixel values having an R ratio higher (larger) than this threshold is the fifth range.

  Note that the nostril extraction unit 155 may include a counter that counts the number of times the nostril extraction process illustrated in FIG. 11 is executed. The nostril extraction unit 155 may change the threshold value determination method in step S1115 according to the count value of the counter. This is because whether or not the feature region can be detected correctly depends on the threshold value.

  The nostril extraction unit 155 identifies nostril candidate regions based on the threshold values determined in steps S1113 and S1115 (S1117). For example, the nostril extraction unit 155 extracts a pixel having a pixel value smaller than the threshold value determined in step S1113 and having an R ratio higher (larger) than the threshold value determined in step S1115 as a feature region. The feature area having the largest area and the feature area having the second largest area may be used as the nostril candidate area among the plurality of extracted feature areas.

  The nostril extraction unit 155 determines whether the extracted candidate region is correct according to the relative positional relationship between the two nostril candidate regions and / or the relative positional relationship with other facial parts that have already been extracted. Is determined (S1119).

  For example, the nostril extraction unit 155 may determine the presence / absence of a relative positional shift in the vertical direction of the two nostril candidate regions and the positional relationship between the eyes, the nostrils, and the mouth as the correctness determination. For example, the nostril extraction unit 155 determines whether or not the region extracted as the nostril has the same height in the face region. For example, the nostril extraction unit 155 determines whether the upper end or the lower end of the reference region is between the upper end and the lower end of the other region with respect to any one region. The nostril extraction unit 155 determines that a vertical misalignment occurs when the upper end or the lower end of any region is not between the upper end and the lower end of the other region, regardless of which region is the reference region. . In addition, the nostril extraction unit 155 may determine, for example, whether the distance from the eye region to the nostril region and the distance from the nostril region to the mouth region are correct. For example, the nostril extraction unit 155 compares the distance from the lower end of the eye region to the upper end of the nostril region with the distance from the upper end of the nostril region to the upper end of the mouth region, and may be positive when the former is long.

  When it is determined that these nostril candidate regions are correct in the correctness determination (S1119: correct), the nostril extraction unit 155 determines the two regions to be determined as nostril regions (S1131).

  When it is determined that these nostril candidate regions are not correct in the correct / incorrect determination (S1119: incorrect), the nostril extraction unit 155 determines whether there is a relative overlap between the two nostril candidate regions in the left-right direction (S1121). For example, the nostril extraction unit 155 determines whether or not the region extracted as the nostril has an overlapping position in the left-right direction within the face region. For example, the nostril extraction unit 155 determines whether the left end or the right end of the reference region is between the left end and the right end of the other region with respect to any one region. The nostril extraction unit 155 determines that there is an overlap in the left-right direction when the left end or the right end is between the left end and the right end of the other region when any one is set as the reference region.

  When there is no left-right overlap (S1121: no overlap), it is determined that no region can be a nostril region candidate, and the process returns to step S1113.

  When there is an overlap on the left and right (S1121: with overlap), one of the upper regions can be a nostril region candidate, so the other candidate region is extracted by the following process. That is, first, the nostril extraction unit 155 re-extracts nostril candidate regions (S1123). For example, the nostril extraction unit 155 has an area of a plurality of regions formed by connecting pixels whose brightness is in the fourth range and whose pixel value of the R ratio image is in the fifth range. The fourth area may be specified in accordance with the positional relationship between the four areas having the higher dimensions.

  More specifically, in this re-extraction, the nostril extraction unit 155 extracts the third and fourth largest areas of the feature area (the largest and second largest areas have already been extracted in step S1117). ). And about the area | region where the area extracted here is the 3rd and 4th largest, the same right / wrong determination as step S1119 is performed (S1125).

  When it is determined that the positional relationship between these nostril candidate regions is correct in the correctness determination (S1125: correct), the nostril extraction unit 155 determines that no region can be a nostril region candidate and returns to step S1113. Then repeat the subsequent processing.

  When it is determined that the positional relationship between these nostril candidate regions is incorrect in the correct / incorrect determination (S1125: incorrect), the nostril extraction unit 155 performs the same left and right overlap as in step S1121 for the third and fourth largest areas. Whether or not there is is determined (S1127).

  When there is no left-right overlap (S1127: no overlap), it is determined that no region can be a nostril region candidate, and the process returns to step S1113.

  When there is an overlap on the left and right (S1127: with an overlap), the nostril extraction unit 155 specifies a nostril candidate region again (S1128). That is, the region determined to be on the upper side in step S1119 and the region determined to be on the upper side in step S1125 are set as nostril candidate regions.

  The nostril extraction unit 155 performs the same correct / incorrect determination as in step S1119 for the nostril candidate region specified in step S1128 (S1129).

  When it is determined in step S1129 that the positional relationship between the two regions targeted here is correct (S1129: correct), the nostril extraction unit 155 uses the two regions targeted for determination as nostril regions. Determine (S1131).

  If it is determined in step S1129 that the positional relationship between the two regions targeted here is not correct (S1129: incorrect), the nostril extraction unit 155 returns to step S1113 and repeats the subsequent processing. .

  According to the present embodiment, face parts can be automatically and accurately excluded from a person's face image regardless of conditions at the time of shooting.

  Further, according to the present embodiment, in order to perform skin image analysis, it is possible to automatically specify a target region for skin analysis from a human face image.

  The above-described embodiments of the present invention are examples for explaining the present invention, and are not intended to limit the scope of the present invention only to those embodiments. Those skilled in the art can implement the present invention in various other modes without departing from the gist of the present invention.

DESCRIPTION OF SYMBOLS 1 Image processing apparatus 5 Camera 10 Image processing apparatus main body 11 Pre-processing part 13 Face extraction part 15 Face part extraction part 17 Mask generation processing part 19 Skin analysis processing part 111 Gray image generation part 113 Hue image generation part 115 Ratio image generation part 117 Ratio image generation unit 119 Differential image generation unit 151 Lip extraction unit 153 Eye / brow extraction unit 155 Nostril extraction unit 300 Original image data storage unit 311 Gray image data storage unit 312 Hue image data storage unit 313 Ratio image data storage unit 314 Ratio image Data storage unit 315 Differential image data storage unit 350 Facial region data storage unit 360 Lip region data storage unit 370 Eye / brow region data storage unit 380 Nostril region data storage unit 390 Mask region data storage unit

Claims (14)

A storage unit for storing image data of an original image having RGB components obtained by photographing the face of the subject;
Means for extracting pixels whose brightness and hue are within a predetermined range in the original image, and specifying the facial region of the subject;
Means for extracting a pixel having a G (green) component or hue normalized in the original image within a predetermined range, and identifying a first region including the lips in the facial region;
Means for specifying a second region including an eyebrow in the facial region in accordance with image data of a differential image obtained by differentiating the original image or image data of a grayscale image obtained by grayscale converting the original image;
Means for extracting a pixel having a lightness within a predetermined range in the original image and identifying a third region including a closed eye in the face region;
In the original image, a pixel in which a normalized R (red) component in a region obtained by removing at least one of the first region to the third region from the facial region is extracted within a predetermined range; Means for identifying a fourth region including a nostril in the facial region;
Means for extracting a skin region obtained by removing at least one of the first region to the fourth region from the facial region;
A skin image analysis apparatus comprising: analysis means for analyzing the skin condition of the subject according to the original image data of the skin region. The means for specifying the facial area is:
A first range is determined according to a distribution of pixel values of the grayscale image,
A second range is determined according to a distribution of pixel values of a hue image that is an image obtained by converting pixel values of the original image into a hue,
The skin image analysis apparatus according to claim 1, wherein pixels whose pixel values of the gray scale image are within a first range and whose pixel values of the hue image are within a second range are extracted. The means for specifying the first area including the lips in the face area is:
A third range is determined according to a distribution of pixel values of a G ratio image that is an image obtained by converting the pixel value of the original image into G / (R + G + B),
A region including pixels whose pixel values of the G ratio image component are within the third range is defined as a first lip candidate region,
A pixel whose pixel value of the G ratio image component is in the third range is specified and a wrinkle elimination process is applied, and a region including the remaining pixels is set as a second lip candidate region,
The skin image analysis apparatus according to claim 1 or 2, wherein one of the first lip candidate area and the second lip candidate area is the first area. The means for specifying the second region including the eyebrows in the facial region is
Generating a differential binarized image that is an image obtained by binarizing the differential image according to a threshold value determined according to a distribution of pixel values of the differential image;
Generating a smoothed image that is an image obtained by smoothing the differentiated binary image;
The skin image analysis apparatus according to claim 1, wherein a second region is determined according to the smoothed image. The means for specifying the second region including the eyebrows in the facial region is
Generating a differential binarized image that is an image obtained by binarizing the differential image according to a threshold value determined according to a distribution of pixel values of the differential image;
Generating a smoothed image that is an image obtained by smoothing the differentiated binary image;
A third eyebrow candidate area is determined according to the smoothed image,
Generating a grayscale binarized image that is an image obtained by binarizing the grayscale image according to a threshold value determined according to a distribution of pixel values of the grayscale image;
A fourth eyebrow candidate area is determined according to the grayscale binarized image,
The skin image analysis apparatus according to any one of claims 1 to 3, wherein one of the third eyebrow candidate area and the fourth eyebrow candidate area is the second area. The means for identifying the fourth region including the nostril in the facial region is:
A fourth range is determined according to the brightness of the original image,
A fifth range is determined according to a distribution of pixel values of an R ratio image, which is an image obtained by converting the pixel value of the original image into R / (R + G + B),
The skin according to any one of claims 1 to 5, wherein a region including a pixel having a lightness within a fourth range and a pixel value of the R ratio image within a fifth range is defined as a fourth region. Image analysis device. The means for identifying the fourth region including the nostril in the facial region is:
Of the plurality of regions formed by connecting pixels whose lightness is within the fourth range and whose pixel value of the R ratio image is within the fifth range, the top four areas are wide. The skin image analysis apparatus according to claim 6, wherein the fourth area is specified according to the mutual positional relationship of the areas. A computer program for causing a skin image analysis apparatus provided with a storage unit to store image data of an original image having RGB components obtained by photographing a face of a subject,
Means for extracting pixels whose brightness and hue are within a predetermined range in the original image, and specifying the facial region of the subject;
Means for extracting a pixel having a G (green) component or hue normalized in the original image within a predetermined range, and identifying a first region including the lips in the facial region;
Means for specifying a second region including an eyebrow in the facial region in accordance with image data of a differential image obtained by differentiating the original image or image data of a grayscale image obtained by grayscale converting the original image;
Means for extracting a pixel having a lightness within a predetermined range in the original image and identifying a third region including a closed eye in the face region;
In the original image, a pixel in which a normalized R (red) component in a region obtained by removing at least one of the first region to the third region from the facial region is extracted within a predetermined range; Means for identifying a fourth region including a nostril in the facial region;
Means for extracting a skin region obtained by removing at least one of the first region to the fourth region from the facial region;
A computer program for constructing, on the skin image analysis device, analysis means for analyzing the skin condition of the subject according to the original image data of the skin region. A skin image analysis method performed by a skin image analysis apparatus including a storage unit that stores image data of an original image having RGB components obtained by photographing a face of a subject,
A face area extracting unit extracting pixels within a predetermined range of brightness and hue in the original image, and specifying the face area of the subject;
A lip extraction unit extracting a pixel having a G (green) component or hue normalized by the original image within a predetermined range, and identifying a first region including the lips in the facial region;
The eye / brow extraction unit identifies a second area including the eyebrows in the face area according to image data of a differential image obtained by differentiating the original image or image data of a grayscale image obtained by converting the original image to gray scale Extracting a pixel having a lightness within a predetermined range from the original image and identifying a third region including a closed eye in the face region;
Pixels whose normalized R (red) component is within a predetermined range in a region obtained by removing at least one of the first region to the third region from the face region in the original image by the nostril extraction unit. And identifying a fourth region including a nostril in the facial region;
A step of analyzing a skin condition of the subject according to original image data of a skin area obtained by removing at least one of the first area to the fourth area from the face area; How to do. A storage unit for storing image data of an original image having RGB components obtained by photographing the face of the subject;
Means for extracting pixels whose brightness and hue are within a predetermined range in the original image, and specifying the facial region of the subject;
Means for extracting a pixel having a G (green) component or hue normalized in the original image within a predetermined range, and identifying a first region including the lips in the facial region;
Means for specifying a second region including an eyebrow in the facial region in accordance with image data of a differential image obtained by differentiating the original image or image data of a grayscale image obtained by grayscale converting the original image;
Means for extracting a pixel having a lightness within a predetermined range in the original image and identifying a third region including a closed eye in the face region;
In the original image, a pixel in which a normalized R (red) component in a region obtained by removing at least one of the first region to the third region from the facial region is extracted within a predetermined range; Means for identifying a fourth region including a nostril in the facial region;
Image processing and means divided at least one region of said first region to the fourth region from the face region. A computer program for causing an image processing apparatus provided with a storage unit to store image data of an original image having RGB components obtained by photographing a subject's face,
Means for extracting pixels whose brightness and hue are within a predetermined range in the original image, and specifying the facial region of the subject;
Means for extracting a pixel having a G (green) component or hue normalized in the original image within a predetermined range, and identifying a first region including the lips in the facial region;
Means for specifying a second region including an eyebrow in the facial region in accordance with image data of a differential image obtained by differentiating the original image or image data of a grayscale image obtained by grayscale converting the original image;
Means for extracting a pixel having a lightness within a predetermined range in the original image and identifying a third region including a closed eye in the face region;
In the original image, a pixel in which a normalized R (red) component in a region obtained by removing at least one of the first region to the third region from the facial region is extracted within a predetermined range; Means for identifying a fourth region including a nostril in the facial region;
Computer program for constructing means for extracting realm removing at least one region of said first region to the fourth region from the face region, the prior-outs image analyzer on. An image processing method performed by an image processing apparatus including a storage unit that stores image data of an original image having RGB components obtained by photographing a face of a subject,
A face area extracting unit extracting pixels within a predetermined range of brightness and hue in the original image, and specifying the face area of the subject;
A lip extraction unit extracting a pixel having a G (green) component or hue normalized by the original image within a predetermined range, and identifying a first region including the lips in the facial region;
The eye / brow extraction unit identifies a second area including the eyebrows in the face area according to image data of a differential image obtained by differentiating the original image or image data of a grayscale image obtained by converting the original image to gray scale Extracting a pixel having a lightness within a predetermined range from the original image and identifying a third region including a closed eye in the face region;
Pixels whose normalized R (red) component is within a predetermined range in a region obtained by removing at least one of the first region to the third region from the face region in the original image by the nostril extraction unit. And identifying a fourth region including a nostril in the facial region;
How mask generation processing unit performs the steps of: generating a mask by removing at least one region of said first region to the fourth region from the face region. A storage unit for storing image data of an original image having RGB components obtained by photographing the face of the subject;
Means for extracting pixels whose brightness and hue are within a predetermined range in the original image, and specifying the facial region of the subject;
It means for normalized G (green) Ingredient in the original image by extracting pixels in a predetermined range, identifying a first region including a lip of the facial area,
Means it divided at least the first region from said facial region, images processing device Ru comprising a. A storage unit for storing image data of an original image having RGB components obtained by photographing the face of the subject;
Means for extracting pixels whose brightness and hue are within a predetermined range in the original image, and specifying the facial region of the subject;
In the original image, and means for the facial area or RaTadashi-normalized been R (red) component and extracting pixels in a predetermined range and identifies a fourth region including a nostril of the facial area,
Means you divided at least the fourth region from the face region, images processing device Ru comprising a.

Images