JP2009006089A - Face imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a face imaging apparatus which evaluates the skin with high precision on the basis of imaged face image data irrespective of difference in surface temperatures of the face. <P>SOLUTION: The face imaging apparatus includes an imaging camera 4 for imaging a face image, a thermography device 6 for measuring temperature distribution data of the face surface, calculation means for carrying out calculation for skin diagnosis using face image data imaged by the imaging camera 4, and correction processing means 36f for correcting calculation contents of the calculation on the basis of the measurement result of the thermography device 6. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a face photographing apparatus for evaluating a skin condition using face image data photographed by an imaging means.

  Such a face image device is used in cosmetic dermatology, cosmetic surgery, plastic surgery, etc., and a face photographing device observes a change in skin condition over time by photographing a person's face regularly, It is used for the purpose of confirming the effects of cosmetics and skin treatment (for example, Patent Document 1 below). Specifically, evaluation of spots, wrinkles, pores, color unevenness, and the like is performed from the obtained face image, and further, porphyrins, hide-and-spots using ultraviolet rays are also evaluated. In order to evaluate such a skin condition, a face image shot before and a face image shot this time are compared using an image processing technique (software). Therefore, it is necessary that the previous face image and the current face image are taken under the same conditions.

Japanese Patent No. 3219550

  However, if the face surface temperature (temperature distribution) when previously photographed is different from the current face surface temperature, accurate skin condition evaluation may not be possible. For example, pores tend to close when the temperature is cold and open when the temperature is warm. This also has the same tendency when sweating. Therefore, when measuring the pore size from the face image data and evaluating the skin condition, if the surface temperature of the face is different between the previous time and the current time, there is a possibility that an accurate diagnosis cannot be performed.

  The present invention has been made in view of the above circumstances, and the problem is that it is possible to accurately evaluate the skin based on the photographed face image data, regardless of the difference in the surface temperature of the face. It is to provide a face photographing device.

In order to solve the above-described problem, a face photographing apparatus according to the present invention includes:
Imaging means for photographing a face image;
Temperature measuring means for measuring temperature distribution data on the face surface;
Arithmetic means for performing arithmetic processing for skin diagnosis using face image data photographed by the imaging means;
And correction processing means for correcting the calculation contents when performing the calculation processing based on the measurement result by the temperature measurement means.

  The operation and effect of the face photographing apparatus having such a configuration will be described. An imaging means for photographing a face image and a temperature measuring means for measuring temperature distribution data on the face surface are provided. An arithmetic process for skin diagnosis is performed by using, for example, a technique such as image processing using the face image data captured by the imaging unit. When this calculation process is performed, the calculation contents are corrected based on the measurement result of the temperature measurement means, so that it is possible to correct the difference in the surface temperature of the face and perform accurate skin diagnosis. . As a result, it is possible to provide a face photographing apparatus capable of performing accurate skin evaluation based on photographed face image data regardless of the difference in the surface temperature of the face.

  The arithmetic processing according to the present invention preferably includes a process for determining the size of pores based on a threshold value, and the correction data generating means preferably changes the threshold value.

  When determining the size of pores, a threshold value can be set to determine whether the pore is larger than a predetermined size, but if the surface temperature changes, the size of the pore itself depends on the temperature. Change. Therefore, accurate skin diagnosis can be performed by correcting so that an appropriate threshold is set based on the value of the surface temperature.

  A preferred embodiment of a face photographing apparatus according to the present invention will be described with reference to the drawings. FIG. 1 is a cross-sectional view showing the internal configuration of the face photographing apparatus, and FIG. 2 is a longitudinal sectional view of the face photographing apparatus shown in FIG.

  The face photographing apparatus H is used in cosmetic dermatology, cosmetic surgery, plastic surgery and the like. It is used for the purpose of observing changes in skin condition over time by photographing a person's face at regular or moderate intervals and confirming the effects of cosmetics and skin treatment. This apparatus has a function of evaluating spots, wrinkles, pores, color unevenness, and the like from the obtained face image, and can also evaluate porphyrins, hide spots, etc. using ultraviolet rays.

  In FIG. 1, the face photographing apparatus H includes a substantially cube-shaped housing 1, and when photographing a face, the face is held in a predetermined position in the housing 1, and indoor light is emitted. A structure that does not easily enter the housing 1 is employed. An opening 1a is formed on the front surface of the housing 1, from which a face is inserted. A curtain 1b is provided to prevent outside light from entering from the opening 1a. As face holding means for holding a person's face in a predetermined position, a chin rest 2 and a forehead press 3 are provided. The chin rest 2 and the forehead retainer 3 are configured to be manually position-adjustable so as to correspond to the size and shape of a person's face.

  Specifically, as shown in FIG. 2, the presser portion 3a of the forehead presser 3 is rotatable around the shaft 3b so as to be able to cope with a change in the posture of the face. The chin mount 2 can be rotated around the shaft 2b via the connecting portion 2a so as to be able to cope with a change in the posture of the face. The shaft 3b is provided slightly behind the position of the presser portion 3a. Similarly, the shaft 2b is also provided slightly behind the position where the jaw of the jaw mounting base 2 is placed. The shaft 3b and the shaft 2b are provided coaxially. Thereby, even if the posture of the face is changed to landscape, the face is positioned at the center of the screen.

  A photographing camera 4 is provided as a photographing means for photographing the face, and is held on the housing 1 via a camera support 5. When the face faces the front position (first posture), the position of the photographing camera 4 is fixed so that the face just faces the photographing camera 4. However, the position of the photographing camera 4 may be finely adjustable.

  The camera support 5 is also equipped with an infrared thermography 6 that functions as a temperature measuring means, and can measure the surface temperature of the face. When a face image is taken by the imaging camera 4, the temperature distribution data of the face surface is also acquired at the same time by the thermography 6.

  When taking a side image of the face, the face holding means is adjusted so that the face is in a sideways posture (second posture). As shown in FIG. 1, three shooting positions of the face by the shooting camera 4 are set, that is, a front position P0 and left and right side positions P1, P2. When the face line of sight faces the front position P0 (first posture), a front image is taken, and when the face line of sight faces the side positions P1 and P2 (second posture), A side image is taken.

  The side positions P1 and P2 are positions set at θ ° with respect to the front position P0. This θ is set between 30 ° and 75 °, and is generally set between 40 ° and 45 °. Further, the side surface positions P1 and P2 are not limited to two locations, and more side surface positions may be set. For example, the side images may be taken at 30 °, 45 °, 60 °, and 75 ° positions.

  As the photographing camera 4, a digital camera is used, but any type of camera may be used. If necessary, one having a video function may be used, or a video camera capable of taking a still image may be used.

  Light sources 8 for illuminating the face are arranged at a plurality of locations in the housing 1. The light source 8 is arranged so as to face the face along the circumferential direction with the position of the face as a center, and is also arranged at a plurality of locations above and below as shown in FIG. The light source 8 includes a light source that irradiates visible light and a black light that irradiates ultraviolet rays. Either one or both are used depending on the purpose. As the light source 8 for irradiating visible light, an appropriate one such as a halogen lamp, a fluorescent lamp, an LED, or a flash light source can be used. A flash light source may be used as the light source 8.

  A diffusion plate 9 for uniformly diffusing light emitted from the light source 8 is disposed between the light source 8 and the face. The cross-sectional shape of the diffusion plate 9 shown in FIG. As shown in FIG. 2, the diffusion plate 9 is arranged not only on the front surface of the face but also on the ceiling portion and the bottom surface portion, and thus the face is surrounded by the diffusion plate 9. The diffusion plate 9 does not have to be semicircular, and may have a polygonal shape or an elliptical shape. A rectangular opening 9a is formed to enable photographing at the front position P0 where the photographing camera 4 is installed.

  The control device 20 has a function of controlling the operation of the face photographing device H and a function of performing image processing relating to a face image photographed by the photographing camera 4. The captured face image is displayed on the monitor 21. The keyboard 22 inputs various operation commands to the face photographing device H and various operation commands and data for performing image processing. The control device 20, the monitor 21, and the keyboard 22 may be configured by a general-purpose computer (personal computer).

<Control related>
Next, main control functions of the face photographing apparatus H will be described with reference to the control block diagram of FIG. The controller 30 provides a function for comprehensively controlling the face photographing apparatus H, and includes a CPU, a memory, and other necessary programs. The illumination control unit 31 controls the turning on / off of the light source 8. The light source 8 is turned on when taking a face image. When using visible light and ultraviolet light separately, control is performed as to which of them is lit or both of them are lit.

  The camera control unit 34 controls the operation of the photographing camera 4, and specifically takes a face image by releasing the shutter. The face image is taken when the face orientation is the front position P0 and the side positions P1 and P2.

  The thermography control unit 33 controls the operation of the thermography 6. When a face image is taken by the imaging camera 4, the face surface temperature is also measured by the thermography 6. The measurement of the surface temperature is also performed when the face orientation is the front position P0 and the side positions P1 and P2.

  The data storage unit 35 stores the face image data photographed by the photographing camera 4 and the temperature distribution data of the face surface measured by the thermography 6. The face image data is digital color image data, and is stored in an appropriate file format such as JPEG. The face image data is stored for each of the front image and the left and right side images, and is stored together with a person ID (identification information) that identifies a person, shooting date, and surface temperature data.

<Configuration of facial image analysis system>
The image processing unit 36 has a function of performing image processing on the acquired face image data and detecting pores and spots, and can be configured by software or a combination of software and hardware.

  The face area setting unit 36 a has a function of setting a face area included in the face image data photographed by the imaging camera 4. The face area refers to a human skin area in the face, and excludes nostrils, eyebrows, eyes, mouth, etc., which are not related to skin evaluation. The face area may be set by the operator using a mouse or the like while viewing the screen of the monitor 21 or automatically. In the present invention, any method may be adopted, and it is preferable that both methods are provided functionally.

  After the face area is set, the difference calculating unit 36b acquires the pixel value having the maximum frequency for each pixel constituting the face area based on the histogram created for the predetermined area including the target pixel. A function of calculating difference data between the obtained pixel value and the pixel value of the target pixel. Based on this function, the difference between the normal skin color and the color of the skin and pores / stains is calculated to determine whether the skin is a pore or a stain as will be described later.

  The quaternarizing means 36c has a function of quaternizing the difference data calculated by the difference calculating means 36b with three threshold values. Based on this function, the difference data is classified into four types [0, 1, 2, 3].

  Based on the result of the quaternarization, the area classification unit 36d obtains difference data as a first area [1, 2, 3] including pores and spots, and a second area [0] including no pores and spots. Classify into: The second region represents a normal skin region, and the first region is a region estimated to be composed of pores or spots. The first region is actually configured as an aggregate of a large number of clusters (small regions having a small area).

  The determination unit 36e performs an analysis on each cluster obtained by the region classification unit 36d and performs a determination process as to whether it is a pore or a spot. For example, it is possible to analyze the shape / area of the cluster and determine whether it is a pore / stain / wrinkle. Further, by analyzing the existence ratio of the quaternarized pixels [1, 2, 3], for example, it is possible to perform a detailed analysis such as whether the pores are conspicuous or dark pores. Details of each function will be described using specific face images.

  These quaternization means 36c, area classification means 36d, and determination means 36e correspond to arithmetic means for performing arithmetic processing for skin diagnosis.

  The correction processing unit 36f has a function of correcting calculation contents when the quadrature conversion unit 36c, the region classification unit 36d, the determination unit 36e, and the like perform calculations. Specifically, the calculation content is corrected based on the measurement result by the thermography 6. This is because correction is performed so that skin diagnosis can be performed as accurately as possible because the pore size changes when the surface temperature of the face differs. For example, pores tend to close when the temperature is cold and open when the temperature is warm. This also has the same tendency when sweating.

  In order to determine pores / stains, the quaternizing means 36c performs level division based on three threshold values, but the setting of the threshold values can be changed. For example, when the skin surface temperature is higher than that at the previous photographing, the threshold value is corrected to be increased as a whole. Since the face surface temperature may vary depending on the location, an average value is obtained from the temperature distribution data, and correction is performed based on whether the average value is higher or lower than the previous time.

  The display data generation unit 36g generates data for displaying an image obtained as a result of image processing on the monitor 21. Further, the image obtained as a result of the image processing can be appropriately stored in the face image data storage unit 35.

The evaluation area setting unit 34 sets a face area for skin diagnosis, and can be set by, for example, specifying a specific area in the face image displayed on the monitor 21 with a mouse or the like. . Of course, the entire face area may be diagnosed. Based on the setting content by the evaluation area setting means 34, image processing is performed by the image processing unit 36. Even when correction is performed based on the measurement result of the surface temperature, based on the surface temperature data in the set area, Correction is performed.

<Image processing procedure>
Next, a procedure from when a face image is taken to when pores / stains are detected and displayed on the monitor 12 will be described. FIG. 4 is a flowchart showing an outline of the processing procedure.

  First, a face image is taken and the surface temperature (temperature distribution) of the face is measured by the thermography 6 (S1). There are three types of face images to be photographed: a front image and left and right side images. For convenience of explanation, only the front image will be described. The face image data photographed by the imaging camera 4 is stored in the face image data storage unit 35. Similarly, the temperature distribution data of the face is stored (S2). Further, the photographed face image can be displayed on the monitor 21 (see FIG. 5). When shooting is performed with the imaging camera 4, the shooting is performed so that the face is as full as possible.

  Next, the face area is set. The face image data includes a region that is not a face and a region that is not related to the evaluation target of pores and spots even within the face region. Therefore, in order to perform image processing efficiently, the face area is set (S3).

  Therefore, first, the average pixel value of the pixel group included in the rectangle in the central area A of the screen is acquired (S30). The central area A is a preset area, and is set to a size of about 1/4 vertical and 1/4 horizontal with respect to the image size (see FIG. 6). Since the face image is taken so as to fill the screen, it is estimated that the center part of the face is included in this range. Note that a rectangle representing the central area A may be displayed on the monitor screen so that the operator can check it. Moreover, you may comprise so that the position and magnitude | size of the center area | region A can be changed as needed. Note that the average pixel value can be obtained by, for example, an average value of RGB constituting the color image data.

  The average pixel value obtained as described above is estimated to represent the skin color. Next, a pixel having a pixel value within a predetermined range with respect to the average pixel value is searched (S31). For example, pixels within a range of ± 5% with respect to the average pixel value are extracted. The range to be searched is an area outside the central area A in addition to the central area A described above. The detected pixel group is set as a face area (S32), and image processing for detecting pores and spots is performed in the face area. The set face area is schematically shown in FIG. Areas other than the outline of the face (background, etc.) and even inside the face, mouth, nostrils, eyes, eyebrows, etc. are areas that are not related to detection of pores and spots, and are excluded from the face area .

  When the face area is set, the following scanning is performed for each pixel in the face area. That is, a histogram of a predetermined area range is created with the target pixel as the center (S4). The histogram is performed using blue image data among RGB data constituting the color image data. This is because when a stain occurs, the blue component is greatly reduced. In the case of pores, since RGB decreases to approximately the same level, there is no problem in accuracy regardless of which component is used.

  The created histogram is conceptually shown in FIG. A rectangular (square or rectangular) predetermined area C is set around the target pixel B. For example, in the case of an imaging camera of 10 million pixel class, the size of the predetermined area C is set to a size of 51 pixels × 51 pixels. Depending on the number of pixels, etc., this can be performed as appropriate.

  Accordingly, the histogram is created for 51 × 51 = 2601 pixels in the above example, and is shown in FIG. 8B, for example. A pixel value (blue image data) having the maximum frequency is acquired from this histogram (S5). This pixel value is indicated by D. Next, difference data (difference data for blue image data) obtained by subtracting the pixel value of the target pixel B from the pixel value D having the maximum frequency is calculated (S6). The pixel value D having the highest frequency usually represents the skin color that is not a pore / stain, and if the pixel of interest B corresponds to a skin part that is also not a pore / stain, the difference data is zero.

  In addition, if there are pores, spots, etc., the luminance becomes darker, so when the difference data is calculated, it becomes a positive value. Depending on the size of the difference data, it is possible to determine whether it is a stain or a pore, and the level of the pore. If the difference data becomes negative, it is set to 0. This is because the entire face area is not illuminated with a uniform illuminance due to the illumination light source, and this variation is taken into consideration.

  In the above description, the average value may be used instead of the maximum frequency. However, the maximum frequency is used if the average value is taken, especially if there is a large dark spot, the average value may decrease, and accurate pore / spot detection may not be possible. is there.

  When the difference data is calculated, it is determined whether or not scanning has been completed for all the pixels in the face area (S7). If not, the process returns to step S4 and the same calculation is repeated for the next pixel of interest.

  When scanning is completed over the entire face area, the difference data is binarized using three threshold values (S10). However, the threshold value is not a fixed numerical value, but is corrected according to the temperature distribution data of the face measured by the thermography 6. That is, the surface temperature at the previous photographing is compared with the surface temperature at the current photographing (S8), and if there is no change in the surface temperature at the previous photographing and the face temperature, the same threshold as the previous photographing is used. If the surface temperature is different from the previous time, the threshold setting is changed (S9). Specifically, if the temperature is measured in the higher direction, the three threshold values are shifted in the direction of increasing overall, and if measured in the lower direction, it is shifted in the direction of decreasing overall. Since the previous surface temperature data is stored in the data storage unit 35, it can be extracted by searching for data on the person.

  The specific contents of the quaternarization will be described. The difference data is classified into four types [0, 1, 2, 3], and the first area [1, 2, 3 including pores and spots is included. ] And a second region [0] that does not include pores and spots (S9). This is a so-called labeling process. When classified into two areas in this way, the face area is the second area [0], which is a normal skin area, and the first area [1, 2, 3], which is an aggregate of many clusters (small areas). ].

  Note that, for convenience of calculation, a pixel classified as [1] is a pixel having only [1], but a pixel classified as [2] is a pixel having two values [1] and [2]. , [3] are treated as pixels having two values [1] and [3].

  FIG. 9 is a diagram showing the situation of the cluster E classified as described above, and is an enlarged display of a part of the face image. The cluster E represents a small region in which the pixel group represented by the above [1, 2, 3] is gathered. Pixels included in each cluster E are classified as either [1], [2], or [3]. Based on the number of [1,2,3] and the shape of the cluster, pore / stain is determined (S11).

  First, the detection of a spot can be determined based on the shape of the cluster E. This is because the size of the stain is usually larger than the pore and it can be determined that the spot has a predetermined shape. In particular. As shown in FIG. 9, a rectangle F surrounding the cluster E is set. The rectangle F is set so that the outer shape of the cluster E is in contact with the four sides of the rectangle F. Then, based on the lengths of the two sides x and y of the rectangle F, it is determined whether or not it is a stain. That is, if one side (x or y) of the rectangle F is 15 pixels or more, it is determined that the cluster E is a spot. In addition, about the setting of 15 pixels which is a criterion, it can change suitably. This rectangle F is set for all clusters E.

  Next, detection of pores will be described. First, the rectangle F is set as described above, and if the aspect ratio is 1: 0.65 or more, it is determined as a pore. Since the shape of the pore can be estimated to be close to a circle, it is possible to determine that a pore that is closer to a square than the above ratio is a pore.

  Further, the pores are finely classified into two levels: conspicuous pores and dark pores. Whether or not the pore is a conspicuous pore (large pore) is determined by dividing the number of pixels assigned with [2] by the number of pixels assigned with [1] is 0.3 or more. It is determined that the pore is conspicuous. If it is less than 0.3, it is determined to be a thin pore. Here, the number of pixels to which [1] is assigned is a value obtained by adding the number of pixels to which only [1] is assigned and the number of pixels to which [2] is assigned according to the rules described above. Become.

  It is determined whether or not the pore is a dark pore if the value obtained by dividing the number of pixels assigned to [3] by the number of pixels assigned to [1] is 0.5 or more. It is determined that If it is less than 0.5, it is determined to be a thin pore. Here, the number of pixels to which [1] is assigned is a value obtained by adding the number of pixels to which only [1] is assigned and the number of pixels to which [3] is assigned according to the rules described above. Become.

  In the above, the threshold values of 0.3 and 0.5 can be changed as appropriate. Also, it is preferable to change the setting of this threshold according to the difference in the temperature of the face surface.

  Note that a cluster close to a square with an aspect ratio of 1: 0.65 or more is judged as a thin pore (small pore) if it is composed of only [1] pixels without [2] [3] pixels. May be. In this case, pores are determined and classified at three levels. Or you may determine with noise.

  FIG. 10 is a conceptual diagram showing the relationship between the distribution of difference data and the threshold value. Here, three threshold values T1, T2, and T3 are shown, and as a result, they are quaternized to [0, 1, 2, 3]. Moreover, the result of classification (labeling) is shown in the figure, and is classified into thin pores, conspicuous pores, dark pores, and spots.

  When the determination process is completed, a display process for displaying the pore / spot detection results on the monitor screen is performed (S12).

<Another embodiment>
In this embodiment, quaternarization is performed on the difference data, but the present invention is not limited to this. In order to discriminate between pores and spots, binarization is performed at a minimum. Further, ternarization or quinary or more may be employed depending on the level of pores to be detected. For example, a threshold value for further dividing the region indicated by [1] in FIG. 10 into two stages may be set to identify whether it is a thin pore or noise. Further, a threshold value that can further classify the stain may be set.

  In the present embodiment, a method of automatically searching for the setting of the face area based on the average pixel value is adopted. However, when only a part of the face is determined, the operator can display on the monitor screen. The face area may be set with a mouse or the like. Such setting can be performed based on the function of the area setting means 34. When only a part of the face needs to be evaluated, setting in this way can limit the range for image processing and shorten the time required for image processing. It is preferable that any one of a method for automatically searching and setting a face area and a method for setting by an operator can be arbitrarily selected. Further, when the skin condition is determined only for a part of the face area, the threshold setting can be changed based on the surface temperature data for the area.

  Based on the temperature distribution data obtained by the thermography 6, a function for determining whether or not the blood circulation is good may be added.

Cross-sectional view showing the internal configuration of the face photographing device Longitudinal sectional view showing the internal structure of the face photographing device Block diagram showing control functions of face photographing apparatus and face image analysis system Flow chart showing the procedure for detecting pores and spots The figure which shows the example which displayed the photographed face image on the monitor screen Diagram showing the central area The figure which shows the example of the set face area The figure for explaining the processing when obtaining the difference data Diagram showing a specific example of a cluster Conceptual diagram showing the relationship between the distribution of difference data and the threshold

Explanation of symbols

4 photographing camera 5 camera support 6 thermography 21 monitor 30 controller 33 thermography control unit 34 front face area setting means 35 data storage part 36 image processing part 36a face area setting means 36b difference calculation means 36c quaternization means 36d area classification means 36e Determination means 36f Correction processing means 36g Display data generation means A Central area E Cluster F Rectangle H Face photographing device

Claims (2)

Imaging means for photographing a face image;
Temperature measuring means for measuring temperature distribution data on the face surface;
Arithmetic means for performing arithmetic processing for skin diagnosis using face image data photographed by the imaging means;
A face photographing apparatus comprising: correction processing means for correcting calculation contents when performing the calculation processing based on a measurement result by the temperature measurement means.   The face according to claim 1, wherein the calculation process includes a process for determining a pore size based on a threshold value, and the correction processing unit changes a threshold value. Shooting device.

Images