JP2008293327A - Face image display system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a face image display system for emphasizing pores and spots, and for displaying them on a monitor screen by a simple processing procedure. <P>SOLUTION: This face image display system makes a monitor 12 display face image data photographed by a imaging camera 4, and provided with a face region setting means 36a for setting a face area on the basis of photographed face image data; an averaging processing means 36b for performing averaging processing to the set face area; a difference arithmetic means 36c for calculating difference data between the face image data before averaging and the face image data to which the averaging processing has been performed; an emphasis processing means 36d for performing contrast emphasis to the difference data; and a display data generation means 25e for generating display data for making the monitor display the face image data to which the contrast emphasis has been performed. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a face image display system for displaying face image data photographed by an imaging means on a monitor.

  Such a face image display system is used in cosmetic dermatology, cosmetic surgery, plastic surgery, etc., and a face photographing device regularly photographs a person's face to observe changes in the skin condition over time. It is used for the purpose of confirming the effects of cosmetics and skin treatment. Specifically, evaluation of spots, wrinkles, pores, color unevenness and the like is performed while the obtained face image is displayed on a monitor screen.

  Among the above evaluation items, especially pores and wrinkles may be difficult to see due to the condition of illumination light hitting the face, and particularly wrinkles are greatly affected by the state of light and shadow. When the subject is in front of the eyes, it is possible to respond by changing the positional relationship between the imaging camera and the subject, but make a diagnosis or judge the progress of treatment while looking at the face images taken in the past. In the case where it is performed, the above-mentioned measures cannot be taken.

  Therefore, in order to make pores and wrinkles easy to see, a method of performing image processing on face image data and emphasizing the pores and wrinkles and displaying them on a monitor screen can be considered. As a technique for suppressing or enhancing the wrinkle component, for example, image processing using an ε filter disclosed in Patent Document 1 below is known. A method of enhancing pores / wrinkles by edge detection or sharpening processing such as a Laplacian filter is also conceivable.

Japanese Patent Laid-Open No. 2001-118084

  However, the above-described method has problems such as a complicated structure and a point where emphasis is not required, and further improvement has been desired.

  The present invention has been made in view of the above circumstances, and an object thereof is to provide a face image display system capable of highlighting pores and wrinkles and displaying them on a monitor screen by a simple processing procedure. .

In order to solve the above-described problems, a face image display system according to the present invention includes:
A face image display system for displaying face image data captured by an imaging means on a monitor,
Face area setting means for setting the face area based on the captured face image data;
An averaging processing means for performing an averaging process on the set face area;
A difference calculating means for calculating difference data between the face image data before averaging and the face image data subjected to the averaging process;
Enhancement processing means for performing contrast enhancement on the difference data;
And display data generating means for generating display data for displaying the contrast-enhanced face image data on a monitor.

  The operation and effect of the face image display system configured as described above will be described. First, a face area is set in the photographed face image data. This is to set an area where pores, wrinkles and the like exist, and to efficiently perform image processing to be performed below. Also, it is possible to avoid emphasizing unnecessary areas. Next, an averaging process is performed on the set face area. In this averaging process, features such as pores and wrinkles are erased. Next, difference data between the face image data before averaging and the face image data subjected to the averaging process is calculated. Thereby, pores and wrinkles can be extracted as features. Contrast enhancement is further performed on the difference data. Thereby, face image data in which pores and wrinkles are emphasized can be obtained, and display data can be generated and displayed on the monitor screen. Therefore, pores and wrinkles can be emphasized and displayed on the monitor screen by a simple processing procedure.

  The averaging processing means according to the present invention is preferably configured to create a histogram for a predetermined region including a pixel of interest for each pixel constituting the face region, and obtain a pixel value having the maximum frequency.

  Various processing methods can be considered as the averaging process. In the present invention, a histogram is created for the irradiation region including the target pixel, and the pixel value having the maximum frequency is acquired. It is possible to reliably obtain averaged face image data while using the procedure.

  A preferred embodiment of a face image display system according to the present invention will be described with reference to the drawings. FIG. 1 is a cross-sectional view showing an internal configuration of a face photographing apparatus used in the face image display system, and FIG. 2 is a longitudinal sectional view of the face photographing apparatus shown in FIG. FIG. 3 is a diagram showing details of the rotational movement mechanism of the photographing camera.

  The face photographing apparatus H is used in cosmetic dermatology, cosmetic surgery, plastic surgery and the like. It is used for the purpose of observing changes in skin condition over time by photographing human faces at regular or moderate intervals and confirming the effects of cosmetics and skin treatments. This device has a function of evaluating pores, spots, wrinkles, color unevenness, and the like from the obtained face image, and can also evaluate porphyrins, hide spots, etc. using ultraviolet rays. Note that the present invention particularly relates to detection of pores / stains, and therefore the following description is limited to detection of pores / stains.

<Configuration of face photographing device>
In FIG. 1, the face photographing apparatus H includes a substantially cube-shaped housing 1, and when photographing a face, the face is held in a predetermined position in the housing 1, and indoor light is emitted. A structure that does not easily enter the housing 1 is employed. An opening 1a is formed on the front surface of the housing 1, from which a face is inserted. As face holding means for holding a person's face in a predetermined position, a chin rest 2 and a forehead press 3 are provided. The chin rest 2 and the forehead retainer 3 are configured to be manually position-adjustable so as to correspond to the size and shape of a person's face. A curtain 1b is provided to prevent outside light from entering from the opening 1a. The face holding means is not always necessary.

  A photographing camera 4 is provided as photographing means for photographing a face, and is held by a camera support 5. As shown in FIG. 1, three shooting positions of the face by the shooting camera 4 are set, that is, a front position P0 and left and right side positions P1, P2. The front position P0 is a position where the photographing camera 4 faces the face, and the side positions P1 and P2 are positions set to 45 ° with respect to the front position P0. However, the angle of 45 ° is not limited to this value, and may be set to other angles. Also, the side surface positions P1 and P2 are not limited to two locations, and more side surface positions are provided. It may be set. For example, it may be set to 30 °, 45 °, 60 °, and 75 °. P3 indicates a trajectory in which the photographing camera 4 is rotated by a moving mechanism described later.

  As the photographing camera 4, a digital camera is used, but any type of camera may be used. If necessary, one having a video function may be used, or a video camera capable of taking a still image may be used.

  Thus, in order to photograph face images at a plurality of locations, the photographing camera 4 is configured to be able to rotate around the face position. The rotation center B of the photographic camera 4 is set at the position of the chin mount 2.

  Light sources 8 for illuminating the face are arranged at a plurality of locations in the housing 1. The light source 8 is arranged so as to face the face along the circumferential direction with the position of the face as a center, and is also arranged at a plurality of locations above and below as shown in FIG. The light source 8 includes a light source that irradiates visible light and a black light that irradiates ultraviolet rays. Either one or both are used depending on the purpose. As the light source 8 for irradiating visible light, an appropriate one such as a halogen lamp, a fluorescent lamp, an LED, or a flash light source can be used. A flash light source may be used as the light source 8.

  A diffusion plate 9 for uniformly diffusing light emitted from the light source 8 is disposed between the light source 8 and the face. The cross-sectional shape of the diffusion plate 9 shown in FIG. As shown in FIG. 2, the diffusion plate 9 is arranged not only on the front surface of the face but also on the ceiling portion and the bottom surface portion, and thus the face is surrounded by the diffusion plate 9. The diffusion plate 9 does not have to be semicircular, and may have a polygonal shape or an elliptical shape. Since the photographing camera 4 moves to the back side of the diffusion plate 9, a rectangular opening 9a is formed to enable photographing at the front position P0 and the side positions P1 and P2.

  The control device 20 has a function of controlling the operation of the face photographing device H and a function of performing image processing relating to a face image photographed by the photographing camera 4. The captured face image is displayed on the monitor 21. The keyboard 22 inputs various operation commands to the face photographing device H and various operation commands and data for performing image processing. The control device 20, the monitor 21, and the keyboard 22 may be configured by a general-purpose computer (personal computer).

<Camera rotational movement mechanism>
Next, details of the rotational movement mechanism of the photographing camera 4 will be described with reference to FIG. A drive motor 10 and a drive shaft 11 are provided at the rotation center B, and an arm 12 is connected to the drive shaft 11. Two connecting shafts 13 are implanted in the front end portion 12 a of the arm 12 in the vertical direction, and the camera support 5 is coupled to the upper end of the connecting shaft 13. The camera support 5 has a substantially U-shape when viewed from the side, and the photographing camera 4 is disposed inside the U-shape.

  A roller support 14 is attached to the back surface of the arm 12 at the tip end side so as to be rotatable about the vertical direction with respect to the arm tip end portion 12a, and the roller 15 is held rotatably. The roller 15 is configured to be in contact with the bottom surface of the housing 1, whereby the photographing camera 4 can be smoothly rotated. The rotational movement mechanism of the photographing camera 4 is not limited to the present embodiment, and various modifications can be considered.

<Control related>
Next, main control functions of the face photographing apparatus H and the face image display system will be described with reference to the control block diagram of FIG. The controller 30 provides a function for comprehensively controlling the face photographing apparatus H, and includes a CPU, a memory, and other necessary programs. The illumination control unit 31 controls the turning on / off of the light source 8. The light source 8 is turned on when taking a face image. When using visible light and ultraviolet light separately, control is performed as to which of them is lit or both of them are lit.

  The camera position control unit 32 performs movement control for stopping the photographing camera 4 at each of the front position P0 and the side positions P1 and P2, and issues an operation command to the drive motor 10. In order to stop the photographing camera 4 at a predetermined position, a camera position sensor 33 is provided. As the camera position sensor 33, sensors for detecting the above three photographing positions are provided. For example, the position of the arm 12 can be detected by an optical sensor, a micro switch, or the like.

  Further, not only the above three shooting positions but also an intermediate position between them can be detected. For example, the position of the photographing camera 4 can be detected in fine steps based on a signal from an encoder that rotates in conjunction with the drive motor 10, a count value of a drive pulse supplied to the drive motor 10 (pulse motor), or the like. it can. Therefore, position detection can be performed for the entire range in which the photographing camera 4 is movable.

  The camera control unit 34 controls the operation of the photographing camera 4, and specifically takes a face image by releasing the shutter. The face image is taken at three locations, the front position P0 and the side positions P1 and P2. When it is necessary to finely adjust the position of the photographing camera 4, photographing is performed after the fine adjustment is completed.

  The face image data storage unit 35 stores face image data photographed by the photographing camera 4. The face image data is digital color image data, and is stored in an appropriate file format such as JPEG. The face image data is stored for each front image and left and right side images, and is stored together with a person ID (identification information) for identifying a person and a shooting date.

<Configuration of facial image analysis system>
The image processing unit 36 has a function of performing image processing on the acquired face image data and detecting pores and spots, and can be configured by software or a combination of software and hardware. The image processing unit 36 is a core part of the face image display system according to the present invention.

  The face area setting unit 36 a has a function of setting a face area included in the face image data photographed by the photographing camera 4. The face area refers to a human skin area in the face, and excludes nostrils, eyebrows, eyes, mouth, etc., which are not related to skin evaluation. The face area may be set by the operator using a mouse or the like while viewing the screen of the monitor 21 or automatically. In the present invention, any method may be adopted, and it is preferable that both methods are provided functionally.

  The averaging processing means 36b has a function of creating a histogram for a predetermined area including the target pixel for each pixel constituting the face area after the face area is set, and a pixel value having the maximum frequency based on the histogram. Has a function to acquire. That is, since the pixel of interest is replaced with the pixel that exists most in the predetermined region, it can be converted into an averaged image as a whole.

  The difference calculation unit 36c has a function of calculating difference data between the pixel value acquired by the averaging processing unit 36b and the pixel value of the target pixel. Based on this difference data, it is possible to create display image data that has been subjected to enhancement processing such as pores and wrinkles. Moreover, based on this difference data, it can also be determined whether it is a pore or a spot.

  The enhancement processing unit 36d can create image data with enhanced contrast based on the difference data. Contrast enhancement is a process for increasing the brightness difference of an image, and a known enhancement process can be used.

  The determination unit 36e determines pores, spots, wrinkles, and the like based on the difference data obtained by the difference calculation unit 36c. An example of a processing procedure until determination is performed will be described. First, the difference data is binarized using three threshold values. Based on this function, the difference data is classified into four types [0, 1, 2, 3]. Next, based on the quaternarized result, the difference data is divided into a first area [1, 2, 3] including pores, wrinkles, wrinkles and the like, and a second area [0] including no pores / stains. And classify. The second region represents a normal skin region, and the first region is a region estimated to be composed of pores or the like. The first region is actually configured as an aggregate of a large number of clusters (small regions having a small area).

  The determination unit 36e performs an analysis process for each cluster and performs a determination process as to whether it is a pore or a stain. For example, it is possible to analyze the shape / area of the cluster and determine whether it is a pore / stain / wrinkle. Further, by analyzing the existence ratio of the quaternarized pixels [1, 2, 3], for example, it is possible to perform a detailed analysis such as whether the pores are conspicuous or dark pores.

  The display data generation unit 36f generates data for displaying an image obtained as a result of image processing on the monitor 21. Further, the image obtained as a result of the image processing can be appropriately stored in the face image data storage unit 35.

<Image processing procedure>
Next, a procedure for displaying an emphasized image such as pores and wrinkles on the monitor 12 after photographing a face image will be described. FIG. 5 is a flowchart showing an outline of the processing procedure.

  First, a face image is taken (S1). There are three types of face images to be photographed: a front image and left and right side images. For convenience of explanation, only the front image will be described. The face image data photographed by the photographing camera 4 is stored in the face image data storage unit 35 (S2). The photographed face image can be displayed on the monitor 21 (see FIG. 6). When photographing with the photographing camera 4, the photographing is performed so that the face is as large as possible.

  Next, the face area is set. The face image data includes a region that is not a face and a region that is not related to an evaluation target such as pores and wrinkles even within the face region. Therefore, in order to perform image processing efficiently, the face area is set (S3).

  Therefore, first, the average pixel value of the pixel group included in the rectangle in the central area A of the screen is acquired (S30). This central area A is a preset area, and is set to a size of about 1/4 vertical and 1/4 horizontal with respect to the image size. Since the face image is taken so as to fill the screen, it is estimated that the center part of the face is included in this range. Note that a rectangle representing the central area A may be displayed on the monitor screen so that the operator can check it. Moreover, you may comprise so that the position and magnitude | size of the center area | region A can be changed as needed. Note that the average pixel value can be obtained by, for example, an average value of RGB constituting the color image data.

  The average pixel value obtained as described above is estimated to represent the skin color. Next, a pixel having a pixel value within a predetermined range with respect to the average pixel value is searched (S31). For example, pixels within a range of ± 5% with respect to the average pixel value are extracted. The range to be searched is an area outside the central area A in addition to the central area A described above. The detected pixel group is set as a face area (S32), and image processing for enhancing pores, wrinkles and the like is performed in the face area. The set face area is schematically shown in FIG. Even in areas other than the outline of the face or inside the face, mouth, nostrils, eyes, eyebrows and the like are areas that are not related to detection of pores and spots, and are excluded from the face area.

  When the face area is set, the following scanning is performed for each pixel in the face area. That is, a histogram of a predetermined area range is created with the target pixel as the center (S4). This is conceptually illustrated in FIG. A rectangular (square or rectangular) predetermined area C is set around the target pixel B. For example, in the case of an imaging camera of 10 million pixel class, the size of the predetermined area C is set to a size of 51 pixels × 51 pixels. Depending on the number of pixels, etc., this can be performed as appropriate.

  Accordingly, the histogram is created for 51 × 51 = 2601 pixels in the above example, and is shown in FIG. 9B, for example. From this histogram, the pixel value having the maximum frequency is acquired (S5). This pixel value is indicated by D. Next, difference data obtained by subtracting the pixel value of the target pixel B from the pixel value D having the maximum frequency is calculated (S6). The pixel value D having the highest frequency usually represents a skin color that is not pores, wrinkles, spots, etc. If the pixel of interest B corresponds to a skin portion that is also not pores, wrinkles, spots, etc., the difference data Becomes 0.

  In addition, if there are pores or the like, the luminance becomes darker, so when the difference data is calculated, it becomes a positive value. Depending on the size of the difference data, it is possible to determine whether it is a stain or a pore, and the level of the pore. If the difference data becomes negative, it is set to 0. This is because the entire face area is not illuminated with a uniform illuminance due to the illumination light source, and this variation is taken into consideration.

  Note that, by using the maximum frequency in the above, it is possible to easily generate averaged image data. In addition, when evaluating pores, a stain, etc. using difference data, you may acquire by an average value instead of the maximum frequency. However, if the average value is taken, especially if there is a large dark spot, the average value will decrease, and it may be impossible to detect pores or the like with high accuracy.

  When the difference data is calculated, it is determined whether or not scanning has been completed for all the pixels in the face area (S7). If not, the process returns to step S4 and the same calculation is repeated for the next pixel of interest.

  When scanning is completed over the entire face area, the difference data is binarized using three threshold values (S8). Specifically, the difference data is classified into four types [0, 1, 2, 3], and the first region [1, 2, 3] including the pores and spots, and the pores and spots. Is classified into the second area [0] that does not include (S9). This is a so-called labeling process. When classified into two areas in this way, the face area is the second area [0], which is a normal skin area, and the first area [1, 2, 3], which is an aggregate of many clusters (small areas). ].

  Next, contrast enhancement processing is performed on the four-valued face image data (S10). In this example, each pixel value is represented by four values [0, 1, 2, 3] by quaternization. For example, [0, 2, 4, 6] or [0 , 3, 6, 9], a greater brightness difference is produced, so that enhanced image data can be obtained.

  When the enhancement process is completed, a display process for displaying the image on the monitor screen is performed (S11). An example of this display is shown in FIG. The face image data subjected to the enhancement process is stored in the face image data storage unit 35 as necessary (S12).

  The image data that has been binarized in step S8 is also used for determination processing such as pores and spots (S9). This determination process will be briefly described. For example, spot detection can be determined based on the shape of the cluster. This is because the size of the stain is usually larger than the pore and it can be determined that the spot has a predetermined shape. Specifically, a rectangle surrounding the cluster is set, and whether or not it is a stain is determined based on the lengths of the two sides x and y of the rectangle. For example, if one side (x or y) of the rectangle is 15 pixels or more, the cluster is determined to be a stain.

  Next, a detection example of pores will be described. If the rectangle is set as described above and the aspect ratio is 1: 0.65 or more, it is determined that the pore is a pore. Since the shape of the pore can be estimated to be close to a circle, it is possible to determine that a pore that is closer to a square than the above ratio is a pore.

  Further, pores can be detected by classifying them into two levels of conspicuous pores and dark pores based on the existence ratio of [1] [2] [3].

  When the determination process ends, a display process for displaying the detection result of pores or the like on the monitor screen is performed (S12).

<Another embodiment>
As the averaging process, for example, a process using a moving average filter may be performed in addition to the method of the present embodiment.

  In this embodiment, the emphasis process is performed on the quaternarized image data, but the emphasis process may be directly performed on the difference data before the quaternarization.

  In the present embodiment, a method of automatically searching for the setting of the face area based on the average pixel value is adopted. However, when only a part of the face is determined, the operator can display on the monitor screen. The face area may be set with a mouse or the like. When only a part of the face needs to be evaluated, setting in this way can limit the range for image processing and shorten the time required for image processing. It is preferable that any one of a method for automatically searching and setting a face area and a method for setting by an operator can be arbitrarily selected.

  In the present embodiment, the histogram is preferably performed using blue image data out of RGB constituting the color image data. Therefore, the difference data is also calculated for the blue image data. This is because when a stain occurs, the blue component is greatly reduced. In the case of pores, since RGB decreases to approximately the same level, there is no problem in accuracy regardless of which component is used.

  In this embodiment, quaternarization is performed on the difference data, but the present invention is not limited to this. It may be binarized or may be binarized or higher. For this, an appropriate threshold value may be set according to the detection purpose.

Cross-sectional view showing the internal configuration of the face photographing device Longitudinal sectional view showing the internal structure of the face photographing device The perspective view which shows the detailed structure of the rotational movement mechanism of an imaging camera Block diagram showing control functions of face photographing apparatus and face image analysis system Flow chart showing the procedure for detecting pores and spots The figure which shows the example which displayed the photographed face image on the monitor screen Diagram showing the central area The figure which shows the example of the set face area The figure for explaining the processing when obtaining the difference data Figure showing a display example of an emphasized image

Explanation of symbols

4 photographing camera 21 monitor 30 controller 35 face image data storage unit 36 image processing unit (face image display system)
36a Face area setting means 36b Averaging processing means 36c Difference calculation means 36d Enhancement processing means 36e Determination means 36f Display data generation means H Face photographing apparatus

Claims (2)

A face image display system for displaying face image data captured by an imaging means on a monitor,
Face area setting means for setting the face area based on the captured face image data;
An averaging processing means for performing an averaging process on the set face area;
A difference calculating means for calculating difference data between the face image data before averaging and the face image data subjected to the averaging process;
Enhancement processing means for performing contrast enhancement on the difference data;
A face image display system comprising: display data generating means for generating display data for displaying contrast-enhanced face image data on a monitor.   2. The averaging processing unit is configured to create a histogram for a predetermined area including a target pixel for each pixel constituting a face area, and obtain a pixel value having a maximum frequency. The face image display system described in 1.

Images