JP6273640B2 - Captured image display device - Google Patents

Description

  The present application relates to a photographed image display device that photographs a user's face and the like and displays a characteristic portion of the skin.

  In the beauty field, there is a need for an apparatus for observing or evaluating skin conditions. For example, an apparatus for measuring skin spots and wrinkles has been realized.

  Patent Document 1 discloses a method for evaluating local transparency of skin by bringing a probe into contact with or close to the skin. Patent Document 2 discloses an apparatus in which a subject who performs skin measurement inserts a face and a head into a predetermined apparatus and evaluates the skin of the face.

Japanese Patent No. 4105554 Japanese Patent No. 4485837

  In such a conventional technique, it has been required that the user can more easily evaluate or observe the characteristic part of the skin in the wide part of the face. A non-limiting embodiment of the present application provides a captured image display device that can correctly display an image showing a characteristic portion of skin even in various imaging environments.

  A captured image display device according to an embodiment of the present application is a captured image display device that captures at least a part of a user's face and displays the captured image, and that captures at least a part of the user's face. And the first face image for display based on the face image obtained from the camera, or the second face image generated from the first face image, or the first face image or the first face image Generating a feature image indicating a feature portion of the skin appearing in the face image in a region other than the region including the low luminance region and the high luminance region in the skin portion of the face image of the second face image, and the feature in the first face image An image processing unit that generates a third face image on which an image is superimposed, and a display that displays the third face image.

    According to the photographed image display device disclosed in the present application, it is possible to display an image showing a characteristic part of the skin even in various photographing environments.

(A) is a front view of the picked-up image display apparatus by 1st Embodiment, (b) is a block diagram which shows the structure of information processing apparatus. (A) is a block diagram of the captured image display apparatus according to the first embodiment, and (b) is a block diagram of an image processing unit. It is a flowchart explaining operation | movement of the picked-up image display apparatus by 1st Embodiment. (A) to (c) is a diagram for explaining pixel expansion processing in binary image processing. It is a figure explaining the detection method of the characteristic part of skin. (A) is a typical example of the image displayed on the picked-up image display apparatus by 1st Embodiment, (b) has shown the specific example of the image shown on a display. (A) is a block diagram of the image processing part of the picked-up image display apparatus by 2nd Embodiment, (b) is another block diagram of an image processing part. It is a flowchart explaining operation | movement of the picked-up image display apparatus by 2nd Embodiment. It is a typical example of the image displayed on the picked-up image display apparatus by 2nd Embodiment. (A)-(c) is an example which shows the position and shape of an evaluation area in 2nd Embodiment. (A) to (c) are examples of evaluation areas that can be set by the user in the second embodiment. It is a figure which shows an example of the evaluation area set in 2nd Embodiment. It is a figure which shows the example which set the area reduced to the evaluation area shown in FIG. It is a figure which shows an example of the evaluation area, low-luminance area | region, and high-luminance area | region set in 2nd Embodiment. It is a figure which shows an example which set the area reduced to the evaluation area shown in FIG. 14, and excluded the low-intensity area | region and the high-intensity area | region. It is a flowchart explaining operation | movement of the picked-up image display apparatus by the other example of 2nd Embodiment. It is a typical example of the image displayed on the picked-up image display apparatus by the other example of 2nd Embodiment. It is a flowchart explaining operation | movement of the picked-up image display apparatus by other embodiment. It is a block diagram which shows the structure of the image process part by other embodiment. It is a figure which shows an example of the parameter | index image by other embodiment.

  The inventor of the present application examined the prior art in detail. According to the method disclosed in Patent Document 1, the local skin condition is known. However, since it is not obtained from a wide part such as the entire cheek of the face, for example, it is difficult to understand the skin state in a wide non-local area such as the cheek or the entire face, for example, the distribution of wrinkles.

  Further, according to the device disclosed in Patent Document 2, it is necessary to insert the head into a predetermined device or to place the face at a predetermined position of the predetermined device. This is because the skin condition is not accurately evaluated unless predetermined imaging conditions are satisfied.

  In view of such problems, the inventor of the present application has come up with a photographic image display device that can shoot under general or simple conditions and can correctly evaluate the skin condition. Details of the captured image display device according to an embodiment of the present application are as follows.

  A captured image display device according to an embodiment of the present application is a captured image display device that captures at least a part of a user's face and displays the captured image, and that captures at least a part of the user's face. And the first face image for display based on the face image obtained from the camera, or the second face image generated from the first face image, or the first face image or the first face image Generating a feature image indicating a feature portion of the skin appearing in the face image in a region other than the region including the low luminance region and the high luminance region in the skin portion of the face image of the second face image, and the feature in the first face image An image processing unit that generates a third face image on which an image is superimposed, and a display that displays the third face image.

  The image processing unit sets an evaluation area in a part of the first face image or the second face image, and the evaluation area includes a low luminance area and a high luminance area in the evaluation area. A feature image showing a feature portion of the skin appearing in the face image may be generated in a region other than the region including.

  The image processing unit may further generate an image indicating the evaluation area, and generate the third face image in which the feature image and the evaluation area are superimposed on the first face image.

  The image processing unit may further include a user interface that receives an input from the user, and the image processing unit may set the evaluation area based on the input from the user interface.

  The image processing unit may extract feature points of the face from the first face image, and set the evaluation area based on the feature points of the face.

  The image processing unit may generate a mirror image of an image obtained from the camera as the first face image.

  Another captured image display device according to an embodiment of the present application is a captured image display device that captures at least a part of a user's face and displays the captured image, and captures at least a part of the user's face. And the first face image for display based on the face image obtained from the camera, or the second face image generated from the first face image, or the first face image or Generating an index image in which the skin feature portion appearing in the face image is indexed in areas other than the low brightness area and the high brightness area in the skin portion of the second face image; An image processing unit that generates a third face image obtained by combining the image and the index image, and a display that displays the third face image.

  The captured image display device may further include a housing having a display surface, and the display and the camera may be provided on the display surface.

  The photographed image display device may further include a light source provided on the display surface for irradiating the user with light.

  The light may be polarized light, and the camera may include a polarizing filter.

  The photographed image display device further includes a control unit that controls the light source and the camera, and the camera captures the user's face by turning on the light source under the control of the control unit, and the control unit. The image is generated by photographing the user's face with the light source turned off under the control of the image processing unit, and the image obtained by photographing the user's face with the light source turned on is the first face image. An image obtained by subtracting an image obtained by photographing the user's face by turning off the light source from the first face image may be used as the second face image.

  The imaging and image display method according to an embodiment of the present application includes a step of acquiring a facial image by imaging at least a part of a user's face, and a first facial image for display based on the facial image, or Based on the second face image generated from the first face image, except for the first face image or a region other than a region including a region having a high luminance and a region having a high luminance in the skin portion of the second face image. A step of generating a feature image indicating a feature portion of the skin appearing in the face image, a step of generating a third face image in which the feature image is superimposed on the first face image, and the third face Displaying an image.

(First embodiment)
Fig.1 (a) is a typical front view of the picked-up image display apparatus 1 of this embodiment. The captured image display device 1 includes a camera 2, a light source 3, a display 4, and an information processing device (microcomputer) 5. The captured image display device 1 further includes, for example, a housing 6, and the camera 2, the display 4, and the light source 3 are disposed on the display surface 6 a of the housing 6. Further, the information processing apparatus 5 is accommodated in the housing 6. As shown in FIG. 1B, the information processing device 5 includes a memory 5a and an arithmetic device 5b.

  In the present embodiment, the light sources 3 are disposed on both sides of the display 4 in the horizontal direction and emit white light. The captured image display device 1 may not include the light source 3. Further, as described below, the light source 3 may emit polarized light or non-polarized light in order to detect a facial skin feature. Further, polarized lights having different polarization axes may be sequentially emitted.

  When the user operates the captured image display device 1 with the face facing the display surface 6a, the light emitted from the light source 3 illuminates the user's face, and the camera 2 captures the user's face. The captured face image is displayed on the display 4. At this time, if image processing is performed so that the image displayed on the display 4 becomes a mirror image, the user can see the image without feeling uncomfortable as if he is looking at the mirror. The image displayed on the display 4 may not be a mirror image of the photographed image.

  The photographed image display device 1 detects spots, wrinkles, pores, frying lines, and the like, which are the characteristics of the skin seen in the skin area of the face image, generates an image in which these are emphasized, and overlays it on the face image. indicate. Since the camera 2 and the display 4 are arranged on the display surface 6a, the user can use the captured image display device 1 like a mirror (electronic mirror), and the user himself can easily remove facial stains, wrinkles, and the like. Can be observed. In addition, unlike a conventional apparatus, a user only adjusts his / her posture so that his / her face appears on the display 4 without placing his / her head or face at a specific position. Can be observed. At this time, the photographed image display device 1 is affected by light from the outside, but the photographed image display device 1 does not detect stains or wrinkles in a portion where a face stain or wrinkle cannot be correctly detected due to the influence of external light. Do not display.

  Hereinafter, the configuration and operation of the captured image display device 1 will be described in detail. FIG. 2A is a functional block diagram of the captured image display device 1. As illustrated in FIG. 2A, the information processing apparatus 5 includes a function as an image processing unit 5 </ b> A that receives and processes a captured image from the camera 2, and a control unit 5 </ b> B that controls the light source 3 and the camera 2. . These functions are stored in the memory 5a (FIG. 1 (b)), which defines the image processing procedure and the control procedure of the light source 3 and the camera 2, and the software is read from the memory 5a. This is realized by being executed by the arithmetic unit 5b.

  FIG. 2B is a detailed functional block diagram of the image processing unit 5A in the present embodiment. The image processing unit 5 </ b> A includes a mirror image generation unit 11, a low luminance / high luminance region exclusion unit 12, a feature image generation unit 13, and an image synthesis unit 14.

  FIG. 3 is a flowchart for explaining the operation of the captured image display apparatus 1 in the present embodiment. The operation of the photographed image display device 1 will be described with reference to FIGS.

  First, as shown in step S <b> 1, the control unit 5 </ b> B shoots the user's face by turning on the light source 3 and operating the camera 2. As a result, a face image is generated, and the generated image is output from the camera 2 to the information processing apparatus 5. Note that the camera 2 may continuously capture a face and output a moving image to the information processing apparatus 5 or may output a still image to the information processing apparatus 5. A moving image is a set of still images generated at predetermined intervals. The information processing apparatus 5 performs image processing, which will be described below, on one or a plurality of still images, superimposes an image with emphasis on facial spots, wrinkles, etc. on the still image and displays it on the display 4. For this reason, when the image processing capability of the information processing device 5 is high, image processing is performed on all of the still images constituting the moving image, and an image in which facial spots or wrinkles are emphasized in real time is displayed. Good.

  As shown in step S2, the image processing unit 5A configured by the information processing apparatus 5 receives face image information, and the mirror image generation unit 11 displays a first face image that is a mirror image of the face image. Generate face image information. As described above, the captured image may be used as it is as the first face image without performing the mirror image processing. The generated first face image information is output to the low luminance / high luminance region excluding unit 12 and the image synthesizing unit 14.

  As shown in step S3, the low-brightness / high-brightness region excluding unit 12 detects a low-brightness region and a high-brightness region in the skin portion of the first face image, and sets a region including these regions as the first region. Exclude from face image. In the flowchart shown in FIG. 3, detection / exclusion of a low-luminance region and detection / exclusion of a high-luminance region are shown to be performed in parallel, but two processes may be performed in parallel. Either one of the processes may be performed first and the other may be performed later.

  Specifically, in the first face image, for example, it is determined whether or not it is a skin part from the chromaticity of each pixel of the image, and the brightness of the skin part is determined based on a preset brightness threshold. Extract high areas. In addition, a region with low luminance is extracted based on another threshold value of luminance set in advance. A plurality of preset threshold values may be stored so as to be changed according to the subject and the shooting environment, or may be fixed to one value.

  When the excluded high-brightness region and low-brightness region are distributed in minute pixel units and are isolated, the high-brightness region and the low-brightness region are contracted or expanded by binary image processing, and are isolated. The image may be converted so that the region does not exist. For example, as shown in FIG. 4A, consider a case where the brightness of three isolated pixels (hatched pixels) in an image is higher than a threshold value and extracted as an excluded region. In this case, the extracted pixels are expanded in order to combine isolated pixels. For example, when there are pixels extracted in four directions, up and down and left and right, adjacent to the target pixel, the target pixel is also treated as an extracted pixel. Thereby, as shown in FIG.4 (b), the area | region to exclude is expanded. Further, by performing the same process once again, as shown in FIG. 4C, it is possible to obtain one large excluded region in which three isolated pixels are combined. Further, when there are pixels extracted in eight directions including an oblique direction in addition to the vertical and horizontal directions adjacent to the target pixel, the target pixel may be treated as the extracted pixel. Binary image processing is a technique known in the digital image processing field. When performing binary image processing, the excluded area includes an area that does not satisfy a condition based on a preset luminance threshold.

  Next, as shown in step S <b> 4, the feature image generation unit 13 determines the skin portion of the first face image from which the low brightness region and the high brightness region in the skin portion of the face image are excluded. Detecting skin features.

  As described above, the characteristic part of the skin includes a stain, a wrinkle, a pore, a fray line, and the like. With reference to FIG. 5, a method for detecting a characteristic part of the skin will be described. FIG. 5 schematically shows a cross section of the skin. The skin portion of the first face image includes various kinds of information such as skin spots, wrinkles, pores, and frying lines. As shown in FIG. 5, the skin 300 includes an epidermis 300 </ b> A in a range of a depth of about 0.06 mm or more and about 0.2 mm from the surface 300 </ b> S of the skin 300 toward the inside, and a dermis 300 </ b> B existing inside the skin. Skin spots, wrinkles, pores, and frying lines have different shapes on the skin and depth positions at which they exist in the skin 300. Therefore, by obtaining images from different depths of the skin and performing identification by shape, it is possible to extract a characteristic portion in the skin portion.

  Image information from different skin depths can be obtained by using polarized light or color components. For example, when skin is photographed using linearly polarized light parallel to a predetermined direction as a light source, the linearly polarized light is reflected on the skin surface 300S while maintaining the polarization direction. On the other hand, the linearly polarized light reflected inside the skin 300A is emitted from the skin 300A with its polarization direction disturbed by scattering. For this reason, if the light source 3 that emits linearly polarized light is used and the camera 2 is used for photographing with a filter that detects polarized light parallel to the light source (parallel polarization condition), there is a lot of information on the surface of the skin. An image with less can be obtained. Further, if a light source that emits linearly polarized light is used and polarized light orthogonal to the light source is detected (orthogonal polarization condition), an image with a large amount of information inside the skin and a small amount of surface information can be obtained. That is, by using polarized light as a light source, an image that selectively includes information on the inside of the skin and information on the surface can be obtained. In order to change the polarization conditions in this way, for example, one of the two light sources 3 shown in FIG. 1A emits linearly polarized light in the horizontal direction, and the other linearly polarized light in the vertical direction is emitted. Thus, the light source 3 is provided with a polarizing filter. In addition, a polarizing filter may be provided so that linearly polarized light in the horizontal direction or the vertical direction is incident on the camera 2.

  On the other hand, the longer the wavelength, the light from the light source is incident on the inside of the skin 300A and is reflected inside. Accordingly, in the image obtained by photographing the skin, the blue (B) component contains more information on the surface of the skin, and the red (R) and infrared components contain more information inside the epidermis 300A.

  Moreover, a stain, a wrinkle, etc. may have the property of being easy to absorb the light of a specific wavelength range. In such a case, the characteristic part of the skin can be detected by using the light component in the specific wavelength region.

  In this way, by using polarized light as the face image acquired from the camera or by using the color composition included in the first face image, image information including more skin feature portions that are desired to be detected is obtained. Can do.

  In order to extract a characteristic part such as a stain or a wrinkle from the obtained image information, for example, identification by shape is performed. For this purpose, a feature portion can be extracted by performing image processing on the image information using a line detection filter, a point detection filter, or the like. For example, when the characteristic part is a wrinkle, the first face image is subjected to edge detection processing using a line detection filter, and noise can be removed to obtain a wrinkle image.

  As described above, various well-known techniques can be used for image processing for detecting spots, wrinkles, pores, constriction lines, and the like. For example, Japanese Unexamined Patent Application Publication No. 2003-144393 discloses a spot detection method. Japanese Patent Laid-Open No. 2002-200050 discloses a method for detecting a melanin component and a hemoglobin component of the skin.

  The feature image generating unit 13 may output the obtained feature image as it is to the image synthesizing unit 14, or performs image processing for adding a predetermined color to the obtained feature image so that the feature portion of the skin is conspicuous. May be.

  In addition, the feature image generation unit 13 detects a feature portion of the skin in the entire first face image, generates a feature image, and the brightness in the skin portion of the face image is generated from the feature image of the entire face image generated. A feature image may be generated by excluding a region including a low region and a region with high luminance (masking a region including a region with low luminance and a region with high luminance).

  As shown in step 5, the image composition unit 14 superimposes the feature image obtained from the feature image generation unit 13 on the first face image obtained from the mirror image generation unit 11 to generate a third face image. The generated third face image is displayed on the display 4.

  FIG. 6A schematically shows an example of an image shown on the display 4. A feature image showing wrinkles is superimposed and displayed on the first face image, which is an image of the user's face. FIG. 6B is a specific example of an image shown on the display 4, and shows an enlarged portion below the left eye (right eye in the figure) shown in FIG. In the example shown in FIG. 6B, there is a region with high luminance in a portion near the eyes. In this part, wrinkles cannot be detected correctly due to the high brightness of the image. According to this embodiment, since this part is excluded and wrinkles are displayed, the correct skin characteristic part can be displayed to the user.

  The positions of the high brightness area and the low brightness area in the first face image vary depending on the position of the user's head and the environment such as external illumination in which the captured image display device 1 is used. According to the captured image display device 1 of the present embodiment, an image showing spots, wrinkles, etc., excluding high brightness areas and low brightness areas where it is difficult to detect spots, wrinkles, etc. in the captured face image. Therefore, even if the usage environment is different, spots, wrinkles, etc. can be displayed on the face image with high accuracy.

(Second Embodiment)
The captured image display device according to the present embodiment is the first in that it sets an area in which a skin feature such as a stain or wrinkle is to be detected or displayed in the face image, and displays the skin feature in the area. Different from the embodiment. FIG. 7A is a block diagram illustrating a configuration of the image processing unit 5A ′ of the captured image display device of the present embodiment. The image processing unit 5A ′ is different from the image processing unit 5A of the first embodiment in that it further includes an evaluation area setting unit 15.

  FIG. 8 is a flowchart for explaining the operation of the captured image display apparatus according to this embodiment. With reference to FIG. 7A and FIG. 8, the operation of the captured image display apparatus 1 of the present embodiment will be described.

  First, similarly to the first embodiment, in step S1, the user's face is photographed by the camera 2, and in step S2, first face image information indicating a first face image that is a mirror image of the face image is obtained. Generate. The generated first face image information is output to the evaluation area setting unit 15 and the image composition unit 14.

  As shown in step S <b> 6, the evaluation area setting unit 15 determines an evaluation area in which a skin feature is to be detected or displayed in the first face image, and extracts the evaluation area from the first face image. The user's face image includes an area where it is not necessary to detect spots, wrinkles, or the like, and the user may want to observe a characteristic part of the skin only in a specific area. By limiting the region for detecting the skin feature to the evaluation area, it is possible to detect the skin feature at a higher speed and generate a feature image. Further, since the evaluation area is set to the skin part of the face, detection with higher accuracy is possible.

  The evaluation area setting unit 15 first detects facial feature points in the first face image in order to set the evaluation area. As a method for detecting a facial feature point, a facial feature point used in a known face recognition technique and its determination method can be used. For example, edges such as eyebrows, eyes, nose, and mouth are used as feature points, and an evaluation area is determined based on the positions of these feature points.

  FIG. 9 shows an example of the evaluation area 21 set on the first face image. The evaluation area 21 may be an area in which the above-described feature points (indicated by circles in FIG. 9) are connected by a straight line. FIGS. 10A to 10C show other examples of the evaluation area 21. The evaluation area 21 may be arranged at a different position or may have a different shape depending on the characteristics of the skin to be detected. For example, when it is desired to detect wrinkles, the evaluation area 21 may be set so as to include an area in the vicinity of the eyes as much as possible. Further, as shown in FIG. 10B, when it is desired to detect a spot, for example, the evaluation area 21 may be set so as to include as wide a cheek region as possible. Further, when it is desired to detect pores, the evaluation area may be set so as to include the nose as shown in FIG.

  Further, the user may determine the position and shape of the evaluation area 21. Depending on the user, it is also conceivable that the site where the skin condition is anxious differs from the evaluation area set in the captured image display device in advance. Therefore, for example, as shown in FIGS. 11A to 11C, the evaluation area setting unit 15 prepares a plurality of evaluation areas that can be measured based on the facial feature point information. One of the evaluation areas may be selected. Moreover, it may be comprised so that a user can change the shape of an evaluation area. In this case, the display 4 may be a touch panel or the like, and may include a user interface that receives user input.

  As shown in step S3, the low-brightness / high-brightness region excluding unit 12 detects a low-brightness region and a high-brightness region in the evaluation area 21 of the first face image, and determines a region including these regions. Exclude from evaluation area 21.

  FIG. 12 shows an example of the evaluation area 21 when the subject is illuminated from the front and the face is photographed with the eyes closed. The evaluation area 21 is inside the blackened area. By illuminating from the front, illumination light is irradiated obliquely in the vicinity of the outer periphery of the cheek, and the tendency of the reflected light to become small becomes remarkable, and a low-luminance portion is generated. In addition, by selecting the lower part of the eye as the feature point, the eyelashes of the upper eyelid protrude from the evaluation area. Since the eyelashes are black, the luminance is low.

  As described above, when there is a low luminance area to be excluded in the vicinity of the boundary of the evaluation area 21, in this embodiment, the evaluation area 21 is reduced so as not to include the low luminance area to be excluded. For example, as shown in FIG. 13, a new area 22 may be set at a position, for example, 10 pixels inside from the evaluation area 21, and a skin feature may be detected within the area 22. The area 22 is determined by excluding the peripheral area including the low brightness area and the high brightness area from the evaluation area 21.

  As a result, it is possible to exclude an area that is likely to occur in the vicinity of the boundary of the evaluation area and is likely to be erroneously detected. Here, an area 22 having a boundary inside the evaluation area at a certain distance from the boundary is set. That is, the evaluation area 21 and the area 22 are similar to each other, and an area 22 that is evenly contracted from the evaluation area 21 is set. However, the new area 22 may not be similar to the evaluation area 21 as long as it does not include a low luminance area to be excluded or a high luminance area to be excluded. For example, it is possible to set the eyelashes around the eyelashes and the cheeks to have a large amount of contraction, and to reduce the amount of contraction in the other parts. Thereby, the area | region which is easy to misdetect the characteristic part of skin is excluded more reliably.

  As shown in FIG. 14, when a high-luminance area and a low-luminance area exist in the evaluation area 21, the above-described area 22 is set, and the threshold value is set as described in the first embodiment. A region with high brightness and a region with low brightness may be extracted using, and the extracted region may be expanded or contracted to set a region to be excluded. As a result, for example, as shown in FIG. 15, a region including a low luminance region and a high luminance region is excluded from the evaluation area 21.

  Next, as shown in step S <b> 4, similar to the first embodiment, it appeared in the face image in the evaluation area 21 except for the area including the low luminance area and the high luminance area in the evaluation area 21. A feature image showing a feature portion of the skin is generated. Specifically, as described in the first embodiment, the skin that appears in the face image in the evaluation area 21 other than the low brightness area and the high brightness area in the evaluation area 21. The feature portion is detected, and a feature image indicating the feature portion is generated. Alternatively, in the entire first face image, a feature portion of the skin is detected, and a region including a region other than the evaluation area 21 and a low luminance area and a high luminance area in the evaluation area 21 from the generated whole characteristic image. Is excluded (masked) to generate a feature image.

  Further, as shown in step S <b> 5, the image composition unit 14 generates a third face image by superimposing the feature image obtained from the feature image generation unit 13 on the first face image obtained from the mirror image generation unit 11. The generated third face image is displayed on the display 4.

  According to the photographed image display device of the present embodiment, in order to generate a feature image showing a feature portion of the skin in the evaluation area among the photographed face images, more accurately overlaying a stain, wrinkles, etc. on the face image. Can be displayed. In addition, it is possible to detect or display spots, wrinkles, etc. according to the user's intention.

  In the present embodiment, an image indicating the evaluation area may be superimposed on the face image displayed on the display 4. FIG. 7B and FIG. 16 are a block diagram and a flowchart of the image processing unit 5A ″ when the evaluation area is displayed as an image. As illustrated in FIG. 7B, the evaluation area setting unit 15 ′ of the image processing unit 5 </ b> A ″ generates an image of the set evaluation area and outputs it to the image composition unit 14. The image composition unit 14 generates a third image in which the feature image and the image of the evaluation area are superimposed on the first face image, and outputs the third image to the display 4.

  FIG. 17 is a schematic diagram illustrating an example in which a third image on which an evaluation area is superimposed is displayed. By showing the evaluation area 21, it becomes easy to recognize in which part the skin feature is detected. Therefore, since it is not an evaluation area, the user can discriminate between the case where the skin feature is not shown and the case where the skin feature is not detected although it is in the evaluation area, and the skin state is observed more. It becomes easy to do.

(Other embodiments)
In the first and second embodiments, the first face image is used in step S3, and the region including the low luminance region and the high luminance region in the skin portion from the first face image is excluded. However, in order to make it easier to detect the characteristic part of the skin, an image obtained by performing image processing on the first face image may be used, and the procedure after step S3 may be performed.

  For example, as shown in the flowchart of FIG. 18, in step S1 ′, the face image obtained by shooting the face of the user by turning on the light source 3 that emits polarized light, and the face of the user by turning off the light source 3 are taken. To obtain a face image. Next, in step 2 ', specular images of these face images are generated and set as first and first' face images, respectively.

  In step 7, an image obtained by subtracting the first 'face image from the first face image is set as a second face image. Since the second face image obtained in this way includes only face image information obtained by polarized light, it includes only information on the surface of the skin or only information on the inside of the skin, and more accurately detects skin features. It's easy to do. As shown in FIG. 18, step S3 and the subsequent steps may be the same procedure as in the first or second embodiment. The light source 3 may be a light source having a plurality of polarization axes. In this case, the user's face may be photographed by sequentially or simultaneously turning off lights having different polarization axes or turning off all of them. The light source 3 may emit white light that is non-polarized light.

  When the second face image that has been subjected to such image processing is used, spots, wrinkles, and the like are detected more accurately, and an enhanced image can be displayed on the display 4.

  In the first and second embodiments, the skin feature image is displayed on the display 4. However, an index image obtained by indexing the skin feature portion together with or in place of the skin feature image. May be generated and displayed on the display 4. FIG. 19 is a block diagram of an image processing unit 5A ″ ″ that generates an index image.

  As shown in FIG. 19, the image processing unit 5 </ b> A ″ ″ differs from the second embodiment in that it includes a feature index image generation unit 16. The feature index image generation unit 16 includes an index that quantifies the area, length, density, and the like of the skin feature and an image that indicates the index in the evaluation area 21 from which the low luminance region and the high luminance region are excluded. Is generated. For example, as shown in FIG. 20, an image showing a radar chart of one or more feature indexes may be generated. By synthesizing this image with at least the first face image in the image composition unit 14 and displaying it on the display 4, the user can easily grasp how many features such as spots and wrinkles appear on the face. Become.

  The photographed image display device and the photographed image display method disclosed in the present application are suitably used in the beauty field and home beauty equipment.

DESCRIPTION OF SYMBOLS 1 Captured image display apparatus 2 Camera 3 Light source 4 Display 5 Information processing apparatus 5A Image processing part 5B Control part 5a Memory 5b Arithmetic unit 6 Case 6a Display surface 11 Mirror image generation part 12 Low brightness / high brightness area exclusion part 13 Feature image generation Section 14 Image composition section 15 Evaluation area setting section 16 Feature index image generation section 21 Evaluation area 22 Area 300 Skin 300A Epidermis 300B Dermis 300S Surface

Claims (11)

A captured image display device that captures at least a part of a user's face and displays a captured image,
A camera that images at least a portion of the user's face;
An evaluation area is set in a part of the first face image for display based on the face image obtained from the camera, and the face is other than an area including a low luminance area and a high luminance area in the evaluation area. An image processing unit that generates a feature image indicating a feature portion of the skin that appears in the image, and generates a third face image in which the feature image is superimposed on the first face image;
A display for displaying the third face image ,
The image processing unit reduces the evaluation area so as not to include a low-luminance area in the vicinity of the boundary when the low-luminance area exists in the evaluation area and in the vicinity of the boundary of the evaluation area. , Image display device. The image processing section further generates an image showing the evaluation area, according to claim 1, wherein the feature image and the evaluation area in the first face image to generate the third face images superimposed Captured image display device. A user interface for receiving input from the user;
Wherein the image processing unit, the captured image display device according to claim 1 or 2 to set the evaluation zone based on input from the user interface. Wherein the image processing unit, said first extracts feature points of the face from the face image, based on the feature point of the face, the captured image display device according to claim 1 or 2 to set the evaluation area. Wherein the image processing unit, the captured image display device according to any one of claims 1 to 4 for generating a mirror image of the image obtained from the camera as the first facial image. A captured image display device that captures at least a part of a user's face and displays a captured image,
A camera that images at least a portion of the user's face;
An evaluation area is set in a part of the first face image for display based on the face image obtained from the camera, and the face is other than an area including a low luminance area and a high luminance area in the evaluation area. Generate an index image that indexes skin features that appear in the image,
An image processing unit that generates a third face image obtained by combining the first face image and the index image;
A display for displaying the third face image ,
The image processing unit reduces the evaluation area so as not to include a low-luminance area in the vicinity of the boundary when the low-luminance area exists in the evaluation area and in the vicinity of the boundary of the evaluation area. , Image display device. Further comprising the display and the camera are captured image display device according to any one of 6 claims 1 is provided on the display surface a housing having a display surface. The captured image display device according to claim 7 , further comprising a light source that is provided on the display surface and irradiates light to the user.   A captured image display device that captures at least a part of a user's face and displays a captured image,
  A camera including a polarizing filter that images at least a portion of the user's face;
  A light source for irradiating the user with polarized light;
  Based on the second face image including the face image information of the polarized light generated from the first face image for display based on the face image obtained from the camera, the luminance in the skin portion of the second face image A third face in which a feature image indicating a feature portion of the skin that appears in the face image is generated in a region other than a region including a region with low brightness and a region with high brightness, and the feature image is superimposed on the first face image An image processing unit for generating an image;
  A display for displaying the third face image;
With
  The image processing unit reduces the evaluation area so as not to include a low-luminance area in the vicinity of the boundary when the low-luminance area exists in the evaluation area and in the vicinity of the boundary of the evaluation area. , Image display device. A control unit for controlling the light source and the camera;
The camera generates an image in which the light source is turned on under the control of the control unit and the user's face is photographed, and an image in which the light source is turned off under the control of the control unit and the user's face is photographed.
The image processing unit turns the light source on and images the user's face as the first face image, and turns off the light source and images the user's face from the first face image. The captured image display device according to claim 9, wherein an image obtained by subtracting an image is used as the second face image. Acquiring a face image by imaging at least a portion of the user's face;
Skin that appears in the face image in an area other than the area including the low brightness area and the high brightness area in the evaluation area where an evaluation area is set in a part of the first face image for display based on the face image Generating a feature image showing a feature portion of
Generating a third face image in which the feature image is superimposed on the first face image;
Displaying the third face image ,
The step of generating the feature image includes the evaluation area within the evaluation area, and when the low-luminance area exists near the boundary of the evaluation area, the evaluation area is not included so as not to include the low-luminance area near the boundary. Method of shooting and image display to reduce .