JP2009188528A - Face photographing apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a face photographing apparatus capable of more satisfactorily positioning a face. <P>SOLUTION: Waveform data along two or more detection lines L1-L3 in a photographed (extracted) face image are detected as detected waveform data and are compared with reference waveform data along the detection lines L1-L3. Then, positioning information corresponding to the deviation direction and deviation amount of the respective detected waveform data to the reference waveform data is output and the face is positioned on the basis of the positioning information. Thus, the time when the position of the face is shifted in a right and left direction and the time when the angle of the face is shifted are distinguished, the face is more satisfactorily positioned. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a face photographing apparatus that photographs a face image by aligning the face with a reference face image.

  In cosmetic dermatology, cosmetic surgery, plastic surgery, etc., the face of a person (photographed person) is regularly photographed with a face photographing device, and the change in the state of the skin over time is observed. An operation for confirming the effect may be performed (see, for example, Patent Document 1). Specifically, evaluation of spots, wrinkles, pores, color unevenness and the like is performed from the obtained face image, and further, porphyrins and hide spots using ultraviolet rays are also evaluated.

  In order to evaluate such a skin condition, a face image shot before may be compared with a face image shot this time. In such a case, the previous face image and the current face image are preferably photographed under the same conditions as much as possible, and examples of the conditions include the face position and angle. That is, if the face position and angle in the face image taken this time are different from the face image taken previously, the position and size of spots, wrinkles, pores, color unevenness, etc. may not be accurately evaluated. There is.

  Here, as a method of capturing a face image under the same conditions as possible, a method of determining whether or not the head is at the reference position by detecting the top of the head (for example, see Patent Document 2), eyes, ears, nose, etc. There has been proposed a method of detecting a feature portion of a face and determining whether or not it is at a reference position (for example, see Patent Document 3).

JP 2004-302424 A JP 2003-189161 A JP 2005-117316 A

  However, when detecting the top of the head as in the technique disclosed in Patent Document 2, even if the top is at the reference position, the face angles may not match. Further, when detecting facial feature parts such as eyes, ears, and noses as in the technique disclosed in Patent Document 3, it is relatively difficult to accurately detect the positions of the feature parts. For this reason, the conventional technique may not be able to satisfactorily align the face.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a face photographing apparatus that can perform better face alignment.

  A face photographing apparatus according to a first aspect of the present invention is a face photographing apparatus for photographing a face image by aligning a face with respect to a reference face image, wherein two or more predetermined detection lines in the reference face image are detected. Reference waveform data storage means for storing waveform data as reference waveform data, waveform data detection means for detecting waveform data along the two or more detection lines in the photographed face image as detection waveform data, and the reference waveform Waveform data comparison means for comparing the data and the detected waveform data is provided.

  According to such a configuration, the waveform data along two or more detection lines in the captured face image is detected as detection waveform data, and compared with the reference waveform data along those detection lines, Alignment can be performed.

  Here, in the configuration in which only the detected waveform data on one detection line is compared with the reference waveform data, when the face position is shifted in the left-right direction and when the face angle is shifted. Thus, the deviation amount of the detected waveform data with respect to the reference waveform data may be the same. On the other hand, according to the configuration in which the detection waveform data in two or more detection lines is compared with the reference waveform data as in the present invention, the face angle is shifted when the face position is shifted in the left-right direction. Since it can be distinguished from when it is shifted, it is possible to perform face alignment better.

  According to a second aspect of the present invention, there is provided a face photographing apparatus comprising alignment information output means for outputting face alignment information based on a comparison result by the waveform data comparison means.

  According to such a configuration, face alignment information can be output based on the comparison result between the reference waveform data and the detected waveform data, and the face can be aligned based on the alignment information. Therefore, it is easy to understand how to shift the position of the face, so that the face can be aligned better.

  A face photographing apparatus according to a third aspect of the present invention is characterized in that the waveform data is brightness data of a face image along the detection line.

  According to such a configuration, the brightness data along the detection line in the captured face image is detected as detection waveform data, and compared with the reference waveform data composed of the brightness data along the detection line, thereby comparing the face data. Alignment can be performed. Since the feature of the face image can be determined well based on the brightness data, the face can be positioned well by comparing the brightness data.

  A face photographing apparatus according to a fourth aspect of the present invention comprises lightness averaging processing means for averaging lightness data of face images taken on a plurality of lines parallel to the detection line, and the waveform data detecting means comprises the lightness average Brightness data averaged by the image processing means is detected as the detected waveform data.

  According to such a configuration, it is possible to remove noise in the brightness data by averaging the brightness data of the captured face images in a plurality of lines parallel to the detection line. By comparing the detected waveform data composed of the brightness data after the noise is removed in this way with the reference waveform data, it is possible to perform better face alignment.

  A face photographing apparatus according to a fifth aspect of the present invention comprises lightness normalization processing means for normalizing lightness data on the detection line by converting the lightness data on the basis of the maximum value, and detecting the waveform data The means detects the brightness data normalized by the brightness normalization processing means as the detected waveform data.

  According to such a configuration, even if there is an overall brightness difference between the photographed face image and the reference face image, the brightness data on the detection line normalized based on the maximum value of the brightness data is used. The detected waveform data is compared with the reference waveform data, thereby making it possible to perform better face alignment.

  The face photographing apparatus according to a sixth aspect of the present invention is characterized in that the waveform data is composed only of R data along the detection line among the RGB color data constituting the face image.

  According to such a configuration, face alignment is performed by comparing detection waveform data consisting only of R data along the detection line with reference waveform data among the RGB color data constituting the face image. be able to. Since the amount of change in R data among the RGB color data is relatively large in the face contour portion and the like, the face alignment can be performed satisfactorily by comparing the R data.

  According to the present invention, the waveform data along two or more detection lines in the captured face image is detected as detection waveform data, and compared with the reference waveform data along those detection lines, thereby improving the face. Can be aligned.

<First Embodiment>
FIG. 1 is a diagram showing a configuration of a face photographing apparatus H according to the first embodiment of the present invention, and shows a configuration inside the housing 1 in a cross-sectional view. FIG. 2 is a longitudinal sectional view of the housing 1. In the following, the upper side in FIG. 1 will be described as the front, the lower side as the rear, the left side as the left side, and the right side as the right side.

  The face photographing apparatus H is used in cosmetic dermatology, cosmetic surgery, plastic surgery, and the like, and the face photographing apparatus H photographs a person's (photographed person's) face periodically or at an appropriate period of time. By observing changes in the state over time, it is possible to perform an operation to confirm the effects of cosmetics and skin treatment. This face photographing apparatus H has a function of evaluating spots, wrinkles, pores, color unevenness, and the like from the obtained face image, and further has a function of evaluating porphyrins using an ultraviolet ray, and hide-and-seek Jimi. is doing. However, the face photographing apparatus H can be used not only for a photographed person in cosmetic dermatology, cosmetic surgery, plastic surgery and the like, but also as an apparatus for photographing a human face in any other place.

  The face photographing apparatus H includes a substantially cubic shaped housing 1 and a face image is photographed with a face held at a predetermined position in the housing 1. An opening 1a is formed on the back surface of the housing 1, and the face of the subject is inserted through the opening 1a. Further, a curtain 1b is provided on the rear surface of the housing 1 to prevent the outside light from entering from the opening 1a. That is, a structure is employed in which indoor light or the like is less likely to enter the housing 1 by covering the back side of the head with the curtain 1b with the face inserted by the subject through the opening 1a.

  The face of the subject is held at the predetermined position by the chin rest 2 and the forehead press 3 as face holding means. As shown in FIG. 2, the holding portion 3a of the forehead press 3 that contacts the forehead of the subject is configured to be rotatable around the shaft 3b so that it can rotate following the change in the angle of the face in the left-right direction. It has become. Further, the chin rest 2 that comes into contact with the chin of the person to be photographed is configured to be rotatable around the shaft 2b via the connecting portion 2a, and can be rotated following the change of the face angle in the left-right direction. ing. Therefore, the angle of the face can be rotated in the left-right direction while the face is held by the chin mount 2 and the forehead retainer 3, and even in that case, the chin mount 2 and the forehead retainer 3 that rotate following the face can be rotated. The face can be held stably.

  A photographing camera 4 that is a photographing means for photographing a face is provided in front of the housing 1 and is fixed to the housing 1 via a camera support 5. The photographing camera 4 is fixed so that its optical axis extends rearward, and can photograph the face of the subject held rearward by the chin rest 2 and the forehead retainer 3. More specifically, when the face is facing the front direction P0 parallel to the optical axis of the photographing camera 4, a face image viewed from the front (front face image) is photographed. Further, when the face is facing the left tilt direction P1 tilted to the left with respect to the optical axis of the photographing camera 4, a face image (right face image) viewed from the right front is photographed, and the face is photographed by the photographing camera 4. When facing in the right tilt direction P2 tilted to the right with respect to the optical axis, a face image (left face image) viewed from diagonally left front is photographed.

  The left inclined direction P1 is inclined to the left by an angle θ with respect to the front direction P0, and the right inclined direction P2 is inclined to the right by an angle θ with respect to the front direction P0. That is, the left inclined direction P1 and the right inclined direction P2 are inclined in the opposite direction in the horizontal plane with respect to the front direction P0, and the inclination angle θ with respect to the front direction P0 is the same. This inclination angle θ is set between 30 ° and 75 °, and is generally set between 40 ° and 45 °.

  Here, the shaft 3b is provided slightly behind the position of the pressing portion 3a. Similarly, the shaft 2b is also provided slightly behind the position where the jaw of the jaw mounting base 2 is placed. Further, the shaft 3b and the shaft 2b are provided on the same axis, and the optical axis of the photographing camera 4 is orthogonal to these axes. Therefore, even if the face angle is changed in the left-right direction by rotating the face around the shaft 3b and the shaft 2b while the face is held by the chin rest 2 and the forehead press 3, the photographing is performed. It is possible to prevent the center of the face image to be shifted greatly from the optical axis of the photographing camera 4.

  In the housing 1, light sources 8 for illuminating the face are arranged at a plurality of locations. As shown in FIG. 1, each light source 8 is arranged in the circumferential direction centering on the position of the face, and is also provided above and below as shown in FIG. 2, and is arranged so as to face the face. Has been. The light source 8 includes a light source that irradiates visible light and a black light that irradiates ultraviolet rays, and one or both are used depending on the purpose. As the light source 8 for irradiating visible light, an appropriate one such as a halogen lamp, a fluorescent lamp, an LED (Light-Emitting Diode), or a flash light source can be used.

  Between the light source 8 and the face position, a diffusion plate 9 for uniformly diffusing light emitted from the light source 8 is disposed. As shown in FIG. 1, the diffusion plate 9 has a circular cross-sectional shape (semicircular shape). As shown in FIG. 2, the diffusing plate 9 is arranged not only in the horizontal direction with respect to the face but also in the upper and lower portions, so that the face is surrounded by the diffusing plate 9. A rectangular opening 9a is formed on the diffusing plate 9 on the optical axis of the photographing camera 4, and a face can be photographed by the photographing camera 4 through the opening 9a. The diffusion plate 9 is not limited to a semicircular shape, and can be formed in various shapes such as a polygonal shape and an elliptical shape.

  The photographing camera 4 and the light source 8 are connected to the control device 20 and operate under the control of the control device 20. The control device 20 has a function of performing image processing on a face image photographed by the photographing camera 4 in addition to a function of controlling the operation of the face photographing device H. A general-purpose computer having a monitor 21 and a keyboard 22 is connected to the control device 20. The operator of the face photographing apparatus H can perform an input operation such as a shutter operation by operating the keyboard 22, and monitors a face image taken by the photographing camera 4, a face image subjected to image processing, and the like on the monitor 21. Can be confirmed.

  FIG. 3 is a block diagram showing an example of the electrical configuration of the face photographing apparatus H. As shown in FIG. 3, in addition to the above-described photographing camera 4, light source 8, monitor 21, and keyboard 22, the face photographing device H includes a controller 30, an illumination control unit 31, a camera control unit 32, an audio control unit 33, A speaker 34, a data storage unit 35, an image processing unit 36, and the like are provided.

  Here, the controller 30, the illumination control unit 31, the camera control unit 32, the sound control unit 33, and the image processing unit 36 are provided in the control device 20. The data storage unit 35 may be provided in the control device 20 or may be configured by a memory of a computer connected to the control device 20. The speaker 34 is an audio output means for outputting audio, and may be provided in the control device 20 or a computer connected to the control device 20 or may be provided in the housing 1. The voice control unit 33 is voice control means for controlling the voice output from the speaker 34.

  The controller 30 controls the face photographing apparatus H in an integrated manner, and includes a CPU and a memory. The illumination control unit 31 controls turning on and off of the light source 8 based on a control signal from the controller 30 and turns on the light source 8 when photographing a face image. The camera control unit 32 controls the operation of the photographing camera 4 based on a control signal from the controller 30, and a moving image composed of a plurality of still image data by continuously photographing face images with the photographing camera 4. Image data can be generated, and still image data having a higher resolution than the moving image data can be generated based on a shutter operation by the keyboard 22. The moving image data and still image data generated by the photographing camera 4 are stored in the data storage unit 35. The image processing unit 36 is an image processing unit that performs image processing on moving image data and still image data stored in the data storage unit 35, and the image-processed data is displayed on the monitor 21 as necessary.

  In the present embodiment, the front face image is taken by photographing the face of the person to be photographed from the front direction P0, then the right face image is taken with the face facing the left tilt direction P1, and the face is further faced in the right tilt direction P2. By shooting the left face image with the face facing, three still image data composed of front face image data, right face image data, and left face image data are generated. That is, by capturing a face image from the front direction P0, front face image data is generated, and by capturing the face image from directions inclined at the same angle θ in both the left and right directions with respect to the front direction P0, Face image data and left face image data can be generated.

  When the front face image, the right face image, and the left face image are photographed, moving image data is generated by the photographing camera 4, and the moving image data is displayed on the monitor 21 as a moving image data display means in real time. Yes. Here, the real time is a concept including a slight time error due to data transfer or the like, and means that captured moving image data is displayed almost simultaneously with the time of shooting. In the present embodiment, when a front face image, a right face image, and a left face image are shot, the face image can be shot under the same conditions as much as possible by aligning the face with a reference face image that has been shot in advance. It is like that.

  FIG. 4 is a diagram for explaining an aspect when performing face alignment. FIG. 4 shows an aspect in which the face alignment is performed when the front face image is captured, but the face alignment is performed in a similar manner when the right face image and the left face image are captured. It can be carried out.

  At the time of photographing the front face image, the right face image, and the left face image, still image data is extracted from the moving image data photographed by the photographing camera 4 at a predetermined timing, and predetermined in the extracted still image data. Waveform data (cross-sectional waveform data) along two or more detection lines is detected as detected waveform data. Then, by comparing the detected waveform data with reference waveform data composed of waveform data along the detection lines in the reference face image, the face can be aligned. After performing the face alignment in this way, the front face image data, the right face image data, and the left face image data photographed under the same conditions as the reference face image by performing a shutter operation using the keyboard 22. Can be obtained.

  In this example, as the two or more detection lines, a center detection line L1 extending in the left-right direction near the center of the face, an upper detection line L2 extending in parallel upward with respect to the center detection line L1, and a center detection line L1 A lower detection line L3 extending in parallel downward is set. Here, the detection lines L1 to L3 are set so as to cross the feature portion of the face such that the center detection line L1 is near the nose, the upper detection line L2 is near the eyes, and the lower detection line L3 is near the mouth. It may be set so as not to cross the facial feature.

  Note that still image data extracted from moving image data has a lower resolution than front face image data, right face image data, and left face image data. In the following description, it is assumed that not only processing for generating front face image data, right face image data, and left face image data, but also processing for extracting still image data from moving image data is included in “shooting”.

  In this embodiment, lightness data processing such as lightness averaging processing, moving average processing, and lightness normalization processing is performed on the lightness data in each of the detection lines L1 to L3, and the lightness data after these processing is detected waveforms. Detected as data.

  In the lightness averaging process, lightness data of a plurality of lines parallel to the detection lines L1 to L3 is averaged within the predetermined upper and lower widths W around the detection lines L1 to L3. It is processing.

  In the moving average process, for each of the detection lines L1 to L3, the point of interest is sequentially moved in one direction leftward or rightward on the detection line L1 to L3, and within the predetermined width on the left and right of each point of interest. In this process, the average value of the brightness data is calculated and the average value is used as the brightness data of the target point.

  In the brightness normalization process, for each detection line L1 to L3, the brightness data in the detection lines L1 to L3 is converted into brightness data with the maximum value as a reference, thereby setting the maximum value to “1”. It is a process to normalize.

  FIG. 5 is a waveform diagram showing an example of brightness data after the brightness data processing. As shown in FIG. 5, the brightness data after the brightness data processing is converted into a waveform whose maximum value is “1” by the brightness normalization processing while noise is removed by the brightness averaging processing and moving average processing. The

  As can be seen from FIG. 5, the brightness data greatly changes along the width direction of the face parallel to the detection lines L1 to L3, and the features of the face image are reflected particularly in the region where the change is large. For example, in FIG. 5, the edge portions that are one of the features of the face image are reflected in the left and right end regions where the change amount of the brightness data is particularly large. As described above, since the feature of the face image can be determined well based on the brightness data, the face alignment can be performed well by comparing the brightness data.

  In the present embodiment, the noise of the brightness data can be removed by the brightness averaging process and the moving average process. Therefore, by comparing the detected waveform data composed of the brightness data after the noise is removed with the reference waveform data. , It is possible to better align the face. Further, even when there is an overall brightness difference between the photographed face image and the reference face image, the brightness normalization process performs detection on the detection lines L1 to L3 normalized based on the maximum value of the brightness data. By comparing the detected waveform data composed of the brightness data with the reference waveform data, it is possible to perform better face alignment.

  FIG. 6 is a waveform diagram showing an aspect when comparing the reference waveform data and the detected waveform data. In FIG. 6, the reference waveform data is indicated by a broken line and the detected waveform data is indicated by a solid line.

  In this example, only a part of waveform data in the width direction of the face along the detection lines L1 to L3 is compared. More specifically, the reference waveform data and the detected waveform data are compared in the ranges R1, R2 corresponding to the left and right contour portions of the face and the range R3 corresponding to the center portion of the face. . A part of each of the ranges R1 to R3 may or may not overlap.

  However, it is not limited to the configuration in which the reference waveform data and the detected waveform data are compared in the three ranges R1 to R3 as described above, but only the two ranges R1 and R2 corresponding to the left and right contour portions, or the central portion. A configuration in which only one corresponding range R3 is compared may be used, or a configuration in which the reference waveform data and the detected waveform data are compared in four or more ranges may be used. Further, the configuration may be such that the reference waveform data and the detected waveform data are compared over the entire range, rather than a part of the waveform data in the face width direction.

  When comparing the reference waveform data and the detected waveform data, the degree of coincidence can be calculated by a known method using a cross-correlation coefficient. In this case, it can be determined that the degree of coincidence increases as the calculated value approaches “0” and the degree of coincidence decreases as the calculated value moves away from “0”. Further, it can be determined whether the detected waveform data is shifted to the right or left with respect to the reference waveform data depending on whether the calculated value is positive or negative. Therefore, when the calculated value is within a predetermined range centered on “0”, it can be determined that the reference waveform data matches the detected waveform data.

  FIG. 7 is a functional block diagram illustrating a configuration example of the image processing unit 36. As shown in FIG. 7, the image processing unit 36 includes a face image extraction unit 361, a lightness averaging processing unit 362, a moving average processing unit 363, a lightness normalization processing unit 364, a waveform data comparison unit 366, and a registration information output unit. These functional blocks are realized by a computer program executed by a CPU provided in the controller 30. The lightness averaging processing unit 362, the moving average processing unit 363, and the lightness normalization processing unit 364 constitute a waveform data detection unit 368. The waveform data detection unit 368 is waveform data detection means for detecting waveform data along the detection lines L1 to L3 in the photographed face image as detection waveform data.

  As shown in FIG. 7, the data storage unit 35 is assigned with a detection line data storage unit 35A that stores data relating to the detection lines L1 to L3 and a reference face image data storage unit 35B that stores data relating to the reference face image. It has been. The detection line data storage unit 35A stores position information of the center detection line L1, the upper detection line L2, and the lower detection line L2. The position information of the detection lines L1 to L3 may be a fixed value or may be changed to an arbitrary value.

  On the other hand, the reference face image data storage unit 35B stores still image data of a plurality of reference face images taken in advance. The reference face image data storage unit 35B constitutes reference waveform data storage means that stores waveform data along the detection lines L1 to L3 in the reference face image as reference waveform data in association with each reference face image. .

  The face image extraction unit 361 extracts still image data from moving image data generated by the photographing camera 4. The waveform data detection unit 368 performs lightness data on the detection lines L1 to L3 of the extracted still image data based on the position information of the detection lines L1 to L3 stored in the detection line data storage unit 35A. The lightness averaging processing unit 362, the moving average processing unit 363, and the lightness normalization processing unit 364 perform processing, and the waveform data subjected to these processings is detected as detected waveform data. The reference waveform data described above can also be detected as waveform data that has been processed in the same manner as the detected waveform data.

  The lightness averaging processing unit 362 performs lightness averaging processing on the lightness data of the detection lines L1 to L3, thereby averaging lightness data in a plurality of lines parallel to the detection lines L1 to L3. It is. The moving average processing unit 363 is a moving average processing unit that performs a moving average process on the brightness data of the detection lines L1 to L3. The lightness normalization processing unit 364 converts the lightness data in each detection line L1 to L3 into lightness data based on the maximum value by performing lightness normalization processing on the lightness data of each detection line L1 to L3. It is a lightness normalization processing means for normalizing by this.

  The waveform data comparison unit 366 is a waveform data comparison unit that compares the detected waveform data detected by the waveform data detection unit 368 with the reference waveform data stored in the reference face image data storage unit 35B. More specifically, as described above, the reference waveform data and the detected waveform data are compared using a cross-correlation coefficient or the like for a partial range R1 to R3 in the face width direction along the detection lines L1 to L3. Thus, the degree of coincidence of these waveform data is calculated.

  The alignment information output unit 367 is alignment information output means for outputting face alignment information based on the comparison result by the waveform data comparison unit 366. The alignment information is information for approximating each detected waveform data to the corresponding reference waveform data, and is output as information according to the shift direction and shift amount of each detected waveform data with respect to the reference waveform data. The alignment information output from the alignment information output unit 367 is output to the operator of the face photographing apparatus H or the person to be imaged by being output as sound from the speaker 34.

  FIG. 8 is a diagram for explaining a method for determining face alignment information, and shows a case where the face position is shifted in the horizontal direction with respect to the face image based on the reference face image data. FIG. 9 is a diagram for explaining a method for determining face alignment information, and shows a case where the face angle is deviated from the face image based on the reference face image data.

  8 and 9, the face image based on the reference face image data is indicated by a broken line, and the face image based on still image data extracted from the moving image data generated by the photographing camera 4 is indicated by a solid line. . For each of the detection lines L1 to L3, the direction of deviation of the detected waveform data from the reference waveform data in the ranges R1 and R2 corresponding to the left and right contour portions of the face is indicated by the direction of the arrow, and the amount of deviation is indicated by the length of the arrow. It is shown by The face alignment information can also be determined by comparing the reference waveform data and the detected waveform data in the two ranges R1 and R2 as described above, but the reference waveform in another range such as the range R3 described above. The configuration may be such that data and detected waveform data are compared. For example, when a face is photographed from an oblique direction such as a right face image or a left face image instead of a front face image, the reference waveform data and the detected waveform data are compared with each other in the range R3 as well. Good alignment information can be determined.

  FIG. 8A shows a case where the face image based on the extracted still image data is shifted leftward with respect to the face image based on the reference face image data. That is, in the housing 1 shown in FIG. 1, a face image is shown that is taken in a state where the face is horizontally shifted to the right with respect to the position of the face when the reference face image is taken. In this case, the detected waveform data in each of the ranges R1 and R2 in each of the detection lines L1 to L3 is the same in the same shift direction (left direction) with respect to the reference waveform data, as indicated by arrows in FIG. It will be shifted by the amount of shift.

  FIG. 8B shows a case where the face image based on the extracted still image data is shifted to the right with respect to the face image based on the reference face image data. That is, in the housing 1 shown in FIG. 1, a face image is shown that is taken in a state where the face is horizontally shifted to the left with respect to the position of the face when the reference face image is taken. In this case, the detected waveform data in each of the ranges R1 and R2 in each of the detection lines L1 to L3 is the same in the same shift direction (right direction) with respect to the reference waveform data, as indicated by an arrow in FIG. 8B. It will be shifted by the amount of shift.

  FIG. 8C shows a case where the face image based on the extracted still image data is reduced with respect to the center portion with respect to the face image based on the reference face image data. That is, in the housing 1 shown in FIG. 1, a face image is shown that is taken with the face horizontally shifted backward with respect to the position of the face when the reference face image is taken. In this case, as indicated by arrows in FIG. 8C, among the detected waveform data in the ranges R1 and R2 in the detection lines L1 to L3, the detected waveform data on the right side and the detected waveform data on the left side are They are shifted in the opposite direction and in the direction approaching each other with the same shift amount.

  FIG. 8D shows a case where the face image based on the extracted still image data is enlarged with respect to the center portion with respect to the face image based on the reference face image data. That is, in the housing 1 shown in FIG. 1, a face image is shown that is taken in a state where the face is shifted horizontally in the forward direction with respect to the position of the face when the reference face image is taken. In this case, as indicated by arrows in FIG. 8D, among the detected waveform data in the respective ranges R1 and R2 in the respective detection lines L1 to L3, the detected waveform data on the right side and the detected waveform data on the left side are They are shifted in the opposite direction and away from each other by the same shift amount.

  As shown in FIGS. 8A to 8D, when the face position is shifted in the horizontal direction with respect to the face image based on the reference face image data, each range R1, L1 in each detection line L1 to L3. The detected waveform data of R2 is shifted by the same shift amount with respect to the reference waveform data, and only the shift direction is different. Therefore, based on the fact that the detected waveform data in each of the ranges R1 and R2 in the detection lines L1 to L3 have the same amount of deviation, it can be determined that the face is displaced in the horizontal direction, and the amount of deviation Based on the combination of the shift directions of the detected waveform data in the ranges R1 and R2 in the detection lines L1 to L3, the direction and amount of movement of the face (horizontal movement) can be determined.

  Next, FIG. 9A shows a case where the face image based on the extracted still image data is rotated rightward with respect to the face image based on the reference face image data, and the position of the chin is the center. A state where it is rotated and shifted is shown. That is, in the case 1 shown in FIG. 1, the face was photographed with the face rotated to the left about the position of the chin contacting the chin rest 2 with respect to the position of the face when the reference face image was photographed. A face image is shown. In this case, the detected waveform data in each of the ranges R1 and R2 in each of the detection lines L1 to L3 is shifted in the same shift direction (rightward) with respect to the reference waveform data, as indicated by arrows in FIG. 9A. At the same time, the amount of deviation differs depending on the ranges R1 and R2 in the detection lines L1 to L3. More specifically, in each of the right ranges R1, R2 and the left ranges R1, R2, the shift amount is larger in the detected waveform data in the upper range.

  FIG. 9B shows a case where the face image based on the extracted still image data is rotated to the right with respect to the face image based on the reference face image data, and is rotated around the forehead position. The state of being shifted is shown. That is, in the case 1 shown in FIG. 1, a face photographed with the face rotated to the left about the position of the forehead in contact with the forehead retainer 3 with respect to the position of the face at the time of photographing the reference face image An image is shown. In this case, the detected waveform data in each of the ranges R1 and R2 in each of the detection lines L1 to L3 is shifted in the same shift direction (leftward) with respect to the reference waveform data, as indicated by arrows in FIG. At the same time, the amount of deviation differs depending on the ranges R1 and R2 in the detection lines L1 to L3. More specifically, in each of the right ranges R1, R2 and the left ranges R1, R2, the shift amount is larger in the detection waveform data in the lower range.

  FIG. 9C shows a case where the face image based on the extracted still image data is rotated leftward with respect to the face image based on the reference face image data, and is rotated around the position of the chin. The state of being shifted is shown. That is, in the case 1 shown in FIG. 1, the face was photographed in a state where the face was rotated to the right with respect to the position of the face contacting the chin rest 2 with respect to the face position at the time of photographing the reference face image. A face image is shown. In this case, the detected waveform data in each of the ranges R1 and R2 in each of the detection lines L1 to L3 is shifted in the same shift direction (leftward) with respect to the reference waveform data, as indicated by arrows in FIG. 9C. At the same time, the amount of deviation differs depending on the ranges R1 and R2 in the detection lines L1 to L3. More specifically, in each of the right ranges R1, R2 and the left ranges R1, R2, the shift amount is larger in the detected waveform data in the upper range.

  FIG. 9D shows a case where the face image based on the extracted still image data is rotated leftward with respect to the face image based on the reference face image data, and is rotated around the position of the forehead. The state of being shifted is shown. That is, in the housing 1 shown in FIG. 1, a face photographed with the face rotated to the right around the position of the forehead that contacts the forehead retainer 3 with respect to the position of the face at the time of photographing the reference face image An image is shown. In this case, the detected waveform data in each of the ranges R1 and R2 in each of the detection lines L1 to L3 is shifted in the same shift direction (rightward) with respect to the reference waveform data as indicated by arrows in FIG. 9D. At the same time, the amount of deviation differs depending on the ranges R1 and R2 in the detection lines L1 to L3. More specifically, in each of the right ranges R1, R2 and the left ranges R1, R2, the shift amount is larger in the detection waveform data in the lower range.

  As shown in FIGS. 9A to 9D, when the face angle is shifted with respect to the face image based on the reference face image data, the detection of the ranges R1 and R2 in the detection lines L1 to L3 is performed. The waveform data is shifted in the same shift direction with respect to the reference waveform data, and the shift amounts differ depending on the ranges R1 and R2 in the detection lines L1 to L3. Therefore, it is possible to determine that the face is rotated and shifted based on the difference between the detection waveform data in each of the ranges R1 and R2 in the detection lines L1 to L3. The direction and amount of movement of the face can be determined based on the combination of the shift directions of the detected waveform data in the ranges R1 and R2 in the detection lines L1 to L3.

  In the manner described with reference to FIGS. 8 and 9, the direction and amount of movement of the face are determined in order to approximate each detected waveform data to the corresponding reference waveform data, and the information is the position of the face. Output as alignment information. Therefore, by performing face alignment based on the output alignment information and capturing a face image, it is possible to capture the face image under the same conditions as possible when capturing the reference face image.

  In FIG. 9, the case where the face rotates leftward or rightward around the position of the chin or forehead has been described. However, when the face rotates forward or backward around the position of the chin, Similarly, even when the face rotates forward or backward around the position of the forehead, similarly, the amount of deviation of each detected waveform data differs depending on each range R1, R2 in each detection line L1 to L3. Based on that, it can be determined that the face is rotated and shifted, and based on the combination of the shift amount and the shift direction of the detected waveform data of each of the ranges R1 and R2 in each of the detection lines L1 to L3, It is possible to determine the direction and amount of movement to move (rotate) the face.

  FIG. 10 is a flowchart showing a flow of processing when the face photographing apparatus H performs face alignment and photographs a face image. When photographing a face image with the face photographing apparatus H, first, when the operator operates the keyboard 22, a plurality of reference face image data that has been previously captured and stored in the reference face image data storage unit 35B. The desired reference face image data is selected from (step S101). For example, if you want to shoot under the same conditions as possible for a face image taken before to evaluate the skin condition, select still image data of the face image taken before as the reference face image data. It becomes.

  Thereafter, still image data is extracted from the moving image data generated by the photographing camera 4 (step S102), and the lightness averaging process (step S103) is performed on the lightness data in the detection lines L1 to L3 of the still image data. ), Moving average processing (step S104) and brightness normalization processing (step S105) are performed. The waveform data composed of the brightness data after these processes are performed is detected as detected waveform data along the detection lines L1 to L3 (step S106), and the detected waveform data is compared with the reference waveform data. (Step S107).

  As a result of the comparison, when all the detected waveform data matches with the corresponding reference waveform data (Yes in step S107), the operator performs a shutter operation using the keyboard 22 and the face is taken by the photographing camera 4. An image is taken and still image data is generated (step S108). At this time, it is preferable that a sound for stopping the position of the face is output from the speaker 34 such as “Please do not move as it is. Note that the present invention is not limited to a configuration in which shooting is performed based on a shutter operation by an operator, and shooting may be performed automatically.

  On the other hand, if there is detected waveform data that does not match the corresponding reference waveform data among the detected waveform data as a result of the comparison (No in step S107), the mode described with reference to FIGS. Thus, alignment information corresponding to the shift direction and shift amount of each detected waveform data with respect to the reference waveform data is generated, and sound based on the alignment information is output from the speaker 34 (step S109).

  At this time, it is preferable that the sound output from the speaker 34 has a configuration that varies depending on the shift direction and shift amount of the detected waveform data with respect to the reference waveform data. For example, when it is determined that the position of the face is greatly shifted to the left side, a voice “Please move to the right.” Is output. For example, a voice that includes information indicating the amount of movement may be output as the alignment information.

  When the alignment information is output, after a predetermined time has elapsed (Yes in step S110), still image data is extracted again from the moving image data generated by the photographing camera 4 (step S102). The process after step S103 is performed on the brightness data in the detection lines L1 to L3 of the still image data. Therefore, the subject only needs to perform face alignment based on the output position information after the output of the position alignment information and before the next still image data is extracted.

  In the present embodiment, by detecting waveform data along two or more detection lines L1 to L3 in a photographed (extracted) face image as detection waveform data and comparing it with reference waveform data along those detection lines. , Face alignment can be performed.

  Here, in the configuration in which only the detected waveform data on one detection line is compared with the reference waveform data, when the face position is shifted in the left-right direction and when the face angle is shifted. Thus, the deviation amount of the detected waveform data with respect to the reference waveform data may be the same. On the other hand, according to the configuration in which the detected waveform data in the two or more detection lines L1 to L3 is compared with the reference waveform data as in the present invention, when the face position is shifted in the left-right direction, Can be distinguished from when the angle is shifted, so that the face can be aligned more satisfactorily.

  In the present embodiment, face alignment information can be output based on the comparison result between the reference waveform data and the detected waveform data, and the face can be aligned based on the alignment information. Therefore, it is easy to understand how to shift the position of the face, so that the face can be aligned better.

Second Embodiment
In the first embodiment, a configuration has been described in which lightness data along the detection lines L1 to L3 in still image data extracted from moving image data is detected as detection waveform data. On the other hand, in the second embodiment, among the color data of R (red) data, G (green) data, and B (blue) data constituting still image data extracted from moving image data, Only the R data along the detection lines L1 to L3 is detected as detected waveform data.

  FIG. 11 is a functional block diagram showing a configuration example of the image processing unit 36 in the face photographing apparatus H according to the second embodiment of the present invention. In the present embodiment, the waveform data detection unit 368 has the same configuration as that of the first embodiment except that the configuration of the waveform data detection unit 368 is different from that of the first embodiment. Therefore, the description will be omitted.

  In the present embodiment, the waveform data detection unit 368 includes an R data average processing unit 362A, an R data moving average processing unit 363A, an R data normalization processing unit 364A, and the like, and the still image extracted by the face image extraction unit 361. R data along the detection lines L1 to L3 in the image data is detected as detected waveform data.

  The R data averaging processing unit 362A is an R data averaging processing unit that averages R data in a plurality of lines parallel to the detection lines L1 to L3 for the R data of the detection lines L1 to L3. The R data moving average processing unit 363A is an R data moving average processing unit that performs a moving average process on the R data of the detection lines L1 to L3. The R data normalization processing unit 364A is an R data normalization processing unit that normalizes the R data in each of the detection lines L1 to L3 by converting the R data into R data based on the maximum value.

  The R data averaging processing unit 362A, R data moving average processing unit 363A, and R data normalization processing unit 364A perform the lightness averaging processing unit 362, moving average processing unit 363, and lightness normalization in the first embodiment. This can be performed in the same manner as the processing by the processing unit 364. The R data after such processing is detected as detected waveform data and compared with corresponding reference waveform data.

  In the present embodiment, face alignment is performed by comparing detection waveform data consisting only of R data along the detection lines L1 to L3 with reference waveform data among the RGB color data constituting the face image. be able to. Since the amount of change in R data among the RGB color data is relatively large in the face contour portion and the like, the face alignment can be performed satisfactorily by comparing the R data.

  In the above embodiment, a configuration for extracting still image data from moving image data has been described. However, the present invention is not limited to such a configuration, and a higher resolution than moving image data is obtained by capturing a face image based on a shutter operation. The still image data may be generated, and the waveform data along the detection lines L1 to L3 in the still image data may be detected as the detected waveform data.

  The detection lines L1 to L3 are not limited to the configuration extending in the left-right direction, but may be configured to extend in a direction inclined by a predetermined angle with respect to the left-right direction. Further, the number of detection lines is not limited to three, that is, the center detection line L1, the upper detection line L2, and the lower detection line L3, and may be two or four or more.

  The face alignment information is not limited to being output from the speaker 34 by sound, but may be configured to be output by other means such as a display on the monitor 21.

  In the above embodiment, the face photographing apparatus H capable of photographing the front face image, the right face image, and the left face image has been described. However, the present invention is not limited to such a face photographing apparatus H, The present invention can be applied to a face photographing apparatus that can photograph at least one face image.

  In addition, the configuration is not limited to the configuration in which the operator operates the keyboard 22 to perform the shutter operation of the photographing camera 4, and the configuration in which the photographed person performs the shutter operation may be employed. In this case, based on the alignment information output from the speaker 34, it is possible to shoot the face image by performing the shutter operation by the subject himself / herself after aligning the face.

  In the above embodiment, the configuration in which the control device 20 is provided separately from the housing 1 and the general-purpose computer having the monitor 21 and the keyboard 22 is connected to the control device 20 has been described. The configuration is not limited, and each component may be integrally formed.

It is the figure which showed the structure of the face imaging device which concerns on 1st Embodiment of this invention, and has shown the structure in the housing | casing with the cross-sectional view. It is a longitudinal cross-sectional view of a housing. It is the block diagram which showed an example of the electrical structure of this face imaging device. It is a figure for demonstrating the aspect at the time of aligning a face. It is the wave form diagram which showed an example of the brightness data after a brightness data process. It is the wave form diagram which showed the aspect at the time of comparing reference waveform data and detection waveform data. It is the functional block diagram which showed one structural example of the image process part. It is a figure for demonstrating the determination method of the positioning information of a face, and has shown the case where the position of the face has shifted | deviated to the horizontal direction with respect to the face image based on reference | standard face image data. It is a figure for demonstrating the determination method of the positioning information of a face, and has shown the case where the angle of the face has shifted | deviated with respect to the face image based on reference | standard face image data. It is the flowchart which showed the flow of the process at the time of aligning a face and image | photographing a face image with this face imaging device. It is the functional block diagram which showed the example of 1 structure of the image process part in the face imaging device which concerns on 2nd Embodiment of this invention.

Explanation of symbols

H face photographing device L1 center detection line L2 upper detection line L3 lower detection line 4 photographing camera 20 control device 30 controller 31 illumination control unit 32 camera control unit 33 voice control unit 34 speaker 35 data storage unit 36 image processing unit 361 face image extraction Unit 362 lightness averaging processing unit 363 moving average processing unit 364 lightness normalization processing unit 366 waveform data comparison unit 367 alignment information output unit 368 waveform data detection unit

Claims (6)

A face photographing device for photographing a face image by aligning a face with a reference face image,
Reference waveform data storage means for storing waveform data along two or more predetermined detection lines in the reference face image as reference waveform data;
Waveform data detecting means for detecting waveform data along the two or more detection lines in the photographed face image as detected waveform data;
A face photographing apparatus comprising: waveform data comparing means for comparing the reference waveform data and the detected waveform data.   2. The face photographing apparatus according to claim 1, further comprising alignment information output means for outputting face alignment information based on a comparison result by the waveform data comparison means.   The face photographing apparatus according to claim 1, wherein the waveform data includes brightness data of a face image along the detection line. Brightness average processing means for averaging brightness data of face images taken on a plurality of lines parallel to the detection line;
4. The face photographing apparatus according to claim 3, wherein the waveform data detecting means detects the brightness data averaged by the brightness averaging processing means as the detected waveform data. Lightness normalization processing means for normalizing by converting the lightness data on the detection line into lightness data based on the maximum value,
5. The face photographing apparatus according to claim 3, wherein the waveform data detection means detects the brightness data normalized by the brightness normalization processing means as the detected waveform data.   3. The face photographing apparatus according to claim 1, wherein the waveform data is composed only of R data along the detection line among RGB color data constituting a face image.

Images