JP4345440B2 - Outline determination apparatus and outline determination method - Google Patents

Description

  The present invention relates to a technique for determining a physical state of a subject based on a captured image.

  An example of determining the physical state of a subject is to determine whether a woman or a man has succeeded in a diet. For women and men, the purpose of dieting is not only to lose weight itself, but also to make the appearance appear thin due to weight loss. In particular, women who seek diet are often more interested in how their appearance changes as a result of their efforts toward dieting, rather than their weight and body fat values themselves.

In response to such a requirement, there is a technique for determining a body shape using a determination body image obtained by processing a body image according to the degree of brightness (see Patent Document 1). Specifically, first, the body to be determined is imaged to obtain a body image. Next, the body image is subjected to image enhancement processing, in particular, image enhancement processing relating to light and dark, thereby creating a determination body image represented by a plurality of regions distinguished in terms of brightness. Then, based on the determination factors such as the ratio of the two regions in the determination body image, the shape of the contour, the state of the boundary line, etc., the determination of the body shape, property, state, etc. is performed.
JP 2000-37369 A

  By the way, there is a face as a body part that women particularly care about appearance. Certainly, women often care about the appearance of arms, torso and legs. However, unlike the arms, torso, legs, etc., the face is difficult to hide with clothes and the like, and one of the body parts that is particularly problematic is whether or not it appears to be thin for women. For this reason, there are many users who want to obtain an objective judgment as to whether or not their face appears to be thin, or whether or not their face has become thinner than in the past.

  In particular, women may wish to compare themselves with certain others. For this reason, there are many women who want to know what state they are compared to a specific other person as to whether their faces appear to be thin.

  However, in the conventional techniques as described above, the determination is only performed based on the input image, and the user cannot obtain an objective determination as to whether or not his / her face looks thin. It was. For example, in order to help understanding, the user cannot obtain an objective judgment as to whether or not his / her face has become thinner than in the past. Similarly, the user cannot objectively determine whether or not the contour of his / her face appears to be thinner than a specific other person.

  An object of the present invention is to provide an apparatus, a method, a program, and the like that can solve such a problem and allow a user to obtain an objective determination as to whether or not his / her face appears to be thin.

In order to solve the above problem, we focused on the fact that whether or not the face appears to be thin depends largely on the contour of the face. As you can see from the fact that the word “small face” is used,
Many people want to make their faces appear thin by making the facial contours smaller.

  However, in the above-described conventional technique, the contour of the bright and dark area caused by the unevenness of the body is considered, but the contour of the face is not considered. Therefore, in the present invention described below, the problem is solved by considering the contour of the face. The contour determination apparatus according to the first aspect extracts facial contours from a plurality of facial images, and makes an objective determination based on the extracted contours. Specifically, the contour determination apparatus includes an extraction unit and a determination unit.

  The extracting means extracts the outline of the human face from the face image. A “face image” refers to an image in which at least a human face is captured.

  The judging means compares the first contour extracted for the first face image with the second contour extracted for the second face image, and compares the first face image with the second face image. Judge the size of the contour. The determining means may determine which of the first face image and the second face image is a face image having a smaller (or larger) contour. Further, the determination unit may acquire a difference between the contour in the first face image and the contour in the second face image.

  The determination unit may perform determination based on a reference point common to the first face image and the second face image. In this case, the determination unit may determine the distance between a predetermined point on each contour and the reference point and compare the distances. The reference point may be defined by an extraction unit or a determination unit. For example, the midpoint of both eyes detected from the face image is defined as the reference point.

  The first face image and the second face image may be face images in which faces of different persons are captured, or face images in which faces of the same person are captured. When the face image is obtained by capturing a face of a different person, the determination unit determines which person has a face with a smaller contour. For this reason, the user can obtain an objective judgment on the difference in the facial contour between the specific person and himself / herself by inputting the face image of the specific person and the current own face image. .

  On the other hand, in the case of face images obtained by capturing the face of the same person, the determination means determines how the face outline has changed according to the time of image capture. A desirable example of such a face image is a face image of the same person in the past and the present. In this case, the determination means determines how the contour of the face has changed at present compared to the past. For this reason, the user can obtain an objective judgment about the effect of the diet at the present time, for example.

  The extracting means may be configured to extract a predetermined number of points located on the contour of the human face as the contour of the human face. In this case, the determination means is configured to compare which of the point on the first contour and the point on the second contour corresponding to this point is located inside the face.

  The contour determination device may be configured to further include advice means for providing advice to the user based on the determination result by the determination means.

In addition, the contour determination device may be configured to further include an input unit that enables a user to input a desired facial contour. In this case, the determination unit is configured to determine the contour extracted from the input face image and the contour input by the input unit. As a result, the user can obtain an objective determination as to how close the current contour is to the contour desired by the user, for example.

  Further, when the input means as described above is provided, the advice means has a contour desired by the user input by the input means based on such a determination result, for example, how the user applies makeup. You can make it look like your face or give makeup advice. In this case, even before the user obtains the desired facial contour, the makeup method that makes the user look like a face having such a contour, and the diet method for becoming a face having such a contour. And advice on how to maintain health.

  The extraction means may be configured to include a detection means, a normalization means, and a contour extraction means. By being configured in this way, the contour determination device accurately performs determination regarding the contour even when the first face image and the second face image have different face sizes or angles. Is possible.

  Specifically, the detection means detects the first face element and the second face element for each of the first face image and the second face image. The “face element” refers to an organ or the like constituting the face, such as eyes, nostrils, nose, mouth, ears, and eyebrows. The first face element and the second face element do not have to be the same organ. For example, the right eye and the left eye may be used (an example of the same kind of organ), or the right eye and the right nostril may be used (an example of different organs).

  The normalizing means performs image conversion based on the positions of the first face element and the second face element in each of the first face image and the second face image, thereby matching the conditions of both face images.

  The contour extracting means extracts the contour of the human face from the first face image and the second face image after image conversion.

  In the contour determination device configured as described above, the determination regarding the contour is performed on the face image whose condition is unified (for example, normalized) based on the position of the face element. For example, the first face image and the second face image are determined under equal conditions, and the accuracy of the determination is improved. This is particularly effective when a determination is made on images of the same person.

<Second embodiment>
The above problem can also be solved by the second aspect. The contour display device according to the second aspect extracts facial contours from a plurality of facial images, and displays the extracted contours in a distinguishable manner. Specifically, the contour display device includes an extraction unit and a display unit.

  The extraction unit extracts the outline of the human face from the face image obtained by capturing the human face. The display means displays the first contour extracted for the first face image and the second contour extracted for the second face image so that they can be compared. “Comparably displayed” means that when the user displays the first contour and the second contour, any display element indicates the first contour, and any display element indicates the second contour. It is shown that it is possible to discriminate whether it is a display. The user can objectively recognize the difference between the extracted first contour and the second contour by viewing such a display.

  Further, the extraction means of the second aspect may be configured to include a detection means, a normalization means, and a contour extraction means, similarly to the extraction means in the first aspect. With this configuration, the accuracy of the determination made by the user based on the contour displayed by the display unit is improved.

[Others]
The first aspect and the second aspect may be realized by executing the program by the information processing apparatus. That is, the present invention specifies the processing executed by each unit in the first aspect and the second aspect described above as a program for causing the information processing apparatus to execute or a recording medium on which the program is recorded. Can do. Moreover, the process which each means in a 1st aspect and a 2nd aspect performs can be specified as a method which an information processing apparatus performs.

  The user can obtain an objective judgment regarding the difference between the contours of the first face image and the second face image. For example, when the first face image is a face image of a specific other person and the second face image is a face image of itself, the user can objectively determine the difference in facial contour between the specific person and himself / herself. It is possible to obtain a correct judgment.

  Further, for example, when the first face image is a past face image and the second face image is a current face image, the user can objectively determine, for example, the current diet effect. It is possible to obtain a reasonable judgment.

  Next, the contour determination device and the contour display device will be described with reference to the drawings. In addition, the following description about an outline determination apparatus and an outline display apparatus is an illustration, The structure is not limited to the following description.

[First embodiment of contour determination apparatus]
〔〔System configuration〕〕
First, the outline determination apparatus 1a which is 1st embodiment of an outline determination apparatus is demonstrated.

The contour determination device 1a includes a CPU (Central Processing Unit), a main storage device (RAM: Random Access Memory), an auxiliary storage device, and the like connected via a bus in terms of hardware. The auxiliary storage device is configured using a nonvolatile storage device. The nonvolatile storage device referred to here is a so-called ROM (Read-Only Memory: EEPROM (Erasable Programmable Read-Only Memory)) or EEPROM (Electrically Erasable Programmable Read-Only Memory).
), Mask ROM, etc.), FRAM (Ferroelectric RAM), hard disk and the like.

  FIG. 1 is a diagram illustrating functional blocks of the contour determination device 1a. The contour determination device 1a is loaded with various programs (OS, applications, etc.) stored in the auxiliary storage device and executed by the CPU, whereby the image storage unit 2, the extraction unit 3, the determination unit 4a, And functions as a device including the display unit 5a and the like. The extraction unit 3 and the determination unit 4a are realized by executing a program by the CPU. Each of the extraction unit 3 and the determination unit 4a may be configured as a dedicated chip.

<Image storage unit>
The image storage unit 2 is configured using a so-called RAM or ROM. The image storage unit 2 stores the face image input to the contour determination device 1a. It is desirable that the face image input to the contour determination device 1a is a face in front of a person. The image storage unit 2 stores a face image input from an external storage device via an interface. The image storage unit 2 stores a face image captured by an imaging device (not shown).

<Extraction part>
The extraction unit 3 reads the face image stored in the image storage unit 2 and extracts the contour of the face captured in the read face image. The extraction unit 3 reads two predetermined images (hereinafter referred to as “one side”, “the other”, etc.) from the image storage unit 2 and performs processing on each image. The two images read by the extraction unit 3 may be designated in any way. That is, the two images read out by the extraction unit 3 may be predetermined images, images specified by the user, newly input images from other devices, It may be an image designated by the apparatus.

  In addition, the extraction unit 3 may be specifically configured by applying any existing technique. Below, the specific structural example of the extraction part 3 is shown. FIG. 2 is a diagram illustrating functional blocks of the extraction unit 3. The extraction unit 3 includes a detection unit 6, a normalization unit 7, and a contour extraction unit 8. Hereinafter, each functional unit included in the extraction unit 3 will be described.

<<Detection unit>>
The detection unit 6 detects a human face from the input face image, and further detects left and right eyes as face elements from the detected face. Specifically, the detection unit 6 acquires the center coordinates of both eyes of the subject from the input image.

  The detection of a human face may be realized by applying any existing technique. For example, the face may be detected by template matching using a reference template corresponding to the outline of the entire face. Further, the face may be detected by template matching based on face components (eyes, nose, ears, etc.). Further, the vertex of the hair may be detected by chroma key processing, and the face may be detected based on the vertex. Further, the face may be detected based on the detection of the skin color area.

  The detection processing for both eyes may be realized by applying any existing technique. For example, an area having a specific brightness distribution in the horizontal direction in the detected face area may be detected as the left and right eyes. The eyes may be detected by pattern matching using the eye pattern.

  The detection unit 6 determines whether or not the detected face is a front face. FIG. 3 is a diagram for explaining a process for determining whether or not the detected face is a front face. The detection unit 6 acquires an angle between a detected straight line passing through the centers of the left and right eyes (hereinafter referred to as “eye axis”: L1) and the horizontal axis (x axis) of the image. Further, the detection unit 6 detects the mouth in the detected face region, and passes through the center point of the mouth and the middle point (P) of the left and right eyes (hereinafter referred to as “face center line”: L2). And an angle with a straight line passing through the center of the left and right eyes and parallel to the y-axis (hereinafter referred to as “vertical line”: L3). The detection unit 6 determines that the detected face is a front face when the two angles are smaller than the corresponding threshold values. Note that the mouth detection process by the detection unit 6 may also be realized by applying any existing technique as in the eye detection process.

<< Normalization part >>
The normalization unit 7 normalizes the image determined to be a front face by the detection unit 6. For example, the normalization unit 7 normalizes two input face images according to a predetermined standard, thereby matching conditions such as the size, position, and orientation of the face. Alternatively, the normalization unit 7 may be configured to convert the image of the other face image so as to match the condition in the one face image.

  More specifically, the normalization unit 7 is determined based on the distance between the face elements, the angle between the straight line connecting the two face elements and the x-axis (y-axis) of the image, and the position of each face element. Image conversion is performed so that conditions such as the position of a point (such as an intermediate point) match.

When the two face elements are the right eye and the left eye, the distance based on any part such as the distance between the left and right corners of the eyes and the distance between the center points of the left and right eyes can be obtained. good. For example, the normalization unit 7 enlarges / reduces the face image based on the distance. The normalizing unit 7 may normalize both images by image conversion, or may perform image conversion so that one image matches the other image.

  Here, the normalization unit 7 performs normalization according to the distance between the centers of the left and right eyes detected by the detection unit 6 (hereinafter referred to as “eye distance”). Specifically, the normalization unit 7 enlarges or reduces the image so that the eye distance becomes a predetermined size. Further, the normalization unit 7 moves and rotates the image so that the midpoint and eye axis of the left and right eyes in one face image coincide with the midpoint and eye axis of the left and right eyes in the other face image. carry out.

<< Contour Extraction Unit >>
The contour extraction unit 8 extracts the contour of the human face from the face image normalized by the normalization unit 7. The contour extracting unit 8 extracts the left and right cheeks and chin contours of the human face. The contour extraction unit 8 may be realized by applying any contour extraction technique. For example, as a contour extraction technique applicable to the contour extraction unit 8, there is a technique described in the following document.

TFCootes and CJTaylor, “On Representing Edge Structure for Model Matching” Proc. CVPR 2001, vol. 1, pp. 1114--1119.
FIG. 4 is a diagram illustrating an example of a processing result of the contour extraction unit 8. The contour extraction unit 8 extracts, for example, the contour of a human face as a set of a plurality of points. Thus, the method is not limited to the method of extracting as a set of a plurality of points, and for example, a technique of extracting the outline of a human face as a line may be applied.

<Judgment part>
Information on the outline of the face in the two face images extracted by the extraction unit 3 is input to the determination unit 4a. At this time, information on the contour of the face in the two input face images is processed by the normalization unit 7 so that the midpoint and the eye axis of the left and right eyes coincide. The determination unit 4a determines which of the contours of the input face image is a smaller contour. In other words, the determination unit 4a determines which contour is a contour located more inside the face. In addition, the determination unit 4a acquires the difference between the sizes of the two contours (in other words, the difference in the degree of being located inside the face).

  Specifically, the determination unit 4a acquires points (hereinafter referred to as “contour points”) located on the extracted contour lines for the contours in the two face images. When the technique described in the above paper is applied in the contour extraction unit 8, the determination unit 4a acquires all or some of the extracted points as contour points. The determination unit 4a acquires a degree indicating that the contour point is located inside the face with respect to the contour point in each face image. For example, the distance between each contour point and the reference element (eg, face center point, face center line, left and right eye midpoint, lip center point) included in the detected face is located inside It is acquired as a degree indicating. Or the position vector from this reference element to each contour point and the difference between them (the difference between the position vector of the contour point in one image and the position vector of the contour point corresponding to the contour point in one image in the other image) Is acquired as a degree indicating that it is located inside.

  Then, the determination unit 4a determines which of the contours is located on the inner side of the face using the acquired degree value. At this time, the determination unit 4a can calculate the difference in the acquired degree.

FIG. 5 is a diagram illustrating the contents of processing in the determination unit 4a. A specific processing example in the determination unit 4a will be described with reference to FIG. In FIG. 5, the outline of the face is shown by a set of points. FIG. 5 shows an example in which contours are extracted for images A and B. Image A
The contour extracted for is indicated by a set of points (contour points) An (n = 1, 2,..., 13: circles indicated by diagonal lines). Similarly, the contour extracted for the image B is indicated by a set of points (contour points as in An) Bn (n = 1, 2,..., 13: white circle).

  In the comparison process of the contour points An and Bn described below, a comparison based on the corresponding contour points is performed. An and Bn may be associated based on any standard. For example, when the values of n are equal, the contour point (An) in the image A and the contour point (Bn) in the image B may be handled as corresponding contour points. Further, for example, contour points having the same angle with respect to the eye axis with the middle point of both eyes as a vertex may be treated as corresponding contour points. In addition, for example, contour points having the same angle with respect to an arbitrary axis (for example, the x-axis, a straight line connecting the right and left nostrils, etc.) with the representative point (for example, the middle point of the left and right nostrils, the center point of the lips, etc.) as the vertices corresponded. It may be treated as a contour point.

The determination unit 4a acquires distances (DAn and DBn, respectively) between the contour points An and Bn and the intermediate point (black point) between the left and right eyes (points indicated by crosses). Next, the determination unit 4a obtains the difference between DAn and DBn as Dn. The determination unit 4a counts the number of positive Dn (PP) and the number of negative Dn (NP). And the value which subtracted PP from NP is acquired as a point. When the point is a positive number, the determination unit 4a determines that the face outline of the image A is smaller than the face outline of the image B. On the other hand, when the point is a negative number, the determination unit 4a determines that the face outline of the image A is larger than the face outline of the image B.

<Display section>
The display unit 5a is configured using a display device such as a CRT (Cathode Ray Tube) or a liquid crystal display. The display unit 5a displays the points determined by the determination unit 4a. FIG. 6 is a diagram showing a display example by the display unit 6a. In FIG. 6, a point is displayed on the upper left of the display screen by the display unit 5a. The user can know how small the contour of the face of the images A and B is by looking at the displayed points.

  In addition, the display unit 5a can display the contour points extracted by the extraction unit 3. At this time, the display unit 5a can display the contour points extracted for the image A and / or the image B. When the contour points extracted for the images A and B are displayed at the same time, the contour points in the image A and the contour points in the image B are displayed in different display forms. Specifically, the display elements (contours, figures such as circles) indicating the contour points are displayed in different shapes, colors, patterns, and the like. In FIG. 6, the contour points extracted for the image A and the image B are displayed as a circle filled with diagonal lines and a white circle, respectively. The display unit 5a can display the image A or the image B. In FIG. 6, an image A is displayed. The display content of the display unit 5a can be set by the user.

[[Operation example]]
Next, an operation example of the contour determination device 1a will be described. FIG. 7 is a flowchart illustrating an operation example of the contour determination apparatus 1a. Hereinafter, a case where the size of the outline is determined for the image A and the image B will be described.

  When two images to be compared are specified by the user from the images stored in the image storage unit 2 (S01), the extraction unit 3 reads the two specified images, and about these two images The face contour extraction process is started (S02). The designation input in S01 is input via a user interface (not shown). FIG. 8 is a flowchart illustrating an example of face contour extraction processing by the extraction unit 3.

  First, the detection unit 6 detects a face from the read image (S05). Next, the detection unit 6 detects the left and right eyes from within the detected face area, and acquires the center point of each of the left and right eyes (S06). Next, the detection unit 6 determines whether or not the detected face is a front face. Specifically, the detection unit 6 first determines whether the angle between the eye axis and the horizontal axis of the image is within a threshold value (here, 10 degrees). When it is larger than 10 degrees (S07-No), the detection unit 6 determines that the detected face is not a front face, and outputs an error (S13). In this case, the display unit 5 displays that an error has occurred because the detected face is not a front face.

  On the other hand, when the magnitude of the angle between the eye axis and the horizontal axis is within 10 degrees (S07-Yes), the detection unit 6 acquires the midpoint (P) of the left and right eyes (S08). Next, the detection unit 6 detects the mouth from the detected face, and acquires the center point of the detected mouth (S09). The detection unit 6 determines whether the angle between the face center line and the vertical line is within a threshold (here, 10 degrees). When it is larger than 10 degrees (S10-No), the detection unit 6 determines that the detected face is not a front face and outputs an error. Also in this case, the display unit 5 displays that an error has occurred because the detected face is not a front face.

  On the other hand, when the size of the angle between the face center line and the vertical line is within 10 degrees (S10-Yes), the normalization unit 7 determines the size of the eye distance between the left and right eyes detected by the detection unit 6. Based on this, the image is normalized (S11). Then, the contour extraction unit 8 extracts the contour of the human face from the normalized image (S12).

  Returning to the description using FIG. After the extraction unit 3 executes the face outline extraction process, the determination unit 4a executes a face outline comparison process based on the result of the face outline extraction process (S03). FIG. 9 is a flowchart illustrating an example of face contour comparison processing by the determination unit 4a.

  The determination unit 4a uses An (n = 1, 2,... 13) for each contour point extracted for the image A and Bn (n = 1, 2,... 13) for each contour point extracted for the image B. ) Are defined respectively (S14). Next, the determination unit 4a acquires the distance DAn between the intermediate point P and each contour point An (S15). Similarly, the determination unit 4a acquires the distance DBn between the intermediate point P and each contour point Bn (S16). Next, the determination unit 4a acquires Dn, which is the difference between DAn and DBn, for the corresponding contour point (S17). The determination unit 4a acquires a point based on the acquired sign of each Dn (S18), and determines which of the image A and the image B is the contour located inside the face. Then, the determination unit 4a outputs the determination result (including the point value) to the display unit 5 (S19).

  Returning to the description using FIG. When receiving the determination result from the determination unit 4a, the display unit 5 displays the determination result (S04).

[[Action / Effect]]
By using the contour determination device 1a, the user can obtain an objective evaluation (determination result by the determination unit 4a) regarding the contour of the face of the person captured in the images A and B.

Here, specific examples of the image A and the image B will be described. The image A can be a user's own face image, and the image B can be a face image of a person other than the user (for example, a user's diet rival, a person having a face contour desired by the user). In this case, the user's face outline is smaller than the face outline of a person other than the user, in other words, whether the face appears to be thin, or is a contour located more inside the face. An objective judgment can be obtained about whether or not. Similarly, by knowing the point displayed, the user can see how much smaller, in other words how thin the face contour of the person other than the user is, or how much is located inside the face. It is possible to obtain an objective judgment as to whether the outline is to be performed.

  As a specific example of the image A and the image B, the image A can be a user's current face image, and the image B can be a user's past face image. In this case, the user can obtain an objective determination as to whether or not the current face contour has become smaller than the user's past face contour. In addition, the user can obtain an objective determination as to how small the current facial contour is by knowing the displayed point. Based on this determination, the user can obtain an objective determination regarding the effect of his / her diet.

  In the contour determination device 1a, the normalization unit 7 normalizes the image A and the image B. For this reason, even if faces having different face shapes and sizes (such as different people or faces of the same person but at different times), it is possible to accurately obtain an objective judgment on the contour. It becomes possible.

  In the display unit 5a, the contour point An and the contour point Bn are displayed in a distinguishable manner. For this reason, the user can know not only the point but also the state and change of his / her contour from the image. Therefore, it is possible to know which part of the face is different between An and Bn (for example, which part is thinned and which part is fat relative to others).

  The contour determination device 1a can be applied to an information processing device having an imaging device (for example, a mobile phone with a camera, a digital still / video camera, a PDA with a camera, a health device with a camera). Further, even if the contour determination device 1a is an information processing device that does not have an imaging device, the contour determination device 1a can acquire an image by communicating with other devices wirelessly / wiredly or by reading data from another recording medium. Any device (eg, personal computer, workstation, mobile phone, PDA) is applicable.

[[Modification]]
The determination unit 4a may be configured to obtain the value of Dn for all the contour points and acquire the average value thereof. When the average value is a positive value, the determination unit 4a determines that the face outline in the image A is larger than the face outline in the image B. On the other hand, when the value of Dn is a negative value, the determination unit 4a determines that the face outline in the image A is smaller than the face outline in the image B. At this time, the determination unit 4a acquires a value obtained by multiplying the average value by -1 as a point.

  Further, a user may input a desired contour point by a user interface (not shown). In this case, only one image is extracted from the contour point by the extraction unit 3. The determination unit 4a performs determination on the contour points extracted by the extraction unit 3 and the extraction points input by the user. When configured in this manner, the user can input, for example, a contour point desired by the user (a facial contour desired by the user if desired after a diet). Then, the user can obtain an objective determination as to how close the current contour of his / her face is to the contour point desired by the user.

  The contour points extracted by the contour extracting unit 8 need not be limited to 13 points, and any number of contour points may be extracted. The determination unit 4a performs processing according to the number of contour points extracted by the contour extraction unit 8.

Further, the contour determination device 1a may be configured to include another form of output device instead of the display unit 5a. Examples of such an output device include a voice output device (for example, a speaker and headphones) that conveys the points acquired by the determination unit 4a to the user by voice, and a printing device such as a printer.

[Second Embodiment of Contour Determination Device]
〔〔System configuration〕〕
Next, the outline determination apparatus 1b which is 2nd embodiment of an outline determination apparatus is demonstrated.

  FIG. 10 is a diagram illustrating functional blocks of the contour determination device 1b. The contour determination device 1b differs from the contour determination device 1a in that it further includes an extraction result storage unit 9 and a designation unit 10, and in that it includes a determination unit 4b instead of the determination unit 4a. Hereinafter, the difference between the contour determination device 1b and the contour determination device 1a will be described.

<Extraction result storage unit>
The extraction result storage unit 9 is configured using a so-called RAM or ROM. The extraction result storage unit 9 stores the processing result in the extraction unit 3 in association with the image or the ID of the image. Specifically, the extraction result storage unit 9 stores the left and right eye midpoint (P) and each contour point for each image. At this time, the extraction result storage unit 9 can further store the normalized image.

<Designated part>
The designation unit 10 is configured using an input device such as a microphone, a pointing device (mouse, trackball, touch panel, etc.), a keyboard, and the like. The designation unit 10 is used by the user to designate a face image whose extraction result is already stored in the extraction result storage unit 9.

<Judgment part>
The determination unit 4b is different from the determination unit 4a in that the extraction result related to the face image designated by the designation unit 10 is read from the extraction result storage unit 9 and the face outline comparison process is executed using the read extraction result.

  The determination unit 4b may be configured to use the one related to the face image designated by the designation unit 10 for one extraction result and use the latest extraction result for the other extraction result. Further, the determination unit 4b may be configured to use the one relating to the face image designated by the designation unit 10 for both one and the other extraction results.

  Further, the determination unit 4b may be configured to store an identifier indicating the extraction result of the face image designated by the designation unit 10, or the extraction result itself. Hereinafter, an image in which the identifier indicating the extraction result or the extraction result itself is stored by the determination unit 4b is referred to as a “designated storage image”. In this case, the determination unit 4b receives the new extraction result or a newly designated value when a new extraction result is obtained by the extraction unit 3 or when a new face image is designated by the designation unit 10. Contour comparison processing is executed for the extraction result related to the face image and the extraction result stored in itself.

  The content of the face contour comparison process performed by the determination unit 4b is the same as the process performed by the determination unit 4a.

[[Operation example]]
FIG. 11 is a flowchart illustrating an operation example of the contour determination apparatus 1b. Hereinafter, an operation example of the contour determination device 1b will be described with reference to FIG. In the following description, when the determination unit 4b is configured to use the one related to the face image specified by the specifying unit 10 for one extraction result and use the latest extraction result for the other extraction result. Will be described. Further, in the following description, a case will be described in which the determination unit 4b is configured to store an extraction result ID related to the designated storage image. Furthermore, in the following description, a case will be described in which the determination unit 4b is configured to execute the contour comparison process when a new extraction result is obtained by the extraction unit 3. By referring to the following description, those skilled in the art can easily implement other operation patterns of the determination unit 4b described in the system configuration.

  The extraction unit 3 waits until a new image is stored in the image storage unit 2 (S20-No). When a new image is stored in the image storage unit 2 (S20-Yes), the extraction unit 3 executes a face contour extraction process (S21). And the extraction part 3 writes the extraction result by a face outline comparison process in the extraction result memory | storage part 9 (S22).

  Next, the determination unit 4b determines whether there is a designated storage image. When there is no designated storage image (S23-No), the process returns to S20. On the other hand, when there is a designated storage image (S23-Yes), the determination unit 4b extracts the extraction result of the designation storage image and the recently stored extraction result (that is, the extraction result obtained by the contour comparison process by the recent extraction unit). Read from the result storage unit 9 (S24). The determination unit 4b performs a face contour comparison process using the read extraction result (S25). And the display part 5a displays the determination result by the determination part 4b (S26).

[[Action / Effect]]
In the contour determination device 1b, even when the user is not instructed to perform the face contour extraction process or the face contour comparison process, the face contour comparison process for the image is performed when a new image is stored in the image storage unit 2. The extraction result is stored in the extraction result storage unit 9. For this reason, when the user desires to execute the face outline comparison process, it is not necessary to execute the face outline extraction process for the image already stored in the image storage unit 2. Therefore, it is possible to shorten the processing time. For example, when the configuration is such that the user designates two images by the designation unit 10, the configuration is effective when the user designates one of the images to be compared by the designation unit 10.

  Further, in the contour determination device 1b, when a user designates a designated storage image in advance, a point in time when a new image is stored in the image storage unit 2 (for example, an image captured by an imaging device (not shown) is stored. At the time when the image is input and stored from an external storage device (not shown), the face contour comparison process can be executed immediately (see the description of the operation example). This eliminates the need for the user to designate an image to be compared with a new image each time.

  In the contour determination device 1b, the user can designate the designated storage image in advance, so that the face contour comparison process can be executed immediately when the designation unit 10 designates a new image. Specifically, in the process of S20 of the operation example of the contour determination device 1b, the extraction unit 3 does not detect that a new image is stored, but detects that the image is specified by the specification unit 10. What is necessary is just to be comprised. With this configuration, the user does not need to designate an image to be compared with the designated image each time.

[[Modification]]
The image normalized by the normalization unit 7 of the extraction unit 3 may be configured to be stored in the image storage unit 2. In this case, the extraction result storage unit 9 stores the normalized image identifier (ID, address, etc.) stored in the image storage unit 2 in association with the extraction result.

  Further, when a user designates a face image using the designation unit 10, the face image that can be designated may be read from the image storage unit 2 and displayed on the display unit 5a.

  Further, as described in the modification of the contour determination device 1a, when the user is configured to be able to input a desired contour point, the extraction result storage unit 9 is configured to store the contour point input by the user. May be. At this time, the eye axis and / or the middle point of the left and right eyes may be input by the user, or values of other images already stored may be applied.

[Third embodiment of contour determination apparatus]
〔〔System configuration〕〕
Next, the outline determination apparatus 1c which is 3rd embodiment of an outline determination apparatus is demonstrated.

  FIG. 12 is a diagram illustrating functional blocks of the contour determination device 1c. The contour determination device 1c is different from the contour determination device 1a in that it includes an advice unit 11 and a display unit 5b instead of the display unit 5a. Hereinafter, the difference between the contour determination device 1c and the contour determination device 1a will be described.

<Advising Department>
The advice unit 11 is realized by the CPU executing a program for causing the information processing apparatus to operate as the contour determination apparatus 1c. The advice unit 11 may be configured as a dedicated chip.

  Examples of advice given by the advice unit 11 include diet-related items, makeup-related items, health-related items, and photo-related items. The advice unit 11 provides advice on diet, makeup, health, and how to take a photo based on the extraction result by the extraction unit 3 and / or the determination result by the determination unit 4.

  Specific examples of such advice are given below. For example, when the user obtains a determination result by inputting his / her own face images of the past and the present, it may be found that the diet effect is not obtained as much as the user desires. In such a case, the advice unit 11 can give advice based on the determination result as to how the user puts on the makeup so that the user's face appears to be thinner. Conversely, when the user has become too thin, the advice unit 11 can also advise makeup that makes the user look healthier and plump. In addition, for example, when an image shot with various face orientations is input by the user, it is advised that the best shooting method is to take a picture when the face image with the smallest outline is taken. Is possible. When it is determined that the cheek portion is larger than the ideal contour, it is possible to advise that the portion is covered with hair or the like. Moreover, when it is larger than the ideal outline as a whole, it can be advised that a brown shadow is applied along the face line to make it appear thin.

<Display section>
The display unit 5b is different from the display unit 5a in that it further displays the advice content retrieved by the advice unit 11 in addition to the display content displayed by the display unit 5a.

[[Operation example]]
In the contour determination device 1c, the advice unit 11 executes the advice content search processing between the face contour comparison processing (S03) by the determination unit 4 of the contour determination device 1a and the display of the determination result by the display unit 5a (S04). This is different from the operation example of the contour determination device 1a.

[[Action / Effect]]
By using the contour determination device 1c, the user can receive advice based on the execution result of the face contour extraction process and the face contour comparison process in the images A and B. For example, by inputting / selecting the user's current and past face images as images A and B, he / she can receive advice according to changes in the contour of the face from the past to the present, particularly advice on diet and advice on health. it can.

  Further, when the user can input a desired contour point as described in the modification of the contour determination device 1a, the following effects can be expected. That is, by inputting / selecting the user's current face image and an ideal contour point, advice according to the difference between the current actual face contour and the ideal contour, especially for approaching such an ideal contour Get advice on dieting and makeup that should be done.

[[Modification]]
The contour determination device 1c may be configured to further include an extraction result storage unit 9 in the contour determination device 1b.

[Outline display device]
〔〔System configuration〕〕
FIG. 13 is a diagram illustrating functional blocks of the contour display device 12. The contour display device 12 includes an image storage unit 2, an extraction unit 3, and a display unit 5c. The image storage unit 2 and the extraction unit 3 included in the contour display device 12 have the same configuration as the image storage unit 2 and the extraction unit 3 included in the contour determination device 1a. For this reason, the description about the image storage part 2 and the extraction part 3 is abbreviate | omitted.

<Display section>
The display unit 5c is different from the display unit 5a in that only the contour points extracted by the extraction unit 3 are displayed. That is, since the outline display device 12 does not include the determination unit 4 or the advice unit 11, the display unit 5c does not display the determination result by the determination unit 4, advice for the user, or the like. However, the display unit 5c displays the contour point An extracted in the image A and the contour point Bn extracted in the image B so as to be distinguishable for the user, as described in the description of the display unit 5a.

[[Operation example]]
FIG. 14 is a flowchart illustrating an operation example of the contour display device 12. An operation example of the contour display device 12 will be described with reference to FIG.

  When two images to be compared are designated by the user from the images stored in the image storage unit 2 (S27), the extraction unit 3 reads the two designated images, and the two images are read out. The face contour extraction process is started (S28). The specific processing of the extraction unit 3 in S28 is the same as the processing of the extraction unit 3 in the contour determination device 1a. That is, the contents of the processing in S28 are as shown in FIG. After the face contour extraction process in S28, the display unit 5c displays the extraction result (S29).

[[Action / Effect]]
In the contour display device 12, the result of face contour extraction processing by the extraction unit 3 is displayed on the display unit 5c for the images A and B. At this time, the display unit 5c displays the outline of the person's face in the image A and the outline of the person's face in the image B in different display forms so that the user can compare. For this reason, the user can objectively confirm the outline of the face in the image A and the image B, and can determine which is smaller.

  The contour display device 12 does not include a functional unit corresponding to the determination unit 4. For this reason, the contour display device 12 does not execute the contour comparison process. Therefore, the time until the process is completed is reduced. Furthermore, the time and cost required for the design can be reduced as compared with the contour determination devices 1a, 1b, and 1c.

[[Modification]]
The contour display device 12 may be configured such that a user can input a desired contour point by a user interface (not shown). In this case, only one image is extracted from the contour point by the extraction unit 3. When configured in this manner, the user can input, for example, a contour point desired by the user (an ideal contour of the face that the user desires to be after the diet). Then, the user can compare the contour point desired by the user with the contour point extracted from the actual image by the extraction unit 3 on the display screen. For example, by selecting and inputting the current face image as an image on which contour points are extracted by the extraction unit 3, it is possible to determine how close the current face contour is to the ideal contour, etc. Judgment can be made objectively.

  Similarly to the contour determination device 1a, the contour points extracted by the contour extracting unit 8 need not be limited to 13, and any number of contour points may be extracted.

  The contour determination device 1c may be configured to include another form of output device instead of the display unit 5c. An example of such an output device is a printing device such as a printer.

  Moreover, the outline display apparatus 12 may be comprised so that the extraction result storage part 9 may be provided like the outline determination apparatus 1b. In this case, when the contour to be displayed is stored in the extraction result storage unit 9, the display unit 5c reads out and displays the contour data from the extraction result storage unit 9.

It is a figure which shows the functional block of 1st embodiment of an outline determination apparatus. It is a figure which shows the functional block of an extraction part. It is a figure for demonstrating the judgment process whether the detected face is a front face. It is a figure which shows the example of the process result of an outline extraction part. It is a figure which shows the content of the process in a judgment part. It is a figure which shows the example of a display by a display part. It is a flowchart which shows the operation example of 1st embodiment of a contour judgment apparatus. It is a flowchart which shows the example of the face outline extraction process by an extraction part. It is a flowchart which shows the example of the face outline comparison process by a judgment part. It is a figure which shows the functional block of 2nd embodiment of an outline determination apparatus. It is a flowchart which shows the operation example of 2nd embodiment of a contour judgment apparatus. It is a figure which shows the functional block of 3rd embodiment of an outline determination apparatus. It is a figure which shows the functional block of an outline display apparatus. It is a flowchart which shows the operation example of an outline display apparatus.

Explanation of symbols

1a, 1b, 1c Contour determination device 2 Image storage unit 3 Extraction unit 4 Determination unit 5a, 5b, 5c Display unit 6 Detection unit 7 Normalization unit 8 Contour extraction unit 9 Extraction result storage unit 10 Designation unit 11 Advice unit 12 Contour display apparatus

Claims (9)

Extraction means for extracting a point located on the contour of the person's face as a contour point from a face image obtained by imaging a human face;
A plurality of first contour points extracted for the first face image and a plurality of second contour points extracted for the second face image are compared, and the first face image and the second face image and a determination means for determining the size of the contour,
The determination means includes
Performing an association between the plurality of first contour points and the plurality of second contour points;
By comparing the distance from a predetermined reference element included in the face to each contour point for the corresponding two contour points, the first contour point and the second contour point, which contour point is located inside the face Determine if it is located,
If the number of first contour points determined to be located inside the face is larger than the first contour point, it is determined that the contour of the first face image is smaller than the contour of the second face image;
If the number of the second contour points determined to be located inside the face is larger than that of the second face image, it is determined that the contour of the first face image is larger than the contour of the second face image. Judgment device. The contour determination apparatus according to claim 1, further comprising advice means for giving advice to a user based on a determination result by the determination means. It further comprises an input means that allows a user to input a desired facial contour,
The determining means includes a contour which is extracted for the input facial image, the outline determination apparatus according to any one of claims 1 to 2 for determining the the inputted contour by said input means. The extraction means includes
Detecting means for detecting the first face element and the second face element for each of the first face image and the second face image;
By performing image conversion based on the positions of the first face element and the second face element in the first face image and the second face image, respectively, the condition of the face image in both images is set. A normalization means to match,
The contour determination according to any one of claims 1 to 3 , further comprising contour extracting means for extracting a contour of a human face from the first face image and the second face image after image conversion by the normalizing means. apparatus. A first face image composed of a plurality of first contour points extracted for the first face image and a second contour point composed of a plurality of second contour points extracted for the second face image . Display means for displaying the contour of the second face image in a comparable manner
The contour determination apparatus according to claim 1, comprising: Extracting a point located on the contour of the person's face as a contour point from a face image obtained by capturing the human face;
A plurality of first contour points extracted for the first face image and a plurality of second contour points extracted for the second face image are compared, and the first face image and the second face image and determining the size of the contour a program to be executed by the information processing apparatus,
The step of determining includes
Performing an association between the plurality of first contour points and the plurality of second contour points;
By comparing the distance from a predetermined reference element included in the face to each contour point for the corresponding two contour points, the first contour point and the second contour point, which contour point is located inside the face Determine if it is located,
If the number of first contour points determined to be located inside the face is larger than the first contour point, it is determined that the contour of the first face image is smaller than the contour of the second face image;
When the number determined to be located inside the face is larger in the second contour point, the step is to determine that the contour of the first face image is larger than the contour of the second face image. A featured program . Furthermore, it is composed of a contour of the first face image composed of a plurality of first contour points extracted for the first face image and a plurality of second contour points extracted for the second face image. second and outline of the face image, the program according to claim 6 for performing the steps <br/> that comparably displayed on the information processing apparatus that. An information processing device extracting, as a contour point, a point located on the contour of the human face from a face image obtained by capturing a human face;
The information processing apparatus compares the plurality of second contour points extracted for a plurality of first contour points extracted for the first face image and the second face image, and the first face image No. Determining the size of the contour between the second face image ,
The step of determining includes
Performing an association between the plurality of first contour points and the plurality of second contour points;
By comparing the distance from a predetermined reference element included in the face to each contour point for the corresponding two contour points, the first contour point and the second contour point, which contour point is located inside the face Determine if it is located,
If the number of first contour points determined to be located inside the face is larger than the first contour point, it is determined that the contour of the first face image is smaller than the contour of the second face image;
When the number determined to be located inside the face is larger in the second contour point, the step is to determine that the contour of the first face image is larger than the contour of the second face image. A characteristic contour determination method. Information processing device, first constituted by a plurality of first contour points extracted for the first face image
The contour of one face image, and the contour of the second face image composed of a plurality of second contour points extracted for a second face image, the steps <br/> that comparably displayed The contour judging method according to claim 8 including.

Images