JP2009172181A - Health checkup method and health checkup apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a health checkup method and a health checkup apparatus capable of judging a health condition accurately and easily by using of a face image. <P>SOLUTION: In an image of a user face, the color information of the eye part is difficult to be affected by the lighting condition and the health condition and the face color information is corrected from the eye color information to perform color correction of the face color information (step S270). Then the health condition of the user is determined corresponding to the face color information after correction (step S280). By using of the eye color information which is difficult to be affected by the lighting condition, the color correction of the face color information can be performed without setting the predetermined color standard differently, and the determination of health condition from the face color can be performed accurately and easily. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a health checkup method and a health checkup apparatus that determine a health condition using an image of a person.

  Conventionally, it has been known that there is a close relationship between the user's face color and health condition, such as poor physical condition when the face color is blue, etc. A technique for determining a person's health state using a face image has been proposed (for example, see Non-Patent Document 1).

  In the technique described in Non-Patent Document 1, a predetermined color reference and a person are photographed in one frame, and the face color is calibrated using the photographed color reference color, thereby determining the health condition. In this case, the influence of the illumination condition at the time of photographing is excluded.

Nobuyuki Ishibashi and two others, “How to simplify facial color measurement to become“ a measure of health ”” [online], Internet, <http://chihara.aist-nara.ac.jp/gakkai/VIR/PDF /A-08.pdf>

  However, in the technique described in Non-Patent Document 1, since it is necessary to set a color reference for color calibration at the time of shooting, it takes more time for shooting for a user who wants to receive a health condition determination. There was a problem of becoming inconvenient. In particular, in the case of mobile devices such as mobile phones, it is often difficult to set color standards outdoors, etc., so even if the mobile device is equipped with a health condition determination function that requires setting of color standards, It is conceivable that it becomes a function that is difficult to use and lacks convenience.

  SUMMARY An advantage of some aspects of the invention is to solve at least a part of the problems described above, and the invention can be implemented as the following forms or application examples.

  Application Example 1 A health diagnosis method for diagnosing a health condition using a face image, wherein a detection step of detecting pupil color information from the face image, and based on the pupil color information, A health diagnosis method comprising: a calibration step of calibrating color information; and a determination step of determining a health state based on color information of the face image after color calibration.

  In this way, the color of the face image used to determine the health condition is calibrated based on the color of the pupil that is not easily affected by the lighting conditions. The state can be accurately determined. In addition, since color calibration is performed based on the pupil color, unlike the technique described in Non-Patent Document 1, there is no need to set a color reference. Therefore, it is possible to accurately and easily determine the health state performed using the face image.

  [Application Example 2] In the health check method, in the calibration step, the difference between the color information of the pupil photographed under the reference illumination condition and the detected color information of the pupil is reduced or eliminated. A health diagnostic method comprising correcting the color of the face image.

  Application Example 3 In the health diagnosis method, in the determination step, a health condition is determined based on color information of an eye area in the face image.

  Application Example 4 In the above health checkup method, an image showing a subject to be photographed and a pupil alignment mark are displayed in an overlapping manner, and the pupil to be photographed is aligned with the position of the alignment mark. A health diagnosis method, further comprising: a photographing step of obtaining the face image by photographing the photographing object in a state.

  Application Example 5 A health diagnostic apparatus for diagnosing a health condition using a face image, the detection means for detecting color information of a pupil from the face image, and the detection of the face image based on the color information of the pupil A health diagnostic apparatus comprising: calibration means for calibrating color information; and determination means for determining a health state based on color information of the face image after color calibration.

  Hereinafter, embodiments of the present invention will be described based on examples.

  FIG. 1 is a block diagram illustrating a configuration of a health diagnosis apparatus according to the present embodiment. As shown in FIG. 1, the health diagnosis apparatus 1 includes a digital camera unit 10, a color information detection unit 20, a reference registration unit 30, a database unit 40, a color calibration unit (calibration means) 50, and a health condition determination. Unit (determination means) 60, an operation panel 70 including various switches for the user to operate the health diagnostic apparatus 1, a display panel 80 such as an LCD (Liquid Crystal Display) or an organic EL display, the digital camera unit 10, And a control unit 90 for controlling operations of the operation panel 70 and the display panel 80. The health diagnosis apparatus 1 of the present embodiment captures a user's face as a health condition diagnosis target with the digital camera unit 10, and determines the user's health condition based on the photographed face color. Specific examples of the health diagnostic apparatus 1 include portable devices such as a mobile phone and a PHS (Personal Handyphone System) that have a digital camera function. Hereinafter, each component of the health checkup apparatus 1 will be described.

  The color information detection unit 20 includes a pupil color information detection unit (detection means) 21 and a face color information detection unit 22. The pupil color information detection unit 21 specifies a part such as an eye and a nose by performing image processing such as contour extraction on the face image captured in the image data captured by the digital camera unit 10, and the eye region Among these, processing for detecting color information of the pupil part (hereinafter referred to as “pupil color information”) is performed. The face color information detection unit 22 detects color information representing the face color (hereinafter referred to as “face color information”) in the face image. In this embodiment, the color information of the eye area is detected as face color information. The reason why the color information of the eye area is the face color information is that the color of the eye area is likely to change in accordance with the change of the health condition and is less susceptible to the shadows due to illumination than the nose and cheeks.

  The reference registration unit 30 is a part for registering reference data RD to be used for color calibration, which will be described later, in the database unit 40, and when registering the reference data RD, in addition to the detected pupil color information, the user name Information such as the shooting date and time is registered in the database unit 40 as reference data RD.

  In the database unit 40, a state determination table HT indicating the relationship between the person's face color and the health state is registered in advance. In the state determination table HT, as shown in FIGS. 2A and 2B, color gamuts T1 to T5 on the xy chromaticity coordinates and representative colors when the face color is within the range of each color gamut. Health status is associated. For example, for a color gamut T1 corresponding to a yellowish complexion, a user who has a yellowish complexion is considered to be in good health, and for a color gamut T2 corresponding to a white complexion, such as lungs and nose It is said to be a health condition that requires attention to the respiratory system. Similarly, for the color gamut T3 corresponding to the red color, the health state requires attention to the heart, and for the color gamut T4 corresponding to the green color, the health condition requires attention to the gastrointestinal tract and the blue color. The corresponding color gamut T5 is considered to be a visceral state or a healthy state that requires attention to stress. Thus, the state determination table HT shows the relationship between the human face color and the health state. Of course, the contents of the color gamuts T1 to T5 shown in FIG. 2A and the state determination table HT shown in FIG. 2B should be set as appropriate according to various health conditions. It is not limited to the example. In the present embodiment, the health states are classified on the xy chromaticity coordinates. However, for example, the health states may be classified based on values of saturation and lightness in the HSV color system.

  The color calibration unit 50 performs color calibration of the face color information using the pupil color information with reference to the reference data RD when determining the health state of the user. Then, the health condition determination unit 60 refers to the condition determination table HT and performs a process of determining a health condition corresponding to the face color information after color calibration. The reason why the pupil color information is used for the color calibration of the face color information is that the color of the pupil is not affected by the health condition and is stable and is suitable as a color reference for color calibration.

  Then, the health condition determination unit 60 refers to the condition determination table HT to determine the health condition based on the face color information after color calibration. That is, the color gamut corresponding to the face color information after color calibration is selected from the color gamuts T1 to T5, and the health state corresponding to the selected color gamut is determined to be the health state of the user to be determined.

  As described above, the health diagnostic apparatus 1 performs facial color information color calibration using pupil color information that is not easily affected by the health condition in the captured image data, so that the facial color according to the illumination condition at the time of imaging is determined. The health condition is determined after eliminating or reducing the impact on the health.

  Next, the flow of processing for determining a health condition by the health diagnostic apparatus 1 described above will be described with reference to a flowchart. Hereinafter, first, the processing at the time of reference data registration will be described according to the flowchart of FIG.

  Registration of the reference data RD is performed under a predetermined illumination environment as a reference, such as a standard light source D65. When the user in the reference lighting environment operates the operation panel 70 and inputs an instruction to register the reference data RD, the process of FIG. 3 is started. When the processing is started, first, as shown in FIG. 4 as an example, the display panel 80 is provided with an alignment mark M for aligning the pupil position on the display panel 80 under the control of the control unit 90. It is displayed (step S100). The user confirms the position of the alignment mark M displayed on the display panel 80 and adjusts the positional relationship between itself and the digital camera unit 10 so that the position of his / her pupil matches the alignment mark M. To do. When the photographing switch included in the operation panel 70 is pressed (step S110: Yes), the digital camera unit 10 photographs the user's face and takes in image data in which the pupil position is adjusted to a predetermined position ( Step S120).

  Next, the pupil color information detection unit 21 performs image processing on the image of the captured image data to identify the region of the face and the contour of the face such as the eyes and nose, as shown in FIG. Then, the color information of the pupil region is detected as the reference pupil color information R in the face image FP representing the face in the captured image data (step S130). Then, it is determined whether or not the detected pupil color information R is an appropriate color as a pupil color based on whether or not the detected pupil color information R is included in a predetermined color range corresponding to the pupil color (step S140). For example, when a pupil with a contact lens is photographed with red eyes or when a color cast due to flash light is intense, the color of the pupil is not appropriate (step S140: No). Returning to step S100, the user's face is photographed again.

  On the other hand, when the pupil color is appropriate (step S140: Yes), the reference registration unit 30 uses the pupil color information R, user name, and shooting date / time information detected in step S130 as reference data. The RD is registered in the database unit 40 (step S150). When the reference data RD is registered in step S150, the process in FIG.

  As described above, at the time of reference data registration, the color information of the pupil imaged in the reference illumination environment is registered as the reference data RD. Next, the flow of processing performed when determining the health state will be described with reference to the flowchart of FIG.

  For example, when the user inputs an instruction to input the user name and determine the health condition by operating the operation panel 70 under lighting conditions different from the environment used as a reference when registering the reference, such as outdoors, FIG. The process is started. When the processing is started, first, as shown in FIG. 4 as an example, the alignment mark M for aligning the pupil position is displayed on the display panel 80 (step S200). The user confirms the position of the alignment mark M displayed on the display panel 80, and adjusts the positional relationship between the user and the digital camera unit 10 so that the position of his / her pupil is aligned with the alignment mark M. adjust. When the photographing switch is pressed (step S210: Yes), the digital camera unit 10 photographs the user's face as a subject and takes in image data in which the pupil position is adjusted to a predetermined position (step S220). .

  Next, the pupil color information detection unit 21 specifies the region of the part such as the eyes and the nose in the image of the captured image data, and as shown in FIG. It is detected as color information H (step S230). Then, the face color information detection unit 22 detects the color information of the eye area as the face color information F (step S240). As a method for specifying the eye area, for example, an area having a predetermined width that extends below the eye position and to the left and right with respect to the nose position may be used as the eye area. Further, as the face color information F, the color information of the eye area distributed to the left and right with respect to the position of the nose is calculated separately, and is set as the face color information F1 and F2, respectively.

  Then, it is determined whether or not the detected pupil color information H and face color information F have appropriate values (step S250). For example, when the user is wearing a contact lens and the pupil has red eyes, or when a strong color cast is generated in the entire captured image, the color is not appropriate (step S250: No), returning to step S200, the user's face is photographed again.

  On the other hand, when the pupil color information H and the face color information F are appropriate colors (step S250: Yes), the color calibration unit 50 refers to the reference data RD registered in the database unit 40, and in step S230. A difference between the detected pupil color information H and the reference pupil color information R is detected (step S260). Here, as shown in FIG. 8, the difference ΔH between the reference pupil color information R (= (Rx, Ry)) and the detected pupil color information H (= (Hx, Hx)) in the xy chromaticity coordinates. (= (Rx−Hx, Ry−Hx)) is calculated. Since this value ΔH can be assumed to be a difference caused by the difference between the lighting condition at the time of reference registration and the lighting condition of the place where the user who is going to determine the health condition, the value ΔH depends on the difference in the lighting condition. The correction value is used to correct the influence.

  Next, the color proofing unit 50 corrects the chromaticity coordinates corresponding to the correction value ΔH for each of the face color information F1 and F2 (step S270). Here, as shown in FIG. 8, the color calibrated face color information F1 ′ (= F1 + ΔH), F2 is obtained by correcting the chromaticity coordinates corresponding to the correction value ΔH with respect to the chromaticity coordinates of the face color information F1, F2. '(= F2 + ΔH) is acquired.

  Next, the health condition determination unit 60 refers to the condition determination table HT registered in the database unit 40 to determine the health condition corresponding to the corrected face color information F1 'and F2' (step S280). Here, the health state corresponding to the average value of the face color information F1 ′ and F2 ′ is determined as the user's health state. For example, the average value of the face color information F1 ′ and F2 ′ is included in the range of the color gamut T4. In this case, it is determined that the gastrointestinal needs attention according to the state determination table HT. If the color is different in the left and right eye area of the face and the color gamut including one face color information is different from the color gamut including the other face color information, the average value of the face color information F1 ′ and F2 ′ is the color. Since it can be assumed that the area is near the boundary of the area, for example, it may be determined that the degree of attention is lowered, for example, “Slight attention is required”. Thus, when the user's health condition is determined, the control unit 90 displays the determination result on the display panel 80 (step S290), thereby notifying the user of the determination result. If the process of step S290 is performed, the process of FIG. 6 will be complete | finished.

  Note that the processing in steps S200 to S220 is the photographing step described in the claims, the processing in step S230 is the detection step in the claims, the processing in steps S260 and S270 is the calibration step in the claims, and the processing in step S280 is This corresponds to the determination step recited in the claims.

  According to the health diagnostic apparatus 1 described above, color calibration is performed using the color of a pupil that is not easily affected by the health condition and has a stable color as a color reference. Without being able to accurately determine the state of health. Thereby, the user is not limited to the illumination conditions when the reference is photographed, and can receive a health condition diagnosis without choosing a place. Further, unlike the technique described in Non-Patent Document 1, since face color information can be color calibrated without taking the time and effort of setting a color reference, there are many scenes in which the health diagnosis apparatus 1 is carried and the location is moved. Even in the environment, the user can easily receive an accurate diagnosis of his / her health condition, which is convenient for the user. From the above, it is possible to accurately and easily determine the health condition based on the face color. Note that this health checkup method is particularly useful when applied to a portable device whose shooting location is not fixed.

  In addition, since the health condition is determined using color information of the eye area where changes in the health condition are likely to appear, changes in the health condition can be determined with high sensitivity, and shadows due to illumination are less likely to occur in the eye area. Therefore, the health condition can be accurately determined.

  Further, when the reference is registered and when the health condition is determined, the position of the pupil is aligned with the alignment mark M, and the image of the light source is reflected on the pupil by capturing the face of the person from an oblique direction. Therefore, the health condition can be determined more accurately using the pupil color information.

  As mentioned above, although one Example of the health-diagnosis apparatus was demonstrated, it is good also as the following modifications.

  (Modification 1) In the above embodiment, the health diagnosis device 1 performs reference registration and health status determination. However, the reference registration device and the health diagnosis device may be configured separately. For example, the reference data RD may be generated by a reference registration system that is installed in an environment with a reference illumination condition and includes a personal computer and a digital still camera. In this case, by transferring the reference data RD from the personal computer to the health diagnostic apparatus 1 which is a portable device and registering the reference data RD in the database unit 40 of the health diagnostic apparatus 1, the health diagnostic apparatus 1 stores the reference data RD. The used health condition can be determined.

  (Modification 2) In the above embodiment, the determination result and the diagnosis result are notified by displaying the determination result on the display panel 80. However, the determination result may be notified to the user by voice or the like. Alternatively, the determination result may be transmitted to a hospital via the Internet or a wireless communication network and notified to a doctor in charge.

  (Modification 3) The health diagnosis apparatus 1 of the above embodiment is a portable device, but is not limited thereto, and may be a personal computer to which a digital camera unit such as a digital still camera or a web camera is connected. Good. Further, a digital camera unit may be provided in a dresser or the like so as to have a function as a health diagnostic apparatus.

The block diagram which showed the structure of the health-diagnosis apparatus which concerns on a present Example. It is a figure for demonstrating a state determination table, (a) is a figure which showed the color gamut corresponding to a healthy state in xy chromaticity coordinates, (b) was a figure which showed an example of the state determination table. The flowchart which showed the flow of the process at the time of registering reference data. The figure which showed an example of the mark for alignment. The figure for demonstrating the detection method of the pupil color information used as a reference. The flowchart which showed the flow of the process which determines a health state. The figure for demonstrating the detection method of pupil color information and face color information. The figure for demonstrating color correction.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Health diagnostic apparatus, 10 ... Digital camera part, 20 ... Color information detection part, 21 ... Pupil color information detection part as a detection means, 22 ... Facial color information detection part, 30 ... Reference registration part, 40 ... Database part, 50 Color calibration unit as calibration means 60 Health state determination unit as judgment means 70 Operation panel 80 Display panel RD Reference data HT State determination table H Eye pupil color information R Reference Pupil color information, F ... face color information, M ... positioning mark.

Claims (5)

A health diagnosis method for diagnosing a health condition using a face image,
A detection step of detecting pupil color information from the face image;
A calibration step of calibrating the color information of the face image based on the color information of the pupil;
And a determination step of determining a health state based on color information of the face image after color calibration. The health check method according to claim 1,
In the calibration step, the color of the face image is corrected so as to reduce or eliminate a difference between color information of a pupil photographed under a reference illumination condition and color information of the detected pupil. Health checkup method. The health check method according to claim 1 or 2,
In the determination step, a health condition is determined based on color information of an eye area of the face image. In the health checkup method according to any one of claims 1 to 3,
By displaying an image of a subject to be photographed and a pupil alignment mark in an overlapping manner, and photographing the subject in a state where the pupil to be photographed and the position of the alignment mark are aligned. And a photographing step for acquiring the face image. A health diagnosis device for diagnosing a health condition using a face image,
Detecting means for detecting color information of the pupil from the face image;
Calibration means for calibrating the color information of the face image based on the color information of the pupil;
And a determination unit that determines a health state based on color information of the face image after color calibration.

Images