JP2008259676A - Information processing device and color determining method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To inform a user of a health condition by estimating it with accuracy without being affected by external light when a face image of the user has obstacles. <P>SOLUTION: This information processing device is equipped with a light source part 12 for emitting rays to a subject, a camera part 11 for acquiring images of the subject, an obstacle determining part 23 for determining whether there are obstacles or not, a target region determining part 24 for determining a region where the obstacles are avoided in the image of the subject as a target region, a difference image generating part 25 for generating difference images between an image acquired by emitting rays from the light source part 12 and an image acquired by emitting no rays from the light source part 12, and a complexion determining part 26 which compares with a threshold value based on an expectation value set in advance in the target region of the difference image generated by the part 25 and determines a degree of changes in the color of the subject. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an information processing apparatus including light source means such as a backlight of a liquid crystal screen and photographing means such as a camera function, and in particular, a user's health state based on an image such as a user's face photographed by the photographing means. The present invention relates to an information processing apparatus that estimates and notifies the above.

  As such an information processing apparatus having a function of notifying the user's health state and psychological state, a technique such as a mobile phone disclosed in Patent Document 1 has been conventionally proposed.

Such a conventional mobile telephone analyzes a display panel, an input face image using a face image feature analysis function, and determines a mind-body state based on the result, and a determination result is obtained. Display control means for displaying on the display panel. With such a configuration, a conventional mobile phone can check the health and psychological state of the user and the other party using various kinds of information that can be collected on a daily basis by a mobile phone.
JP 2005-27225 A JP 2003-150944 A

  However, for example, when a face image of a user is photographed as in the conventional example and the feature analysis of the face image is performed, the face color or shadow to be photographed changes due to external light such as sunlight or lighting. There has been a problem that it is difficult to accurately determine the health condition from the image. Furthermore, when trying to capture a user's face image, for example, the user may be wearing a mask or sticking a plaster on the face, which becomes an obstacle and the analysis of the face image is accurate. There was a possibility that it could not be implemented.

  The present invention has been made to solve such a problem. For example, in a face image of a user, the face color can be acquired without being affected by external light and without being affected by an obstacle. It is an object of the present invention to provide an information processing apparatus and a color determination method for accurately estimating and notifying a user's health condition.

  In order to achieve the above-described object, an information processing apparatus according to the present invention includes a light source unit that irradiates a subject with light rays, a photographing unit that photographs the subject and acquires image data of the subject, and an image of the subject. In the data, an obstacle determination unit that determines the presence or absence of an obstacle and detects an area of the obstacle, a target region determination unit that determines a region avoiding the obstacle in the image data of the subject as a target region, and a light source unit A difference image generation unit that generates difference image data between image data acquired by irradiating light from the image data acquired without irradiating light from the light source means, and difference image data generated by the difference image generation unit In the target area, a color determination unit that compares the expected value set in advance and determines the degree of change in the color of the subject is provided.

  The information processing apparatus according to the present invention further includes a telephone function, wherein the difference image generation unit generates difference image data and the color determination unit determines the degree of color change during a telephone call using the telephone function. .

  In addition, the information processing apparatus of the present invention further includes a telephone function, and at the same time when the telephone function is started, the photographing unit shoots image data in a state in which light is emitted from the light source and a state in which light is not emitted from the light source The difference image generation unit generates difference image data, and the color determination unit uses the difference image data to determine the degree of color change.

  In addition, the information processing apparatus of the present invention has a configuration having means for notifying the user that there is an obstacle when the obstacle determination unit determines that there is an obstacle.

  The information processing apparatus of the present invention further includes a call control unit having a function of detecting a change in the call position when it is determined that there is an obstacle, and the call position is changed by the function of detecting the change in the call position. When detected, the image capturing unit again acquires image data captured in a state in which light is emitted from the light source unit and image data captured in a state in which light is not irradiated from the light source unit, and the difference image generation unit receives the difference image data. And the color determination unit controls to determine the degree of color change using the difference image data.

  In the information processing apparatus according to the present invention, the obstacle determination unit divides the image data in units of blocks including a plurality of pixels, a predetermined range of threshold values, and an average value of each pixel in the block And a block whose average value is outside the threshold range of a predetermined range is defined as a first excluded block, which is a first excluded block, and a predetermined number of first excluded blocks that are consecutive in the surrounding blocks. If there are blocks, each is a second exclusion block, each outermost block of the image is a third exclusion block, the second exclusion block and the third exclusion block are obstacle regions, the target region determination unit, In the image data, a region excluding an obstacle region is set as a target region.

  In addition, the information processing apparatus of the present invention further includes a correction unit that generates a correction value for the expected value, and the color determination unit compares the expected value corrected with the correction value in the target region and changes the color of the subject. It is the structure which determines the degree of.

  The information processing apparatus according to the present invention is configured such that the correction unit generates a correction value based on a user input.

  The information processing apparatus according to the present invention has a configuration in which the correction value is a value corresponding to a temporary change in the skin color of the user.

  Further, the information processing apparatus of the present invention has a configuration in which the correction value is a value corresponding to the date and time, the season, and the life pattern.

  The color determination method of the present invention is a color determination method by an information processing apparatus including light source means for irradiating a subject with light rays and photographing means for photographing the subject and acquiring image data of the subject. In the step of photographing the subject by irradiating light from the light source means and acquiring image data, in the step of photographing the subject without irradiating the light from the light source means and acquiring image data, and in the image data of the subject, An image obtained by irradiating light from the light source means, determining whether there is an obstacle, detecting the area of the obstacle, determining an area avoiding the obstacle in the image data of the subject as the target area, and Generating difference image data between the data and image data acquired without irradiating light from the light source means, and in the target area of the generated difference image data , Compared with the expected value set in advance, and a determining the degree of change in the color of the object.

  The program of the present invention is a program for an information processing apparatus comprising light source means for irradiating a subject with light rays and photographing means for photographing the subject and acquiring image data of the subject. Shooting a subject by irradiating light and acquiring image data, capturing a subject without irradiating light from the light source means, obtaining image data, and whether there is an obstacle in the subject image data Detecting the area of the obstacle, determining the area avoiding the obstacle in the image data of the subject as the target area, image data acquired by irradiating light from the light source means, and a light source In the step of generating difference image data with image data acquired without irradiating light from the means, and in the target area of the generated difference image data, Luo compared to beforehand set expected value, and a step of determining the degree of change in the color of the object.

  The information processing apparatus and the color determination method according to the present invention include, for example, image data acquired by irradiating a light beam from a light source unit and image data acquired without irradiating a light beam from a light source unit for a user's face color determination. Since the difference image data is used, the influence of external light can be suppressed. Furthermore, the information processing apparatus and the color determination method of the present invention determine the user's face color in a target area that is an area that avoids an obstacle that is an obstacle for face color determination when a user's face image is captured. For this reason, according to the information processing apparatus or the color determination method of the present invention, it is not affected by outside light, and even in the case where there is an obstacle in the user's face image, the area where the obstacle is avoided. An information processing apparatus and a color determination method can be provided that can determine a health condition and the like, thereby accurately estimating and notifying a user's health condition.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

(Embodiment)
FIG. 1 is a block diagram showing a configuration of an information processing apparatus according to an embodiment of the present invention.

  In this embodiment, an example of an information processing device such as a mobile phone device that has a telephone function, a light source such as a backlight for a display function, and a camera function will be described. This information processing apparatus is characterized by having a function of determining the user's skin color. For example, when the user is making a call, the user can use the camera function to detect the user's skin color. For example, a cheek portion of the face is photographed, and the user's face color is determined. In the present embodiment, an example will be described in which a face color such as a cheek is determined as the user's skin color.

  As shown in FIG. 1, the information processing apparatus generally includes a light source unit 12 as a light source unit that irradiates a subject with light rays, and a camera unit as an imaging unit that captures a subject and acquires image data of the subject. 11, a photographing unit 10 that performs processing related to photographing, a signal processing unit 21 that performs image data processing of an image photographed by the camera unit 11, and the presence or absence of an obstacle in the image data of a subject, An image obtained by irradiating light from the light source unit 12, an obstacle determination unit 23 that detects an obstacle region, a target region determination unit 24 that determines a region avoiding an obstacle in the image data of the subject as a target region, A difference image generation unit 25 that generates difference image data between the data and image data acquired without irradiating light from the light source unit 12, and generates an expected value for face color determination A color determination unit that determines the degree of color change of the subject by comparing with the expected value generated in advance by the initialization unit 22 in the target area of the difference image data generated by the initialization unit 22 and the difference image generation unit 25 A facial color determination unit 26, a correction unit 28 that corrects the expected value generated by the initialization unit 22, a call control unit 27 that performs processing related to a telephone call, and an instruction from a user A user interface unit (hereinafter referred to as a user IF unit) 30 that notifies the user of information, a display unit 31 that displays information for the user, a key unit 32 that receives an instruction from the user, an expected value, And a memory 20 for storing various information and data including correction values. It should be noted that the obstacle determination unit 23 may be configured to detect obstacle candidates, or may be erroneously detected, for example, in the detection of the obstacle region.

  The imaging unit 10 controls the camera unit 11 and the light source unit 12 in accordance with an instruction from the call control unit 27, and acquires image data for generating difference image data. The light source unit 12 is, for example, a backlight disposed on the back surface of the liquid crystal panel when the display unit 31 performs display using a liquid crystal panel. That is, in the information processing apparatus, when the user is making a call, the photographing unit 10 controls the camera unit 11 to photograph the user's face. In particular, in such shooting, first, the shooting unit 10 controls the camera unit 11 so as to obtain cheek image data in a state where the light source unit 12 does not emit light. Next, the photographing unit 10 controls the camera unit 11 so that cheek image data irradiated by the light source unit 12 is obtained. The imaging unit 10 generates two types of image data by performing such imaging processing and light source control processing, and supplies the generated two types of image data to the signal processing unit 21. Such two types of images are generated in order to suppress the influence of external light such as a fluorescent lamp for illumination. This principle will be described in detail below.

  In the present embodiment, the light source unit 12 is described as a backlight such as a liquid crystal panel as described above. However, for example, an information processing apparatus that does not have a backlight because it is self-luminous, such as an organic EL. In this case, for example, two types of image data may be acquired under the condition of the white light emission state and the light emission stop state.

  Two types of image data are supplied to the signal processing unit 21. The signal processing unit 21 converts each image data into, for example, luminance data Y and two color difference data UV for data processing.

  The initialization unit 22 is provided in order for the user to create a reference for face color determination in the information processing apparatus in advance. That is, the user generates, for example, two types of image data as described above by photographing in a healthy state, and at this time, the initialization unit 22 generates a difference value between the two image data as an expected value, Further, various parameters for face color determination are generated from the expected values and recorded in the memory 20.

  The obstacle determination unit 23 determines the presence or absence of an obstacle in the cheek image data captured by the camera unit 11 and the area on the image when there is an obstacle. That is, this information processing apparatus automatically photographs the user's cheek while the user is talking, and determines the face color. For example, the user may have a mask by catching a cold, an adhesive bandage etc. Are stuck on the cheek, they become obstacles and cannot be determined accurately. For this reason, in the information processing apparatus, the obstacle determination unit 23 first performs an obstacle determination process for determining a region where an obstacle exists on the image. The obstacle determination method will be described in detail below. The obstacle determination unit 23 also performs processing for calculating the ratio of the area occupied by the obstacle on the image.

  In addition, the target area determination unit 24 uses the information related to the area of the obstacle determined by the obstacle determination unit 23 to obtain an area avoiding the obstacle on the image, and generates a difference image as the target area in the image data. Notify unit 25.

  The difference image generation unit 25 includes two types of image data as described above, that is, image data acquired by irradiating light from the light source unit 12 and image data acquired without irradiating light from the light source unit 12. The difference is obtained, and the difference image data in the target area is notified to the face color determination unit 26.

  The face color determination unit 26 obtains an average value of the difference image data in the target area in the difference image data generated by the difference image generation unit 25, and sets each parameter such as a threshold based on the average value and an expected value stored in the memory 20. And the degree of color change of the subject is determined. Further, the face color determination unit 26 creates a message corresponding to the color change based on such a determination result, and notifies the user IF unit 30 of the message. Further, by displaying such a message on the display unit 31, for example, the user IF unit 30 notifies the user of the facial color determination result. That is, for example, when the value of the difference image data tends to be blue with respect to the expected value, the face color determination unit 26 creates a message that the health condition is not good and displays it on the display unit 31.

  The correction unit 28 generates a correction value used for the determination process of the face color determination unit 26. That is, for example, the face color is tanned and darkened in summer or the like, and also changes depending on the difference between morning and night and changes in lifestyle patterns. Therefore, the correction unit 28 creates a correction value used for the determination process of the face color determination unit 26 and stores such a correction value in the memory 20. The correction unit 28 also performs processing for clearing the correction value stored in the memory 20.

  The call control unit 27 performs processing related to the telephone call, determines the start and end of the call, and notifies the photographing unit 10. Further, the call control unit 27 performs a call position determination process for determining whether the information processing apparatus is in close proximity to the right side or the left side of the face.

  Next, the principle of face color determination as color determination of the information processing apparatus in the present embodiment will be described.

  FIG. 2 is a diagram illustrating the principle of face color determination by the information processing apparatus. 2A shows an outline when the expected value is generated by the initialization unit 22, and FIG. 2B shows an outline when the face color is determined by the face color determination unit 26. FIG. Is shown.

  First, since this information processing apparatus is likely to be used in a different light environment each time, when determining such a face color, the influence of external light as shown by the light source 1 and the light source 2 in FIG. 2 is suppressed. There is a need. For this reason, in this information processing apparatus, the face color is determined based on the difference image data between the image data with irradiation from the light source unit 12 such as the backlight shown in FIG. 2 and the image data without irradiation. That is, in the case of FIG. 2A, when there is irradiation from the light source unit 12, a light beam added with the light amounts of the light source 1 and the light source unit 12 is irradiated onto the cheek of the face. In addition, when there is no irradiation from the light source unit 12, a light beam of only the light source 1 is irradiated onto the cheek of the face. By obtaining such a difference between the two types of image data, the image component corresponding to the light amount of the light source 1 is canceled, and the image data of the cheek of the face due to the light irradiation of the light source unit 12 can be extracted. Similarly in the case of FIG. 2B, the image component by the light source 2 can be canceled by obtaining the difference between the two types of image data. In this way, the information processing apparatus suppresses the influence of external light and acquires facial cheek image data by the light source unit 12 that is always in the same condition as a backlight. Therefore, it is possible to determine the face color at the time of each call based on the expected value generated as shown in FIG.

  Next, a method of detecting an obstacle area by the obstacle determination unit 23 and determining a target area avoiding the obstacle by the target area determination unit 24 will be described.

  FIG. 3 is a diagram for explaining a procedure for extracting a target area in the information processing apparatus. In the present embodiment, such an obstacle is assumed to be continuous, and the area is obtained. In this embodiment, the image is divided into blocks each including a predetermined number of pixels in order to detect an obstacle area. FIG. 3 shows an example in which one block is composed of N × M pixel blocks.

  First, the obstacle determination unit 23 compares the predetermined upper and lower thresholds for obstacle determination with the average value of each block of image data captured from the signal processing unit 21. Then, the obstacle determination unit 23 sets a block whose average value is outside the range of both thresholds as a first excluded block. The determination image 1 in FIG. 3 shows an example in which such a first exclusion block is set. In the determination image 1, “0” indicates a block whose average value is within the range of both thresholds, and “1” indicates a block whose average value is outside the range of both thresholds, that is, the first excluded block. ing.

  Next, the obstacle determining unit 23 sets a block that is a first excluded block and further includes a predetermined number of consecutive first excluded blocks in the surrounding blocks as a second excluded block. For example, when there are three or more consecutive first excluded blocks in the eight surrounding blocks, these blocks are set as second excluded blocks. That is, since the obstacle is usually larger than the block, the area where the first excluded block is continuous is determined as the obstacle. The determination image 2 in FIG. 3 shows an example in which such a second exclusion block is set. In the determination image 2, “2” indicates the second exclusion block.

  Next, the obstacle determination unit 23 sets the outermost block on the image as a third excluded block. The determination image 3 in FIG. 3 shows an example in which such a third excluded block is set. In the determination image 3, “3” indicates a third excluded block.

  The obstacle determination unit 23 determines the area of the obstacle by the procedure as described above. Further, the obstacle determination unit 23 performs such an obstacle determination process on the luminance data and the color difference data of the image data with and without the irradiation from the light source unit 12.

  Next, the target region determination unit 24 sets a region excluding the second exclusion block and the third exclusion block set by the obstacle determination unit 23 as the target region. That is, as illustrated in the determination image 3 of FIG. 3, an area having blocks other than “2” and “3” is set as a target area. Since the face color determination unit 26 compares the value of the difference image data with the expected value in the target region that is not affected by the obstacle, and determines the face color, the face color can be determined with high accuracy. Become.

  Next, the operation of the information processing apparatus of the present embodiment configured as described above will be described. FIG. 4 is a flowchart showing the procedure of each process for face color determination as color determination in the information processing apparatus. FIG. 5 is a flowchart illustrating a procedure of face color determination processing in the information processing apparatus. FIG. 6 is a diagram for explaining the values of parameters for the face color determination process in the information processing apparatus. The detailed operation of the information processing apparatus according to the present embodiment will be described below with reference to FIGS.

  The information processing apparatus determines whether or not a call is started in the call control unit 27. When the information processing apparatus determines that the call is started by the information processing apparatus, the information processing apparatus performs facial color determination according to the procedure illustrated in FIG. Start processing for.

  When the information processing apparatus starts such processing, first, under the control of the imaging unit 10, the camera unit 11 captures an image without irradiation from the light source unit 12, such as a backlight. At this time, since the user is in a call using the information processing apparatus, the camera unit 11 captures the cheek of the user's face. The photographing unit 10 takes in image data obtained by photographing with the camera unit 11 and supplies the image data to the signal processing unit 21. The signal processing unit 21 generates luminance data y and color difference data u and v from the image data, and further obtains an average value for each block. That is, the signal processing unit 21 obtains y0avg that is an average value for each block and average values u0avg and v0avg in the same color difference data in the luminance data in a state without irradiation from the light source unit 12 (step S100). ).

  Next, in the information processing apparatus, under the control of the imaging unit 10, the camera unit 11 captures an image with irradiation from the light source unit 12 such as a backlight. The signal processing unit 21 generates luminance data y and color difference data u and v from the image data, and further obtains an average value for each block. That is, the signal processing unit 21 obtains y1avg, which is an average value for each block, and average values u1avg and v1avg in the same color difference data in the luminance data in a state where irradiation from the light source unit 12 is performed (step S102). ).

  Next, the obstacle determination unit 23 executes an obstacle determination process according to the procedure described with reference to FIG. Further, the target area determination unit 24 determines an area excluding the obstacle area, and thereby sets a target area for determining the face color (step S104).

  Next, the information processing apparatus calculates the ratio of the obstacle to the acquired image. That is, in the case of the determination image 3 in FIG. 3, for example, by dividing the number of second excluded blocks indicated by “2” by the total number of blocks indicated by “0”, “1”, and “2”, Calculate the percentage of obstacle blocks. Thereafter, when the obstacle block ratio is larger than the predetermined ratio, the process proceeds to step S110. If the obstacle block ratio is smaller than the predetermined ratio, the process proceeds to step S108 (step S106).

  When the ratio of the obstacle block is smaller than the predetermined ratio, the difference image generation unit 25 calculates the difference between the image data acquired by irradiating light from the light source unit 12 and the image data acquired without irradiation in the target region. And the difference image data in the target area is obtained. That is, the difference image generation unit 25 obtains difference data ydiff in luminance data and difference data udiff and vdiff in color difference data (step S108).

  Thereafter, the face color determination unit 26 performs a face color determination process (step S200). The face color determination process will be described in detail below with reference to FIG.

  If it is determined in step S106 that the obstacle block ratio is greater than the predetermined ratio, the process proceeds to step S110. The information processing apparatus has a function in which the call control unit 27 detects a change in the call position. In step S110, the information processing apparatus detects the call position, that is, the information processing apparatus by the call control unit 27. It is determined whether the phone is talking near the right side or the left side of the face. If the information processing apparatus determines that the position of the ear where the user is bringing the information processing apparatus close has changed based on the determination of the call position, the process returns to step S100. If the information processing apparatus determines that the position of the ear has not changed, the information processing apparatus proceeds to the process of step S112 (step S110).

  As an example of a function for detecting a change in the call position, the information processing apparatus is configured to include a geomagnetic sensor, and it is determined that the call position has been changed when the orientation detected by the geomagnetic sensor changes by, for example, 120 degrees or more. By providing such a means, such a function can be realized. In addition, as another example of the function for detecting the change of the call position, the information processing apparatus includes a motion sensor. The motion sensor detects the inclination of the information processing apparatus, and the inclination detected by the motion sensor. For example, there may be provided means for determining that the call position has been changed if it has changed by, for example, 10 degrees or more. Furthermore, as another example of the function for detecting a change in the call position, the information processing apparatus includes a motion detection unit, and the call position is changed when a stationary state is detected after the movement is detected by the motion detection unit. Means for determining may be provided. Furthermore, as another example of the function for detecting the change of the call position, the information processing apparatus includes a touch panel, and the touch panel is in the order of “contact”, “non-contact”, and then “contact”. A means may be provided for determining that the call position has been changed when contact is detected.

  Next, in the information processing apparatus, the call control unit 27 determines whether or not the call is ended at a predetermined time interval such as once per minute. If the call has not ended, the process returns to step S110. If the call is terminated, the process proceeds to step S114 (step S112).

  When it is determined that the call has ended, the information processing apparatus presents a message such as “Please remove the obstacle” by the user IF unit 30. That is, since there is a large obstacle and the user has not changed the call position of the information processing apparatus, there is a high possibility that the face color cannot be accurately determined. Therefore, such a message prompts the user to remove the obstacle (step S114).

  Next, in the same manner as step S100, the information processing apparatus controls the photographing unit 10 to photograph the camera unit 11 without irradiation from the light source unit 12 such as a backlight (step S116), and further to step S102. Similarly to the above, the camera unit 11 captures an image with irradiation (step S118). Further, similarly to step S104, the obstacle determination unit 23 executes an obstacle determination process (step S120). Thereafter, as in step S106, the information processing apparatus calculates the ratio of the obstacle to the acquired image. If the ratio is large, the process proceeds to step S124. If the ratio is small, The process proceeds to S108 (step S122). That is, when the ratio is small in step S122, the information processing apparatus determines that the user can remove the obstacle by the message in step S114 and thereby can accurately determine the face color, and proceeds to the processing in step S108. On the other hand, if the ratio is large in step S122, the information processing apparatus determines that the user has not removed the obstacle or that the facial color may be bad enough to determine that it is an obstacle even if the obstacle is removed. Proceed to processing.

  When the information processing apparatus determines that the ratio is large in step S122, the information processing apparatus determines that the abnormality is as described above. Next, the user IF unit 30 determines, for example, “If the obstacle has not been removed, The message “There is a possibility of being bad” is presented, and then the processing is terminated (step S124).

  Note that, when the expected value is generated by the initialization unit 22, steps S100 to S108 in FIG. 4 are executed, thereby generating the expected value.

  Next, the face color determination process executed in step S200 in FIG. 4 will be described in detail with reference to FIGS.

  First, each parameter used for the face color determination process will be described with reference to FIG. In FIG. 6, parameters for the luminance data Y are illustrated, but the same applies to the color difference data U and V. In the present embodiment, in order to determine the face color, a range in which the face color is normal or a range in which the face color is temporarily normal is determined in advance and stored in the memory 20.

  In the present embodiment, a lower width y0 and an upper width y1 are set with respect to the expected value acquired by the initialization unit 22 as a range in which the face color is normal. The normal range is from 1 to the upper threshold 2. That is, when the difference luminance data in the target area is a value within this range, it is determined that the face color is normal.

  Further, in the present embodiment, a temporary correction value y_tmp0 is set in the downward direction of the threshold 1 and the temporary correction value y_tmp1 is set in the upward direction of the threshold 2 as a range in which the face color is temporarily normal. Thus, the temporary normal range is from the lower limit threshold value tmp1 to the upper limit threshold value tmp2. Such a temporary normal range is set as an allowable range even when the face color changes due to the influence of sunburn, drinking or bathing.

  Next, the face color determination process for determining the face color using each parameter shown in FIG. 6 will be described.

  As shown in FIG. 5, when the face color determination process is started, the face color determination unit 26 first calculates an average value in the target area of the difference luminance data and the difference color difference data. Further, the face color determination unit 26 determines whether or not the calculated average value is within the range from the threshold value 1 to the threshold value 2 shown in FIG. When the average value is within the range from the threshold value 1 to the threshold value 2, the face color determination unit 26 proceeds to the process of step S212. If the average value is not within the range from the threshold value 1 to the threshold value 2, the face color determination unit 26 proceeds to the process of step S220 (step S210).

  When the process proceeds to step S212, the face color determination unit 26 determines whether or not the temporary correction value is stored in the memory 20 (step S212). When the temporary correction value is stored, the face color determination unit 26 proceeds to the process of step S214, clears the temporary correction value stored in the memory 20, and ends the face color determination process (step S214). If the temporary correction value is not stored, the face color determination unit 26 proceeds to the end of the process and ends the face color determination process. As described above, when the color of the face image data captured by the user during a call is within the normal range indicated by the threshold value 1 to the threshold value 2 by the processing from step S210 to step S214, The processing ends with “the face color is normal”.

  When it is determined in step S210 that the average value is not within the range from the threshold value 1 to the threshold value 2, the face color determination unit 26 first determines whether the temporary correction value is stored in the memory 20 ( Step S220). If the temporary correction value is stored, the face color determination unit 26 proceeds to the process of step S222. If the temporary correction value is not stored, the face color determination unit 26 proceeds to the process of step S240.

  If it is determined in step S220 that the temporary correction value is stored, the face color determination unit 26 next determines whether the calculated average value is within the range from the threshold value tmp1 to the threshold value tmp2 shown in FIG. To do. When the average value is within the range from the threshold value tmp1 to the threshold value tmp2, the face color determination unit 26 proceeds to the end of the process and ends the face color determination process. On the other hand, if the average value is not within the range from the threshold value tmp1 to the threshold value tmp2, the face color determination unit 26 proceeds to the process of step S224 (step S222). When the processing proceeds to step S224, the information processing apparatus presents a message such as “Is my physical condition okay?” By the user IF unit 30 (step S224) and requests user input (step S226). ). If there is an input such as “YES” from the user, the process proceeds to end, and the face color determination process ends. If the user inputs “NO”, the process proceeds to step S228, and the user IF unit 30 presents a message such as “I recommend a health checkup” ( Step S228), the face color determination process is terminated.

  If it is determined in step S220 that the temporary correction value is not stored, the user IF unit 30 presents a message such as “Your face looks bad, but after exercise, bathing, or drinking?” (Step S240) and user input is requested (step S242). If the user inputs “YES”, the process proceeds to step S248. If the user inputs “NO”, the process proceeds to step S244, and the user IF unit 30 displays, for example, “Your face looks bad, but is it tanned?” A new message is presented (step S244), and further a user input is requested (step S246). In step S246, if the user inputs “YES”, the process proceeds to step S248. If the user inputs “NO”, the process proceeds to step S224, and a message regarding “facial color is bad” such as step S224 or step S228 is presented. finish. Further, when the processing proceeds to step S248, a temporary correction value is determined from the current image data and stored in the memory 20 so that the current image data is temporarily within a normal range (step S248). Thereafter, the face color determination process is terminated.

  In the above description, an example in which the facial color is determined using each parameter calculated from the expected value acquired once has been described. However, the facial color is as described above after bathing or drinking, or tanning. In addition to temporary changes such as the above, there is also a possibility that changes may occur depending on, for example, the season, day of the week, time zone, temperature, lifestyle pattern, and the like. FIG. 7 shows an example of each parameter set to cope with each change in the information processing apparatus. That is, as shown in FIG. 7, the initialization unit 22 obtains the expected values for the season, day of the week, time zone, and temperature, and obtains each parameter as shown in FIG. save. Then, by performing the face color determination process in accordance with the season, day of the week, etc., it becomes possible to perform more accurate face color determination.

  In addition, by executing the steps described above with reference to FIGS. 4 and 5, the processing of each step in the color determination method of the present invention is performed.

  As described above, the information processing apparatus of the present invention includes the light source unit 12 that irradiates a face that is a subject with light rays, the camera unit 11 that captures a face and acquires face image data, and the face image. In the data, the presence / absence of an obstacle is determined, an obstacle determination unit 23 that detects an area of the obstacle, a target region determination unit 24 that determines a region avoiding the obstacle in the face image data as a target region, The difference image generation unit 25 that generates difference image data between the image data acquired by irradiating light from the light source unit 12 and the image data acquired without irradiation, and the difference image data generated by the difference image generation unit 25 The target area includes a face color determination unit 26 that compares the threshold value 1 and threshold value 2 based on preset expected values and determines the degree of change in face color. Therefore, according to the information processing apparatus of the present invention, difference image data between image data acquired by irradiating light from the light source unit 12 and image data acquired without irradiating light from the light source unit 12 is used. Therefore, the influence of external light can be suppressed in the determination of the skin color such as the face color. In addition, according to the information processing apparatus of the present invention, when the user's face image is captured for the user's face color determination, the obstacle determination unit 23 determines an area that becomes an obstacle for the face color determination, and the target area The determination unit 24 notifies a target area that is an area avoiding an obstacle, and the face color determination unit 26 determines the face color of the user in the target area. Therefore, according to the information processing apparatus of the present invention, it is possible to determine a health condition or the like in an area avoiding an obstacle without being affected by external light and even when the user's face image has an obstacle. Thus, it is possible to provide an information processing apparatus that accurately estimates and notifies a user's health condition and the like. Similarly, according to the color determination method of the present invention, it is possible to provide a color determination method for accurately estimating and notifying a user's health condition.

  Although the present invention has been described based on the above embodiment, it is needless to say that the present invention is not limited to the above embodiment. The following cases are also included in the present invention.

  That is, some or all of the constituent elements constituting the information processing apparatus may be included in one system LSI (Large Scale Integration). The system LSI is an ultra-multifunctional LSI manufactured by integrating a plurality of components on a single chip.

  In addition, each part of the constituent elements constituting the above circuit may be individually made into one chip, or may be made into one chip so as to include a part or all of them.

  Although the system LSI is used here, it may be called IC, LSI, super LSI, or ultra LSI depending on the degree of integration. Further, the method of circuit integration is not limited to LSI's, and implementation using dedicated circuitry or general purpose processors is also possible. An FPGA (Field Programmable Gate Array) that can be programmed after manufacturing the LSI or a reconfigurable processor that can reconfigure the connection and setting of circuit cells inside the LSI may be used.

  Furthermore, if integrated circuit technology comes out to replace LSI's as a result of the advancement of semiconductor technology or a derivative other technology, it is naturally also possible to carry out function block integration using this technology. Biotechnology can be applied as a possibility.

  Further, the present invention may be the method described above. Further, the present invention may be a computer program that realizes these methods by a computer, or may be a digital signal composed of the computer program.

  The present invention also provides a computer-readable recording medium such as a flexible disk, hard disk, CD-ROM, MO, DVD, DVD-ROM, DVD-RAM, BD (Blu-ray Disc). ), Recorded in a semiconductor memory or the like. Further, the digital signal may be recorded on these recording media.

  In the present invention, the computer program or the digital signal may be transmitted via an electric communication line, a wireless or wired communication line, a network represented by the Internet, a data broadcast, or the like.

  The present invention may be a computer system including a microprocessor and a memory, wherein the memory stores the computer program, and the microprocessor operates according to the computer program.

  In addition, the program or the digital signal is recorded on the recording medium and transferred, or the program or the digital signal is transferred via the network or the like, and executed by another independent computer system. It is good.

  Further, the above embodiment and the above modification examples may be combined.

  Since the present invention can accurately estimate and notify a user's health state even when there are obstacles in the user's face image, the present invention can be applied to an information processing device such as a mobile phone device. Is possible.

The block diagram which shows the structure of the information processing apparatus in embodiment of this invention The figure which shows the principle of the face color determination by the information processing apparatus The figure for demonstrating the procedure which extracts the target area | region in the information processing apparatus The flowchart which shows the procedure of each process for the face color determination in the information processing apparatus The flowchart which shows the procedure of the face color determination process in the information processing apparatus The figure for demonstrating the value of each parameter for the face color determination process in the information processing apparatus The figure which shows an example of each parameter set in order to respond also to each change in the same information processing apparatus

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Image | photographing part 11 Camera part 12 Light source part 20 Memory 21 Signal processing part 22 Initialization part 23 Obstacle determination part 24 Target area | region determination part 25 Difference image generation part 26 Facial color determination part 27 Call control part 28 Correction | amendment part 30 User IF part 31 Display section 32 Key section

Claims (12)

Light source means for irradiating a subject with light rays;
Photographing means for photographing the subject and obtaining image data of the subject;
In the image data of the subject, an obstacle determining unit that determines the presence or absence of an obstacle and detects the area of the obstacle;
A target area determination unit that determines, as a target area, an area that avoids the obstacle in the image data of the subject;
A difference image generation unit that generates difference image data between image data obtained by irradiating light from the light source means and image data obtained without irradiating light from the light source means;
A color determination unit that compares the target area of the difference image data generated by the difference image generation unit with a preset expected value and determines the degree of color change of the subject. Information processing device. 2. The telephone apparatus according to claim 1, further comprising a telephone function, wherein the difference image generation unit generates the difference image data and the color determination unit determines the degree of color change during a call by the telephone function. The information processing apparatus described in 1. A telephone function, and together with the start of a telephone call by the telephone function, the image capturing unit captures image data captured in a state where light is emitted from the light source unit, and image data captured in a state where the light source unit does not emit light. The difference image generation unit generates the difference image data, and the color determination unit determines the degree of color change using the difference image data. The information processing apparatus described in 1. The information according to any one of claims 1 to 3, further comprising means for notifying a user that there is an obstacle when the obstacle determination unit determines that the obstacle is present. Processing equipment. When it is determined that there is an obstacle, the camera further includes a call control unit having a function of detecting a change of the call position, and when the change of the call position is detected by the function of detecting the change of the call position, However, the image data photographed in a state in which light is emitted from the light source means and the image data photographed in a state in which no light is emitted from the light source means are acquired, and the difference image generation unit generates the difference image data. The information processing apparatus according to claim 2, wherein the color determination unit performs control so as to determine the degree of color change using the difference image data. The obstacle determination unit
The image data is divided into blocks each composed of a plurality of pixels,
A predetermined threshold value in a predetermined range is compared with an average value of each pixel in the block, and a block in which the average value is outside the predetermined threshold value range is defined as a first exclusion block.
If there are more than a predetermined number of consecutive first excluded blocks in the surrounding blocks, each is a second excluded block,
Each outermost block of the image is a third excluded block,
The second exclusion block and the third exclusion block are obstacle regions,
The information processing apparatus according to any one of claims 1 to 3, wherein the target area determination unit sets a target area as an area excluding the obstacle area in the image data. A correction unit that generates a correction value for the expected value;
4. The color determination unit according to claim 1, wherein the color determination unit compares the expected value corrected with the correction value in the target area and determines the degree of change in the color of the subject. The information processing apparatus described in 1. The information processing apparatus according to claim 7, wherein the correction unit generates the correction value based on a user input. The information processing apparatus according to claim 8, wherein the correction value is a value corresponding to a temporary skin color change of the user. The information processing apparatus according to claim 7, wherein the correction value is a value according to a date and time, a season, and a life pattern. A color determination method by an information processing apparatus comprising light source means for irradiating a subject with light rays and photographing means for photographing the subject and acquiring image data of the subject,
Irradiating light from the light source means to photograph the subject and obtaining image data;
Photographing the subject without irradiating light from the light source means, and obtaining image data;
Determining the presence or absence of an obstacle in the image data of the subject and detecting the area of the obstacle;
Determining an area avoiding the obstacle in the image data of the subject as a target area;
Generating difference image data between image data acquired by irradiating light from the light source means and image data acquired without irradiating light from the light source means;
A color determination method comprising: a step of comparing the target area of the generated difference image data with a predetermined expected value to determine the degree of color change of the subject. A program for an information processing apparatus comprising light source means for irradiating a subject with light rays and photographing means for photographing the subject and acquiring image data of the subject,
Irradiating light from the light source means to photograph the subject and obtaining image data;
Photographing the subject without irradiating light from the light source means, and obtaining image data;
Determining the presence or absence of an obstacle in the image data of the subject and detecting the area of the obstacle;
Determining an area avoiding the obstacle in the image data of the subject as a target area;
Generating difference image data between image data acquired by irradiating light from the light source means and image data acquired without irradiating light from the light source means;
A program comprising the step of comparing a target value of the generated difference image data with a predetermined expected value and determining a degree of color change of a subject.

Images