JP6452395B2 - Psychological state estimation method, psychological state estimation system, and care system using psychological state estimation method - Google Patents

Description

  The present invention relates to a psychological state estimation method and a psychological state estimation system that estimate the psychological state of a person shown in image data based on image data. The present invention also relates to a care system technique using a psychological state estimation method.

  Conventionally, a technique for estimating the state of a person shown in image data based on image data or the like has been publicly known. For example, as described in Patent Document 1.

  The technique described in Patent Document 1 detects the voice data and image data of the returnee, and based on the normal voice data and image data of the returnee and the voice data and image data at the time of detection, the human body state at the time of detection Are analyzed, and the analysis result is displayed to inform the home-staying person.

  However, the technique described in Patent Document 1 only analyzes a superficial state such as a returnee's facial expression and fatigue level, and analyzes an internal state of a person, that is, a psychological state until it is objectively determined. It has not reached. Regarding the technique for estimating the state of a person shown in image data, there is room for improvement in this respect.

JP 2007-129312 A

  The present invention has been made in view of the above situation, and provides a psychological state estimation method, a psychological state estimation system, and a care system that can objectively determine the psychological state of a target person.

  The problem to be solved by the present invention is as described above. Next, means for solving the problem will be described.

That is, in claim 1, an image acquisition step for acquiring image data by photographing a face of the target person, and a psychological state estimation for estimating a psychological state related to positive emotion and negative emotion of the target person based on the image data And the psychological state estimation step includes a smile degree estimation step for estimating a smile level of the target person based on the image data, and an age for estimating the age of the target person based on the image data. Based on an estimation step, a calculation step of calculating an age difference by comparing the estimation result of the age with a preset reference estimated age of the target person, the estimation result of the smile degree, and the calculation result of the age difference And a composite estimation step of estimating the psychological state of the target person .

According to a second aspect of the present invention, the method further includes a setting step of previously capturing image data serving as a reference of the target person and setting a reference estimated age of the target person based on the captured image data serving as the reference. .

4. The psychological state estimation step according to claim 3, wherein the psychological state estimation step includes a personal identification step of identifying an individual based on the image data, and a reference estimated age of all target persons set in advance based on the result of identifying the individual. An acquisition step of acquiring the reference estimated age of the identified individual from among the above .

According to a fourth aspect of the present invention, in the image acquisition step, the face of the target person is photographed using a camera attached to an entrance phone of an interphone .

According to a fifth aspect of the present invention, a photographing means for photographing the face of the target person to acquire image data, a smile degree estimating means for estimating the smile degree of the target person based on the image data, and the image data The age estimation means for estimating the age of the target person, the estimated result of the age is compared with a preset reference estimated age of the target person to calculate an age difference, and the smile degree estimation result and the age Psychological state estimating means for estimating a psychological state related to the positive emotion and negative emotion of the target person based on the difference calculation result .

In Claim 6, it is a care system using the psychological state estimation method according to any one of claims 1 to 4, wherein notification based on the estimation result of the psychological state of the target person, Notification means for performing to a person other than the target person is provided .

In Claim 7, It is a care system using the psychological state estimation method of Claim 4, Comprising: Based on the result image | photographed using the said camera, the door of the entrance in which the said entrance cordless handset is provided is automatically An automatic unlocking means for unlocking is provided.

  As effects of the present invention, the following effects can be obtained.

In claim 1, the psychological state of the target person can be objectively determined. In addition, the psychological state of the target person can be estimated with high accuracy.

  In claim 2, the psychological state of the target person can be estimated with high accuracy.

In claim 3, the psychological state of the target person can be efficiently estimated.

According to the fourth aspect, the face of the target person can be efficiently photographed .

In claim 5, the psychological state of the target person can be objectively determined .

In claim 6, it is possible for a person other than the target person to grasp that the target person has negative emotion .

According to the seventh aspect, the face of the target person can be efficiently photographed .

The figure which showed the whole structure of the psychological state estimation system which concerns on one Embodiment of this invention. The figure which showed the flow of the psychological state estimation method which concerns on one Embodiment of this invention. The figure which showed the image data image | photographed with the camera of the entrance phone of the intercom. The figure which showed standard face image data. The figure which showed the reverse correlation of the smile degree and age difference when a psychological state changed. The figure which showed the relationship between a smile degree, an age difference, and a psychological state.

  Below, the psychological state estimation method which concerns on one Embodiment of this invention is demonstrated.

  The psychological state estimation method estimates the psychological state of the target person. The psychological state estimation method according to the present embodiment estimates the psychological state of a family including parents and children. The psychological state estimation method is implemented by adopting the care system 10 shown in FIG. 1 in the building where the family resides.

  First, the configuration of the care system 10 will be described with reference to FIG.

  The care system 10 is for assisting family communication using the estimation result of the psychological state estimation method. The care system 10 mainly includes an interphone 20, an electronic lock device 30, a personal computer 40, and the like.

  The interphone 20 according to this embodiment is an interphone with a camera that can visually recognize a visitor in front of the entrance of the building. The interphone 20 includes an entrance handset 21 and a base unit 22.

  The entrance cordless handset 21 is provided outdoors, more specifically, in the vicinity of the door D of the entrance. A camera 21a and the like for photographing a visitor's face are attached to the front door 21.

  The base unit 22 is installed indoors by being attached to the inner wall or the like of the building. The master unit 22 is configured to be able to transmit and receive signals to and from the entrance slave unit 21 by wireless communication means or wired communication means. The master unit 22 is provided with a monitor 22a for displaying an image taken by the camera 21a of the entrance slave unit 21.

  The electronic lock device 30 automatically locks and unlocks the front door D. The electronic lock device 30 includes an electronic lock 31 and a control panel 32.

  The electronic lock 31 is attached to the door D of the entrance, and locks and unlocks the entrance door D by moving a dead bolt by driving a motor. The electronic lock 31 includes an input unit such as a numeric keypad and an IC card reading unit.

  The control panel 32 is disposed in the vicinity of the base unit 22 of the interphone 20 by being attached to the inner wall or the like of the building. The control panel 32 is configured to be able to transmit and receive signals to and from the electronic lock 31 by wireless communication means or wired communication means. The control panel 32 is provided with a button 32a or the like that can be operated by inputting an appropriate signal to the electronic lock 31 to lock and unlock the door D. A signal or the like for notifying that the electronic lock 31 has been unlocked is input to the control panel 32. The control panel 32 is configured to be able to transmit and receive signals to and from the base unit 22 of the interphone 20 by wireless communication means or wired communication means. A signal or the like for unlocking the door D is input from the master unit 22 to the control panel 32.

  The personal computer 40 is for performing arithmetic processing and the like necessary for estimating the psychological state of the family. The personal computer 40 includes an arithmetic device (such as a CPU), a storage device (a main storage device such as a RAM and a ROM, and a storage such as an SSD and an HDD), an input device (such as a keyboard and a mouse), An output device (liquid crystal display, organic EL display, etc.) is provided. The personal computer 40 provides various functions when the arithmetic device calls and executes an OS or program stored in the storage device.

  The personal computer 40 is configured to be able to transmit and receive signals to and from the base unit 22 of the interphone 20 through a LAN cable and a hub H. The personal computer 40 is input with the photographing result of the camera 21 a of the front door slave 21 via the master 22 and the hub H. In addition, a signal for notifying that the door D is unlocked is input to the personal computer 40 via the hub H, the master unit 22, the control panel 32, and the like. Further, the personal computer 40 is configured to be able to input a signal for unlocking the door D to the electronic lock 31 via the hub H, the master unit 22, the control panel 32, and the like.

  The personal computer 40 according to the present embodiment has a personal identification unit 41, a facial expression recognition unit 42, a psychological state estimation unit 43, and an estimation result storage area 44 as means for performing arithmetic processing necessary for estimating the psychological state of the family. Etc.

  The personal identification unit 41 is a program for specifying who is a person shown in predetermined image data (identifying an individual). The personal identification unit 41 compares the face reflected in the predetermined image data with the image data obtained by photographing the family face stored in the storage device in advance, and which family face is captured by the predetermined image data. Check if it matches the image data. Then, the personal identification unit 41 uses the image data obtained by photographing the matched family face as a key, and sets family information (for example, the family name and family one by one) stored in advance in the storage device. ID is identified to identify the individual.

  The facial expression recognition unit 42 is a program for estimating the smile level and age of a person shown in predetermined image data.

  The degree of smile is a numerical value indicating how much a person shown in predetermined image data is laughing. The facial expression recognizing unit 42 determines the eyes of the face in the predetermined image data, the state of the mouth corners, and the like, and estimates the smile level based on the determined result. It is assumed that the smile level in this embodiment is higher (laughing) as the smile level is higher.

  In addition, the facial expression recognition unit 42 determines the contour of the face shown in the predetermined image data, the positions of eyes, nose and mouth, and the like, and estimates the age based on the determined result.

  The psychological state estimation unit 43 is a program for estimating the psychological state of a person shown in predetermined image data. The contents of the calculation process by the psychological state estimation unit 43 will be described in detail later.

  The estimation result storage area 44 is a storage area for storing the estimation result of the psychological state, the image data used for estimating the psychological state, the date and time when the psychological state is estimated, and the like, and is secured in the storage device. The In the estimation result storage area 44, information relating the estimation result of the psychological state, image data, and the like is stored in folders (directories) 44a for each family.

  Next, the operation of the care system 10, that is, the flow of the psychological state estimation method will be described.

  First, as shown in FIGS. 1 and 2, the visitor presses the interphone 20 (call button provided on the entrance handset 21) (step S10). Thereby, the front cordless handset 21 of the interphone 20 photographs the visitor's face with the camera 21a.

  After the visitor presses the interphone 20, the care system 10 confirms whether the visitor is a family (step S20). At this time, the entrance cordless handset 21 of the interphone 20 creates image data obtained by photographing the face of the visitor, and inputs the image data to the personal identification unit 41 of the personal computer 40 via the master unit 22 or the like. The personal identification unit 41 identifies an individual from the input image data.

  If the visitor is not a family, that is, if the individual identification unit 41 cannot identify the individual, the care system 10 waits until the interphone 20 is pressed again without estimating the psychological state (step S20: NO).

  On the other hand, when the visitor is a family, that is, when the family comes home and the individual identification unit 41 can identify the individual, the care system 10 captures the face of the family who has come home with the camera 21a of the entrance terminal 21 a plurality of times. Then, a plurality of image data obtained by photographing the face of the family that has returned home is created (step S20: YES, step S30). In step S30, the entrance child device 21 inputs the plurality of created image data to the personal computer 40 via the parent device 22 or the like. The personal computer 40 acquires only the front-view image data (image data with the face facing the front with respect to the camera 21a) from the plurality of input image data and the image data input in step S20. The personal computer 40 acquires the image data of the front view as follows, for example.

  That is, the personal computer 40 measures the orientation of the face shown in the input image data, and the measurement result indicates the orientation of the face that can estimate the smile level and the age (for example, facing the camera 21a. If the state is 0 ° and the orientation of the face tilted within a range of ± 5 ° in the left-right direction), the image data is acquired. In this way, the personal computer 40 acquires a predetermined number of front-view image data, and inputs the acquired front-view image data to the facial expression recognition unit 42 as a plurality of image data 100 (see FIG. 3).

  The psychological state estimation method according to the present embodiment uses the camera 21a of the entrance cordless handset 21 to photograph the face of the family that has returned home, without using a device or the like that is separately provided for photographing the family face. The face in front of the family can be photographed (using equipment already installed in the building).

  After photographing the face of the family that has returned home, the care system 10 obtains one image data 100 appropriately extracted from the plurality of image data 100 acquired in step S30 from the personal computer 40 (not shown). , And the like (step S40). At this time, the care system 10 sends the image data 100 by the personal computer 40 as follows, for example.

  That is, the storage device of the personal computer 40 stores in advance information that associates family information with information (such as an email address) of a personal terminal that sends the image data 100. When the personal computer 40 proceeds to step S40, the personal computer 40 extracts the personal terminal that sends the image data 100 from the information that associates the family information identified in step S20 with the family information and the personal terminal information. To do. Then, the personal computer 40 sends an email with the image data 100 attached to the extracted personal terminal.

  After sending the image data 100, the care system 10 confirms whether or not there is a person (hereinafter referred to as “responder”) who responds to the visitor (family who has returned home) (step S50). At this time, for example, the care system 10 confirms whether or not there is an attendant by the master unit 22 and the personal computer 40 of the interphone 20 as follows.

  That is, the base unit 22 of the interphone 20 inputs a signal for notifying the personal computer 40 when it is operated by the respondent in step S50. The personal computer 40 determines that there is an attendant by confirming that the signal has been input. On the other hand, the personal computer 40 determines that there is no respondent if the signal is not input even after a preset time has elapsed since the transition to step S50.

  If it is determined that there is an attendant, the care system 10 waits until the front door D is unlocked by the attendant (step S50: YES, step S60). In other words, when the process proceeds from step S50 to step S60, the respondent confirms the visitor on the monitor 22a of the base unit 22 of the interphone 20, and automatically unlocks the door D via the control panel 32 or opens the entrance. Until the door D is unlocked manually. The electronic lock 31 inputs a signal notifying that the door D is unlocked to the personal computer 40. The care system 10 confirms that the door D has been unlocked thereby, and proceeds to step S80.

  On the other hand, when it is determined that there is no respondent, the care system 10 automatically unlocks the door D (step S50: NO, step S70). That is, the care system 10 unlocks the door D by inputting a signal for unlocking the door D from the personal computer 40 to the electronic lock 31, and proceeds to Step S80.

  The care system 10 according to the present embodiment is configured to automatically unlock the door D in step S <b> 70, so that the family that has returned home is aware that the camera 21 a of the front door device 21 faces the front. be able to. Moreover, the care system 10 can photograph a family face every time the door D is unlocked. That is, since the care system 10 can reuse the image data taken for unlocking the door D for estimation of the psychological state, it is not necessary to separately photograph the face data (or photograph the family in step S30). Can be reduced).

  As shown in FIGS. 1, 2, and 3, the care system 10 that has shifted to step S <b> 80 estimates the smile level and the age by the facial expression recognition unit 42. At this time, the personal computer 40 inputs the plurality of image data 100 acquired in step S30 to the facial expression recognition unit 42. The facial expression recognition unit 42 estimates a smile level for each of a plurality of input image data 100, and calculates a simple average of the smile level estimation results. The facial expression recognition unit 42 sets the simple average calculation result as the smile degree estimation result S. Similarly, the facial expression recognition unit 42 calculates a simple average of the age estimation results of the plurality of image data 100 and sets the calculation result as an age estimation result A.

  After estimating the smile level and the age, the care system 10 estimates the psychological state of the family who has returned home by the psychological state estimating unit 43 (step S90).

  Below, step S90 which is the process of estimating a psychological state is demonstrated in detail.

  When the process proceeds to step S <b> 90, the personal identification unit 41 inputs information on the family that has returned (identified) to the psychological state estimation unit 43.

  As illustrated in FIG. 4, the psychological state estimation unit 43 acquires the standard face estimated age S0 of the family that has returned home based on the family information input from the personal identification unit 41.

  The standard face estimated age S0 is an age set in advance for each family, and is stored in advance in the storage device of the personal computer 40 in association with family information. The standard face estimated age S0 is set by inputting the standard face image data SD in which the family shows a normal expression to the expression recognition unit 42 and estimating the age of the standard face image data SD by the expression recognition unit 42. . The psychological state estimation unit 43 acquires the estimated standard face age S0 of the family who has come home from information relating the family information and the estimated standard face age S0 using the family information as a key.

  When the process proceeds to step S90, as shown in FIGS. 1 and 3, the facial expression recognition unit 42 inputs the estimation result S of the smile level and the estimation result A of the age to the psychological state estimation unit 43.

  As shown in FIG. 3, FIG. 4 and FIG. 5, the psychological state estimation unit 43 subtracts the standard face estimated age S0 from the input age estimation result A to obtain the age estimation result A and the standard face estimated age. A difference from S0 (hereinafter referred to as “age difference”) is calculated. FIG. 5 is a graph showing the estimation result S of the smile degree and the estimation result A of the age in time series, and the value at the right end is the estimation result S of the smile degree and the estimation result of the age input in step S90. A.

  When the input age estimation result A is an age that is somewhat lower than the standard face estimation age S0 (when the age difference is a value that is somewhat smaller than 0), the family that has come home appears to be younger than usual. It is. On the other hand, when the input age estimation result A is an age that is somewhat higher than the standard face estimation age S0 (when the age difference is a value that is somewhat larger than 0), the family that has returned home is older than usual. Is visible.

  The psychological state estimator 43 uses the age difference calculation result and the smile degree estimation result S to estimate whether the family that has come home has a positive emotion or a negative emotion.

In the present embodiment, the positive emotion is an emotion that is held when a stress such as joy and happiness is not felt.
In this embodiment, the negative emotion is an emotion held when feeling stress such as sadness, anger and fear.

When the family that has come home has positive emotions, it tends to look young, so the age estimation result A tends to be low and the age difference tends to be smaller than zero. In this case, since the facial expression is likely to be bright, the smile degree estimation result S is likely to be a high value.
On the other hand, when the family that has returned home has negative emotions, it tends to be old, so the age estimation result A tends to be a high age and the age difference tends to be a value greater than zero. In this case, since the facial expression tends to be dark, the estimation result S of the smile level is likely to be a low value.

That is, as shown in FIG. 5 and FIG. 6, when the state of holding a positive emotion is changed to the state of holding a negative emotion, the estimation result S of the smile degree is likely to decrease and the age difference is more than zero. It tends to be a large value (see symbol C shown in FIG. 5 and region R4 shown in FIG. 6).
On the other hand, when the state of holding a negative emotion changes to the state of holding a positive emotion, the estimation result S of the smile level is likely to increase and the age difference tends to be a value smaller than 0 (the region shown in FIG. 6). R1).

  The psychological state estimation unit 43 estimates whether the family that has returned home has positive emotions or negative emotions using the above inverse correlation.

Specifically, the psychological state estimation unit 43 estimates that the family that has come home has a positive emotion when the estimation result S of the smile degree is a high value and the age difference is a value smaller than 0. (See region R1 shown in FIG. 6).
On the other hand, when the estimation result S of the smile level is a low value and the age difference is a value greater than 0, the psychological state estimation unit 43 estimates that the family that has returned home has negative emotions (FIG. 6). Area R4).

  In addition, the psychological state estimation unit 43 estimates that there is a possibility that the family that has come home has negative emotions when the family that has come home does not have such positive emotions and negative emotions. In such a case, the estimation result S of the smile level is a high value and the age difference is a value larger than 0, and the estimation result S of the smile level is a low value and the age difference is smaller than 0. It may be a value (see the regions R2 and R3 shown in FIG. 6).

  The psychological state estimation unit 43 estimates whether the family that has returned home as described above has positive emotions or negative emotions. Thereby, the psychological state estimation method can estimate the psychological state of the family such as the internal information of the family that changes due to the influence of stress or the like, that is, positive emotion and negative emotion.

  According to this, the psychological state estimation method can objectively determine the psychological state of the family from superficial information such as the degree of smile that can be estimated from the image data 100.

  Moreover, the psychological state estimation method and the care system 10 can estimate the psychological state of the family with high accuracy by estimating the psychological state based on the smile level and the age difference.

  The psychological state estimation method and care system 10 does not set the actual age as the standard face estimated age S0, but determines whether the age difference is larger or smaller than 0 based on the normal estimated age (standard face estimated age S0). Judging. Thereby, the psychological state estimation method and the care system 10 can more objectively determine the psychological state of the family.

  In addition, the psychological state estimation method and care system 10 obtains the standard face estimated age S0 based on the result of identification of the individual by the individual identification unit 41, so that the family returning to the personal computer 40 can manually process his / her information. The standard face estimated age S0 can be acquired (effectively) without inputting with.

  As shown in FIG.1 and FIG.2, after estimating a psychological state, the care system 10 preserve | saves the estimation result of a psychological state by the psychological state estimation part 43 (step S100). In step S100, the psychological state estimation unit 43 associates the psychological state estimation result with the input image data 100, the smile degree estimation result S, the age difference calculation result, the date and time when the psychological state is estimated, and the like. It is stored in the estimation result storage area 44. At this time, the psychological state estimation unit 43 stores the information in the folder 44a in the estimation result storage area 44 corresponding to the family that has returned home.

  After storing the estimation result of the psychological state, the care system 10 confirms whether the alert definition is met by the personal computer 40 (step S110). For example, the personal computer 40 checks whether or not the alert definition is satisfied as follows.

  That is, the personal computer 40 checks whether the smile degree estimation result S and the age difference calculation result are included in the regions R1 to R4 shown in FIG. If the psychological state estimation results are included in the regions R2 to R4, the personal computer 40 determines that the alert definition is met. On the other hand, if it is not included in the areas R2 to R4, the personal computer 40 determines that it does not correspond to the alert definition.

  If it is determined that the alert definition is met, the care system 10 sends an alert and a reminder from the personal computer 40 to the personal terminal T (step S110: YES, step S120). For example, the care system 10 sends alerts and alerts by the personal computer 40 as follows.

That is, the storage device of the personal computer 40 stores in advance information that associates family information with information (such as an email address) of the personal terminal T that sends alerts and alerts. When the personal computer 40 proceeds to step S110, the personal computer T extracts the personal terminal T that sends alerts and alerts from the information that associates the family information with the information of the personal terminal T using the identified family information as a key. To do. Then, the personal computer 40 sends an email describing the alert and alerting to the extracted personal terminal T (mobile terminal in this embodiment).
The personal computer 40 sends an alert when the psychological state estimation result is included in the region R4. Further, the personal computer 40 sends an alert when the estimation result of the psychological state is included in the regions R2 and R3.

  The specific contents (text) of such an alert include “the family who came home may have mental stress”. In addition, the specific contents of the alert include “the family who came home may be doing something unreasonable”, “the family who came home may have anxiety”, and the like. The contents of the alert and alert are stored in advance in the storage device in association with the psychological state estimation result, and are appropriately extracted by the personal computer 40 in step S120.

  The care system 10 can send an alert and alert regarding the psychological state of the child who has returned home to the parents' personal terminal T by performing step S120 as described above. Thereby, care system 10 can tell parents about deterioration of a child's psychological state. Therefore, the care system 10 can assist communication between the parents and the child, and can promptly notice the deterioration of the child's psychological state.

  On the other hand, when it is determined that the alert definition is not met, the care system 10 sends a periodic report from the personal computer 40 to the personal terminal T (step S110: NO, step S130). Even in this case, the personal computer 40 extracts the personal terminal T that sends the periodic report using the information of the family that has returned home as a key, as in step S120, and sends an email describing the periodic report to the extracted personal terminal T. send.

  Specific contents of such a periodic report include “the family who has come home is in a relaxed state”. The content of the periodic report is stored in advance in the storage device in association with the psychological state estimation result, and is appropriately extracted by the personal computer 40 in step S130.

  Thus, the care system 10 ends its operation.

  The care system 10 according to the present embodiment associates the standard face estimated age S0 with the standard face image data SD, family information, actual age, date of birth, and date on which the standard face estimated age S0 is set. It is stored in advance in 40 storage devices (see FIG. 4). The personal computer 40 according to the present embodiment corrects the standard face estimated age S0 as appropriate based on the date of birth and the date on which the standard face estimated age S0 is set.

  Specifically, the personal computer 40 determines the standard face estimated age when the birthday is reached after setting the standard face estimated age S0 or when one year has passed since the date when the standard face estimated age S0 is set. Correction is made so that the age obtained by adding 1 year old to S0 becomes the standard face estimated age S0. The personal computer 40 performs such processing regularly (for example, once a month).

  Thereby, even if the care system 10 does not reset the standard face estimated age S0 for a long period of time, it can prevent the age difference from fluctuating due to the influence of aging. That is, since the age difference changes only when the psychological state changes, the care system 10 can estimate the psychological state more objectively.

  As described above, in the psychological state estimation method according to the present embodiment, the target person is acquired based on the image acquisition step (steps S20 and S30) of acquiring the image data 100 by photographing the face of the target person. A psychological state estimating step (steps S20, S80, and S90) for estimating a psychological state related to positive emotions and negative emotions.

  With this configuration, the psychological state of the target person can be objectively determined.

  The psychological state estimation step (steps S20, S80, and S90) is based on a smile level estimation step (step S80) that estimates the smile level of the target person based on the image data 100, and the image data 100. The age estimation step (step S80) for estimating the age of the target person is compared with the age estimation result A and the preset standard face estimated age S0 (reference estimated age) of the target person to determine the age difference. A calculation step (step S90) to calculate, and a composite estimation step (step S90) to estimate the psychological state of the target person based on the estimation result S of the smile degree and the calculation result of the age difference, Is.

  With this configuration, the psychological state of the target person can be estimated with high accuracy.

  In addition, the psychological state estimation method according to the present embodiment captures the target person's standard face image data SD (reference image data) in advance, and the target person's standard based on the captured standard face image data SD. A setting step for setting the estimated face age S0 is further provided.

  With this configuration, the psychological state of the target person can be estimated with high accuracy.

  The psychological state estimating step (steps S20, S80, and S90) is set in advance based on a personal identification step (step S20) for identifying an individual based on the image data 100 and the result of identifying the individual. An acquisition step (step S90) of acquiring the standard face estimated age S0 of the identified individual from the standard face estimated age S0 of all target persons.

  By comprising in this way, the psychological state of a target person can be estimated efficiently.

  In the image acquisition step (steps S20 and S30), the face of the target person is photographed by using a camera 21a attached to the entrance terminal 21 of the interphone 20.

  With this configuration, the target person's face can be efficiently photographed.

  In addition, the care system 10 according to the present embodiment estimates a smile level of the target person based on the camera 21a (imaging unit) that acquires the image data 100 by capturing the face of the target person and the image data 100. Smile degree estimation means, age estimation means for estimating the age of the target person based on the image data 100, the estimated result A of the age and the standard face estimated age S0 of the target person set in advance (reference estimated age) ) To calculate the age difference, and based on the smile degree estimation result S and the age difference calculation result, the psychological state estimation unit 43 estimates the psychological state regarding the positive emotion and negative emotion of the target person. (Psychological state estimation means).

  With this configuration, the psychological state of the target person can be objectively determined.

  In addition, the care system 10 according to the present embodiment is a care system 10 using a psychological state estimation method, and performs notification based on a result of estimation of the psychological state of the target person to a person other than the target person. A computer 40 (notification means) is provided.

  With this configuration, it is possible for a person other than the target person to grasp that the target person has a negative emotion.

  In addition, the care system 10 according to the present embodiment includes an electronic lock device 30 (automatic opening) that automatically unlocks the door D of the entrance where the entrance slave unit 21 is provided based on the result of photographing using the camera 21a. A locking means).

  With this configuration, the target person's face can be efficiently photographed.

The standard face estimated age S0 according to the present embodiment is an embodiment of the reference estimated age according to the present invention.
The standard face image data SD according to the present embodiment is an embodiment of image data serving as a reference according to the present invention.
Moreover, the care system 10 which concerns on this embodiment is one Embodiment of the psychological state estimation system which concerns on this invention.
Moreover, the camera 21a of the interphone 20 according to the present embodiment is an embodiment of the photographing means according to the present invention.
Moreover, the facial expression recognition unit 42 according to the present embodiment is an embodiment of a smile degree estimation unit and an age estimation unit according to the present invention.
Moreover, the psychological state estimation part 43 which concerns on this embodiment is one Embodiment of the psychological state estimation means which concerns on this invention.
Moreover, the personal computer 40 according to the present embodiment is an embodiment of the notification means according to the present invention.
The electronic lock device 30 according to the present embodiment is an embodiment of the automatic unlocking means according to the present invention.

  The embodiment of the present invention has been described above, but the present invention is not limited to the above-described configuration, and various modifications can be made within the scope of the invention described in the claims.

  For example, in the present embodiment, the psychological state of the family who has returned home in the building where the family lives is estimated, but the environment (situation) for estimating the psychological state in the present invention is not limited to this, The psychological state of a person other than the family may be estimated at a place other than the building where the family resides. That is, the psychological state estimation method and the psychological state estimation system according to the present invention may estimate the psychological state of a company employee at the entrance of the company, or may estimate the psychological state of the driver at the driver's seat of the automobile.

  Moreover, although the psychological state estimation method according to the present embodiment estimates the psychological state based on the smile degree estimation result S and the age difference calculation result, the psychological state estimation method according to the present invention is limited to this. Not. The psychological state estimation method according to the present invention may estimate the psychological state based on one of the smile degree estimation result S and the age difference calculation result. Moreover, the psychological state estimation method according to the present invention may estimate the psychological state based on the age estimation result (without calculating the age difference).

  Moreover, although the mental state estimation method and care system 10 according to the present embodiment estimate the smile level based on the state of the eyes and the corner of the mouth, the means for estimating the smile level according to the present invention is limited to this. It is not what is done.

  Moreover, although the psychological state estimation method and care system 10 according to the present embodiment estimate the age based on the contour of the face, the positions of the eyes, nose and mouth, etc., the age according to the present invention is estimated. The means is not limited to this.

  In addition, the psychological state estimation method according to the present invention may estimate a change in the psychological state of a family member who has returned home using a past smile degree estimation result S or the like. In this case, the psychological state estimation method changes from a state in which the emotion of the family that has returned home has a negative emotion to a state in which a positive emotion is held when the estimation result S of the smile level changes from a low value to a high value. It is estimated that In addition, the psychological state estimation method changes from a state in which a family emotion that has returned home has a positive emotion to a state in which a negative emotion is held when the estimation result S of the smile level changes from a high value to a low value. It is estimated that

  In the psychological state estimation method and the care system 10 according to the present embodiment, the simple average of the smile level estimation results of the plurality of image data 100 is set as the smile level estimation result S, but the smile level estimation according to the present invention is performed. The result is not limited to this. The smile level estimation results according to the present invention are the smiles excluding the median (median), the largest multiple (for example, 5) smile levels, and the smallest multiple (for example, 5) smile levels of the plurality of image data 100. It may be a simple average of degrees. The same applies to the age estimation result.

  Moreover, although the psychological state estimation method and care system 10 which concern on this embodiment shall acquire multiple image data 100, the psychological state estimation method and psychological state estimation system which concern on this invention are not limited to this. . The psychological state estimation method and the psychological state estimation system according to the present invention may acquire one image data 100 and estimate a psychological state.

  The standard face estimated age S0 according to the present embodiment is set based on the estimation result of the age of the face shown in the standard face image data SD, but the standard face estimated age according to the present invention is limited to this. It is not what is done. The standard face estimated age according to the present invention may be set based on the actual age.

  In the psychological state estimation method according to the present embodiment, an individual is automatically identified by the individual identification unit 41. However, the psychological state estimation method according to the present invention is not limited to this. The psychological state estimation method according to the present invention may be such that the family member who returns home inputs information identifying the individual to the personal computer 40. In this case, the family who has returned home inputs information for identifying the individual via, for example, the input unit of the electronic lock 31.

  Moreover, although the personal terminal T according to the present embodiment is a portable terminal, the personal terminal according to the present invention is not limited to this. The personal terminal according to the present invention may be a personal computer installed on the go.

  Moreover, although the care system 10 which concerns on this embodiment shall send an alert, alerting, and a regular report by email, the care system which concerns on this invention is not limited to this. The care system according to the present invention may be configured to indirectly send an alert or the like by uploading the alert or the like to a cloud server that can be used by sharing data with the personal terminal T. The same applies to the case where image data is sent in step S40.

  In addition, the care system 10 according to the present embodiment is configured to send the estimation result of the psychological state to the personal terminal T regardless of the state where the family that has returned home has a positive emotion and a negative emotion. The care system according to the present invention is not limited to this. The care system according to the present invention may send the estimation result of the psychological state to the personal terminal T only when the family that has returned home has a negative emotion.

  Moreover, although the psychological state estimation method and care system 10 according to the present embodiment acquire the image data 100 using the camera 21a of the front door device 21 of the interphone 20, the psychological state estimation method according to the present invention and The psychological state estimation system is not limited to this. The psychological state estimation method and the psychological state estimation system according to the present invention may be a monitoring camera that images the front of the entrance and a camera that is separately installed on the door D in order to acquire the image data 100 necessary for estimating the psychological state. good.

  Further, although the care system 10 according to the present embodiment estimates the psychological state using the personal computer 40, the psychological state estimation system according to the present invention is not limited to this. The psychological state estimation system according to the present invention may estimate the psychological state using a mobile terminal. The same applies to estimation of smile level, estimation of age, and calculation of age difference.

  In the care system 10 according to the present embodiment, the interphone 20, the electronic lock device 30, and the personal computer 40 are electrically connected by a LAN cable or the like. It is not limited. The care system according to the present invention may be electrically connected using wireless communication means.

  In this embodiment, the estimation result storage area 44 is secured in the storage device of the personal computer 40. However, the present invention is not limited to this, and the Internet is connected to the personal computer 40. It may be secured in a storage device of a cloud server that can transmit and receive data through a line.

  In this embodiment, the psychological state estimation result is stored in the individual folder 44a. However, the present invention is not limited to this, and the psychological state estimation result is stored in one folder 44a. You may remember.

DESCRIPTION OF SYMBOLS 10 Care system 20 Intercom 30 Electronic lock device 40 Personal computer 100 Image data A Age estimation result S Smile degree estimation result

Claims (7)

An image acquisition step of acquiring the image data by photographing the face of the target person;
A psychological state estimating step of estimating a psychological state related to positive emotion and negative emotion of the target person based on the image data;
Equipped with,
The psychological state estimation step includes
A smile level estimation step of estimating a smile level of the target person based on the image data;
An age estimation step of estimating the age of the target person based on the image data;
A calculation step of calculating an age difference by comparing the estimation result of the age with a preset reference estimated age of the target person;
Based on the estimation result of the smile degree and the calculation result of the age difference, a composite estimation step of estimating the psychological state of the target person,
Comprising
Psychological state estimation method. It further includes a setting step of capturing image data serving as a reference of the target person in advance and setting a reference estimated age of the target person based on the captured reference image data.
The psychological state estimation method according to claim 1. The psychological state estimation step includes
A personal identification step of identifying an individual based on the image data;
Based on the result of identifying the individual, an acquisition step of acquiring the reference estimated age of the identified individual from the reference estimated age of all target persons set in advance;
Further comprising
The psychological state estimation method according to claim 1 or 2. The image acquisition step includes
Photographing the face of the target person using a camera attached to the front door of the interphone,
Psychological state estimation method as claimed in any one of claims 1 to 3.   Photographing means for photographing the face of the target person and acquiring image data;
  Smile level estimation means for estimating the smile level of the target person based on the image data;
  Age estimation means for estimating the age of the target person based on the image data;
  An age difference is calculated by comparing the estimation result of the age with a preset reference estimated age of the target person, and based on the estimation result of the smile degree and the calculation result of the age difference, the target person's positive Psychological state estimating means for estimating a psychological state related to emotion and negative emotion;
  Comprising
  Psychological state estimation system.   A care system using the psychological state estimation method according to any one of claims 1 to 4,
  Notification means for performing notification based on the estimation result of the psychological state of the target person to a person other than the target person,
  Care system. A care system using the psychological state estimation method according to claim 4,
An automatic unlocking means for automatically unlocking the door of the entrance where the entrance cordless handset is provided based on a result photographed using the camera ;
Care system.

Images