JP2014013448A - Unpleasantness degree estimation system and unpleasantness degree estimation method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an unpleasantness degree estimation system and an unpleasantness degree estimation method capable of accurately estimating the unpleasantness degree of a person.SOLUTION: The unpleasantness degree estimation system includes: environment information detection means for detecting information on the environmental change of the indoor space of a room 10; unpleasant expression detection means for detecting unpleasant expressions (frown and blink) shown by a person 20 who exists in the indoor space when the environment of the indoor space changes; and unpleasantness degree determination means for constructing a Bayesian network by preliminarily accumulating information on the frequency of the unpleasant expressions shown by the person 20 when the environment of the indoor space changes and information on how far the person 20 actually feels unpleasant. The unpleasantness degree determination means estimates how far the person 20 feels unpleasant from the Bayesian network by using the information on the environmental change of the indoor space detected by the environment information detection means and the frequency of the unpleasant expressions shown by the person 20 detected by the unpleasant expression detection means when the environment changes.

Description

  The present invention relates to a technique of a discomfort degree estimation system and a discomfort degree estimation method capable of estimating how much a person is uncomfortable.

  2. Description of the Related Art Conventionally, a technique in which a lighting device is controlled based on information about a space environment (particularly, the illuminance of light from a lighting fixture) to make the illuminance of the space a desired illuminance is publicly known. For example, as described in Patent Document 1.

In the technique described in Patent Document 1, a lighting fixture having a large influence on the illuminance measured at a desired place in a space (for example, a place where a person exists) is selected, and the lighting fixture is controlled to increase or decrease in a predetermined order. By doing so, the illuminance at the desired location can be controlled to the desired illuminance. At this time, by setting the illuminance that a person will generally feel comfortable as the desired illuminance in advance, it is possible to provide a comfortable lighting environment for the person.
That is, in this case, when the illuminance reaches the desired illuminance, it is estimated that the person feels comfortable.

  However, in actuality, the illuminance that is felt comfortably varies depending on the person, so even if the illuminance in the space can be controlled to a desired illuminance, the illuminance is not necessarily comfortable for the person. That is, accurately estimate how comfortable a person feels (in other words, how uncomfortable they feel (discomfort level)) with only environmental information (light intensity in the above example). Was difficult.

JP 2008-16291 A

  The present invention has been made in view of the above situation, and a problem to be solved is to provide a discomfort degree estimation system and a discomfort degree estimation method capable of accurately estimating a person's discomfort degree. It is.

  The problem to be solved by the present invention is as described above. Next, means for solving the problem will be described.

  That is, according to claim 1, environmental information detecting means for detecting information related to an environmental change in the indoor space, and an unpleasant facial expression for detecting an unpleasant facial expression that occurs when the person existing in the indoor space changes in the environmental environment of the indoor space. Discomfort in which a Bayesian network is constructed by pre-accumulating detection means and information on the frequency of unpleasant facial expressions generated by the person when the environment of the indoor space changes and how much the person is actually uncomfortable at that time Degree judgment means, and the discomfort degree judgment means is detected by the unpleasant expression detection means at the time of the environment change, and information on the environmental change of the indoor space detected by the environment information detection means. The frequency of the unpleasant facial expression that the person emits is used to estimate how unpleasant the person is from the Bayesian network.

  According to a second aspect of the present invention, the unpleasant facial expression detection means includes face frowning detection means for detecting whether or not the person has frowned.

  According to a third aspect of the present invention, the environmental information detection means includes illuminance detection means for detecting the illuminance of light irradiated on the person.

  According to a fourth aspect of the present invention, the unpleasant facial expression detection means includes blink detection means for detecting whether or not the person has blinked.

  In Claim 5, the environmental information detection process which detects the information regarding the environmental change of indoor space, and the unpleasant facial expression detection process which detects the unpleasant facial expression which the person who exists in the said indoor space emits at the time of the environmental change of the said indoor space And the information on the environmental change of the indoor space detected by the environmental information detection means, and the frequency of the unpleasant facial expression emitted by the person detected by the unpleasant facial expression detection means at the time of the environmental change, The frequency of the unpleasant facial expression that the person emits when the environment of the indoor space changes, and how much the person is from the Bayesian network constructed by pre-accumulating information on how uncomfortable the person is An uncomfortable degree determination step for estimating whether it is unpleasant.

  The present invention has an effect that it is possible to accurately estimate the degree of discomfort of a person by using the frequency of an unpleasant facial expression that a person emits in addition to information on environmental changes.

The schematic diagram which showed the structure of the discomfort degree estimation system. The figure which showed the structure of the Bayesian network which concerns on this embodiment. The figure which showed the judgment standard of the grimace by visual observation. The flowchart which showed the learning process. The figure which showed a mode that the organ of the face was converted into a coordinate. The figure which showed the coordinate used as the reference | standard when normalizing. The figure which showed a mode that a coordinate was converted into 5-dimensional distance information. The flowchart which showed the imaging process and the frown determination process. The figure which showed a mode that a coordinate was converted into two-dimensional distance information.

  Below, the structure of the discomfort degree estimation system 30 which concerns on this embodiment is demonstrated using FIG.

  The discomfort degree estimation system 30 estimates how much the person 20 existing in the room 10 installed as experimental equipment is uncomfortable (discomfort degree). The discomfort level estimation system 30 mainly includes a camera 31, a ceiling light 32, a window light 33, a discomfort level measurement dial 34, and a control device 35.

  The camera 31 serving as an imaging unit captures the face of the person 20. The camera 31 is placed in front of a person 20 who is seated and performing a predetermined work, and can always capture the face of the person.

  The ceiling lighting 32 is a lighting device for the indoor space of the room 10. The ceiling illumination 32 is arranged on the ceiling of the room 10 and above the person 20. The illumination by the ceiling lighting 32 is configured to be changeable.

  The window illumination 33 is a luminaire for reproducing a window provided in the room 10 in a pseudo manner. The window illumination 33 is disposed on the side wall surface of the room 10 and in front of the person 20. The brightness of the window illumination 33 is configured to be changeable. By causing the window illumination 33 to emit light, a state in which sunlight is inserted from a window provided in the room 10 can be simulated.

  The discomfort degree measurement dial 34 is for measuring how discomfort the person 20 is (discomfort degree). The discomfort measurement dial 34 is constituted by a dial-type switch, and is disposed in a range that can be reached by a person 20. The discomfort level measurement dial 34 is provided with a scale indicating the discomfort level within a predetermined numerical range. The person 20 operates the discomfort measurement dial 34 so that the numerical value (scale) indicated by the discomfort measurement dial 34 becomes a value corresponding to the degree of discomfort felt by the person 20.

  The control device 35 controls the illuminance and luminance of the ceiling illumination 32 and the window illumination 33 based on various information, and performs predetermined arithmetic processing and storage. The control device 35 includes a storage unit such as a RAM and a ROM, an arithmetic processing unit such as a CPU, and the like.

The control device 35 is connected to the camera 31 and can acquire image data captured by the camera 31.
The control device 35 is connected to the ceiling illumination 32 and can control the illuminance by the ceiling illumination 32 to be an arbitrary illuminance.
The control device 35 is connected to the window illumination 33 and can control the luminance of the window illumination 33 to be an arbitrary luminance.
The control device 35 is connected to the discomfort level measurement dial 34 and can detect a numerical value (scale) indicated by the discomfort level measurement dial 34.

  The discomfort degree estimation system 30 configured as described above includes an unpleasant expression detection means, an environment information detection means, and a discomfort degree determination means. Below, each said means is demonstrated.

The unpleasant expression detecting means detects an unpleasant expression (in this embodiment, frowning and blinking) that occurs when the person 20 existing in the indoor space of the room 10 changes the environment of the indoor space.
The unpleasant facial expression detection means according to this embodiment is mainly composed of a camera 31 and a control device 35.

  Further, the unpleasant facial expression detecting means includes a frown detection means (face fringe detection system) and a blink detection means.

The frown detection means (face fringe detection system) detects whether or not the person 20 has frowned.
The frowning detection means (face fringe detection system) according to the present embodiment is mainly composed of a camera 31 (imaging means) and a control device 35 (face fringe determination means described later).
A method of detecting the frowning of the person 20 by the frowning detection means (face fringe detection system) will be described later.

The blink detection means detects whether or not the person 20 blinks.
The blink detection unit according to the present embodiment is mainly configured by the camera 31 and the control device 35.
A method for detecting the blink of the person 20 by the blink detection means will be described later.

The environmental information detection means detects information related to the environmental change in the indoor space of the room 10 (in this embodiment, the illuminance of light irradiated on the person 20 from the ceiling lighting 32 and the luminance change of the window lighting 33). It is.
The environmental information detection means according to the present embodiment is mainly composed of a ceiling illumination 32, a window illumination 33, and a control device 35.

  Furthermore, the environmental information detection means includes illuminance detection means and luminance detection means.

The illuminance detection means detects the illuminance of light emitted from the ceiling illumination 32 to the person 20.
The illuminance detection means according to the present embodiment is mainly composed of the ceiling illumination 32 and the control device 35.
Since the control device 35 controls itself so that the illuminance of the light irradiated to the person 20 from the ceiling illumination 32 becomes an arbitrary illuminance, the illumination by the ceiling illumination 32 is detected without using a separate sensor or the like. be able to.

The brightness detecting means detects the brightness of the window illumination 33.
The luminance detection unit according to the present embodiment is mainly configured by the window illumination 33 and the control device 35.
Since the control device 35 controls itself so that the brightness of the window illumination 33 becomes an arbitrary brightness, the brightness of the window illumination 33 can be detected without using a separate sensor or the like.

The discomfort degree judging means estimates how much the person 20 is uncomfortable (discomfort degree).
The discomfort degree determination unit according to the present embodiment is mainly configured by the control device 35.
A method for estimating the discomfort level of the person 20 by the discomfort level determination means will be described later.

  Below, the method of estimating the discomfort degree of the person 20 by the discomfort degree estimation system 30 is demonstrated using FIG.2 and FIG.3.

  In the discomfort degree determination means (that is, the control device 35) provided in the discomfort degree estimation system 30, a Bayesian network constructed as shown in FIG. 2 is stored in advance, and the Bayesian network is used. The discomfort level of the person 20 is estimated. A Bayesian network is a model of probabilistic reasoning in which a causal relationship between a plurality of variables is represented by an acyclic directed graph and a relationship between individual variables is represented by a conditional probability. In the figure, the node on the start point side of the arrow indicates the cause, and the node on the end point side indicates the result. Also, the degree of influence of the cause on the result is quantified with a conditional probability.

  As shown in FIG. 2, the Bayesian network according to the present embodiment includes a frown recognition frequency node N101, a frown expression frequency node N102, a blink recognition frequency node N201, a blink expression frequency node N202, and illuminance ( It is constructed by a node N301 before the ceiling) change, a node N401 before the change in luminance (window), a node N501 after the change in illuminance (ceiling), a node N601 after the change in luminance (window), and a discomfort level node N701.

  In the Bayesian network according to this embodiment, the control device 35 controls the ceiling illumination 32 and the window illumination 33 to change a plurality of patterns of illumination changes (the illuminance due to the ceiling illumination 32 and the luminance of the window illumination 33 are large). Is constructed based on the result of the learning experiment that gives (change), and is stored in the control device 35. Specifically, the probabilities (prior probabilities or conditional probabilities) of the above nodes are obtained based on the learning experiment and stored in the control device 35.

  Below, each node of the Bayesian network which concerns on this embodiment is demonstrated.

  The frown recognition frequency node N101 is a node representing the frown frequency of the person 20. The frowning of the person 20 (whether or not the face is frowned) is detected by the frowning detection means (face frowning detection system). In the learning experiment, whether or not the person 20 is frowning is detected by the face fringing detection means, and the frequency of fringing of the person 20 (frequing frequency) is calculated by the control device 35. The frequency of the frowning is classified into a plurality of numerical ranges, and the probability (prior probability) that the frequency of frowning is included in each numerical range is stored in the control device 35.

The frowning expression frequency node N102 is a node representing the fringing frequency of the actual person 20. In the learning experiment, whether or not the person 20 actually grimaced is confirmed by visual observation or the like, and the frequency of grimacing (frequency of grimacing) of the actual person 20 is calculated.
Here, in the learning experiment, for example, as shown in FIG. 3, the eye E is narrowed (FIG. 3 (b)) with respect to the expression (FIG. 3 (a)) in which the face is not frowned. If the eye E is narrowed by bringing B close (FIG. 3 (c)), the eye E is narrowed by pulling the eyebrow B, and the mouth M is pulled sideways (FIG. 3 (d)), the person 20 faces It is judged that it was frowned.
The calculated frequency of frowning is classified into a plurality of numerical ranges, and the probability (conditional probability) that the frequency of the actual frowning caused by the face fraud recognition frequency node N101 is included in each numerical range is as follows: It is stored in the control device 35.

  The blink recognition frequency node N201 is a node representing the blink frequency of the person 20. The blink of the person 20 is detected by the blink detection means. In the learning experiment, whether or not the person 20 blinks is detected by the blink detection means, and the blinking frequency of the person 20 is calculated by the control device 35. The blink frequency is classified into a plurality of numerical ranges, and the probability (prior probability) that the blink frequency is included in each numerical range is stored in the control device 35.

  The blinking expression frequency node N202 is a node representing the blinking frequency of the actual person 20. In the learning experiment, whether or not the person 20 actually blinks is confirmed by visual observation or the like, and the blink frequency of the actual person 20 (blink frequency) is calculated. The actual blink frequency is classified into a plurality of numerical ranges, and the probability (conditional probability) that the actual blink frequency caused by the blink recognition frequency node N201 is included in each numerical range is stored in the control device 35. Is done.

  The illuminance (ceiling) pre-change node N301 is a node representing the illuminance before the illuminance by the ceiling illumination 32 changes significantly. The illuminance is detected by the illuminance detection means. In the learning experiment, the illuminance before the illuminance due to the ceiling illumination 32 is largely changed is classified into a plurality of numerical ranges, and the probability (prior probability) that the illuminance before the significant change is included in each numerical range is given to the control device 35. Remembered.

  The node N401 before change in luminance (window) is a node representing the luminance before the luminance of the window illumination 33 changes greatly. The brightness is detected by the brightness detecting means. In the learning experiment, the luminance before the luminance of the window illumination 33 is largely changed is classified into a plurality of numerical ranges, and the probability that the luminance before the large change is included in each numerical range (prior probability) is given to the control device 35. Remembered.

  The illuminance (ceiling) post-change node N501 is a node representing the illuminance after the illuminance by the ceiling illumination 32 has changed significantly. The illuminance is detected by the illuminance detection means. In the learning experiment, the illuminance after the illuminance due to the ceiling illumination 32 is greatly changed is classified into a plurality of numerical ranges, and the probability (prior probability) that the illuminance after the significant change is included in each numerical range is given to the control device 35. Remembered.

  The post-change luminance (window) node N601 is a node that represents the luminance after the luminance of the window illumination 33 has changed significantly. The brightness is detected by the brightness detecting means. In the learning experiment, the luminance after the luminance of the window lighting 33 is largely changed is classified into a plurality of numerical ranges, and the probability (prior probability) that the luminance after the large change is included in each numerical range is given to the control device 35. Remembered.

  The discomfort level node N701 is a node representing how much the person 20 is uncomfortable (discomfort level). In the learning experiment, how much the person 20 is actually uncomfortable (discomfort level) is measured. Specifically, in the learning experiment, the control device 35 detects the numerical value (scale) of the discomfort measurement dial 34 operated by the person 20 when the illuminance by the ceiling lighting 32 and the brightness of the window lighting 33 change greatly. Thus, it is possible to measure how unpleasant (discomfort level) the person 20 is. The measured discomfort level is classified into a plurality of numerical ranges, such as a frowning expression frequency node N102, a blinking expression frequency node N202, an illuminance (ceiling) pre-change node N301, a luminance (window) pre-change node N401, and an illuminance ( A probability (conditional probability) that the measured discomfort level caused by the node N501 after the change in the ceiling and the node N601 after the change in the brightness (window) is included in each numerical value range (conditional probability) is stored in the control device 35.

  As described above, the environment of the indoor space of the room 10 has changed using the Bayesian network constructed in advance by the learning experiment (specifically, the illuminance by the ceiling lighting 32 and the luminance of the window lighting 33 have changed greatly). ) The unpleasant degree of the person 20 can be estimated as a probability theory.

Specifically, first, when the illuminance by the ceiling illumination 32 in the room 10 and the luminance of the window illumination 33 change, the environmental information detection unit detects information related to the change. That is, the environmental information detection means (the illuminance detection means and the luminance detection means) has high illuminance before the illuminance by the ceiling illumination 32 changes significantly (node N301 before illuminance (ceiling) change) and the luminance of the window illumination 33. The luminance before the change (node N401 before luminance (window) change), the illuminance after the illuminance by the ceiling illumination 32 changes greatly (node N501 after the illuminance (ceiling) change), and the luminance of the window illumination 33 after the change greatly The brightness (node N601 after brightness (window) change) is detected.
Hereinafter, this process is referred to as an environmental information detection process.

Further, the unpleasant expression detected by the person 20 when the environment of the indoor space of the room 10 (illuminance by the ceiling illumination 32 and luminance of the window illumination 33) changes is detected by the unpleasant expression detection means. That is, the unpleasant facial expression detection means (the frown detection means and the blink detection means) detects the frown and blink of the person 20 when the environment of the indoor space of the room 10 changes.
Hereinafter, this process is referred to as an unpleasant facial expression detection process.

  Next, the frequency (frequency) of the unpleasant facial expression which the person 20 detected by the unpleasant facial expression detection means emits is calculated by the unpleasant degree judgment means. That is, the discomfort degree determination means (control device 35) is configured to detect the fraud frequency of the person 20 detected by the frown detection means (face fraud recognition frequency node N101) and the person 20 detected by the blink detection means. The blink frequency (blink recognition frequency node N201) is calculated.

Then, the discomfort degree determination means (control device 35) is configured so that each of the above nodes (face fraud recognition frequency node N101, blink recognition frequency node N201, illuminance (ceiling) pre-change node N301, luminance (window) pre-change node N401, illuminance Using the conditions of the (ceiling) post-change node N501 and the luminance (window) post-change node N601), the conditional probability of the frowning expression frequency node N102, the blinking expression frequency node N202, and the discomfort degree node N701 A plurality of values (numerical value ranges) that the discomfort level can take and respective probabilities are calculated. The discomfort degree determination means (control device 35) multiplies each of a plurality of possible values of the discomfort level of the person 20 by the probability of the value, and further estimates the numerical value obtained by adding them as the discomfort degree of the person 20 To do.
Hereinafter, this process is referred to as a discomfort degree determination process.

  As described above, the discomfort degree estimation system 30 according to the present embodiment uses the frequency of the unpleasant facial expression generated by the person 20 in addition to the information on the environmental change, and therefore can accurately estimate the discomfort degree of the person 20. .

  In the following, a method for detecting the frowning of the person 20 by the frowning detection means (face fringing detection system) used in the above-described unpleasant facial expression detection step will be described with reference to FIGS.

  As described above, the frown detection means (face fringe detection system) detects whether or not the person 20 has frowned (whether there is an expression). The frown detection means (face fringe detection system) according to the present embodiment is mainly configured by an image pickup means (camera 31) and a face fringe determination means (control device 35).

  An identification function for identifying whether or not the face is frowned by learning the position information of the organ of the face when the person 20 frowns and when the face is frowned. Whether or not the person 20 imaged by the camera 31 frowns from the identification function created in advance and the positional information of the organ of the face of the person 20 imaged by the imaging means (camera 31). (Whether or not there is an expression).

First, using FIG. 3 to FIG. 7, whether or not the control device 35 frowns by learning the position information of the organ of the face when the person 20 frowns and when the face is frowned. A process of creating an identification function for identifying these will be described.
This process (from step S101 to step S106 in FIG. 4) is hereinafter referred to as a learning process.
The learning step is performed (in advance) before determining whether the person 20 has frowned (whether there is an expression).

In step S101 of FIG. 4, the control device 35 acquires a plurality of image data of the face (expression) of the person 20 captured by the camera 31 for the learning.
Specifically, the control device 35 changes the illuminance by the ceiling illumination 32 and the luminance of the window illumination 33 into a plurality of patterns. At this time, the control device 35 acquires a plurality of facial expression image data when the person 20 declares unpleasant (glare) and a plurality of facial expression image data when the person 20 does not declare unpleasant (glare). To do.
Note that the presence or absence of a report indicating that the person 20 is uncomfortable (dazzling) can be determined using the discomfort degree measurement dial 34 or other switches.
After performing the process of step S101, the control device 35 proceeds to step S102.

In step S102, the plurality of image data of the facial expression of the person 20 acquired by the control device 35 is classified into two types.
Specifically, a plurality of image data of the facial expression of the person 20 acquired in step S101 are (A) an image that is declared that the person 20 is uncomfortable and that is visually frowned, ( B) Two types of images (the above (A)): an image that is declared that the person 20 is uncomfortable and that is visually confirmed to be frowned, or an image that is not declared that the person 20 is unpleasant Or (B)).
Whether or not the person 20 is frowning is confirmed by the experimenter's visual observation, and data related to the classification (A) or (B) is input (stored) into the control device 35 by the experimenter.
Here, the determination as to whether or not the face is frowned is made based on the change in facial expression as shown in FIG.
In the classification, image data of facial expressions when the person 20 blinks is omitted. Also, the same number of image data is classified into (A) and (B), and as much image data as possible is classified.
After performing the process of step S102, the control device 35 proceeds to step S103.

In step S <b> 103, the control device 35 converts the facial organs (eyes, eyebrows, and nose) of the person 20 appearing in each acquired image data into coordinates as position information.
Specifically, as shown in FIG. 5, among the faces of the person 20 shown in each acquired image data, the end point located on the innermost side of the right eye is on the point P1 and the upper eyelid. The point located (more specifically, the end point located at the top of the upper eyelid) is the point P2 and the point located above the lower eyelid of the right eye (more specifically, located at the bottom of the lower eyelid) Point P3, the innermost end point of the right eyebrow is the point P4, the innermost end point of the left eye is the point P5, and the upper eyelid is on the upper eyelid A point (more specifically, a point on the uppermost end of the upper eyelid) is a point P6 and a point located on the lower eyelid of the left eye (more specifically, the lowest point of the lower eyelid) Point P7, the innermost end point of the left eyebrow is the point P8, and the lowermost end point of the nose A point P9, to determine, respectively.
The coordinate conversion (determination from the point P1 to the point P9) is executed by a program stored in the control device 35 in advance.
After performing the process of step S103, the control apparatus 35 transfers to step S104.

In step S <b> 104, the control device 35 normalizes the coordinates of the facial organ of the person 20 appearing in each acquired image data.
Specifically, first, the most ideal frontal face image data is selected as the reference image data from among the plurality of acquired image data. Next, as shown in FIG. 6, a total of three points including the coordinates of the inside of both eyes and the lower end of the nose (points P1, P5, and P9) are selected as reference coordinates. Then, the three points (reference coordinates) are compared with the three points (points P1, P5, and P9) in the other image data, and the coordinates of the other image data are normalized. That is, using affine transformation or the like, the influence (measurement error) exerted on the coordinates (point P1 to point P9) by the difference in the face direction and angle of the person 20 in the other image data is canceled.
After performing the process of step S104, the control apparatus 35 transfers to step S105.

In step S105, the control device 35 converts each acquired image data into five-dimensional position information (distance information).
Specifically, as shown in FIG. 7, the control device 35 converts the normalized coordinates of each image data into five pieces of distance information, that is, (1) the distance from the point P1 to the point P4, and (2) the point. The distance is converted into the distance from P5 to point P8, (3) the distance from point P4 to point P8, (4) the distance from point P2 to point P3, and (5) the distance from point P6 to point P7.
After performing the process of step S105, the control device 35 proceeds to step S106.

In step S106, the control device 35 creates an identification function for identifying whether or not the person 20 has frowned (whether or not there is an expression).
Specifically, the control device 35 first creates a database of the five-dimensional distance information converted in step S105 according to the classification (above (A) or (B)) in step S102. Next, using the database as training data, an identification function for identifying whether the person 20 has frowned is created. As the identification function, a support vector machine polynomial kernel method or the like can be used.

  In this way, the control device 35 learns the position information of the organ of the face when the person 20 frowns and when the face is frowned, and whether or not the person 20 frowns (there is an expression) Or not) is created (from step S101 to step S106).

  Next, using FIG. 4 and FIG. 8, the control device 35 compares the position information of the facial organ of the person 20 learned in advance with the positional information of the facial organ of the person 20 captured by the camera 31. The manner in which it is determined whether or not the person 20 imaged by the camera 31 frowns (whether or not there is an expression) will be described.

In step S201 of FIG. 8, the control device 35 obtains image data of the face (expression) of the person 20 to be determined as to whether or not the face imaged by the camera 31 has been grimaced (whether there is an expression). get.
Hereinafter, the step S201 is referred to as an imaging step.
After performing the process of step S201 (imaging process), the control device 35 proceeds to step S202.

In step S202, the control device 35 coordinates the facial organs (eyes, eyebrows, nose) of the person 20 shown in the acquired image data.
The coordinate method is the same as that in step S103 of FIG.
After performing the process of step S202, the control device 35 proceeds to step S203.

In step S203, the control device 35 normalizes the coordinates of the facial organ of the person 20 shown in the acquired image data.
The normalization method is the same as that in step S104 in FIG.
After performing the process of step S203, the control device 35 proceeds to step S204.

In step S204, the control device 35 converts the acquired image data into five-dimensional distance information.
The method for converting to the distance information is the same as that in step S105 in FIG.
After performing the process of step S204, the control device 35 proceeds to step S205.

In step S205, the control device 35 determines whether or not the person 20 has frowned (whether or not there is an expression).
Specifically, the control device 35 applies the five-dimensional distance information converted in step S204 to the identification function created in step S106 in FIG. Or not).
Hereinafter, the step S205 is referred to as a frown determination step.

  In this way, the control device 35 determines whether or not the person imaged by the camera 31 has frowned from the identification function created in advance and the position information of the organ of the person's face imaged by the camera 31. (From step S201 to step S205).

  Hereinafter, a method for detecting the blink of the person 20 by the blink detection unit used in the above-described unpleasant facial expression detection process will be described with reference to FIGS. 4, 8, and 9.

  As described above, the blink detection means detects whether or not the person 20 has blinked. The blink detection unit according to the present embodiment is mainly configured by the camera 31 and the control device 35.

  First, the control device 35 acquires the image data of the face (expression) of the person 20 to be determined as to whether or not the blink captured by the camera 31 has been made.

Next, the control device 35 converts the facial organs (eyes, eyebrows, nose) of the person 20 shown in the acquired image data into coordinates as position information.
The coordinate method is the same as that in step S202 of FIG.

Next, the control device 35 normalizes the coordinates of the facial organ of the person 20 shown in the acquired image data. For the normalization, the reference coordinates selected in step S104 in FIG. 4 can be used.
The normalization method is the same as that in step S203 in FIG.

  Next, the control device 35 converts the acquired image data into two-dimensional position information (distance information). Specifically, as shown in FIG. 9, the control device 35 determines the normalized coordinates of the image data as two pieces of distance information, that is, (1) the distance from the point P2 to the point P3 and (2) the point P6. To the distance from point P7.

  Finally, the control device 35 identifies whether or not the converted two-dimensional distance information is zero (or a value close to zero). If the two-dimensional distance information is zero (or a value close to zero), it can be determined that the person 20 is blinking.

  In this way, the control device 35 can detect whether or not the person 20 has blinked from the two-dimensional distance information.

As described above, the discomfort level estimation system according to the present embodiment includes the environment information detection unit that detects information related to the environmental change in the indoor space of the room 10 and the person 20 existing in the indoor space. Unpleasant facial expression detection means for detecting an unpleasant facial expression (face frowning and blinking), frequency of unpleasant facial expression produced by the person 20 when the environment of the indoor space changes, and how much the person 20 actually does Discomfort level determination means that builds a Bayesian network by preliminarily storing information on whether or not it is uncomfortable, and the discomfort level determination means relates to an environmental change of the indoor space detected by the environment information detection means From the Bayesian network, using information and the frequency of the unpleasant facial expression generated by the person 20 detected by the unpleasant facial expression detection means during the environmental change One in which 20 to estimate how much it is uncomfortable.
As described above, by using the frequency of unpleasant facial expressions generated by a person in addition to information on environmental changes, the degree of unpleasantness of the person can be accurately estimated.

Further, the unpleasant facial expression detection means includes a frown detection means for detecting whether or not the person 20 has frowned.
With this configuration, it is possible to detect a frown that is likely to be generated by the person 20 when the surroundings become bright (blur), as an unpleasant expression.

The environmental information detection means includes illuminance detection means for detecting the illuminance of light irradiated on the person 20.
By comprising in this way, the discomfort degree of the person 20 by the change of the illumination intensity of light can be detected.

The unpleasant facial expression detection means includes blink detection means for detecting whether or not the person has blinked.
By configuring in this way, it is possible to detect a blink that is considered to be generated by the person 20 particularly when the surroundings become bright (dazzling) or dark, as an unpleasant facial expression.

Also, the discomfort level estimation method according to the present embodiment includes an environment information detection step for detecting information related to an environmental change in the indoor space, and an unpleasant expression that a person existing in the indoor space emits when the environmental change in the indoor space occurs. Uncomfortable facial expression detection step, information on environmental change of the indoor space detected by the environmental information detection means, and unpleasant facial expression generated by the person detected by the unpleasant facial expression detection means at the time of the environmental change Bayesian constructed by pre-accumulating information on the frequency of unpleasant facial expressions that the person emits when the environment of the indoor space changes, and how much the person is actually uncomfortable at that time An uncomfortable degree determination step of estimating how unpleasant the person is from the network.
As described above, by using the frequency of unpleasant facial expressions generated by a person in addition to information on environmental changes, the degree of unpleasantness of the person can be accurately estimated.

Further, as described above, the frowning detection means (face fringing detection system) according to the present embodiment includes the camera 31 (imaging means) that picks up the face of the person 20 to be determined whether or not the face is frowned. An identification function for identifying whether or not the person 20 is frowned by learning positional information of the organ of the face when the person 20 frowns and when the face is frowned is created in advance, and the created identification function And face fringe judging means (control device 35) for judging whether or not the person 20 imaged by the camera 31 is frowned from the position information of the organ of the face of the person 20 imaged by the camera 31. Then, the face freckling determination means uses the distance information from the inner edge of the right eyebrow to the inner edge of the right eye (from point P1 to point P4) as the position information of the facial organ, and from the inner edge of the left eyebrow to the left eye. To the inner edge of (from point P5 P8), eyebrow distance (from point P4 to point P8), right eye from upper eyelid to lower eyelid (from point P2 to point P3), and left eye upper eyelid to lower eyelid (from point P6 to point) Distance to P7).
With this configuration, since only five distances are used as the position information of the facial organ, it can be detected in a short time whether or not the person 20 is frowning.

In addition, after acquiring the position information of the facial organs, the frowning determination means determines the inner end (point P1) of the right eye, the inner end (point P5) of the left eye, and the lower end (point P9) of the nose. As a reference, the position information of the facial organ is normalized.
With this configuration, it is possible to reduce an error in the position information of the facial organ due to the difference in the orientation of the face of the imaged person 20.

Further, when determining whether or not the person 20 imaged by the camera 31 is frowning, the face frowning determination means uses an identification function created from position information of the facial organ of the same person as the person 20. Is.
Thus, since the identification function created based on the data of the person 20 itself is used, it is possible to detect more accurately whether the person 20 is frowning.

Further, the face fraud detection method according to the present embodiment identifies whether or not the face 20 is frowned by learning position information of the organ of the face when the person 20 frowns and when the face is frowned. A learning process for creating an identification function in advance, an imaging process for imaging a face of a person 20 to be judged whether or not a face is grimaced, an identification function created in advance, and the face of the person 20 imaged in the imaging process A frown determination step for determining whether or not the person 20 imaged in the imaging step is frowning from the position information of the organ of the right side of the right eyebrow as the position information of the organ of the face Distance from the inner part to the inner edge of the right eye (from point P1 to point P4), distance from the inner edge of the left eyebrow to the inner edge of the left eye (from point P5 to point P8), between the eyebrows (from point P4 to point P8) Distance), from the upper eyelid to the lower eye It is to use the distance from the upper eyelid of the distance and the left eye in Tama (from point P2 to point P3) to the lower eyelid (from the point P6 to the point P7).
With this configuration, since only five distances are used as the position information of the facial organ, it can be detected in a short time whether or not the person 20 is frowning.

In the present embodiment, the room 10 installed as experimental equipment has been described as an example. However, the present invention is not limited to this, and the above-mentioned frowning is performed using a room used in daily life. It is also possible to perform detection and estimation of discomfort.
In the present embodiment, the camera 31 is used as the imaging means, but the present invention is not limited to this. For example, a camera or the like built in a personal computer on which the person 20 is working can be used as the imaging means.
Further, although the window illumination 33 is used in the present embodiment, the present invention is not limited to this, and a normal window (a window through which light outside the room 10 enters) and a sensor that detects the luminance of the window are used. It is also possible to detect the above-mentioned frowning and estimate the degree of discomfort using.
In the present embodiment, the unpleasant facial expression detection means detects frowning and blinking as an unpleasant facial expression generated by the person 20, but the present invention is not limited to this. That is, other unpleasant facial expressions (for example, closing eyes for a long time, tearing, etc.) may be detected.
In the present embodiment, the environment information detection means detects the illuminance of light irradiated to the ceiling lighting 32 persons 20 and the luminance of the window lighting 33 as information related to the environmental change of the indoor space of the room 10. However, the present invention is not limited to this. That is, other environmental changes (for example, temperature change, humidity change, etc.) can be detected.

10 rooms 20 people 30 discomfort estimation system 31 camera 32 ceiling lighting 33 window lighting 34 discomfort measurement dial 35 control device

Claims (5)

Environmental information detection means for detecting information on environmental changes in the indoor space;
Unpleasant facial expression detection means for detecting an unpleasant facial expression that occurs when a person existing in the indoor space changes the environment of the indoor space;
Discomfort level judging means for constructing a Bayesian network by preliminarily storing information on the frequency of an unpleasant facial expression generated by the person at the time of an environmental change of the indoor space and how much the person is actually uncomfortable at that time ,
Comprising
The discomfort level judging means
The Bayesian network using the information on the environmental change of the indoor space detected by the environmental information detecting means and the frequency of the unpleasant facial expression generated by the person detected by the unpleasant facial expression detecting means at the time of the environmental change. Estimating how unpleasant the person is from
Discomfort estimation system. The unpleasant facial expression detecting means is
Including a frowning detection means for detecting whether or not the person has frowned;
The discomfort degree estimation system according to claim 1. The environmental information detection means includes
Including illuminance detection means for detecting the illuminance of the light irradiated to the person,
The discomfort degree estimation system according to claim 1 or 2. The unpleasant facial expression detecting means is
Including blink detection means for detecting whether or not the person has blinked,
The discomfort degree estimation system according to any one of claims 1 to 3. An environmental information detection process for detecting information on environmental changes in the indoor space;
An unpleasant facial expression detection step for detecting an unpleasant facial expression that occurs when a person existing in the indoor space changes the environment of the indoor space;
Using information on the environmental change of the indoor space detected by the environmental information detection means, and the frequency of the unpleasant facial expression emitted by the person detected by the unpleasant facial expression detection means at the time of the environmental change,
What is the person from the Bayesian network constructed by pre-accumulating information on the frequency of unpleasant facial expressions that the person emits when the environment of the indoor space changes and how uncomfortable the person is actually at that time? An uncomfortable degree determination process for estimating whether the degree of unpleasantness,
A discomfort level estimation method comprising:

Images