JP2017012730A - Human state estimation method and human state estimation system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve accuracy of estimation of a human state.SOLUTION: A human state estimation method for estimating a human state which is a state of a human is configured to: acquire (S101) a warm/cold sense index indicating a warm/cold sense of a human in a regulated range; determine (S102) whether or not the acquired warm/cold sense index is in a prescribed range of the regulated range; and estimate (S103) the human state, based on a physiological amount which is acquired when it is determined that the acquired warm/cold sense index is in the prescribed range, and in which activity of an autonomic nerve system is reflected.SELECTED DRAWING: Figure 11

Description

  The present invention relates to a human state estimation method and a human state estimation system.

  It is said that a state (hereinafter also referred to as a human state) such as stress or sleepiness felt by a person is related to a physiological amount based on the activity of the person's autonomic nerve. Examples of physiological quantities based on autonomic nerve activity include skin temperature, variations in heartbeat intervals, and respiratory waveforms. For example, regarding the skin temperature as a physiological quantity, when a person feels stress, the sympathetic nerve among the autonomic nerves is enhanced, the peripheral blood vessels contract and the blood flow decreases, This is explained by the mechanism that the skin temperature decreases. In addition, with regard to sleepiness as a physiological quantity, the peripheral sympathetic nerves in the autonomic nerves are enhanced to relax the peripheral blood vessels and increase the blood flow, thereby increasing the peripheral skin temperature. Explained. A dozing detection device using this fact is disclosed (see Patent Document 1 and Patent Document 2).

JP-A-9-154835 JP 2002-120591 A

  However, since the activity of the autonomic nerve is affected by effects other than stress or drowsiness, further studies are required to accurately estimate the human state such as the degree of stress or drowsiness.

  Therefore, the present invention provides a human state estimation method and the like that improve the accuracy of human state estimation.

  A human state estimation method according to an aspect of the present invention is a human state estimation method for estimating a human state which is a human state, wherein the estimated thermal sensation of the person is indexed within a specified range. A feeling index is acquired, it is determined whether the acquired thermal sensation index is within a predetermined range of the specified range, and it is determined that the acquired thermal sensation index is within the predetermined range. The human state is estimated on the basis of the physiological quantity acquired at the time, and the physiological quantity reflecting the activity of the person's autonomic nerve.

  Note that these comprehensive or specific modes may be realized by a system, method, integrated circuit, computer program, or recording medium such as a computer-readable CD-ROM, and the system, method, integrated circuit, computer. You may implement | achieve with arbitrary combinations of a program and a recording medium.

  The human state estimation method of the present invention improves the accuracy of human state estimation.

FIG. 1A is a conceptual diagram of an automobile on which the human state estimation device according to Embodiment 1 is mounted. FIG. 1B is a schematic view seen from the upper surface of the automobile on which the human state estimation device according to Embodiment 1 is mounted. FIG. 2 is an explanatory diagram illustrating an example of a human face image taken by the thermal image sensor according to the first embodiment. FIG. 3 is a schematic configuration diagram of the thermal sensation estimator in the first embodiment. FIG. 4 is an explanatory diagram showing an example of the thermal sensation index in the first embodiment. FIG. 5 is a block diagram illustrating the function of the thermal sensation estimation unit in the first embodiment. FIG. 6 is a block diagram illustrating functions related to human state estimation in the first embodiment. FIG. 7A is an explanatory diagram illustrating an example of a case where the thermal sensation is 2 or more with respect to temporal variation of the thermal sensation, the outside air temperature, and the nasal skin temperature in the first embodiment. FIG. 7B is an explanatory diagram showing an example of a thermal sensation in the vicinity of 0 with respect to temporal variation in thermal sensation, outside air temperature, and nasal skin temperature in the first embodiment. FIG. 8 is an explanatory diagram showing an example of a sleepiness index in the first embodiment. FIG. 9A is an explanatory diagram illustrating an example in which the thermal sensation is −2 or less with respect to temporal variation of the thermal sensation, the outside air temperature, and the nasal skin temperature in the first embodiment. FIG. 9B is an explanatory diagram illustrating an example of a thermal sensation in the vicinity of 0 with respect to temporal variation of the thermal sensation, the outside air temperature, and the nasal skin temperature in Embodiment 1. FIG. 10 is a block diagram illustrating functions of the human state estimation device according to the first embodiment. FIG. 11 is a flowchart showing a human state estimation method by the human state estimation device in the first embodiment. FIG. 12A is an explanatory diagram of nasal skin temperature measurement in the first embodiment. FIG. 12B is an explanatory diagram of nostril temperature fluctuations in nasal skin temperature measurement in the first exemplary embodiment. FIG. 12C is a schematic diagram illustrating a pulse wave measurement method according to Embodiment 1. FIG. 12D is a schematic diagram illustrating a pulse wave measurement method according to Embodiment 1. FIG. 13A is a conceptual diagram of an automobile on which the human state estimation device according to Embodiment 2 is mounted. FIG. 13B is a schematic view of a person wearing the human state estimation device according to Embodiment 2 as viewed from the front. FIG. 14A is an explanatory diagram illustrating an example of a measured pulse wave in the second embodiment. FIG. 14B is an explanatory diagram of time-series fluctuations in the heartbeat interval obtained from the pulse wave shown in FIG. 14A. FIG. 15A is an explanatory diagram showing a frequency component of a heartbeat interval and a small amount of HF component in the second embodiment. FIG. 15B is an explanatory diagram showing a frequency component of a heartbeat interval and a large amount of HF components in the second embodiment. FIG. 16 is a correlation diagram illustrating an example of the correlation between the earlobe skin temperature and the thermal sensation in the second embodiment. FIG. 17 is a block diagram illustrating functions of the human state estimation device according to the second embodiment. FIG. 18 is a flowchart showing a human state estimation method by the human state estimation device in the second embodiment. FIG. 19A is an explanatory diagram of a respiratory frequency according to the processing load reduction method using the respiratory component in the second embodiment. FIG. 19B is a block diagram illustrating wavelet transform according to the processing load reduction method using the respiratory component in the second embodiment. FIG. 20 is a flowchart showing a human state estimation method to which the processing load reduction method using the respiratory component in the second embodiment is applied. FIG. 21 is a block diagram illustrating a function of correcting the human state determination result in the second embodiment.

(Knowledge that became the basis of the present invention)
Human organs are not actively controlled by human intentions, but are controlled via the autonomic nerves by the center of the brain based on information from receptors throughout the body. Autonomic nerves are composed of two systems, sympathetic nerves and parasympathetic nerves, and both sympathetic nerves and parasympathetic nerves often control one organ.

  For example, regarding the heart rate of the heart, when the sympathetic nerve is increased, the heart rate increases and the variation in the heart rate interval is reduced. On the other hand, when the parasympathetic nerve is increased, the heart rate decreases and the variation in the heart rate interval increases. In peripheral blood vessels, when the sympathetic nerve is enhanced, the blood vessels contract and blood flow is inhibited. As a result, the peripheral skin temperature decreases. On the other hand, when parasympathetic nerves are enhanced, blood vessels relax and blood flow is improved, resulting in an increase in the skin temperature at the peripheral portion.

  Peripheral skin temperature decreases under stress by increasing sympathetic nerves to collect blood flow deep in the body, and by increasing heart rate, a large amount of blood flows to the brain and muscles to supply oxygen As a result, it is thought to activate brain and muscle activity. In addition, the increase in peripheral skin temperature due to sleepiness is because it is necessary to lower the deep body temperature slightly during sleep, so the parasympathetic nerves are enhanced to spread the blood flow to the peripheral part, thereby reducing the body temperature. It is thought to make it easier to dissipate heat.

  The enhancement of the sympathetic nerve or the parasympathetic nerve is influenced by the thermal sensation (hot or cold sensation felt by a person) in addition to the effects of stress and sleepiness described above. For example, when a person feels hot, it is necessary to promote heat dissipation from the body, so that parasympathetic nerves are enhanced to relax peripheral blood vessels. As a result, the heart rate decreases and the variation in the heartbeat interval increases. Conversely, when a person feels cold, it is necessary to suppress heat dissipation from the body, so the sympathetic nerves are enhanced in order to contract peripheral blood vessels. As a result, the heart rate increases and the variation in the heartbeat interval also decreases. Therefore, when estimating a human condition such as stress or sleepiness based on a physiological quantity based on autonomic nerve activity such as skin temperature or heartbeat interval, the physiological quantity is influenced by stress or sleepiness. It is necessary to determine whether it is due to the influence of fluctuations in thermal sensation.

  In order to solve such a problem, a human state estimation method according to an aspect of the present invention is a human state estimation method for estimating a human state that is a human state, and the estimated thermal sensation of the person The thermal sensation index that is indexed within the specified range is acquired, it is determined whether the acquired thermal sensation index is within a predetermined range of the specified range, and the acquired thermal sensation index Is the physiological quantity acquired when it is determined that the human condition is within the predetermined range, and the human state is estimated based on the physiological quantity reflecting the activity of the autonomic nerve of the person.

  According to this, it is possible to estimate the human state with higher accuracy by reducing the false detection during the human state estimation. Specifically, in the human state estimation method according to one aspect of the present invention, the human state is estimated from the physiological quantity under the condition that the acquired thermal sensation information is within a predetermined range. Therefore, it is possible to reduce erroneous detection in estimating the human state, compared to estimating the human state regardless of whether this condition is satisfied. Therefore, the human state estimation method can improve the accuracy of human state estimation. Furthermore, by reducing the false detection described above, it is possible to obtain the effects of reducing the processing amount and processing load and reducing power consumption, such as eliminating the need to perform detection processing again.

  For example, the predetermined range is a part of the specified range including a thermal neutral point related to thermal sensation.

  According to this, the human state is estimated when the thermal sensation of the person is within a range relatively close to the thermal neutral point that the person does not feel hot or cold. When a person feels hot or warm, the skin temperature rises by promoting heat dissipation from the body. On the other hand, when a person feels cold or cool, the skin temperature is lowered by suppressing heat dissipation from the body. That is, when the thermal sensation of a person is within a range relatively close to the thermal neutral point, there is no effect of promotion or suppression of heat dissipation from the human body, or a relatively small state. Therefore, in such a case, by estimating the human state from the physiological amount, the influence of the promotion or suppression of heat dissipation from the human body included in the estimation result can be reduced, and detection error and estimation accuracy can be reduced. Contribute to improvement.

  For example, the predetermined range is a partial range that does not include the point indicating the hottest feeling as the thermal sensation and the point indicating the coldest as the thermal sensation.

  According to this, the human state is estimated when the human thermal sensation is in a range excluding the time when the person feels very hot or very cold. When a person feels very hot, heat dissipation from the person's body is greatly promoted. Also, when a person feels very cold, heat dissipation from the person's body is greatly suppressed. By excluding such cases from the case of estimating the human state, the influence of the promotion or suppression of heat dissipation from the human body included in the estimation result can be reduced, and the detection accuracy and estimation accuracy can be reduced. Contribute to improvement.

  For example, the physiological amount is the nasal skin temperature of the person, and the human state includes the degree of sleepiness of the person.

  According to this, when estimating the degree of sleepiness of a person based on the nose skin temperature of the person, it is possible to reduce the influence of promotion or suppression of heat dissipation from the human body included in the estimation result. . Further, by using the nasal skin temperature, even when there is a disturbance, it can be made less susceptible to the influence of the disturbance.

  For example, in the human state estimation method, the thermal sensation index and the nasal skin temperature are acquired a plurality of times, and when the determination is made, each of the acquired thermal sensation indices is within the predetermined range. In the estimation of the human state, the degree of sleepiness of the person is estimated based on the increase widths of the plurality of acquired nasal skin temperatures over time.

  According to this, a person's sleepiness degree can be specifically estimated based on a person's nose skin temperature.

  For example, the physiological amount includes a heartbeat interval of the person, and the human state estimation method estimates the human state based on the fluctuation of the heartbeat interval as the acquired physiological amount.

  According to this, the human state can be specifically estimated based on the person's heartbeat interval.

  For example, the physiological amount includes the respiratory waveform of the person, and the human state estimation method estimates the human state further based on the acquired respiratory waveform as the acquired physiological amount.

  According to this, a human state can be specifically estimated based on a person's respiratory waveform. It can also contribute to reduction of processing load and speeding up of processing.

  For example, when acquiring the skin temperature of the earlobe of the person and acquiring the thermal sensation index, the correlation between the skin temperature of the earlobe and the thermal sensation index is used to obtain the acquired temperature of the earlobe part. When acquiring the thermal sensation index estimated from skin temperature and acquiring the physiological quantity, a pulse wave measured from the earlobe of the person is acquired as the physiological quantity, and the human state is estimated In this case, the human state is estimated based on the obtained frequency analysis of the pulse wave.

  According to this, the human state can be specifically estimated based on the skin temperature and pulse wave of the human earlobe. The earlobe has a feature that it is easy to measure a pulse wave. Therefore, by acquiring the pulse wave from the earlobe and acquiring the skin temperature as well, information necessary for estimating the human state can be collectively acquired from the earlobe.

  For example, in the human state estimation method, the thermal sensation index estimated by PMV (Predicted Mean Vote) is acquired.

  For example, the specified range is expressed by a PMV seven-level evaluation scale, and the predetermined range is a range in which the PMV value in the specified range is −2 or more and +2 or less.

  According to this, the thermal sensation of a person can be estimated more accurately from the temperature, humidity, airflow, radiation, amount of clothes, and amount of activity, which are the six elements of heat, by estimation using PMV.

  For example, the physiological amount is the person's nasal skin temperature, the human state includes a degree of stress of the person, and in the human state estimation method, the acquired plurality of the obtained It is determined whether each of the thermal sensation index is within the predetermined range, and when estimating the human state, based on the descending width with the passage of time of the plurality of acquired nasal skin temperature, Estimating the degree of stress of the person.

  According to this, the person's stress level can be specifically estimated based on the person's nose skin temperature.

  For example, the physiological amount is the person's skin blood flow, blood pressure, or pulse wave propagation time, and the person state includes the degree of sleepiness of the person.

  According to this, the degree of sleepiness of a person can be specifically estimated based on the person's skin blood flow, blood pressure, or pulse wave propagation time.

  Moreover, the human state estimation system according to one aspect of the present invention is a human state estimation system that estimates a human state that is a human state, and the estimated thermal sensation of the person is indexed within a specified range. A thermal sensation estimation unit that acquires a thermal sensation index, a determination unit that determines whether the acquired thermal sensation index is within a predetermined range of the specified range, and the acquired thermal sensation A human state estimation unit that estimates a human state based on a physiological amount that is acquired when an index is determined to be within the predetermined range and reflects an activity of the human autonomic nerve; Prepare.

  According to this, there exists an effect similar to said human state estimation method.

  Moreover, the program which concerns on 1 aspect of this invention is a program which makes a computer perform the said human state estimation method.

  According to this, there exists an effect similar to said human state estimation method.

  Note that these comprehensive or specific modes may be realized by a system, method, integrated circuit, computer program, or recording medium such as a computer-readable CD-ROM, and the system, method, integrated circuit, computer. You may implement | achieve with arbitrary combinations of a program or a recording medium.

  Hereinafter, embodiments will be specifically described with reference to the drawings.

  It should be noted that each of the embodiments described below shows a comprehensive or specific example. The numerical values, shapes, materials, constituent elements, arrangement positions and connecting forms of the constituent elements, steps, order of steps, and the like shown in the following embodiments are merely examples, and are not intended to limit the present invention. In addition, among the constituent elements in the following embodiments, constituent elements that are not described in the independent claims indicating the highest concept are described as optional constituent elements.

(Embodiment 1)
In the present embodiment, a human state estimation method and a human state estimation device that improve the accuracy of human state estimation will be described. As described above, the human state is a concept including the degree of human sleepiness or the degree of stress. The human state estimation device is sometimes referred to as a human state estimation system.

  An example in which the human state estimation device of the present embodiment is mounted on an automobile will be described with reference to FIGS. 1 to 12D.

  FIG. 1A is a conceptual diagram of an automobile on which the human state estimation device according to the present embodiment is mounted. FIG. 1B is a schematic view seen from the upper surface of the automobile on which the human state estimation device according to the present embodiment is mounted.

  A person 102 is in the driver's seat of the automobile 100. The automobile 100 is equipped with a thermal image sensor 101 facing the person 102 in front of the driver's seat, and can photograph the heat distribution between the face of the person 102 and its surroundings two-dimensionally. The thermal image sensor 101 is a device in which elements having sensitivity to infrared rays, such as a bolometer or a thermopile, are arranged in a two-dimensional matrix, and the amount of infrared rays emitted according to the temperature distribution on the object surface is arranged in the matrix by a lens. The temperature distribution on the object surface can be visualized by forming an image in a shape.

  FIG. 2 is an explanatory diagram illustrating an example of a human face image taken by the thermal image sensor 101 according to the present embodiment.

  When there is a positional relationship between the thermal image sensor 101 and the person 102 as shown in FIG. 1, for example, a thermal image as shown in FIG. 2 is taken. In the thermal image shown in FIG. 2, the portion (pixel) where the temperature of the object is higher is displayed so that the density becomes higher. That is, in the thermal image shown in FIG. 2, a pixel having a higher temperature is displayed in a color closer to black, and it can be recognized that the temperature around the forehead is higher. The display of the thermal image is not limited to this. Further, an image acquired by photographing with the thermal image sensor 101 may be a still image or a moving image.

  FIG. 3 is a schematic configuration diagram of the thermal sensation estimator 108 in the present embodiment. FIG. 4 is an explanatory diagram showing an example of the thermal sensation index in the present embodiment. FIG. 5 is a block diagram showing the function of the thermal sensation estimation unit 108 in the present embodiment. The thermal sensation estimation unit 108 disposed inside the automobile 100 will be described with reference to these drawings.

  The thermal sensation of a person can be quantified using, for example, a seven-stage index as shown in FIG. Here, the thermal sensation of 0 is a hot neutral point that is neither hot nor cold, and it takes a positive and large absolute value as it gets hot. On the other hand, it becomes a negative and large absolute value as it gets colder. It is quantified to take. Such a quantification (indexation) of thermal sensation is called thermal sensation index. The thermal sensation index may be simply a thermal sensation within a range that does not cause confusion.

  PMV associates this thermal sensation with the six thermal elements. Therefore, knowing the temperature, humidity, radiation, airflow, amount of human activity, and amount of clothes that are the six elements of heat, it is possible to estimate the thermal sensation of the person using the PMV calculation formula. It has been. Hereinafter, a method for obtaining the thermal sensation from the PMV will be specifically described. It should be noted that the index of human thermal sensation can be set at 9 levels by adding “+4 very hot” and “−4 very cold” to the 7 levels shown in FIG.

  As shown in FIG. 3, the thermal sensation estimation unit 108 includes a camera 103, a globe thermometer 104a, a thermometer 104b, an anemometer 105a provided near the louver 105b, a hygrometer 107, and And a thermal sensation estimation control unit 106 connected to.

  The thermal sensation estimation unit 108 is a processing unit that acquires a thermal sensation index obtained by indexing the estimated thermal sensation of the person 102 within a specified range. Specifically, the thermal sensation estimation unit 108 acquires a thermal sensation index estimated by PMV. In this case, the specified range is expressed by a PMV seven-level evaluation scale, and the predetermined range is a range in which the PMV value in the seven steps is −2 or more and +2 or less. Hereinafter, an example is shown in which the thermal sensation estimation unit 108 acquires the thermal sensation index by estimating the thermal sensation index. However, the thermal sensation estimation unit 108 uses the thermal sensation index estimated by another device or the like as the other You may acquire from the apparatus of this.

  The configuration of the thermal sensation estimation unit 108 will be described with reference to FIG.

  The temperature in the six elements of heat can be obtained from the thermometer 104b.

  Radiation of the six elements of heat can be obtained from the globe thermometer 104a and the thermometer 104b. The globe thermometer is a thermometer in which a glass thermometer is inserted into a copper ball having a black surface. Since the globe thermometer measures not only the ambient temperature but also the temperature including ambient radiation using a copper ball coated in black, the effect of radiation is determined from the difference in measured values between the globe thermometer 104a and the thermometer 104b. Can be estimated.

  The airflow among the six elements of heat is measured by the anemometer 105. The anemometer 105 corresponds to the anemometer 105a.

  Humidity of the six elements of heat is measured by a hygrometer 107.

  The amount of clothing among the six elements of heat can be obtained by analyzing the image of the person 102 photographed by the camera 103. That is, the clothing amount estimation unit 106a of the thermal sensation estimation control unit 106 obtains an image of the person 102 taken by the camera 103 by calculation. For example, the clothing amount estimation unit 106a obtains the area between the clothing part and the exposed part, that is, the skin part from the image photographed by the camera 103, thereby determining the ratio between the part without the clothing of the person 102 and the part with the clothing. The amount of clothing may be obtained, the amount of clothing may be obtained from the ratio of the wrist as an exposed portion to the arm portion with clothing, or the temperature difference between the clothing portion and the exposed portion. This is because when the amount of clothing is low, the temperature of the clothing surface tends to be close to the body surface temperature. Of course, the amount of clothes may be obtained by other means, or may be entered and entered by the person himself / herself.

  Of the six elements of heat, the amount of activity of a person is close to sitting in a chair when driving a car, and is usually considered to be about 1.1 mets without any particular measurement. Even when the human state estimation device is used in a place other than a car, for example, the camera 103 shoots a moving image of the person 102, specifies the human area of each image, and determines the amount of activity from the fluctuation amount of the human area. It is also possible to estimate.

  By the above method, the six thermal elements related to the person 102 can be extracted. The method for obtaining the six elements of heat shown here is merely an example, and the present invention is not limited to this.

  Next, with respect to the PMV calculation unit 106b of the thermal sensation estimation control unit 106, the clothing amount obtained by the clothing amount estimation unit 106a, the temperature and radiation obtained from the globe thermometer 104a and the thermometer 104b, and the anemometer 105 Is input to the PMV calculation unit 106b, and the PMV calculation unit 106b calculates the thermal sensation of the person 102 from the PMV equation. The operation mechanism of the thermal sensation estimation unit 108 has been described above.

  Next, regarding the human state estimation method, here, a method for estimating using the nasal skin temperature will be described.

  FIG. 6 is a block diagram showing functions relating to human state estimation in the present embodiment.

  This function is realized by the thermal image sensor 101, the physiological quantity acquisition unit 109A, and the human state estimation unit 110.

  The thermal image sensor 101 acquires a thermal image of the person 102.

  The physiological amount acquisition unit 109 </ b> A is a processing unit that acquires a physiological amount from the thermal image acquired by the thermal image sensor 101. The physiological quantity acquisition unit 109A includes an image processing unit 109, and acquires the physiological quantity using the image processing unit 109. Here, an example in which the skin temperature of the nose of the person 102 is used as the physiological amount will be described.

  The image processing unit 109 extracts the nose from the thermal image of the person 102 acquired by the thermal image sensor 101, and calculates the nasal skin temperature that is the extracted temperature of the nose. An example of the method for extracting the nasal skin temperature will be described later. The nasal skin temperature calculated by the image processing unit 109 is input to the human state estimation unit 110 by the physiological quantity acquisition unit 109A, and the human state estimation unit 110 estimates the human state based on the nasal skin temperature.

  An example of processing when the human state estimation unit 110 detects the degree of sleepiness as the human state will be described.

  FIG. 7A is an explanatory diagram illustrating an example in which the thermal sensation is 2 or more with respect to temporal variation of the thermal sensation, the outside air temperature, and the nasal skin temperature in the present embodiment.

  7A shows the nasal skin temperature calculated by the physiological quantity acquisition unit 109A, the ambient temperature (outside temperature) of the person 102 measured by the thermometer 104b, and the thermal sensation estimation control unit 106 (PMV calculation unit 106b). It shows the transition of the thermal sensation of the person 102. Now, as shown in FIG. 7A, it is assumed that the outside air temperature is constant at about 28 ° C. and the thermal sensation exceeds +2. Further, it is assumed that the nasal skin temperature is increased by about 1.5 ° C. within the time range shown in the figure.

  When people feel warm, they try to expand peripheral blood vessels and promote blood flow in order to release heat from the body. Therefore, the peripheral skin temperature rises, but as described above, humans feel sleepy and dilate peripheral blood vessels to improve blood flow. It is difficult to determine whether it is caused solely by feeling or whether it is also drowsy.

  FIG. 7B is an explanatory diagram illustrating an example of the thermal sensation, the outside air temperature, and the time variation of the nasal skin temperature in the present embodiment when the thermal sensation is near zero.

  As shown in FIG. 7B, for example, the outside air temperature drops to about 25 ° C., which is lower than about 28 ° C. in the case of FIG. 7A, and the thermal sensation exists near zero, which is the thermal neutral point. Further, it is assumed that the nasal skin temperature rises by about 1.5 ° C. within the illustrated time range (the temperature is different) as in FIG. 7A.

  In this case, since it is not necessary to promote heat dissipation from the body, the increase in peripheral skin temperature such as the nose due to an increase in peripheral blood flow rate is not observed. Therefore, if the thermal sensation is in the vicinity of the thermal neutral point as shown in FIG. 7B, it can be estimated that the increase in the nasal skin temperature as shown in FIG. 7B is caused by sleepiness. .

  Here, it is difficult to estimate the thermal sensation according to which temperature range the outside air temperature is in. Specifically, for example, due to a difference in the amount of clothes even when the outside temperature is the same (for example, when only wearing a T-shirt or wearing a down jacket or the like), naturally, the warmth of the person may be felt. Because it is different, it is difficult to estimate the thermal sensation according to which temperature range the outside air temperature is in. Therefore, it is necessary to make a judgment based not on the outside air temperature but on the thermal sensation of a person.

  Note that the degree of sleepiness may be divided into five stages according to the temperature range of the fluctuating nasal skin temperature as shown in FIG. If the temperature fluctuation range of the nasal skin temperature is large, it can be determined that the patient is very sleepy. If the temperature fluctuation range of the nasal skin temperature is small, the stage may be slightly sleepy. The relationship between the width and the drowsiness level may be determined as appropriate depending on the difference in the temperature fluctuation range.

  Next, an example of processing when the human state estimation unit 110 detects the stress level as the human state will be described.

  FIG. 9A is an explanatory diagram showing an example in which the thermal sensation is −2 or less with respect to the temporal variation of the thermal sensation, the outside air temperature, and the nasal skin temperature in the present embodiment.

  FIG. 9A illustrates the calculation of the nasal skin temperature calculated by the physiological quantity acquisition unit 109A, the ambient temperature (outside temperature) of the person 102 measured by the thermometer 104b, and the thermal sensation estimation control unit 106 (PMV calculation unit 106b). It shows the transition of the thermal sensation of the person 102. Now, as shown in FIG. 9A, it is assumed that the outside air temperature is constant at about 18 ° C. and the thermal sensation is below −2. When a person feels cold, he tries to suppress blood flow by contracting peripheral blood vessels in order to suppress the release of heat from the body. Therefore, the peripheral skin temperature will decrease, but as described above, as humans feel stress, they try to constrict peripheral blood vessels and suppress blood flow, so this decrease in nasal skin temperature is It is difficult to determine whether it is caused solely by feeling cold or whether you feel stress.

  FIG. 9B is an explanatory diagram showing an example of the thermal sensation in the present embodiment when the thermal sensation is near 0 with respect to temporal variations in the thermal sensation, the outside air temperature, and the nasal skin temperature.

  As shown in FIG. 9B, for example, when the outside air temperature rises to about 22 ° C., which is higher than about 18 ° C. in the case of FIG. 9A, and the thermal sensation exists in the vicinity of zero, which is a thermal neutral point, heat is released from the body. Therefore, there is no decrease in the peripheral skin temperature such as the nose due to a decrease in peripheral blood flow. Therefore, if the thermal sensation is in the vicinity of the thermal neutral point as shown in FIG. 9B, it can be estimated that the decrease in the nasal skin temperature as shown in FIG. 9B is caused by stress. .

  Here, it is difficult to estimate the thermal sensation according to which temperature range the outside air temperature is in. Specifically, for example, due to a difference in the amount of clothes even when the outside temperature is the same (for example, when only wearing a T-shirt or wearing a down jacket or the like), naturally, the warmth of the person may be felt. Because it is different, it is difficult to estimate the thermal sensation according to which temperature range the outside air temperature is in. Therefore, it is important to make a judgment based not on the outside air temperature but on the thermal sensation of the person.

  It should be noted that the degree of stress may be divided according to the stage according to the temperature range of the fluctuating nasal skin temperature, like the sleepiness in FIG. If the temperature fluctuation range of the nasal skin temperature is large, it can be judged that a strong stress is felt, and if the temperature fluctuation range of the nasal skin temperature is small, it is classified by the feeling that a slight stress is felt. The relationship between the fluctuation temperature range and the stress level may be appropriately determined according to the difference in temperature fluctuation range.

  Next, the configuration of the human state estimation device 113 in the present embodiment will be described.

  FIG. 10 is a block diagram illustrating functions of the human state estimation device 113 in the present embodiment. The human state estimating device 113 can also be called a human state estimating system.

  The human state estimation device 113 includes a thermal sensation estimation unit 108, a thermal image sensor 101, and a control unit 112. The human state estimation device 113 is connected to the speaker 114.

  The thermal sensation estimator 108 and the thermal image sensor 101 are the same as the functional blocks having the same names as described above, and thus description thereof is omitted.

  The control unit 112 includes a physiological quantity acquisition unit 109A, a human state estimation unit 110, and a human state estimation execution determination unit 111.

  109 A of physiological quantity acquisition parts are the processing parts which acquire the physiological quantity in which the activity of a person's autonomic nerve was reflected, when it determines with the thermal sensation index which the thermal sensation estimation part 108 acquired within the predetermined range. It is. The physiological quantity acquisition unit 109A acquires the nasal skin temperature as the physiological quantity by processing the thermal image acquired by the thermal image sensor 101 using the image processing unit 109.

  The human state estimation unit 110 is a processing unit that estimates a human state based on the physiological amount acquired by the physiological amount acquisition unit 109A. Specifically, the human state estimation unit 110 is a processing unit that estimates the degree of stress or sleepiness that is a human state from the nasal skin temperature calculated by the physiological quantity acquisition unit 109A.

  The human state estimation execution determination unit 111 is a processing unit that determines whether or not the thermal sensation index acquired by the thermal sensation estimation unit 108 is within a predetermined range of the specified range. This determination is used for determining whether or not the human state estimation unit 110 performs estimation of the human state. The human state estimation execution determination unit 111 is connected to the thermal sensation estimation control unit 106 (PMV calculation unit 106b) of the thermal sensation estimation unit 108, and the result of the thermal sensation estimation of the person 102 is determined as the human state estimation execution determination. Is input to the unit 111. The human state estimation execution determination unit 111 is connected to the physiological quantity acquisition unit 109A. The human state estimation execution determination unit 111 corresponds to a determination unit.

  Note that the human state estimation execution determination unit 111 may be connected to the human state estimation unit 110. In this case, the human state estimation execution determination unit 111 may not be connected to the physiological quantity acquisition unit 109A. In this case (when the human state estimation execution determination unit 111 is not connected to the physiological amount acquisition unit 109A but is connected to the human state estimation unit 110), the physiological amount acquisition unit 109A always acquires a human physiological amount. Perform the process. On the other hand, the human state estimation unit 110 determines the physiological amount at the timing when the human state estimation execution determination unit 111 determines that “the thermal sensation index acquired by the thermal sensation estimation unit 108 is within a predetermined range of the specified range”. The human state is estimated using the physiological amount acquired by the acquisition unit 109A. That is, the human state estimation unit 110 determines that the human state estimation execution determination unit 111 does not determine that “the thermal sensation index acquired by the thermal sensation estimation unit 108 is within a predetermined range of the specified range”. In addition, the physiological quantity acquired by the physiological quantity acquisition unit 109A is not used for estimating the human state.

  The speaker 114 is an output device that notifies the person 102 according to the estimation result of the human state by the human state estimation unit 110. Note that the speaker 114 may be a part of the human state estimation device 113.

  Note that the human state estimation device 113 may be realized as one device by accommodating each functional block in one housing, or each functional block described above is distributed and each functional block is You may implement | achieve by transmitting / receiving information via a communication line.

  Hereinafter, the flow of processing of the human state estimation device 113 will be described.

  FIG. 11 is a flowchart showing a human state estimation method by human state estimation apparatus 113 in the present embodiment.

  In step S101, data obtained by each sensor (camera 103, globe thermometer 104a, thermometer 104b, anemometer 105, hygrometer 107) is input to the PMV calculation unit 106b to estimate the thermal sensation of the person 102. The

  In step S <b> 102, the thermal sensation of the person 102 estimated in step S <b> 101 is input to the human state estimation execution determination unit 111, and the human state estimation execution determination unit 111 determines the thermal sensation. Specifically, the human state estimation execution determination unit 111 determines whether the thermal sensation is within a predetermined range. The predetermined range can be, for example, a range where the thermal sensation is −2 or more and +2 or less, and this case will be described below. The predetermined range may be a part of the specified range including a thermal neutral point related to the thermal sensation. In addition, the predetermined range may be a partial range that does not include the point indicating the hottest feeling as the thermal sensation and the point indicating the coldest as the thermal sensation.

  If it is determined in step S102 that the thermal sensation is greater than +2 or smaller than −2, the process proceeds to step S101, and the PMV calculation unit 106b again estimates the thermal sensation of the person 102. Made. In this case, steps S103 and S104 described later are not performed. When it is determined that the thermal sensation is greater than +2 or smaller than −2, that is, the person 102 with the thermal sensation “very hot” or “very cold” is known to be less prone to sleepiness. In this case, the human state estimation unit 110 may estimate the sleepiness level as 1.

  If it is determined in step S102 that the thermal sensation is in the range of not less than −2 and not more than +2, the process proceeds to step S103.

  In step S103, the human state estimation unit 110 estimates a sleepiness level as a human state based on the nasal skin temperature acquired by the physiological quantity acquisition unit 109A based on the thermal image acquired by the thermal image sensor 101. Specifically, the human state estimating unit 110 estimates the degree of drowsiness of the person 102 based on the increase width with time of the nasal skin temperature. For example, when the nasal skin temperature rises by 1 ° C. within the time range in which the nasal skin temperature is measured (for example, the time range on the horizontal axis shown in FIG. 7A or the like), the human state estimation unit 110 sets the drowsiness level. 2 is determined. Further, when the nasal skin temperature rises by 2 ° C. within the above time range, the sleepiness level is determined to be 4.

  Further, the estimated sleepiness level is determined, and processing is performed as follows according to the determination result.

  That is, when it is determined in step S103 that the drowsiness level is 1, the process proceeds to step S101, and then a series of processes shown in the flowchart is performed again. In this case, it is considered that there is no need to notify the person 102 since the sleepiness level is such that the person 102 does not interfere with the driving of the car.

  On the other hand, when it is determined in step S103 that the drowsiness level is 2 or more, a notification to the person 102 is performed. In this case, since the sleepiness level is considered to hinder the driving of the automobile by the person 102, the above notification is performed to notify the person 102 of the fact. The notification is performed by, for example, informing the person 102 that the person is getting sleepy by the speaker 114 or prompting to take a rest. After the notification, the process proceeds to step S101, and a series of processes shown in the flowchart is performed again.

  In the case where the degree of stress is estimated as the human state, the processing by the human state estimation unit 110 in step S103 is different. Specifically, the human state estimation unit 110 estimates the degree of stress as a human state based on the nasal skin temperature acquired by the physiological quantity acquisition unit 109A based on the thermal image acquired by the thermal image sensor 101. . Specifically, the human state estimation unit 110 estimates the degree of stress of the person 102 based on the descending width of the nasal skin temperature with the passage of time. Other processes are the same as described above.

  As described above, since it is possible to distinguish whether the fluctuation of the skin temperature is caused by human thermal sensation or sleepiness, it is possible to provide a highly accurate sleepiness degree estimating means. Of course, the same applies to the case of the degree of stress as the human state, and the same thing makes it possible to provide highly accurate stress level estimation means. That is, it is possible to provide a highly accurate human state estimation means. In addition, when detecting the degree of sleepiness, for example, when the thermal sensation is greater than +2 or less than −2, that is, when feeling hot or cold, people rarely feel sleepiness. Since useless degree of sleepiness estimation in the control unit 112 can be omitted, the processing load can be reduced and energy consumption can be reduced.

  In addition, as a result of estimating sleepiness in the human state estimation unit 110, for example, it has been described that the person 102 is notified through the speaker 114 when the sleepiness level is 2 or more. The user 102 may be notified by prompting the user to wake up by tightening the belt, or displayed on a display or the like, and the method is not particularly limited.

  In addition, although the physiological quantity acquisition unit 109A (image processing unit 109) calculates the nasal skin temperature from the thermal image captured by the thermal image sensor 101, this is not limited to the nasal skin temperature. Any part corresponding to a similar peripheral part may be used. For example, the back of the hand or the earlobe may be used. However, when human state estimation is performed in the automobile 100, the lower the body of the person 102 (in other words, the closer to the foot), the more susceptible to solar radiation, the skin temperature above the neck. It is desirable to use human eye, and by measuring at the nose or earlobe, it is possible to estimate a human condition that is not easily affected by solar radiation.

  Furthermore, in the human state estimation execution determination unit 111, the upper and lower limits of the thermal sensation range obtained by the PMV calculation unit 106b are set to +2 and -2, but these values may be different from these values. For example, the human state may be estimated when the thermal sensation is +1 or less and −1 or more. Moreover, if the range is narrow in the range including the thermal neutral point, the human state can be estimated with higher accuracy. Of course, instead of an integer, a decimal number such as +1.5 or -1.5 may be used as the threshold value.

  In the above description, the thermal sensation of the person is automatically estimated by the thermal sensation estimation unit 108 as an example. However, for example, the person 102 may directly input the thermal sensation of the person 102. The means is not limited as long as a value that can determine the thermal sensation of the person 102 can be provided to the human state estimation execution determination unit 111.

  In addition, when the nasal skin temperature is calculated by the physiological quantity acquisition unit 109A (image processing unit 109), it is necessary to specify the nose from the thermal image acquired by the thermal image sensor 101. One example will be described. FIG. 12A is an explanatory diagram of nasal skin temperature measurement in the present embodiment. FIG. 12B is an explanatory diagram of nostril temperature fluctuations in nasal skin temperature measurement in the present embodiment.

  FIG. 12A is a schematic diagram of a human nose, and the temperature of the nostril portion varies depending on the breathing cycle as shown in FIG. 12B due to respiration. This temperature fluctuation is caused by the rise in temperature caused by the warming of the nostril when breathing out the breath through the nose due to breathing, and the heat of the nostril due to inhaling the outside air when breathing in. This is due to a decrease in temperature due to deprivation. Therefore, the cycle of this temperature fluctuation is approximately equal to the respiratory cycle (usually about 0.2 to 0.3 Hz). Therefore, in the image processing unit 109, if there are two parts that change with a period of about 0.2 to 0.3 Hz, the part is estimated as a nostril part. When the nostril portion is known, the nose portion to be measured can be easily specified from the position. From the above, it becomes possible to specify the nose.

  When the nostril portion cannot be specified, the person 102 may be notified from the display or the like that the nostril portion cannot be extracted and the human state cannot be estimated. At the same time, you may be encouraged to wear glasses. Glasses normally do not pass infrared rays, so when a person wearing glasses is photographed with the thermal image sensor 101, the glasses part is not the eye temperature but the temperature of the glasses themselves, and is closer to the ambient temperature than the skin temperature. , Easy to detect the position. By estimating and measuring the position of the nose from the detected eye position, the position of the nose can be estimated with high accuracy.

  In addition, it may be urged to take a deep breath through a display or the like. By taking a deep breath, the amplitude of the nostril temperature shown in FIG. 12B is increased, so that the position of the nostril is more easily extracted. Of course, the nose may be specified by a method other than this, for example, the face may be extracted and the nose may be estimated from the contour, and the method is not limited.

  Further, when measuring the nasal skin temperature for human state estimation, the measurement timing may be synchronized with the breathing phase. For example, as shown by an arrow in FIG. 12B, the nasal skin temperature may be measured at the timing when the temperature of the nostril becomes highest in each breath. Since the nasal skin temperature fluctuates due to breathing, the nasal skin temperature is also affected. Therefore, by measuring the nasal skin temperature in synchronization with the breathing phase, precise measurement with little variation becomes possible. Here, the measurement is illustrated at the timing when the temperature of the nostril portion is highest in each breath. However, of course, other phases may be used, the timing when the temperature of the nostril portion is lowest, or other phases may be used. However, it is not limited here.

  In addition, the thermal image sensor 101 is used here to measure the nasal skin temperature. However, this is not a limitation of course, and any device that can measure the skin temperature may be used. A pyroelectric sensor or a monocular infrared sensor (such as a bolometer sensor or a thermopile sensor) may be used, and its means is not limited.

  The human state may be obtained from the skin blood flow. A method for obtaining the human state from the skin blood flow will be described below.

  When sleepiness as a human state becomes strong, there is a correlation that the blood flow rate of the peripheral part (particularly the nose) increases, and therefore the human state can be estimated from the skin blood flow rate based on this correlation. There are various methods for measuring the blood flow of the skin. For example, light of a specific wavelength (for example, infrared light) is received by a camera, and the amount of hemoglobin measured based on the received light is measured. Skin blood flow can be calculated.

  Further, the human state may be obtained from blood pressure. A method for obtaining the human state from the blood pressure will be described below with reference to FIGS. 12C and 12D.

  Since there is a correlation that blood pressure decreases when sleepiness as a human state increases, the human state can be estimated from the skin blood flow based on this correlation. The blood pressure may be obtained continuously with a cuff or the like, or may be obtained from the pulse wave propagation time. The pulse wave propagation time is the time until blood flow from the heart reaches a predetermined end.

  Between blood pressure and pulse wave propagation time, there is a correlation that when the blood pressure decreases, the pulse wave propagation time becomes longer. Therefore, based on these correlations, the human state can be estimated from the pulse wave propagation time via the blood pressure.

  There are various methods for measuring the pulse wave propagation time. For example, as shown in FIG. 12C, a moving image including a human face and a portion other than the face (neck, hand, etc.) is captured by a camera, and a portion P1 included in the face and a portion P2 other than the face are extracted from the moving image. Can be obtained from the deviation amount T of the peak time of the pulse wave at (FIG. 12D). The deviation amount T is a value that can vary in about 0.2 msec.

  Further, instead of using the face and the part other than the face, for example, the difference may be obtained from the shift amount of the peak time of the pulse wave at two different places (for example, jaw and forehead) in the face. In addition, the heart vibration is measured with a millimeter wave sensor, etc., and the face pulse wave is detected from subtle color fluctuations of the face image taken from the camera, and the heart vibration peak and the face pulse wave peak are detected. Blood pressure fluctuations may be estimated using the amount of time shift as the pulse wave propagation time.

  As described above, according to the human state estimation method according to the present embodiment, it is possible to estimate the human state with higher accuracy by reducing false detection during human state estimation. Specifically, in the human state estimation method according to one aspect of the present invention, the human state is estimated from the physiological quantity under the condition that the acquired thermal sensation information is within a predetermined range. Therefore, it is possible to reduce erroneous detection in estimating the human state, compared to estimating the human state regardless of whether this condition is satisfied. Therefore, the human state estimation method can improve the accuracy of human state estimation. Furthermore, by reducing the false detection described above, it is possible to obtain the effects of reducing the processing amount and processing load and reducing power consumption, such as eliminating the need to perform detection processing again.

  The human state is estimated when the human thermal sensation is within a range relatively close to a thermal neutral point at which the human does not feel hot or cold. When a person feels hot or warm, the skin temperature rises by promoting heat dissipation from the body. On the other hand, when a person feels cold or cool, the skin temperature is lowered by suppressing heat dissipation from the body. That is, when the thermal sensation of a person is within a range relatively close to the thermal neutral point, there is no effect of promotion or suppression of heat dissipation from the human body, or a relatively small state. Therefore, in such a case, by estimating the human state from the physiological amount, the influence of the promotion or suppression of heat dissipation from the human body included in the estimation result can be reduced, and detection error and estimation accuracy can be reduced. Contribute to improvement.

  In addition, when a person's thermal sensation is in a range excluding a time when the person feels very hot or very cold, the human state is estimated. When a person feels very hot, heat dissipation from the person's body is greatly promoted. Also, when a person feels very cold, heat dissipation from the person's body is greatly suppressed. By excluding such cases from the case of estimating the human state, the influence of the promotion or suppression of heat dissipation from the human body included in the estimation result can be reduced, and the detection accuracy and estimation accuracy can be reduced. Contribute to improvement.

  Moreover, when estimating the degree of sleepiness of a person based on a person's nose skin temperature, the influence of the promotion or suppression of the heat radiation from a person's body contained in an estimation result can be made small. Further, by using the nasal skin temperature, even when there is a disturbance, it can be made less susceptible to the influence of the disturbance.

  In addition, the degree of sleepiness of a person can be specifically estimated based on the skin temperature of the person's nose.

  Moreover, by the estimation by PMV, it is possible to more accurately estimate the thermal sensation of a person from the six elements of temperature, that is, temperature, humidity, airflow, radiation, amount of clothes, and amount of activity.

  In addition, it is possible to specifically estimate a person's degree of stress based on the person's nose skin temperature.

  Further, it is possible to specifically estimate the degree of sleepiness of a person based on the person's skin blood flow, blood pressure, or pulse wave propagation time.

(Embodiment 2)
An example in which the human state estimation apparatus according to the present embodiment is mounted on an automobile will be described with reference to FIGS.

  FIG. 13A is a conceptual diagram of an automobile 200 on which the human state estimation device according to the present embodiment is mounted. FIG. 13B is a schematic view of a person 202 wearing the human state estimation device in the present embodiment as seen from the front. The person 202 shown in FIG. 13B corresponds to the person 202 riding in the automobile 200 shown in FIG. 13A.

  As shown in FIG. 13B, the person 202 attaches the skin temperature pulse wave sensor 203 to the earlobe. Skin temperature pulse wave sensor 203 is an integrated temperature sensor and pulse wave sensor, and can measure skin temperature and pulse wave simultaneously. In the present embodiment, an example is shown in which the skin temperature of the earlobe is used for thermal sensation estimation and the pulse wave is used for human state estimation.

  Hereinafter, a method for estimating a human state using a pulse wave will be described.

  FIG. 14A is an explanatory diagram showing an example of a measured pulse wave in the present embodiment. FIG. 14B is an explanatory diagram of time-series fluctuations in the heartbeat interval obtained from the pulse wave shown in FIG. 14A. FIG. 15A is an explanatory diagram showing a frequency component of a heartbeat interval and a small HF component in the present embodiment. FIG. 15B is an explanatory diagram showing a frequency component of a heartbeat interval and a large amount of HF components in the present embodiment.

  FIG. 14A shows an example of a pulse wave measured by the skin temperature pulse wave sensor 203. The pulse wave is a waveform obtained by capturing a change in blood pressure or volume in the peripheral vasculature accompanying the heartbeat as a waveform from the body surface, and typically shows a sawtooth waveform as shown in FIG. 14A. When heartbeat intervals are extracted from the waveform of the pulse wave and arranged in time series, fluctuations in the heartbeat interval (variation in heartbeat interval) with the passage of time are observed as shown in FIG. 14B.

  It is known that there are mainly two types of fluctuations in the heartbeat interval, one is blood pressure fluctuation and the other is respiratory fluctuation. The heart beat is controlled via the autonomic nerve by the center of the brain, and is controlled by two systems, the sympathetic nerve and the parasympathetic nerve. When accelerating the heartbeat, the sympathetic nerve is increased and the heartbeat is increased. When late, parasympathetic nerves are increased. The central part of the brain determines whether to speed up or slow down the heart rate based on the state of the body. Among these factors are blood pressure and respiration. Regarding blood pressure, when the blood pressure decreases, the heart rate is increased to activate the heart activity, and when the blood pressure increases, the heart rate is decreased to suppress the heart activity. Breathing also functions to increase the heart rate while inhaling as the lungs expand, and to slow the heart rate when exhaling.

  The blood pressure fluctuates at about 0.1 Hz, and this fluctuation is called a Mayer wave. Respiration is about 0.2 to 0.3 Hz at rest. Therefore, the heartbeat interval has a waveform with a period of about 0.2 to 0.3 Hz as shown in FIG. 14B or a waveform with a period of about 0.1 Hz.

  Next, the speed (response speed) at which the sympathetic nerve and the parasympathetic nerve follow a command related to the speed of the heartbeat from the center in the brain will be described. It has been found that the parasympathetic nerve can follow the command even with fluctuations of about 0.2 to 0.3 Hz. On the other hand, it is known that the sympathetic nerve can follow even if there is a fluctuation of about 0.1 Hz in the command, but cannot follow if there is a high-frequency fluctuation of about 0.2 to 0.3 Hz in the command. Therefore, when the sympathetic nerve is enhanced, when the frequency component of the waveform of the heartbeat interval is obtained, the frequency component of about 0.2 to 0.3 Hz is relatively low as shown in FIG. When the frequency increases, the frequency component of about 0.2 to 0.3 Hz increases as compared to FIG. 15A as shown in FIG. 15B.

  Therefore, a ratio LF / HF between a low frequency component (Low Frequency: LF) and a high frequency component (High Frequency: HF) can be used as an index indicating which of the sympathetic nerve and the parasympathetic nerve is superior. That is, the sympathetic nerve is dominant when LF / HF is high, and the parasympathetic nerve is dominant when LF / HF is low. Therefore, when sleepiness or the like as a human state occurs and parasympathetic nerves increase, the HF component increases and LF / HF decreases as shown in FIG. 15B. On the other hand, when stress or the like occurs and the sympathetic nerve is enhanced, the HF component decreases and LF / HF increases as shown in FIG. 15A. Here, for example, LF is a frequency component of 0.15 Hz or less, and HF is a frequency component of 0.2 Hz to 0.3 Hz. These boundary values are examples and are not limited to the above values.

  Using the above, it is possible to estimate the human state based on the autonomic nerve from the pulse wave detected by the skin temperature pulse wave sensor 203. Here, the pulse wave is detected from the earlobe as a means for detecting the heartbeat interval, but the method is not limited to this. For example, a sensor may be provided on the steering or the like to detect a pulse wave from the fingertip or the like. Moreover, an electrocardiogram may be measured instead of the pulse wave, and a heartbeat interval may be extracted from the electrocardiogram, and the method is not limited as long as the heartbeat interval can be measured.

  Next, a method for estimating thermal sensation using the human peripheral skin temperature will be described.

  FIG. 16 is a correlation diagram showing an example of the correlation between the earlobe skin temperature and the thermal sensation in the present embodiment.

  When a person feels cold, the brain contracts the blood vessels in the peripheral part to prevent the body temperature in the deep part where there are many vital organs from decreasing. Try to reduce the amount. When the amount of blood reaching the peripheral part decreases, the skin temperature in the peripheral part decreases.

  On the other hand, when a person feels hot, the brain tries to increase the amount of blood that reaches the peripheral part of the body by expanding the blood vessels in the peripheral part to prevent the deep body temperature from rising above a certain level. To do. When the amount of blood reaching the peripheral portion increases, the skin temperature in the peripheral portion increases.

  As a result, there is a correlation between the skin temperature and the thermal sensation in the peripheral part. For example, as shown in FIG. 16, a linear correlation between the skin temperature and the thermal sensation in the earlobe part which is the peripheral part. A relationship arises. From this, the thermal sensation of a person can be estimated by detecting the skin temperature of the peripheral part. In this case, the earlobe is used as the peripheral part, but other parts may of course be used, and other parts such as the palm and nose may be used as long as the peripheral part. However, when estimating the human state in the automobile 200, it is more likely to be affected by solar radiation in the body of the person 202, so it is desirable to use the skin temperature above the neck. By measuring at the earlobe, it is possible to estimate a human condition that is not easily affected by solar radiation.

  Further, as shown in FIG. 16, the fluctuation range of the peripheral skin temperature due to the fluctuation of thermal sensation (−3 to +3) is as large as about 20 ° C. (about 15 ° C. to about 35 ° C.). This is larger than the temperature fluctuation range (usually about 1 to 2 ° C.) due to the influence of stress or drowsiness that is a factor of the above, and thus does not significantly affect the estimated thermal sensation. Further, for example, the thermal sensation may be estimated from the difference between the skin temperature of the forehead or the like close to the skin temperature of the trunk and the skin temperature of the peripheral part, instead of the peripheral skin temperature alone. By doing so, the individual difference at the time of thermal sensation estimation can be reduced.

  Next, the configuration of the human state estimation device 213 in the present embodiment will be described.

  FIG. 17 is a block diagram illustrating functions of the human state estimation device 213 according to the present embodiment.

  The human state estimation device 213 includes a skin temperature pulse wave sensor 203, a thermal sensation estimation unit 208, and a control unit 212.

  The control unit 212 includes a human state estimation unit 210 and a human state estimation execution determination unit 211.

  The human state estimation unit 210 is a processing unit that estimates the degree of stress or sleepiness that is a human state from the pulse wave acquired by the skin temperature pulse wave sensor 203. More specifically, the human state estimation unit 210 acquires the heart rate interval of the person 202 obtained from the pulse wave as a physiological quantity, and estimates the human state of the person 202 based on the obtained fluctuation of the heart rate interval.

  The human state estimation execution determination unit 211 is a processing unit that determines whether or not the human state estimation unit 210 performs human state estimation. The human state estimation execution determination unit 211 is connected to the thermal sensation estimation unit 208, and the result of the thermal sensation estimation of the person 202 is input to the human state estimation execution determination unit 211. The human state estimation execution determination unit 211 is connected to the human state estimation unit 210.

  Hereinafter, the flow of processing of the human state estimation device 213 will be described.

  FIG. 18 is a flowchart showing a human state estimation method by human state estimation apparatus 213 in the present embodiment.

  In step S201, the skin temperature data obtained by the skin temperature pulse wave sensor 203 is input to the thermal sensation estimation unit 208, and the thermal sensation of the person 202 is estimated.

  In step S202, the thermal sensation of the person 202 estimated in step S201 is input to the human state estimation execution determination unit 211, and the human state estimation execution determination unit 211 determines the thermal sensation.

  When it is determined in step S201 that the thermal sensation is greater than +2 or smaller than −2, the process proceeds to step S201, and the thermal sensation estimation unit 208 again determines the thermal sensation of the person 202. An estimate is made.

  If it is determined in step S201 that the thermal sensation is in the range of −2 or more and +2 or less, the process proceeds to step S203.

  In step S203, the human state estimation unit 210 estimates the sleepiness level as a human state based on the pulse wave measured by the skin temperature pulse wave sensor 203. Further, the estimated sleepiness level is determined, and processing is performed as follows according to the determination result.

  That is, when it is determined in step S203 that the drowsiness level is 1, the process proceeds to step S201, and then a series of processes shown in the flowchart is performed again. In this case, it is considered that there is no need to notify the person 202 because the sleepiness level is such that the person 202 does not interfere with the driving of the car.

  On the other hand, if it is determined in step S203 that the drowsiness level is 2 or more, a notification is given to the person 202. In this case, since the sleepiness level is considered to hinder the driving of the automobile by the person 202, the above notification is performed to notify the person 202 of the fact. The notification is performed, for example, by notifying the person 202 that the person 202 is getting sleepy or prompting the person to take a rest. After the notification, the process proceeds to step S201, and a series of processes shown in the flowchart is performed again.

  Even when the degree of stress is estimated as the human state, the process is the same as that in the case of drowsiness except that the determination criteria in the human state estimation unit 210 are different, and thus a description thereof will be omitted.

  As described above, since it is possible to determine whether the fluctuation of the pulse wave is due to human thermal sensation or sleepiness, it is possible to provide highly accurate sleepiness level estimation means. Of course, the same applies to the case of the degree of stress as the human state, and the same thing makes it possible to provide highly accurate stress level estimation means. That is, it is possible to provide a highly accurate human state estimation means. In addition, when detecting the degree of sleepiness, for example, when the thermal sensation is greater than +2 or less than −2, that is, when feeling hot or cold, people rarely feel sleepiness. Since useless degree of sleepiness estimation in the control unit 112 can be omitted, the processing load can be reduced and energy consumption can be reduced.

  In addition, as a result of estimating sleepiness in the human state estimation unit 210, for example, it has been described that the person 202 is notified through the speaker 114 when the sleepiness level is 2 or more. The user 202 may be alerted by tightening the belt tightly or displayed on a display or the like, and the method is not particularly limited.

  Furthermore, in the human state estimation execution determination unit 211, the upper and lower limits of the thermal sensation range obtained by the thermal sensation estimation unit 208 are +2 and -2, but this value is different from these values. For example, the human state may be estimated when the thermal sensation is +1 or less and −1 or more. Moreover, if the range is narrow in the range including the thermal neutral point, the human state can be estimated with higher accuracy. Of course, instead of an integer, a decimal number such as +1.5 or -1.5 may be used as the threshold value.

  In the above description, the thermal sensation of the person is automatically estimated by the thermal sensation estimation unit 208. However, for example, the person 202 may directly input the thermal sensation of the person 202. The means is not limited as long as a value that can determine the thermal sensation of the person 202 can be provided to the human state estimation execution determination unit 211.

  Although the human state has been estimated from the pulse wave so far, a method for detecting at a higher speed using the respiration mentioned in the first embodiment together with the pulse wave will be described.

  FIG. 19A is an explanatory diagram of the respiratory frequency according to the processing load reduction method using the respiratory component in the present embodiment. FIG. 19B is a block diagram illustrating wavelet transform according to the processing load reduction method using the respiratory component in the present embodiment.

  In the description of FIGS. 15A and 15B, it is shown that the frequency characteristics of FIGS. 15A and 15B are obtained from the waveform of the heartbeat interval shown in FIG. 14B. Usually, such a frequency characteristic is obtained by using Fourier transform, discrete Fourier transform, or the like, and is often executed by fast Fourier transform on a computer. However, when performing a Fourier transform of a pulse wave, only one piece of data can be obtained in about 1 second, and therefore data of about several minutes is usually required. However, in order to analyze LF / HF for the analysis of autonomic nerves, it is important to detect how much the HF component is, and it is not always necessary to detect all frequency components.

  Therefore, as shown in FIG. 19B, a respiratory waveform can be used as a physiological quantity. That is, it is possible to estimate the human state based on the respiratory waveform in addition to the heartbeat interval. As described in the first embodiment, the respiration waveform may be obtained from the temperature fluctuation of the nostril, or other than that, for example, the expansion / contraction of the abdomen due to respiration may be detected from the tension of the seat belt. For example, the position change of the abdomen may be detected in a non-contact manner. By performing a respiratory cycle analysis 1902 on the obtained respiratory waveform, for example, by analyzing the respiratory cycle from the interval between the peak values, the frequency of the respiratory frequency component of the person 202 is obtained. Next, with respect to the obtained pulse waveform, the intensity of the respiratory frequency component obtained from the respiratory waveform is obtained by wavelet transformation 1901.

  By extracting the respiratory component intensity of the pulse wave based on the respiratory waveform in this way, it is possible to determine how much the respiratory frequency component is included in the pulse wave in less than one minute. Since the amount of calculation by this method is smaller than the amount of calculation by Fourier transform, there is also an effect that the processing load of the human state estimation device 213 can be reduced and energy consumption can be reduced.

  A human state estimation method including a respiratory sensor will be described.

  FIG. 20 is a flowchart showing a human state estimation method to which the processing load reduction method using the respiratory component in the present embodiment is applied.

  In step S203A, as an input for estimating the human state by the human state estimating unit 210, in addition to the pulse wave obtained by the skin temperature pulse wave sensor 203, a respiration waveform obtained from the respiration sensor is inputted.

  The rest is the same as the processing step of the same name in FIG.

  13A and 13B, the skin temperature pulse wave sensor 203 is attached to the left ear of the person 202. However, when the automobile 200 is a right-hand drive car, the person wearing it on the left ear is more susceptible to disturbance such as solar radiation. It is preferable that the pulse wave is not easily influenced, and the pulse wave is optically detected because the pulse wave is not easily affected by disturbance such as solar radiation. Of course, if it is a left-hand drive vehicle, it is preferable to attach it to the right ear.

  Further, in the present embodiment, a method for detecting a pulse wave from the earlobe has been described. However, for example, when detecting the human state of a person working on a personal computer or the like in an office from the pulse wave, Alternatively, the pulse wave may be detected through a mouse which is an input device of a personal computer. Since the part where the finger is placed with the mouse is almost determined, the human state can be detected without actively wearing the sensor by optically reading the pulse wave at that part. Other than that, as a wearable sensor, for example, an element that detects a person's pulse wave may be attached to a shirt or the like, and the pulse wave waveform may be transferred wirelessly to a smartphone or the like to perform processing using a cloud. . By doing so, the human state can be detected even when the person is moving. In addition, for example, by treating and analyzing the human state of a large number of people as big data, it is possible to identify a place or time zone where a lot of stress is applied, so a place or time zone where an accident or the like is likely to occur is extracted. I can do it.

  Next, a method for reducing individual differences in the human state will be described.

  FIG. 21 is a block diagram illustrating a function of correcting the human state determination result in the present embodiment.

  When the human state estimation unit 210 analyzes the pulse wave to determine the human state, the pulse wave analysis unit 250 performs the above-described Fourier transform and wavelet transform to obtain LF / HF, and the human state determination unit 251 determines the human state from the result. The level of sleepiness as a state is determined from 1 to 5. A speaker or the like outputs the result of the determination as a sound.

  The person 202 knows by listening to the result of the determination. If the person 202 feels that the state and the degree of sleepiness that the person 202 feels are different, the person 202 inputs the degree of sleepiness that the person 202 feels through the correction value input unit 253. The human state correction unit 252 recognizes and stores individual differences from the difference between the degree of sleepiness determined by the human state determination unit 251 and the degree of sleepiness of the person received by the correction value input unit 253. Thereafter, the corrected human state is estimated for each person 202 by correcting the result determined by the human state determining unit 251 using the value stored in the human state correcting unit 252. . By doing so, it is possible to provide the human state estimation device 213 with little individual difference.

  Further, the human state detection unit using the skin temperature and the pulse wave has been described so far, but the present invention is not limited to these, and the skin blood flow rate may be detected, for example, other than the skin temperature and the pulse wave. The means is not limited as long as it is a physiological quantity controlled by an autonomic nerve such as line of sight, blinking, cerebral blood flow, or electroencephalogram.

  As described above, according to the human state estimation method according to the present embodiment, it is possible to estimate the human state with higher accuracy by reducing false detection during human state estimation. Specifically, in the human state estimation method according to one aspect of the present invention, the human state is estimated from the physiological quantity under the condition that the acquired thermal sensation information is within a predetermined range. Therefore, it is possible to reduce erroneous detection in estimating the human state, compared to estimating the human state regardless of whether this condition is satisfied. Therefore, the human state estimation method can improve the accuracy of human state estimation. Furthermore, by reducing the false detection described above, it is possible to obtain the effects of reducing the processing amount and processing load and reducing power consumption, such as eliminating the need to perform detection processing again.

  In addition, the human state can be specifically estimated based on the person's heartbeat interval.

  In addition, the human state can be specifically estimated based on the breathing waveform of the person. It can also contribute to reduction of processing load and speeding up of processing.

  In addition, the human state can be specifically estimated based on the skin temperature and pulse wave of the human earlobe. The earlobe has a feature that it is easy to measure a pulse wave. Therefore, by acquiring the pulse wave from the earlobe and acquiring the skin temperature as well, information necessary for estimating the human state can be collectively acquired from the earlobe.

  In each of the above embodiments, each component may be configured by dedicated hardware or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory. Here, the software that realizes the human state estimation device according to each of the above embodiments is the following program.

  That is, this program is a human state estimation method for estimating a human state that is a human state in a computer, and obtains a thermal sensation index obtained by indexing the estimated thermal sensation of the person within a specified range. And determining whether the acquired thermal sensation index is within a predetermined range of the specified range, and acquiring when the acquired thermal sensation index is determined to be within the predetermined range. The human state estimation method for estimating the human state based on the physiological amount that reflects the activity of the person's autonomic nerve is executed.

  As mentioned above, although the human state estimation apparatus etc. which concern on the one or several aspect were demonstrated based on embodiment, this invention is not limited to this embodiment. Unless it deviates from the gist of the present invention, various modifications conceived by those skilled in the art have been made in this embodiment, and forms constructed by combining components in different embodiments are also within the scope of one or more aspects. May be included.

  For example, the present invention includes the following cases.

  (1) In the above embodiment, the human state estimation method including at least the sensor, the thermal sensation estimation unit, and the control unit has been described. However, some components of the sensor, the thermal sensation estimation unit, the control unit, and the human state estimation method may be configured separately as software. In this case, the subject that processes the software may be a calculation unit of a human state estimation method, a calculation unit included in a PC (personal computer), a smart phone, or the like. It may be a cloud server connected to the apparatus via a network.

  Further, the arrangement or configuration of each device is not limited to the arrangement or configuration of the device as shown in FIG. A part or all of each sensor (camera, thermometer, globe thermometer, anemometer, hygrometer) may be incorporated in an integrated module, or may be arranged as a single unit (separately). Further, as shown in FIG. 10, the thermal sensation estimator 108 may be incorporated as an integral configuration. Further, the thermal sensation estimation control unit 106 (a process included therein) may be provided separately as software. Further, the thermal sensation estimation unit 108 and the control unit 112 (processing included therein) may be provided as integrated software. The thermal image sensor 101, the thermal sensation estimation control unit 106, and the control unit 112 may be provided as an integrated module. The arrangement or configuration of each device is not limited to this, and includes any form in which each configuration is combined and provided as software or a module.

  (2) Each of the above devices is specifically a computer system including a microprocessor, ROM, RAM, a hard disk unit, a display unit, a keyboard, a mouse, and the like. A computer program is stored in the RAM or hard disk unit. Each device achieves its functions by the microprocessor operating according to the computer program. Here, the computer program is configured by combining a plurality of instruction codes indicating instructions for the computer in order to achieve a predetermined function.

  (3) A part or all of the constituent elements constituting each of the above-described devices may be constituted by one system LSI (Large Scale Integration). The system LSI is an ultra-multifunctional LSI manufactured by integrating a plurality of components on a single chip, and specifically, a computer system including a microprocessor, ROM, RAM, and the like. . A computer program is stored in the RAM. The system LSI achieves its functions by the microprocessor operating according to the computer program.

  (4) A part or all of the constituent elements constituting each of the above devices may be configured as an IC card that can be attached to and detached from each device or a single module. The IC card or the module is a computer system including a microprocessor, a ROM, a RAM, and the like. The IC card or the module may include the super multifunctional LSI described above. The IC card or the module achieves its function by the microprocessor operating according to the computer program. This IC card or this module may have tamper resistance.

  (5) The present disclosure may be the method described above. Further, the present invention may be a computer program that realizes these methods by a computer, or may be a digital signal composed of the computer program.

  The present disclosure also relates to a computer-readable recording medium such as a flexible disk, hard disk, CD-ROM, MO, DVD, DVD-ROM, DVD-RAM, BD (Blu-ray). (Registered trademark) Disc), or recorded in a semiconductor memory or the like. The digital signal may be recorded on these recording media.

  In addition, the present disclosure may transmit the computer program or the digital signal via an electric communication line, a wireless or wired communication line, a network represented by the Internet, a data broadcast, or the like.

  The present disclosure may be a computer system including a microprocessor and a memory, the memory storing the computer program, and the microprocessor operating according to the computer program.

  In addition, the program or the digital signal is recorded on the recording medium and transferred, or the program or the digital signal is transferred via the network or the like and executed by another independent computer system. You may do that.

  (6) The above embodiment and the above modifications may be combined.

  The present disclosure can be used for a human state estimation device, and in particular, can be used for a human state estimation device that estimates a human state in a driver's seat such as an automobile or a train or an office.

DESCRIPTION OF SYMBOLS 100, 200 Car 101 Thermal image sensor 102, 202 Person 103 Camera 104a Globe thermometer 104b Thermometer 105, 105a Anemometer 105b Louver 106 Thermal sensation estimation control part 106a Clothing amount estimation part 106b PMV calculation part 107 Hygrometer 108, 208 Thermal sensation estimation unit 109 Image processing unit 109A Physiological quantity acquisition unit 110, 210 Human state estimation unit 111, 211 Human state estimation execution determination unit 112, 212 Control unit 113, 213 Human state estimation device 114 Speaker 203 Skin temperature pulse wave sensor 250 Pulse wave analysis unit 251 Human state determination unit 252 Human state correction unit 253 Correction value input unit P1, P2 part

For example, in the person state estimation method, the Tokushi preparative and thermal sensation index the nasal skin temperature Prefecture, upon said determining, in the each predetermined range of the obtained pre-Symbol thermal sensation index determines whether the time of estimation of the person condition, based on the rise with time of Kihana part skin temperature before acquired, to estimate the degree of the person of sleepiness.

For example, the physiological amount is the nasal skin temperature of the person, the human state includes a degree of stress of the person, and the human state estimation method includes the thermal sensation index and the nasal skin temperature. It acquires the in the determination, the previous SL thermal sensation indicators acquired to determine whether or not it is within the predetermined range, the time of estimation of the person condition, the obtained pre-Symbol The degree of stress of the person is estimated based on the descending width of the nasal skin temperature with time.

Claims (14)

A human state estimation method for estimating a human state which is a human state,
Obtaining a thermal sensation index obtained by indexing the estimated thermal sensation of the person within a specified range;
It is determined whether the acquired thermal sensation index is within a predetermined range of the specified range,
The physiological state acquired when it is determined that the acquired thermal sensation index is within the predetermined range, and the human state is estimated based on the physiological amount reflecting the activity of the human autonomic nerve. Human state estimation method. The human state estimation method according to claim 1, wherein the predetermined range is a partial range including a thermal neutral point related to a thermal sensation in the specified range. The predetermined range is a part of the specified range that does not include a point indicating that it is the hottest as a thermal sensation and a point indicating that it is the coldest as a thermal sensation. Human state estimation method. The physiological amount is the nasal skin temperature of the person,
The human state estimation method according to claim 1, wherein the human state includes a degree of sleepiness of the person. In the human state estimation method,
Obtaining the thermal sensation index and the nasal skin temperature multiple times,
In the determination, it is determined whether each of the acquired plurality of thermal sensation indices is within the predetermined range,
5. The human state estimation method according to claim 4, wherein, when estimating the human state, the degree of sleepiness of the person is estimated based on an increase width of the plurality of acquired nasal skin temperatures over time. The physiological amount includes a heartbeat interval of the person,
In the human state estimation method,
The human state estimation method according to claim 1, wherein the human state is estimated based on the fluctuation of the heartbeat interval as the acquired physiological amount. The physiological amount includes a respiratory waveform of the person,
In the human state estimation method,
The human state estimation method according to claim 6, wherein the human state is estimated further based on the acquired respiratory waveform as the acquired physiological quantity. Obtaining the skin temperature of the earlobe of the person,
When acquiring the thermal sensation index, the correlation between the skin temperature of the earlobe and the thermal sensation index is used to acquire the thermal sensation index estimated from the acquired skin temperature of the earlobe part. ,
When acquiring the physiological amount, the pulse wave measured from the earlobe of the person is acquired as the physiological amount,
The human state estimation method according to claim 1, wherein when estimating the human state, the human state is estimated based on a frequency analysis of the acquired pulse wave. The human state estimation method according to any one of claims 1 to 8, wherein the human state estimation method acquires the thermal sensation index estimated by PMV (Predicted Mean Vote). The specified range is expressed by a PMV seven-level rating scale,
The human state estimation method according to claim 9, wherein the predetermined range is a range in which the PMV value in the prescribed range is −2 or more and +2 or less. The physiological amount is the nasal skin temperature of the person,
The human condition includes the degree of stress of the person,
In the human state estimation method,
In the determination, it is determined whether each of the acquired plurality of thermal sensation indices is within the predetermined range,
The degree of stress of the person is estimated based on the descending widths of the plurality of acquired nasal skin temperatures with time when the human state is estimated. The human state estimation method described. The physiological quantity is the person's skin blood flow, blood pressure, or pulse wave propagation time,
The human state estimation method according to claim 1, wherein the human state includes a degree of sleepiness of the person. A human state estimation system for estimating a human state that is a human state,
A thermal sensation estimation unit for obtaining a thermal sensation index obtained by indexing the estimated thermal sensation of the person within a specified range;
A determination unit that determines whether the acquired thermal sensation index is within a predetermined range of the specified range;
The physiological state acquired when it is determined that the acquired thermal sensation index is within the predetermined range, and the human state is estimated based on the physiological amount reflecting the activity of the human autonomic nerve. A human state estimation system comprising a human state estimation unit.   The program which makes a computer perform the human state estimation method of any one of Claims 1-12.