JP6473110B2 - Brain activity estimation device - Google Patents

Description

  The present invention relates to a brain activity estimation device for estimating human brain activity.

  Conventionally, an electroencephalogram measurement method (EEG), magnetic resonance imaging (fMRI), or near infrared spectroscopy (NIRS) as disclosed in Patent Document 1 (Japanese Patent Laid-Open No. 2013-176406). Attempts have been made to estimate human brain activity using data detected by.

  However, when an electroencephalogram measurement method or near-infrared spectroscopy is adopted as a detection method, it is necessary to attach an electrode or a probe that requires pretreatment to the subject. In addition, when magnetic resonance imaging is adopted as a detection method, measurement can be performed only in the MRI room. In other words, when data is detected using any of the detection methods of electroencephalography, magnetic resonance imaging, and near infrared spectroscopy, the work required at the preparation stage is complicated or the conditions at the time of detection are limited. There is a problem of being.

  Accordingly, an object of the present invention is to provide a brain activity estimation apparatus that can easily estimate human brain activity.

  The brain activity estimation apparatus according to the first aspect of the present invention includes facial skin temperature acquisition means and brain activity estimation means. The facial skin temperature acquisition means detects the skin temperature of at least a part of the human face. The facial skin temperature acquisition means acquires facial skin temperature data including the detected temperature data and the position data of the detected part in time series. The brain activity estimation means estimates human brain activity based on the facial skin temperature data acquired by the facial skin temperature acquisition means. The brain activity estimating means decomposes the facial skin temperature data into a plurality of components by singular value decomposition, principal component analysis, or independent component analysis. Further, the brain activity estimating means extracts, as a determination component, a component whose amplitude of the component waveform is correlated with a change at rest of the brain and activation of the brain. Then, the brain activity estimating means determines whether or not the determination component is related to the human brain activity based on the presence or absence of a temperature change in a predetermined part of the human face.

  In the brain activity estimation apparatus according to the first aspect of the present invention, human brain activity is estimated based on time-series facial skin temperature data acquired by the facial skin temperature acquisition means. For this reason, in this brain activity estimation apparatus, it is possible to estimate human brain activity based on the skin temperature of the face without wearing a sensor that requires processing before wearing an electroencephalogram electrode, a probe, or the like. Therefore, human brain activity can be easily estimated as compared with the case where conventional detection methods such as electroencephalogram measurement, magnetic resonance imaging, and near infrared spectroscopy are used.

  The brain activity estimation apparatus according to the second aspect of the present invention is the brain activity estimation apparatus according to the first aspect. The facial skin temperature data includes data of a period during which a brain activity activation task is given to a human. . The brain activity estimating means sets a period when the brain activity activation task is not given to a human as a resting state of the brain. The brain activity estimating means sets a period during which a brain activity activation task is given to a human as brain activation. Then, the brain activity estimation means evaluates whether or not there is a correlation between the plurality of components. Furthermore, the brain activity estimating means extracts a component evaluated as being correlated among the plurality of components as a determination component. For this reason, this brain activity estimation device actually creates a situation in which the human brain is activated or calmed by giving or not giving a brain activity activation task to a human, and a correlation is made based on this. The relationship is evaluated and a determination component is extracted. Therefore, it is possible to reduce the possibility that a component having low relevance to human brain activity is extracted from a plurality of components as an extraction component.

  The brain activity estimation apparatus according to the third aspect of the present invention is the brain activity estimation apparatus according to the first aspect or the second aspect, wherein the brain activity estimation means includes temperature data included in the acquired facial skin temperature data for each predetermined time. The facial skin temperature data is converted into relative temperature data. Then, the brain activity estimating means decomposes each of the acquired facial skin temperature data and the created facial skin temperature data into a plurality of components by singular value decomposition, principal component analysis, or independent component analysis.

  Here, when the cool air, warm air, sunlight, or the like of the air conditioner hits the human face, the skin temperature of the face changes, and the detected skin temperature of the face may change. In such a case, a change (noise) in the facial skin temperature due to an external factor not related to the brain activity is mixed into the facial skin temperature data.

  In the brain activity estimation apparatus according to the third aspect of the present invention, the facial skin temperature converted into relative temperature data from the facial skin temperature data for each predetermined time corresponding to the temperature data detected by the facial skin temperature acquisition means. Data is created. In addition to facial skin temperature data corresponding to detected temperature data, facial skin temperature data converted to relative temperature data is also converted into a plurality of components by singular value decomposition, principal component analysis, or independent component analysis. Disassembled. That is, in this brain activity estimation apparatus, the facial skin temperature data converted into the relative temperature data is created, so that the relative change in the facial skin temperature at every predetermined time can be captured. For this reason, it is possible to detect a change in facial skin temperature due to an external factor not related to brain activity.

  The brain activity estimation apparatus according to a fourth aspect of the present invention is the brain activity estimation apparatus according to any one of the first to third aspects, wherein the facial skin temperature acquisition means is an infrared thermography apparatus. In this brain activity estimation apparatus, since the skin temperature of the face is detected by the infrared thermography apparatus, the burden on the human being to be detected is reduced compared to the case where the skin temperature of the face is detected by attaching a sensor or the like to the face. be able to.

  The brain activity estimation device according to the fifth aspect of the present invention is the brain activity estimation device according to any one of the first to fourth aspects, wherein the predetermined part is the periphery of the sinus and / or the forehead part.

  Here, the brain has a mechanism called a selective brain cooling system that cools the brain independently of the body temperature. As a selective brain cooling mechanism, it is known that heat generated by brain activity is exhausted using the forehead portion and the periphery of the sinuses.

  In the brain activity estimation apparatus according to the fifth aspect of the present invention, whether or not there is a change in the temperature around the sinuses and / or the forehead in relation to the determination component correlated with the change at rest and activation of the brain. Based on this, it is determined whether or not the component is related to human brain activity. As a result, it is possible to accurately identify components related to human brain activity.

  The brain activity estimation method according to the sixth aspect of the present invention comprises a facial skin temperature acquisition step and a brain activity estimation step. In the facial skin temperature acquisition step, the skin temperature of at least a part of the human face is detected, and facial skin temperature data including the detected temperature data and the position data of the detected part is acquired in time series. In the brain activity estimation step, human brain activity is estimated based on the facial skin temperature data acquired in the facial skin temperature acquisition step. The brain activity estimation step includes a decomposition step, an extraction step, and a determination step. In the decomposition step, the facial skin temperature data is decomposed into a plurality of components by singular value decomposition, principal component analysis, or independent component analysis. In the extraction step, out of a plurality of components decomposed in the decomposition step, a component whose amplitude of the component waveform is correlated with a change at rest of the brain and activation of the brain is extracted as a determination component. In the determination step, it is determined whether or not the determination component extracted in the extraction step is related to human brain activity based on the presence or absence of a temperature change at a predetermined part of the human face.

  In the brain activity estimation method according to the sixth aspect of the present invention, human brain activity is estimated based on time-series facial skin temperature data acquired in the facial skin temperature acquisition step. For this reason, compared with the case where the conventional detection methods, such as an electroencephalogram measurement method, a magnetic resonance imaging method, and near-infrared spectroscopy, are utilized, human brain activity can be estimated simply.

  The brain activity estimation apparatus according to the first aspect of the present invention can easily estimate human brain activity.

  In the brain activity estimation apparatus according to the second aspect of the present invention, it is possible to reduce the possibility that a component having low relevance to human brain activity is extracted from a plurality of components as an extraction component.

  The brain activity estimation apparatus according to the third aspect of the present invention can detect a change in facial skin temperature due to an external factor not related to brain activity.

  In the brain activity estimation apparatus according to the fourth aspect of the present invention, it is possible to reduce the burden on a human subject to be detected.

  The brain activity estimation apparatus according to the fifth aspect of the present invention can accurately identify components related to human brain activity.

  The brain activity estimation apparatus according to the sixth aspect of the present invention can easily estimate human brain activity.

The figure which shows an example of the relationship between the temperature in face skin temperature data, and time. The figure which shows a part of result of having analyzed the facial skin temperature data according to temperature conversion data. The figure which shows a part of result of having analyzed the facial skin temperature data according to temperature conversion data. The figure which shows a part of result of having analyzed the facial skin temperature data according to relative temperature conversion data. The figure which shows a part of result of having analyzed the facial skin temperature data according to relative temperature conversion data. The figure which shows the amplitude of the component waveform of the component 2, and the amplitude of (beta) wave among the measured brain waves. The figure which shows the amplitude of the component waveform of the component 3, and the amplitude of (beta) wave among the measured brain waves. The figure which shows a part of result of having analyzed the facial skin temperature data obtained by the control experiment. Schematic of the brain activity estimation apparatus which concerns on embodiment of this invention. The flowchart which shows the flow of a process at the time of identifying the component which shows the change of the skin temperature reflecting the brain function in a brain activity estimation apparatus.

  Before describing the embodiments of the present invention, first, the knowledge obtained by the present inventors, which is an important basis for the inventors of the present invention, will be described.

(1) Key points of knowledge by the present inventors It is known that human brain activities reflect human intellectual activities (cognitive activities, etc.) and emotional activities (activity such as pleasure / discomfort). . Conventionally, attempts have been made to estimate human brain activity. In this case, data detected by any one of the electroencephalography, magnetic resonance imaging, and near-infrared spectroscopy is used. There are many.

  By the way, for example, when an electroencephalogram measurement method is adopted as the detection method, it is necessary to attach an electroencephalogram electrode to the subject. And when attaching an electroencephalogram electrode, since it is necessary to make resistance between skin and an electrode small, the process of grind | polishing the skin or apply | coating a paste to an electrode is needed. In addition, when magnetic resonance imaging is adopted, there are restrictions on measurement conditions such as that measurement outside the MRI room is impossible and metal cannot be brought into the measurement room. Furthermore, when near-infrared spectroscopy is used, it is necessary to wear a probe on the subject. However, if the probe is worn for a long time, the subject feels pain or the subject's hair contacts the probe. Depending on the condition, it may not be detected accurately. In this way, when a conventional detection method is adopted to measure human brain activity, pre-processing when wearing an electroencephalogram electrode or a probe is necessary, measurement conditions are limited, etc. The burden on the subject increases.

  Therefore, there is a need for development of means that can reduce the burden on the subject and can easily estimate human brain activity.

  By the way, the brain has a mechanism called a selective brain cooling system that cools the brain independently of the body temperature. As a selective brain cooling mechanism, it is known that heat generated by brain activity is exhausted using the forehead portion and the periphery of the sinuses. If it does so, the change of the facial skin temperature accompanying brain activity will appear in the forehead part and / or the paranasal sinuses (brows and nose part periphery).

  Therefore, the present inventors thought that if the change in the skin temperature of the human face by this selective brain cooling mechanism can be captured, brain activity can be estimated without performing electroencephalogram measurement or the like. And if it is the skin temperature of a human face, the data can be acquired in time series, without attaching the sensor which needs pre-processing.

  However, it is known that the skin temperature of the human face changes under the influence of various factors such as the outside air temperature and / or autonomic nerve activity. For this reason, when it is attempted to estimate brain activity based on facial skin temperature data, it is very difficult to determine whether the detected facial skin temperature reflects only brain activity.

  As a result of intensive studies, the present inventors have detected the skin temperature of the face, and converted the facial skin temperature data including the detected temperature data and the position data (coordinate data) of the detected portion into a singular value decomposition method and a principal component analysis method. Or, it can be decomposed into multiple components using the independent component analysis method, and by analyzing the decomposed multiple components using a selective brain cooling mechanism, components that show changes in skin temperature reflecting brain activity I found that I could identify it. In other words, when estimating human brain activity based on facial skin temperature data, it is effective to decompose facial skin temperature data into a plurality of components by singular value decomposition, principal component analysis, or independent component analysis. It has been found. Then, the inventors of the present invention pay attention to this point, so that the present invention can estimate human brain activity without wearing a sensor that requires processing before wearing an electroencephalogram electrode or a probe. Reached.

(2) Facial skin temperature data acquisition method and facial skin temperature data analysis method FIG. 1A is a diagram illustrating an example of a relationship between temperature and time in acquired facial skin temperature data. FIG. 1B shows the relationship between the temperature and time in the facial skin temperature data created by converting the temperature data included in the facial skin temperature data at predetermined time intervals shown in FIG. 1A into relative temperature data. It is a figure which shows an example of a relationship.

  Next, a method for acquiring facial skin temperature data and a method for analyzing facial skin temperature data used by the inventors to obtain the above knowledge will be described.

  In this study, facial skin temperature data were obtained from six subjects. Specifically, the subject was seated on a chair installed in an artificial weather room maintained at a room temperature of 25 ° C., and facial skin temperature data was obtained from the entire face of the subject using an infrared thermography device. The infrared thermography apparatus detects infrared radiation energy emitted from an object with an infrared camera, converts the detected infrared radiation energy into the temperature of the object surface (in this embodiment, the temperature in degrees Celsius), and the temperature. This is a device capable of displaying and accumulating the distribution as facial skin temperature data (for example, image data representing the temperature distribution). In this test, NEC Avio Infrared Technology Co., Ltd. R300 was used as the infrared thermography device. The infrared camera was installed in front of the subject at a point 1.5 m away from the subject. The facial skin temperature data was acquired for 30 minutes.

  In this test, a brain function activation task was given to the subject while acquiring facial skin temperature data. As a result, facial skin temperature data when the brain was at rest and facial skin temperature data when the brain was activated were obtained.

  Here, as the brain function activation task, the subject performs calculation or psychological work such as recognition of numerical values, shapes and colors, or memory of symbols, characters or languages based on the video displayed on the display device or the like. Is mentioned. In this study, we adopted “multiply mental arithmetic” as a brain function activation task, and asked the subject to calculate the numbers displayed in the writing format on the display device and to input the answers to the keyboard. In this test, a brain function activation task was given to the subject continuously for 10 minutes after 5 minutes from the start of acquisition of facial skin temperature data.

  For analysis of facial skin temperature data, singular value decomposition was performed on the acquired facial skin temperature data using SVLAB (Singular Value Decomposition) of MATLAB (registered trademark) as an analysis tool. In singular value decomposition, all facial skin temperature data (data for 30 minutes) acquired in time series are targeted, the factor is time data every 30 seconds (60 time points in 30 minutes), and the measure is the period (30 Second) facial skin temperature data (240 × 320 pixels). Then, the facial skin temperature data X was decomposed into a plurality of components by singular value decomposition, and a temporal distribution V of each component, a spatial distribution U, and a singular value S indicating the size of each component were calculated. These relationships are expressed by the following equations. V ′ is a matrix in which the rows and columns of V are exchanged.

  And the time distribution V and the spatial distribution U of each component calculated | required by the singular value decomposition were plotted on the graph, and the component waveform diagram and temperature distribution diagram of each component were created.

  Furthermore, the analysis for identifying the component which shows the change of the skin temperature reflecting brain activity was performed about the created component waveform diagram and temperature distribution diagram of each component.

  About the component waveform diagram of each component, the presence or absence of correlation between the amplitude of the component waveform and the resting time and activation of the brain was analyzed. Specifically, it was evaluated whether or not there is a correlation between the amplitude shown in the component waveform diagram of each component and the rest period / activation period of the brain. In this test, during the period of acquiring facial skin temperature data, 5 minutes from the start of data acquisition, which is a period in which no brain function activation task is given to the subject, for 5 minutes. Data acquisition that is a period during which a brain function activation task is given to a subject, with a period of 15 minutes from the start of data acquisition and the time of 15 minutes from the start of data acquisition to the end of data acquisition The period of 10 minutes from the time when 5 minutes passed from the start to the time when 10 minutes passed was defined as the time of brain activation. Then, the presence / absence of a correlation between the amplitude shown in the component waveform diagram of each component and the rest of the brain and the activation of the brain was evaluated. In addition, about the presence or absence of correlation, statistical correlation analysis was conducted and it was judged that there was a correlation when a significance level ((alpha)) is 0.05 or less.

  Regarding the temperature distribution chart of each component, the presence or absence of temperature change at a predetermined part of the face was analyzed. Specifically, in the temperature distribution chart of each component, it was evaluated whether there was a temperature change around the sinuses and the forehead. In addition, regarding the presence or absence of temperature changes around the sinuses and the forehead in the temperature distribution chart, the presence or absence of temperature changes by visual inspection, or the temperature around the sinuses and the forehead is determined from the average temperature of the entire measurement data. Whether there was a difference of 1 standard deviation (SD) or more was used as a reference for the presence or absence of temperature change.

  Since the polarity (plus or minus) of the facial skin temperature data X is determined by the relationship between the spatial distribution U, the singular value S, and the time distribution V, the polarity is inverted in the component waveform diagram and temperature distribution diagram of each component. May appear. For this reason, regarding the evaluation of the component waveform diagram and the temperature distribution diagram, the polarity is not considered as an evaluation target.

  Here, in this infrared thermography apparatus, as described above, infrared radiation energy detected from an object is converted into temperature, and the temperature distribution is used as facial skin temperature data. By the way, when the skin temperature of the face is acquired by using an infrared thermography apparatus for human subjects, temperature changes (so-called noise) that are not related to various brain activities such as facial movement and / or autonomic nerve activity are also faced. It is acquired as skin temperature data (see FIG. 1A). Therefore, in order to detect such a temperature change not related to the brain activity, relative facial skin temperature data in which the total average value of the temperature data included in the facial skin temperature data every 30 seconds is “0” is calculated. The facial skin temperature data created is also subjected to singular value decomposition using the MATLAB® SVD as an analysis tool, and a component waveform diagram and temperature distribution diagram of each component according to the singular value S are created. Then, an analysis was performed to identify a component showing a change in skin temperature reflecting brain activity.

  In the following, for convenience of explanation, the facial skin temperature data acquired by the infrared thermography device is referred to as “facial skin temperature data according to the temperature conversion data”, and is converted into a temperature every predetermined time (every 30 seconds in this test). The relative facial skin temperature data in which the total average value of the temperature data included in the facial skin temperature data according to the data is “0” is referred to as “facial skin temperature data according to the relative temperature conversion data”.

  For one of the six subjects, in addition to detecting the facial skin temperature using an infrared thermography device, electrodes were connected to the subject's scalp to measure brain waves, and during wakefulness and consciousness became tense. The correlation between the amplitude of the β wave (the brain wave having a frequency of 14 to 30 Hz), which is known as the waveform appearing at the time, and the amplitude of the component waveform diagram was also evaluated. In the electroencephalogram measurement, electrodes were arranged at six parts (F3, F4, C3, C4, Cz, Pz) based on the international method 10-20.

  By the way, it is conceivable that the subject's head moves up and down while the subject is given a brain function activation task. Then, the position of the subject's face with respect to the infrared camera changes. In order to verify whether or not the change in the position of the face affects the change in the skin temperature, a control test was performed on one subject. In the control test to verify the influence of the subject's movement when acquiring the facial skin temperature data, the facial skin temperature data of the subject is acquired using the infrared thermography device as in the above test, but the brain function activation task is Subjects were asked to press the keyboard at random times even when they were not given (ie, when the brain was at rest). The facial skin temperature data according to the temperature conversion data obtained by the control experiment and the facial skin temperature data according to the relative temperature conversion data are also subjected to singular value decomposition using the MATLAB® SVD as an analysis tool. Then, a component waveform diagram and a temperature distribution diagram of each component according to the singular value S were created, and an analysis for identifying a component showing a change in skin temperature reflecting brain activity was performed.

(3) Analysis result of facial skin temperature data FIGS. 2 and 3 are diagrams showing a part of the result of analyzing facial skin temperature data according to temperature conversion data. FIG. 2A shows a component waveform diagram of the component 2 of the subject 1. FIG. 2B shows a temperature distribution diagram of the component 2 of the subject 1. FIG. 3A shows a component waveform diagram of the component 3 of the subject 1. FIG. 3B shows a temperature distribution diagram of the component 3 of the subject 1. 4 and 5 are diagrams illustrating a part of the result of analyzing the facial skin temperature data according to the relative temperature conversion data. FIG. 4A shows a component waveform diagram of the component 2 of the subject 1. FIG. 4B shows a temperature distribution diagram of the component 2 of the subject 1. FIG. 5A shows a component waveform diagram of the component 3 of the subject 1. FIG. 5B shows a temperature distribution diagram of the component 3 of the subject 1. 6 and 7 are diagrams showing the relationship between the amplitude of the component waveform and the electroencephalogram. FIG. 6 is a diagram showing the amplitude of the component waveform of the component 2 of the subject 1 and the amplitude of the β wave among the measured brain waves. FIG. 7 is a diagram illustrating the amplitude of the component waveform of the component 3 of the subject 1 and the amplitude of the β wave among the measured brain waves. FIG. 8 is a diagram showing a part of the result of analyzing the facial skin temperature data obtained in the control experiment. FIG. 8A shows a component waveform diagram of component 3. FIG. FIG. 8B shows a temperature distribution diagram of the component 3.

  Table 1 shows the analysis results of facial skin temperature data for each subject.

  Of the plurality of components obtained by decomposing time-series facial skin temperature data by singular value decomposition from the results obtained by the above analysis of facial skin temperature data, component 2 and / or component 3 and human It was confirmed that there was a significant correlation with brain activity.

  Further, as shown in FIGS. 6 and 7, from the results of the electroencephalogram analysis, a significant correlation was confirmed between the amplitude of each component waveform of component 2 and component 3 and the amplitude of the β wave of the electroencephalogram.

  Furthermore, in the control experiment, there was a significant correlation between component 3 and human brain activity even when the subject was in motion while acquiring facial skin temperature data (see FIG. 8). . From this, it was recognized that among the plurality of components, the movement of the subject at the time of acquiring facial skin temperature data was not affected for component 3.

  From these results, the present inventors obtained the following knowledge.

  The time-series facial skin temperature data obtained from the test subject is decomposed into a plurality of components by singular value decomposition, and as a result of analyzing each decomposed component, component 3 of the plurality of components is a component related to brain activity. Admitted. In other words, time-series facial skin temperature data is decomposed into multiple components by singular value decomposition, components that correlate with brain rest / activation are extracted from the decomposed multiple components, and selective brain cooling mechanism is selected for the extracted components It was found that the component showing the change in skin temperature reflecting the brain activity can be identified from a plurality of components by performing the analysis using. From this, the present inventors have obtained the knowledge that brain activity can be estimated based on the skin temperature of the human face.

(4) Brain Activity Estimation Device Next, based on the knowledge described above, a brain activity estimation device according to an embodiment of the present invention that has been completed by the present inventors will be described. The brain activity estimation apparatus according to the present invention is not limited to the following embodiments, and can be appropriately changed without departing from the scope.

  FIG. 9 is a schematic diagram of the brain activity estimation apparatus 10 according to the embodiment of the present invention. FIG. 10 is a flowchart showing the flow of processing when the brain activity estimating apparatus 10 identifies a component indicating a change in skin temperature reflecting the brain function.

  The brain activity estimation device 10 is a device for estimating an individual's brain activity from the skin temperature of the face of the individual (subject). As shown in FIG. 9, the brain activity estimation apparatus 10 includes a facial skin temperature acquisition unit 20 and a brain activity estimation unit 30.

  The facial skin temperature acquisition means 20 detects the skin temperature of at least a part of the individual's face, and acquires facial skin temperature data including the detected temperature data and the position data of the detected part in time series (step S1). In addition, the facial skin temperature acquisition means 20 of this embodiment is an infrared thermography apparatus, and has the infrared camera 21 and the process part 22, as shown in FIG. The infrared camera 21 is for detecting infrared radiant energy emitted from an individual's face. In this embodiment, it is assumed that the infrared camera 21 detects infrared radiation energy from the entire face of the individual. The processing unit 22 converts the infrared radiant energy detected by the infrared camera 21 into temperature to obtain temperature data, and the temperature distribution of the facial skin temperature over the entire face using the position where the infrared radiant energy is detected as position data (coordinate data). A diagram is created, and the created temperature distribution diagram is processed as facial skin temperature data corresponding to the temperature conversion data. The facial skin temperature data corresponding to the temperature conversion data is accumulated in a storage unit (not shown) included in the processing unit 22.

  Here, in the present embodiment, the temperature distribution map of the facial skin temperature in the entire face is created in the processing unit 22, but the present invention is not limited to this, and facial skin including at least the periphery of the sinuses and / or the forehead part is included. A temperature distribution map of temperature may be created, and this may be used as facial skin temperature data according to the temperature conversion data.

  Moreover, in this embodiment, while the facial skin temperature acquisition means 20 is acquiring facial skin temperature data according to temperature conversion data, a brain function activation task is given to a person for a certain period. That is, the facial skin temperature data corresponding to the temperature conversion data acquired by the facial skin temperature acquisition means 20 includes data for a period during which a brain function activation task is given to an individual. The brain function activation task given to an individual is not particularly limited as long as it is estimated that the brain is in an activated state. For example, depending on the purpose of use of the brain activity estimation apparatus 10 You may be comprised so that the content may be determined suitably.

  The brain activity estimation unit 30 estimates human brain activity based on the facial skin temperature data corresponding to the temperature conversion data acquired by the facial skin temperature acquisition unit 20. Specifically, the brain activity estimation unit 30 includes a conversion unit 31, an analysis unit 32, and an estimation unit 33, as shown in FIG.

  The conversion unit 31 converts the temperature data included in the facial skin temperature data corresponding to the temperature conversion data into relative temperature data, and responds to the facial skin temperature data based on the converted relative temperature data, that is, relative temperature conversion data. Facial skin temperature data is created (step S2). Specifically, the conversion unit 31 uses the average value of the temperature data included in the facial skin temperature data corresponding to the temperature conversion data every predetermined time (for example, 30 seconds) as a reference value, and converts the temperature data into a relative value. Convert to temperature data. And the conversion part 31 produces the face skin temperature data according to relative temperature conversion data using the converted relative temperature data and position data.

  The analysis unit 32 converts the facial skin temperature data according to the time-series temperature conversion data and the facial skin temperature data according to the relative temperature conversion data into a plurality of components by singular value decomposition, principal component analysis, or independent component analysis. Decompose (step S3). In the present embodiment, the analysis unit 32 applies the SVLAB of MATLAB (registered trademark) for each of the facial skin temperature data according to the acquired temperature conversion data and the facial skin temperature data according to the converted relative temperature conversion data. Use as an analysis tool to perform singular value decomposition. The singular value decomposition is performed with the factor of time data for a predetermined period (for example, 30 seconds) for the facial skin temperature data according to the temperature conversion data acquired in time series and the facial skin temperature data according to the relative temperature conversion data, The measure is performed as facial skin temperature data corresponding to the temperature conversion data in that period and facial skin temperature data corresponding to the relative temperature conversion data. Then, by singular value decomposition, each of the facial skin temperature data according to the temperature conversion data and the facial skin temperature data according to the relative temperature conversion data is decomposed into a plurality of components, and the time distribution, spatial distribution, and A singular value indicating the magnitude is calculated.

In addition, the analysis unit 32 determines whether or not each component satisfies the first condition and the second condition in order to identify a component indicating a change in skin temperature reflecting brain activity from a plurality of components decomposed by singular value decomposition. (Step S4a, Step S4b, Step S5a, and Step S5b). In the present embodiment, the analysis unit 32 first determines whether or not the first condition is satisfied for each component based on the facial skin temperature data corresponding to the temperature conversion data (step S4a), and in step S4a. It is determined whether or not the second condition is satisfied for the component based on the facial skin temperature data corresponding to the temperature conversion data determined to satisfy the first condition (step S4b). Of the components based on the facial skin temperature data corresponding to the relative temperature conversion data, the first condition is satisfied only for components that match the components determined to satisfy the first condition and the second condition in step S4a and step S4b. (Step S5a), and then the second condition is satisfied for the component based on the facial skin temperature data corresponding to the relative temperature conversion data determined to satisfy the first condition in Step S5a. It is determined whether or not it has been performed (step S5b). However, the order of the determination in the analysis unit 32 is not limited to this embodiment. For example, each component based on the facial skin temperature data corresponding to the temperature conversion data and the facial skin temperature corresponding to the relative temperature conversion data It may be determined whether or not each component based on the data satisfies the first condition and the second condition, and a component having a matching determination result may be finally extracted.

  The first condition is a condition that the amplitude of the component waveform of the component decomposed by the singular value decomposition is correlated with a change at rest of the brain and activation of the brain. The analysis unit 32 extracts a component that satisfies the first condition from the plurality of components as a determination component. In the present embodiment, there is a certain period during which the brain function activation task is given to an individual while acquiring facial skin temperature data corresponding to the temperature conversion data. The analysis unit 32 activates the brain function with the period when the brain function activation task is not given to the individual as the rest of the brain and the period when the brain function activation task is given to the individual as the brain activation. The period during which the assignment is given and the period when the assignment is not given and the component waveform of each component are compared and analyzed. The analysis unit 32 uses the comparison analysis result based on the component waveform data to evaluate whether or not there is a correlation between the component waveform of each component and the resting time and activation of the brain. Among them, the component evaluated as having a correlation is extracted as a determination component that satisfies the first condition. On the other hand, the analysis unit 32 determines that a component evaluated as having no correlation among the plurality of components is not a component that does not satisfy the first condition and shows a temperature change reflecting human brain activity (step S6).

  Here, in the present embodiment, the brain function activation task is given to the individual for a certain period when acquiring the facial skin temperature data according to the temperature conversion data, and based on this, the analysis unit 32 extracts the determination component. However, the content of the first condition, that is, the determination component extraction means in the analysis unit 32 is not limited to this. For example, when a component showing a component waveform correlated with the resting time of the brain and the activation time of the brain is specified among the plurality of components by performing an experiment or the like in advance, The component specified from the components is extracted as a determination component. When the human activity known to be related to the activation / rest of the brain such as eye movement or tapping is detected in the brain activity estimation apparatus, the analysis unit 32 detects the detection result and each component. A determination component may be extracted from a plurality of components by comparing and evaluating the component waveform. Note that the criterion for determining whether or not the first condition is satisfied by the analysis unit 32 is appropriately determined by simulation, experiment, desktop calculation, or the like according to the purpose of use of the brain activity estimation apparatus 10 or the like.

  The second condition is a condition that there is a temperature change in a predetermined part of the human face in the extracted determination component. The analysis unit 32 determines a component that satisfies the second condition among the determination components as a component that is highly likely to be related to human brain activity, and extracts it as a candidate component. That is, the analysis unit 32 determines whether or not the determination component is related to human brain activity based on the presence or absence of a temperature change at a predetermined part of the human face. Specifically, the analysis unit 32 determines whether a temperature change has occurred around the sinuses and / or the forehead based on the extracted temperature distribution data of the determination component, and the temperature change has occurred. In this case, it is determined that the determination component is a component that is highly likely to be related to human brain activity that satisfies the second condition, and is extracted as a candidate component. On the other hand, when no temperature change occurs around the sinuses and / or the forehead part, the analysis unit 32 does not satisfy the second condition and shows a change in skin temperature reflecting brain activity. It is determined that it is not a component (step S6). Note that the criterion for determining whether or not the second condition is satisfied by the analysis unit 32 is appropriately determined by simulation, experiment, desktop calculation, or the like according to the purpose of use of the brain activity estimation apparatus 10 or the like.

  And the analysis part 32 identifies the component determined to satisfy | fill 2nd conditions in step S5b as a component which shows the change of the skin temperature reflecting brain activity (step S7). That is, the component identified as the component indicating the change in skin temperature reflecting the brain activity in step S7 is the candidate component extracted by decomposing and analyzing the facial skin temperature data corresponding to the temperature conversion data by singular value decomposition And the candidate skin component extracted by analyzing the facial skin temperature data corresponding to the relative temperature conversion data by decomposing and analyzing by singular value decomposition. Note that candidate components that do not match in both analyzes are determined not to be components indicating changes in skin temperature reflecting brain activity in step S6.

  The estimation unit 33 estimates the human brain activity based on the component identified by the analysis unit 32 as the component indicating the change in skin temperature reflecting the human brain activity. Specifically, the estimation unit 33 is based on the component waveform data of the component identified by the analysis unit 32, and when the facial skin temperature data is acquired, whether the individual's brain is in an active state or the brain is active. Estimate whether it was not in the state.

  With this configuration, the brain activity estimation apparatus 10 can estimate human brain activity based on the facial skin temperature. Then, the estimation result by the estimation unit 33 is displayed on a display or the like, so that it is possible to know whether the individual's brain is in an active state or in an inactive state.

  Further, when facial skin temperature data is further acquired by the facial skin temperature acquisition means 20 after the component indicating the change in skin temperature reflecting the brain activity is identified in the analysis unit 32, in the brain activity estimation device 10. Furthermore, the acquired facial skin temperature data is decomposed into a plurality of components by singular value decomposition, and only the identified components are analyzed, so that the individual's brain is in an active state when the facial skin temperature data is acquired. It may be estimated whether or not the brain was inactive. By using such a brain activity estimation apparatus 10 to control equipment such as an air conditioner, it is possible to approach an indoor environment suitable for the individual.

(5) Features (5-1)
In order to estimate human brain activity, when data detected by any of the electroencephalography, magnetic resonance imaging, and near-infrared spectroscopy is used, processing is performed before the attachment of electroencephalogram electrodes, probes, etc. Necessary sensors must be used, and the measurement location is limited. Moreover, since the apparatus used for these detection methods is very expensive, if it is going to manufacture the brain activity estimation apparatus provided with these apparatuses, manufacturing cost will become large.

  In the present embodiment, human brain activity is estimated based on facial skin temperature data corresponding to time-series temperature conversion data acquired by the facial skin temperature acquisition means 20. For this reason, it is possible to estimate human brain activity based on the skin temperature of the face without wearing a sensor that requires processing before wearing an electroencephalogram electrode, a probe, or the like. Therefore, human brain activity can be estimated easily compared to the case where conventional detection methods such as electroencephalogram measurement, magnetic resonance imaging, and near infrared spectroscopy are used.

  Moreover, in this embodiment, since it is only necessary to be able to detect the skin temperature of at least a part of the face, the manufacturing cost can be reduced as compared with a brain activity estimation apparatus including a device used in a conventional detection method.

  By the way, as an existing study, an average value of all temperature data included in time-series facial skin temperature data acquired by the facial skin temperature acquisition means 20 is calculated, and analysis of facial skin temperature data according to the calculated average value is performed. The average approach of estimating human brain activity by performing However, the facial skin temperature data contains noise other than what actually reflects brain activity, and the influence of noise becomes relatively large by analyzing the temperature data of a part of the body, The average approach could not accurately estimate brain activity.

  Therefore, as a result of earnest examination, the present inventors decomposed the time series facial skin temperature data acquired by the facial skin temperature acquisition means 20 into a plurality of components by singular value decomposition, principal component analysis or independent component analysis, We came up with a component analysis approach to identify components related to brain activity from the decomposed components. In the component analysis approach, since all temperature data are decomposed, noise-containing components can be removed, and as a result, the brain activity can be accurately estimated as compared with the average value approach.

  Furthermore, by using the brain activity estimation apparatus 10 of the present invention, medical diagnosis such as early detection of dementia can be easily performed. Moreover, the brain activity estimation apparatus 10 of the present invention can be used for objective comfort evaluation, or can be applied to evaluation of learning effect, objective evaluation of interest, and false detection apparatus. . Further, in the brain activity estimation apparatus, the analysis unit 32 performs a comparative analysis on the detection results of human movements known to be related to activation / rest of the brain, such as eye movement or tapping, and the component waveforms of the respective components. When the evaluation component is extracted from the plurality of components by the evaluation, this brain activity estimation device can be used for evaluation of sleepiness during driving of the automobile.

(5-2)
By the way, when time-series facial skin temperature data is acquired, the human brain is activated or rested by being given or not given a brain function activation task. Is a component that is highly likely to be a component that shows a change in skin temperature that reflects brain activity, as a component that has a correlation between the component waveform of each component and when the brain is activated and at rest You can say that.

  In the present embodiment, while facial skin temperature data corresponding to temperature conversion data is acquired by the facial skin temperature acquisition means 20, a brain function activation task is given to the individual for a certain period. That is, in the present embodiment, a situation is created in which the human brain is activated or calmed by actually giving or not giving a brain function activation task to an individual. Then, the facial skin temperature data acquired in this way is decomposed into a plurality of components by singular value decomposition, and the correlation between the component waveform and the brain activation time and rest time is evaluated and correlated for each component. A component is extracted from a plurality of components as a determination component. For this reason, for example, compared with a case where a predetermined component specified in advance by experiments or the like is extracted from a plurality of components as an extraction component, a plurality of components that are less relevant to human brain activity are extracted as a plurality of extraction components. The possibility of being extracted from the components can be reduced.

(5-3)
When cold air, warm air, sunlight, or the like of an air conditioner hits a human face, the skin temperature of the face changes, and the detected skin temperature of the face may also change. If it does so, there exists a possibility that the change (noise) of the facial skin temperature by the external factor unrelated to brain activity may mix in facial skin temperature data.

  In the present embodiment, facial skin temperature data corresponding to the relative temperature conversion data based on the temperature data converted from the facial skin temperature data every predetermined time corresponding to the temperature conversion data into relative temperature data is created. By creating facial skin temperature data corresponding to the relative temperature conversion data, it is possible to capture relative changes in facial skin temperature at predetermined time intervals. For this reason, it is possible to detect a change in facial skin temperature due to an external factor not related to brain activity.

  In the present embodiment, not only the facial skin temperature data corresponding to the temperature conversion data acquired by the facial skin temperature acquisition means 20, but also the relative temperature data based on the temperature data converted to the relative temperature data. The facial skin temperature data is also decomposed into a plurality of components by singular value decomposition, and analysis for each component is performed. For this reason, a component including a change in facial skin temperature due to an external factor not related to brain activity can be removed as a noise component. This makes it possible to accurately identify components related to human brain activity.

(5-4)
The facial skin temperature acquisition means 20 in this embodiment is an infrared thermography apparatus. For this reason, facial skin temperature data can be acquired in a state where the subject individual is not touched, that is, in a non-contact state. Therefore, compared with the case where the sensor is attached to the face and the temperature data of the skin of the face is detected, the burden on the subject individual can be reduced.

(5-5)
The brain has a selective brain cooling mechanism that cools the brain independently of body temperature. As a selective brain cooling mechanism, it is known that heat generated by brain activity is exhausted using the forehead portion and the periphery of the sinuses. If it does so, the change of the facial skin temperature accompanying brain activity will appear in the forehead part and / or the paranasal sinuses (brows and nose part periphery).

  In the present embodiment, it is determined whether or not a temperature change has occurred in the vicinity of the sinuses and / or the forehead for the determination component that correlates the component waveform with the rest of the brain and the activation of the brain. . Therefore, it is possible to accurately determine whether or not the component extracted as the determination component is a component related to human brain activity.

(6) Modification (6-1) Modification A
In the above embodiment, the brain activity estimating means 30 has the conversion unit 31, and facial skin temperature data corresponding to the relative temperature conversion data is created by the conversion unit 31. And the analysis part 32 respond | corresponded not only the face skin temperature data according to the temperature conversion data acquired by the face skin temperature acquisition means 20, but the relative temperature data based on the temperature data converted into relative temperature data. The facial skin temperature data is also decomposed into a plurality of components by singular value decomposition, and each component is analyzed.

  Instead of this, the brain activity estimating means 30 may not have the conversion unit 31. In this case, it is possible to omit processing for creating facial skin temperature data corresponding to the relative temperature conversion data or analyzing data based on the facial skin temperature data corresponding to the relative temperature conversion data.

  However, in order to accurately identify a component related to human brain activity, the brain activity estimation means 30 has a conversion unit 31 as in the above embodiment, and the analysis unit 32 performs facial skin temperature acquisition means. A plurality of facial skin temperature data corresponding to the relative temperature data based on the temperature data converted to the relative temperature data as well as the facial skin temperature data corresponding to the temperature converted data acquired by 20 are obtained by singular value decomposition. It is desirable that each component is analyzed after being decomposed into these components.

(6-2) Modification B
The facial skin temperature acquisition means 20 in the above embodiment is an infrared thermography device that can acquire temperature data in a non-contact state with an object.

  However, if the skin temperature of at least a part of an individual's face is detected and the facial skin temperature data including the detected temperature data and the position data of the detected part can be acquired in time series, the facial skin temperature acquisition means is infrared. It is not limited to a thermography device.

  For example, the facial skin temperature acquisition means may be a device including a temperature sensor. Specifically, a temperature sensor is attached to a predetermined part of an individual's face, and time-series facial skin temperature data is based on temperature data detected by the temperature sensor and position data of the part where the temperature sensor is attached. May be acquired. As described above, even when facial skin temperature data is acquired in a state in which the temperature sensor is in contact with the target individual, the temperature sensor does not require pre-wearing processing like an electroencephalogram electrode. Compared with conventional detection methods such as measurement, magnetic resonance imaging, and near infrared spectroscopy, data can be easily acquired. Thereby, human brain activity can be estimated simply.

  Since the present invention can easily estimate human brain activity, it can be effectively applied to a device that needs to estimate human brain activity.

DESCRIPTION OF SYMBOLS 10 Brain activity estimation apparatus 20 Facial skin temperature acquisition means 30 Brain activity estimation means

JP 2013-176406 A

Claims (7)

A data acquisition unit that acquires time-series data of a thermal image of the subject's face;
A recognizing unit for recognizing that brain function activation information is given to the subject or a providing unit for giving brain function activation information to the subject;
When the providing unit gives brain function activation information to the subject, or when the recognition unit recognizes that brain function activation information is given to the subject, a thermal image obtained by the data obtaining unit An estimation unit that estimates the brain activity of the subject based on series data;
A brain activity estimation device comprising: When the providing unit gives brain function activation information to the subject, or when the recognizing unit recognizes that brain function activation information is given to the subject, the data acquisition unit is the estimation unit Estimating the brain activity of the subject based on time series data of a thermal image of the face including at least the sinus periphery of the subject among the obtained thermal images,
The brain activity estimation apparatus according to claim 1. The estimation unit decomposes a set of time-series data of the thermal image based on a spatial distribution in addition to a time distribution, and estimates the brain activity of the subject based on the decomposed data.
The brain activity estimation apparatus according to claim 1 or 2 . The estimation unit estimates the subject's brain activity based on changes in the spatial distribution, around the sinuses and / or the forehead,
The brain activity estimation apparatus according to claim 3 . The estimation unit extracts a plurality of sets of time distribution and spatial distribution from a set of time-series data of the thermal image, and estimates the brain activity of the subject based on the extracted data .
The brain activity estimation apparatus according to any one of claims 1 to 4 . The estimation unit analyzes the correlation between the brain function activation information and a plurality of time distributions, and then analyzes the spatial distribution to estimate the brain activity of the subject.
The brain activity estimation apparatus according to claim 5. The estimation unit singularly decomposes a set of time-series data of the thermal image , extracts a plurality of sets of time distribution and spatial distribution, and estimates brain activity based on the extracted data .
The brain activity estimation apparatus according to any one of claims 1 to 6 .