JP2018094030A - Pain determination device and medicine effect determination device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a pain determination device determining a pain state in a subject and a medicine effect determination device determining an effect of a medicine to a pain or the like.SOLUTION: A pain determination device 1000 includes a brain function activation information provision unit 1041, a face change information acquisition unit 1042, a face change information decomposition unit 1043, a determination component extraction unit 1044, and a pain determination unit 1045. The face change information acquisition unit 1042 acquires "face change information" representing time series change of face data of a subject 300 to whom brain activation information is provided. The face change information decomposition unit 1043 decomposes the face change information by singular value decomposition or the like into a plurality of components 1, 2, 3, ... The determination component extraction unit 1044 extracts a component associated with the brain function activation information as a "determination component" from the plurality of components 1, 2, 3, ... The pain determination unit 1045 determines a pain state of the subject 300 based on an extraction result by the determination component extraction unit.SELECTED DRAWING: Figure 23

Description

  The present invention relates to a pain determination device and a drug effect determination device.

  In recent years, by electroencephalography (EEG), magnetic resonance imaging (fMRI), and near infrared spectroscopy (NIRS) as disclosed in Patent Document 1 (Japanese Patent Laid-Open No. 2013-176406). Attempts have been made to estimate human brain activity using the detected data. Also, applications such as determination of human physical condition and mental state from the estimated brain activity are being studied.

  However, in the electroencephalogram measurement method and near-infrared spectroscopy, a pretreatment such as the subject having to wear an electrode is necessary. Moreover, in magnetic resonance imaging, it is necessary to measure in a predetermined MRI room. In short, with these methods, there is a problem that work at the preparation stage is complicated, and conditions at the time of measurement are limited. In addition, all of these methods are very expensive. As a result, with these methods, it may be difficult to determine the physical condition and mental state of the subject.

  An object of the present invention is to provide an apparatus and a method for easily determining a mental state or a physiological state of a subject. In particular, an object of the present invention is to provide a pain determination device that determines a pain state in a subject and a drug effect determination device that determines the effect of a drug on pain and the like.

  The pain determination apparatus according to the first aspect of the present invention includes a brain function activation information providing unit, a face change information acquiring unit, a face change information decomposing unit, a determination component extracting unit, and a pain determining unit. The brain function activation information providing unit provides the target person with brain function activation information for activating human brain functions. The face change information acquisition unit acquires face change information indicating a time-series change in the face data of the subject. The face change information decomposing unit decomposes the face change information into a plurality of components by singular value decomposition, principal component analysis, or independent component analysis. The determination component extraction unit extracts a component related to the brain function activation information from the plurality of components as a determination component. The pain determination unit determines the presence or absence of pain in the subject based on the extraction result by the determination component extraction unit.

  In the pain determination apparatus according to the first aspect, the determination component is extracted from a plurality of components obtained by performing singular value decomposition, principal component analysis, and independent component analysis on the facial change information. The presence or absence of the subject's brain activity can be easily estimated without using necessary electrodes or the like. Thereby, the pain state in the subject can be easily determined based on the determination component corresponding to the brain function of the subject.

  The pain determination apparatus according to a second aspect of the present invention is the pain determination apparatus according to the first aspect, wherein the determination component extraction unit is based on a correlation waveform between a determination waveform corresponding to brain function activation information and a plurality of components. Thus, the determination component is extracted. With such a configuration, the determination component corresponding to the brain function of the subject can be specified.

  The pain determination apparatus according to the third aspect of the present invention is the pain determination apparatus according to the first aspect or the second aspect, wherein the brain function activation information providing unit outputs auditory stimulation information or vision for stimulating hearing as brain function activation information. Provide stimulating visual stimulus information to the subject. By using the auditory stimulus information or the visual stimulus information, it is possible to realize a configuration in which there is no correlation when there is pain and there is a correlation when there is no pain. Thereby, the pain state can be determined based on whether or not the activation of the brain function based on the brain function activation information is inhibited (masked) by the pain. And in such a structure, since pain determination can be performed only by providing auditory stimulus information or visual stimulus information at an arbitrary timing, the pain state can be determined with a simple structure.

  In the pain determination device according to the fourth aspect of the present invention, in the pain determination device according to the first to third aspects, the pain determination unit determines a pain state according to the amount of change in the determination component. Thereby, according to the variation | change_quantity from the reference | standard acquired previously, a pain state level can be determined easily.

  The pain determination device according to the fifth aspect of the present invention is the pain determination device according to the first to fourth aspects, further comprising a determination information storage unit. The determination information storage unit calculates the amount of change in a predetermined range of the correlation value of the determination component calculated for the brain function activation information from the reference correlation value of the reference determination component calculated for the brain function activation information. The determination information is stored in association with the pain state level. The pain determination unit calculates a correlation value of the determination component with respect to the brain function activation information, and determines the pain state level of the subject based on the calculated correlation value and determination information. With such a configuration, it is possible to easily determine the pain state level using a reference determination component acquired in advance.

  The pain determination device according to the sixth aspect of the present invention is the pain determination device according to the first to fifth aspects, wherein the pain determination unit calculates a correlation value of the determination component with respect to the brain function activation information, and calculates the correlation The pain state level of the subject is determined based on the value and the determination information. Here, the determination information providing device on the network calculates the correlation value of the determination component calculated for the brain function activation information from the reference correlation value of the reference determination component calculated for the brain function activation information. A determination information storage unit that stores the amount of change in the predetermined range as determination information in association with the pain state level. With such a configuration, it is possible to determine the pain state level of the subject using the determination information providing apparatus on the network.

  The pain determination device according to the seventh aspect of the present invention is the pain determination device according to the first to sixth aspects, wherein the brain function activation information providing unit and the face change information acquiring unit are incorporated in a helmet-type device. is there. With such a configuration, highly reliable face change information can be acquired, and highly reliable pain determination can be easily realized.

  A drug effect determination device according to an eighth aspect of the present invention includes a face change information acquisition unit, a face change information decomposition unit, a determination component extraction unit, and a drug effect determination unit. The face change information acquisition unit acquires face change information indicating a time-series change of the face data of the subject to whom the medicine is administered. The face change information decomposing unit decomposes the face change information into a plurality of components by singular value decomposition, principal component analysis, or independent component analysis. The determination component extraction unit extracts, as a determination component, a component related to the administration of the drug from a plurality of components. The drug effect determination unit determines the effect of the drug based on the extraction result by the determination component extraction unit.

  In the drug effect determination apparatus according to the eighth aspect, the determination component is extracted from a plurality of components obtained by performing singular value decomposition, principal component analysis, and independent component analysis on the face change information, so that pre-processing before wearing Therefore, the presence or absence of the brain activity of the subject can be easily estimated without using the necessary electrodes. Thereby, based on the determination component corresponding to the brain function of the subject to whom the drug is administered, the effect of the drug on the subject can be easily determined.

  The drug effect determination device according to a ninth aspect of the present invention is the drug effect determination device according to the eighth aspect, wherein the determination component extraction unit correlates a determination waveform corresponding to the administration of the drug and a plurality of components. A determination component is extracted based on the value. With such a configuration, the determination component corresponding to the brain function of the subject can be specified.

  The drug effect determination device according to a tenth aspect of the present invention is the drug effect determination device according to the eighth aspect or the ninth aspect, wherein the drug effect determination unit determines the effect of the drug according to the time-series change of the determination component. . Accordingly, the drug effect level can be easily determined according to the amount of change from the reference acquired in advance.

  The drug effect determination device according to the eleventh aspect of the present invention is the drug effect determination device according to the eighth to tenth aspects, further comprising a determination information storage unit. The determination information storage unit associates the change amount of the correlation value of the determination component calculated for the administration of the drug after the predetermined time has elapsed with respect to the reference value after the drug has been administered, with the drug effect level. And stored as determination information. In addition, the drug effect determination unit calculates a correlation value of the determination component with respect to the administration of the drug, and determines a drug effect level for the subject based on the calculated correlation value and determination information. With such a configuration, the drug effect level can be easily determined using a reference value acquired in advance.

  The drug effect determination device according to a twelfth aspect of the present invention is the drug effect determination device according to the eleventh aspect, further comprising a dose calculation unit that calculates the dose of the drug to be administered to the subject according to the drug effect level. Prepare. With such a configuration, an appropriate drug can be administered to the subject.

  The drug effect determination device according to the thirteenth aspect of the present invention is the drug effect determination device according to the eighth to twelfth aspects, wherein the drug effect determination unit calculates a correlation value of the determination component with respect to the administration of the drug. Then, based on the calculated correlation value and determination information, the drug effect level for the subject is determined. Here, the amount of change from the reference value of the correlation value of the determination component calculated for the administration of the drug after a predetermined time has elapsed since the determination information providing apparatus on the network has been administered. Is stored as determination information in association with the drug effect level. With such a configuration, the drug effect level of the subject can be determined using the determination information providing apparatus on the network.

  In the drug effect determination device according to the fourteenth aspect of the present invention, in the drug effect determination device according to the thirteenth aspect, the drug effect determination unit determines the drug effect level and the dose of the drug based on the determination information. However, the determination information providing apparatus further stores the dose of the drug in association with the drug effect level as the determination information. With such a configuration, an appropriate drug can be administered to the subject. In addition, the configuration of the drug effect determination device can be simplified.

  In the pain determination apparatus according to the first aspect, the pain state in the subject can be easily determined based on the determination component corresponding to the brain function of the subject.

  In the pain determination apparatus according to the second aspect, a determination component corresponding to the brain function of the subject can be specified.

  In the pain determination apparatus according to the third aspect, the pain state can be determined with a simple configuration.

  In the pain determination device according to the fourth aspect, the pain state level can be easily determined.

  In the pain determination device according to the fifth aspect, the pain state level can be easily determined.

  In the pain determination device according to the sixth aspect, the pain state level of the subject can be determined using the determination information providing device on the network.

  The pain determination device according to the seventh aspect can easily realize highly reliable pain determination.

  In the drug effect determination device according to the eighth aspect, the effect of the drug on the subject can be easily determined based on the determination component corresponding to the brain function of the subject to whom the drug is administered.

  In the drug effect determination device according to the ninth aspect, the determination component corresponding to the brain function of the subject can be specified.

  In the drug effect determination device according to the tenth aspect, the drug effect level can be easily determined.

  In the drug effect determination device according to the eleventh aspect, the drug effect level can be easily determined.

  In the drug effect determination device according to the twelfth aspect, an appropriate drug can be administered to the subject.

  In the drug effect determination device according to the thirteenth aspect, the drug effect level of the target person can be determined using the determination information providing device on the network.

  In the drug effect determination device according to the fourteenth aspect, an appropriate drug can be administered to the subject.

The figure which shows an example of picked-up image data and the result of having analyzed this. The figure which shows a part of result of having analyzed facial skin temperature data. The figure which shows a part of result of having analyzed facial skin temperature data. The figure which shows the amplitude of the component waveform of the component 2, and the amplitude of (beta) wave among the measured brain waves. The figure which shows the amplitude of the component waveform of the component 3, and the amplitude of (beta) wave among the measured brain waves. The figure which shows a part of result of having analyzed the facial skin temperature data obtained by the control experiment. The figure which shows the component waveform based on the picked-up image data of a face, and the amplitude of (beta) wave among the measured brain waves. The figure which shows the component waveform based on facial skin temperature data, and the amplitude of (beta) wave among the measured brain waves. The figure which shows the component waveform based on the picked-up image data of a face, and the amplitude of (beta) wave among the measured brain waves. The figure which shows the component waveform based on facial skin temperature data, and the amplitude of (beta) wave among the measured brain waves. The figure which shows the component waveform based on the picked-up image data of a face, and the amplitude of (beta) wave among the measured brain waves. The figure which shows the component waveform based on facial skin temperature data, and the amplitude of (beta) wave among the measured brain waves. The figure which shows the component waveform based on the picked-up image data of a face, and the amplitude of (beta) wave among the measured brain waves. The figure which shows the component waveform based on facial skin temperature data, and the amplitude of (beta) wave among the measured brain waves. The figure which shows the component waveform based on the picked-up image data of a face, and the amplitude of (beta) wave among the measured brain waves. The figure which shows the component waveform based on facial skin temperature data, and the amplitude of (beta) wave among the measured brain waves. The figure which shows the component waveform based on the picked-up image data of a face, and the amplitude of (beta) wave among the measured brain waves. The figure which shows the component waveform based on facial skin temperature data, and the amplitude of (beta) wave among the measured brain waves. 1 is a schematic diagram of a brain activity visualization apparatus according to an embodiment of the present invention. The flowchart which shows an example of the flow of a process at the time of identifying the component which shows the change of the skin temperature which reflected the brain function in the brain activity visualization apparatus. 1 is a schematic diagram of a brain activity visualization apparatus according to an embodiment of the present invention. The flowchart which shows an example of the flow of a process at the time of identifying the component which shows the RGB change of the face reflecting a brain function in a brain activity visualization apparatus. The schematic diagram which shows the structure of the pain determination apparatus which concerns on one Embodiment of this invention. The schematic diagram which shows the structure of the determination information database of a pain determination apparatus. The schematic diagram which shows an example of the waveform of a red spot dynamic response function. It is a flowchart which shows operation | movement of a pain determination apparatus. It is a flowchart which shows operation | movement of a pain determination apparatus. The schematic diagram which shows the structure of the modification of a pain determination apparatus. The schematic diagram which shows the structure of the chemical | medical agent effect determination apparatus which concerns on one Embodiment of this invention. The schematic diagram which shows the structure of the determination information database of a chemical | medical agent effect determination apparatus. It is a flowchart which shows operation | movement of a chemical | medical agent effect determination apparatus. The schematic diagram which shows the structure of the determination information database of a chemical | medical agent effect determination apparatus. The figure which shows the verification result of a medicine effect determination. The figure which shows the verification result of a medicine effect determination.

  Before describing the embodiments of the present invention, first, the knowledge obtained by the present inventors, which is an important basis for the inventors of the present invention, will be described.

(1) Key points of knowledge by the present inventors It is known that human brain activities reflect human intellectual activities (cognitive activities, etc.) and emotional activities (activity such as pleasure / discomfort). . Conventionally, attempts have been made to estimate human brain activity. In this case, data detected by any one of the electroencephalography, magnetic resonance imaging, and near-infrared spectroscopy is used. There are many.

  Here, for example, when an electroencephalogram measurement method is employed as the detection method, it is necessary to attach an electroencephalogram electrode to the subject. And when attaching an electroencephalogram electrode, since it is necessary to make resistance between skin and an electrode small, the process of grind | polishing the skin or apply | coating a paste to an electrode is needed. In addition, when magnetic resonance imaging is adopted, there are restrictions on measurement conditions such as that measurement outside the MRI room is impossible and metal cannot be brought into the measurement room. Furthermore, when near-infrared spectroscopy is used, it is necessary to wear a probe on the subject. However, if the probe is worn for a long time, the subject feels pain or the subject's hair contacts the probe. Depending on the condition, it may not be detected accurately. In this way, when a conventional detection method is adopted to measure human brain activity, pre-processing when wearing an electroencephalogram electrode or a probe is necessary, measurement conditions are limited, etc. The burden on the subject increases.

  Therefore, there is a need for development of means that can reduce the burden on the subject and can easily estimate human brain activity.

  And the present inventors thought that human brain activity could be estimated based on the blood circulation state of the face that is considered to be proportional to the skin temperature of the human face or the skin temperature of the face. . If it is the skin temperature of the human face, it can be obtained by using a measuring device such as a thermography, and if the blood circulation state of the face, that is, the blood circulation amount of the face, RGB of the captured image of the face obtained using the imaging device It can be estimated from the data. As described above, a facial skin temperature or a photographed image of a face can be acquired without wearing a sensor that requires processing before wearing an electroencephalogram electrode or a probe.

  On the other hand, it is known that the skin temperature of a human face changes under the influence of various factors such as the outside air temperature and / or autonomic nerve activity. Therefore, when trying to estimate brain activity based on facial skin temperature or based on facial blood flow, which is considered to be proportional to facial skin temperature, it is determined whether the acquired data reflects only brain activity. It seems to be very difficult to do.

  As a result of intensive studies, the present inventors have detected the skin temperature of the face, time-series facial skin temperature data including detected temperature data and position data (coordinate data) of the detected part, or time-series face Time-series facial blood flow data calculated based on RGB data obtained from the captured image data was decomposed into multiple components using the singular value decomposition method, principal component analysis method, or independent component analysis method, and decomposed. By analyzing a plurality of components, it was found that components showing changes in facial skin temperature or facial blood flow reflecting brain activity can be identified. The inventors of the present invention have arrived at the present invention that can visualize the subject's physiological state based on the estimated brain activity by estimating and analyzing the subject's brain activity.

(2) Method for acquiring various facial data, and method for analyzing various acquired data (2-1) Method for acquiring facial skin temperature data, and method for analyzing facial skin temperature data Next, the inventors described above A method of acquiring facial skin temperature data and a method of analyzing facial skin temperature data used for obtaining knowledge will be described.

  In this test, facial skin temperature data was obtained from six subjects. Specifically, the subject was seated on a chair installed in an artificial weather room maintained at a room temperature of 25 ° C., and facial skin temperature data was obtained from the entire face of the subject using an infrared thermography device. The infrared thermography device detects the infrared radiation energy emitted from the object with an infrared camera, converts the detected infrared radiation energy into the temperature of the object surface (in this case, the temperature in degrees Celsius), and converts the temperature distribution. It is a device that can display and accumulate as facial skin temperature data (for example, image data representing a temperature distribution). In this test, NEC Avio Infrared Technology Co., Ltd. R300 was used as an infrared thermography apparatus. The infrared camera was installed in front of the subject at a point 1.5 m away from the subject. The facial skin temperature data was acquired for 30 minutes.

  In this test, a brain function activation task was given to the subject while acquiring facial skin temperature data. As a result, facial skin temperature data when the brain was not activated and facial skin temperature data when the brain was activated were acquired. Examples of the brain function activation task include calculation or recognition of numerical values, shapes and colors, or psychological work such as memory of symbols, characters or languages based on the image displayed on the display device or the like by the subject. . In this test, “Mental calculation of multiplication” was adopted as a brain function activation task, and subjects were asked to calculate numbers displayed in a writing format on the display device and to input the answers to the keyboard. In this test, a brain function activation task was given to the subject continuously for 10 minutes after the start of acquisition of facial skin temperature data for 5 minutes.

  For analysis of facial skin temperature data, singular value decomposition was performed on the acquired facial skin temperature data using SVLAB (Singular Value Decomposition) of MATLAB (registered trademark) as an analysis tool. In singular value decomposition, all facial skin temperature data (data for 30 minutes) acquired in time series are targeted, the factor is time data every 30 seconds (60 time points in 30 minutes), and the measure is the period (30 Second) facial skin temperature data (240 × 320 pixels). Then, the facial skin temperature data X was decomposed into a plurality of components by singular value decomposition, and a temporal distribution V of each component, a spatial distribution U, and a singular value S indicating the size of each component were calculated. These relationships are expressed by the following equations. V ′ is a matrix in which the rows and columns of V are exchanged.

  And the time distribution V and the spatial distribution U of each component calculated | required by the singular value decomposition were plotted on the graph, and the component waveform diagram and temperature distribution diagram of each component were created.

  Furthermore, the analysis for identifying the component which shows the change of the skin temperature reflecting brain activity was performed about the created component waveform diagram and temperature distribution diagram of each component.

  About the component waveform figure of each component, it analyzed about the presence or absence of the correlation of the amplitude of the component waveform, and the time of brain non-activation and the time of brain activation. Specifically, it was evaluated whether there was a correlation between the amplitude shown in the component waveform diagram of each component and the brain non-activation period / brain activation period. In this test, among the period of acquiring facial skin temperature data, a period of 5 minutes from the start of data acquisition, which is a period during which no brain function activation task is given to the subject, Data acquisition is a period in which a brain function activation task is given to a subject, with a period of 15 minutes from the time when 15 minutes have elapsed from the start of data acquisition to the end of data acquisition being the non-activation time of the brain The period of 10 minutes from the time when 5 minutes passed from the start to the time when 10 minutes passed was defined as the time of brain activation. Then, the presence or absence of a correlation between the amplitude shown in the component waveform diagram of each component and the time of brain non-activation and the time of brain activation was evaluated. In addition, about the presence or absence of correlation, statistical correlation analysis was conducted and it was judged that there was a correlation when a significance level ((alpha)) is 0.05 or less.

  Regarding the temperature distribution chart of each component, the presence or absence of temperature change at a predetermined part of the face was analyzed. Here, the brain has a mechanism called a selective brain cooling system that cools the brain independently of the body temperature. As a selective brain cooling mechanism, it is known that the heat generated by the brain activity is exhausted using the forehead and the sinuses (including the eyebrows and the nose). Therefore, in this test, it was evaluated whether or not there was a temperature change around the sinuses and the forehead in the temperature distribution chart of each component. In addition, regarding the presence or absence of temperature changes around the sinuses and the forehead in the temperature distribution chart, the presence or absence of temperature changes by visual inspection, or the temperature around the sinuses and the forehead is determined from the average temperature of the entire measurement data. Whether there was a difference of 1 standard deviation (SD) or more was used as a reference for the presence or absence of temperature change.

  Since the polarity (plus or minus) of the facial skin temperature data X is determined by the relationship between the spatial distribution U, the singular value S, and the time distribution V, the polarity is inverted in the component waveform diagram and temperature distribution diagram of each component. May appear. For this reason, regarding the evaluation of the component waveform diagram and the temperature distribution diagram, the polarity is not considered as an evaluation target.

  Here, in this infrared thermography apparatus, as described above, infrared radiation energy detected from an object is converted into temperature, and the temperature distribution is used as facial skin temperature data. By the way, when the skin temperature of the face is acquired by using an infrared thermography apparatus for human subjects, temperature changes (so-called noise) that are not related to various brain activities such as facial movement and / or autonomic nerve activity are also faced. It is acquired as skin temperature data (see FIG. 1A). Therefore, in order to detect such a temperature change not related to the brain activity, relative facial skin temperature data in which the total average value of the temperature data included in the facial skin temperature data every 30 seconds is “0” is calculated. The facial skin temperature data created is also subjected to singular value decomposition using the MATLAB® SVD as an analysis tool, and a component waveform diagram and temperature distribution diagram of each component according to the singular value S are created. Then, an analysis was performed to identify a component showing a change in skin temperature reflecting brain activity.

  In the following, for convenience of explanation, the facial skin temperature data acquired by the infrared thermography device is referred to as “facial skin temperature data according to the temperature converted data”, and the temperature converted data every predetermined time (in this test every 30 seconds). The relative facial skin temperature data in which the total average value of the temperature data included in the facial skin temperature data corresponding to “0” is “0” is referred to as “facial skin temperature data according to relative temperature conversion data”.

  For one of the six subjects, in addition to detecting the facial skin temperature using an infrared thermography device, electrodes were connected to the subject's scalp to measure brain waves, and during wakefulness and consciousness became tense. The correlation between the amplitude of the β wave (the brain wave having a frequency of 14 to 30 Hz), which is known as the waveform appearing at the time, and the amplitude of the component waveform diagram was also evaluated. In the electroencephalogram measurement, electrodes were arranged at six parts (F3, F4, C3, C4, Cz, Pz) based on the international method 10-20.

  By the way, it is conceivable that the subject's head moves up and down while the subject is given a brain function activation task. Then, the position of the subject's face with respect to the infrared camera changes. In order to verify whether or not the change in the position of the face affects the change in the skin temperature, a control test was performed on one subject. In the control test to verify the influence of the subject's movement when acquiring the facial skin temperature data, the facial skin temperature data of the subject is acquired using the infrared thermography device as in the above test, but the brain function activation task is The subject was forced to press the keyboard at random times even when it was not given (ie, when the brain was not activated). The facial skin temperature data according to the temperature conversion data obtained by the control experiment and the facial skin temperature data according to the relative temperature conversion data are also subjected to singular value decomposition using the MATLAB® SVD as an analysis tool. Then, a component waveform diagram and a temperature distribution diagram of each component according to the singular value S were created, and an analysis for identifying a component showing a change in skin temperature reflecting brain activity was performed.

(2-2) Method for Acquiring Face Shooting Image Data and Method for Analyzing Face Shooting Image Data FIG. 1A is a diagram showing an example of shooting image data around the paranasal sinuses of a subject's face taken by a photographing device. It is. FIG. 1B is a diagram illustrating an example of a blood flow distribution map (image map).

  Next, a method for acquiring face-captured image data and a method for analyzing face-captured image data used by the inventors to obtain the above knowledge will be described.

  In this test, face image data was obtained from six subjects. Specifically, a photographed image around the sinus of the entire face of the subject using a photographing device that allows the subject to sit in a chair installed in an artificial weather room maintained at room temperature of 25 ° C. and acquire images in time series Data was acquired in time series.

  Further, based on the selective brain cooling mechanism described above, a change in facial blood flow that is considered to be proportional to the facial skin temperature accompanying brain activity is considered to appear around the forehead and / or the sinuses. From this, the present inventors considered that brain activity can be estimated with high accuracy if at least changes in blood flow in the face around the forehead and / or sinuses can be captured. In this test, captured image data around the paranasal sinuses of the subject's face was acquired in time series.

  In this test, a photographing device on the liquid crystal screen side of the iPad Air (registered trademark) manufactured by Apple Inc. was used as a photographing device, and color moving image data was acquired as time-series photographed image data. In addition, the photographing apparatus was installed on the front side of the subject and at a point 1.0 m away from the subject. Then, the moving image data of the face was obtained by continuously capturing the captured image data for 30 minutes along the time axis at a capturing period of 30 frames / second by the capturing device.

  Furthermore, in this test, a brain function activation task was given to the subject while acquiring facial moving image data. Thereby, the moving image data of the face when the brain was not activated and the moving image data of the face when the brain was activated were acquired. In this test, as in the case of the above test, the task of activating the brain function is `` multiplying mental arithmetic '', allowing the subject to calculate the numbers displayed in the writing format on the display device, and inputting the answers to the keyboard Imposed. In this test, a brain function activation task was given to the subject continuously for 10 minutes after 5 minutes from the start of facial moving image data acquisition.

For analysis of facial moving image data, blood flow data is calculated based on RGB data obtained from the captured moving image data of the face, and MATLAB (registered trademark) SVD is used for the calculated time-series blood flow data. Singular value decomposition was performed as an analytical tool. Here, according to the CIE-L ^* a ^* b ^* color system, an erythema index “a ^* ” correlated with the redness and hemoglobin amount calculated from the RGB data of the image is obtained, and this is used as blood flow data. . In the singular value decomposition, blood flow volume data (here, erythema index) based on RGB data obtained from all moving image data (30-minute data) acquired in time series is targeted, and the factor is set every 30 seconds. Erythema index calculated from time data (60 time points in 30 minutes) and RGB data in that period (every 30 seconds) (taken out frame data for 1 second every 30 seconds, and RGB values obtained from the frame data) Erythema index calculated from the average value of 240 × 320 pixels). Then, by singular value decomposition, time-series blood flow data based on RGB data obtained from facial moving image data is decomposed into a plurality of components, and each component's time distribution V, spatial distribution U, and each component And a singular value S indicating the magnitude of. In addition, these relationships are represented by the same formula as the above formula (Formula 1).

  Then, the time distribution V and the spatial distribution U of each component obtained by singular value decomposition were plotted on a graph, and a component waveform diagram and a blood flow distribution diagram of each component were created.

  Further, analysis was performed on the component waveform diagram and the blood flow distribution diagram of each component that was created to identify a component indicating a change in facial blood flow that reflects brain activity, that is, a component that indicates a change in facial RGB.

  About the component waveform figure of each component, it analyzed about the presence or absence of the correlation of the amplitude of the component waveform, and the time of brain non-activation and the time of brain activation. Specifically, it was evaluated whether there was a correlation between the amplitude shown in the component waveform diagram of each component and the brain non-activation period / brain activation period. In this test, during the period when the captured image data of the face is acquired, the subject is not given a brain function activation task, and the 5-minute period from the start of data acquisition to the time when 5 minutes have elapsed. Data that is a period in which a brain function activation task is given to a subject, with a period of 15 minutes from the start of data acquisition and the period of 15 minutes from the start of data acquisition to the end of data acquisition being the non-activated state of the brain A period of 10 minutes from the time when 5 minutes passed from the start of acquisition to the time when 10 minutes passed was defined as the time of brain activation. Then, the presence or absence of a correlation between the amplitude shown in the component waveform diagram of each component and the time of brain non-activation and the time of brain activation was evaluated. In addition, regarding the presence or absence of the correlation, statistical correlation analysis was performed, and it was determined that there was a correlation when the significance level (α) was 0.01 or less.

  Regarding the blood flow distribution chart of each component, the presence or absence of a change in blood flow at a predetermined part of the face was analyzed. The blood circulation distribution map is created by arranging the spatial distribution U calculated for each pixel at the position of each pixel. In the blood flow distribution chart of each component thus created, it was evaluated whether there was a change in blood flow around the sinuses and in the forehead. In addition, regarding the presence or absence of changes in blood flow around the sinuses and the forehead in the blood flow distribution diagram, the presence or absence of changes in blood flow by visual inspection, or around the sinuses and the forehead shown in FIG. The reference value for the presence or absence of a change in blood flow was that the value of the blood flow was not “0.000”.

  Note that the polarity (plus or minus) of the blood flow data X is determined by the relationship between the spatial distribution U, the singular value S, and the time distribution V. Therefore, the polarity is reversed in the component waveform diagram and the blood flow distribution diagram of each component. May appear. For this reason, regarding the evaluation of the component waveform diagram and the blood flow distribution diagram, the polarity is not considered as an evaluation target.

  Furthermore, in order to verify the correlation between facial skin temperature and facial blood flow, facial skin temperature data was also obtained by infrared thermography equipment while acquiring facial image data in time series from six subjects. Time-series acquisition, and the acquired facial skin temperature data is also subjected to singular value decomposition using the MATLAB (registered trademark) SVD as an analysis tool, and a component waveform diagram of each component according to the singular value S is created. The presence or absence of a correlation between the amplitude of the component waveform and the brain non-activation and brain activation was analyzed. In this test, an apparatus similar to the above test was used as the infrared thermography apparatus. The infrared camera was installed in front of the subject at a point 1.5 m away from the subject.

  By the way, when acquiring captured image data of a face using a photographing device, light may be reflected by the face when the sunlight hits the face during photographing, and this reflected light may enter the lens of the photographing device. is there. Then, the reflected light is recorded in the photographed image data of the photographed face. Here, in the RGB data obtained from the photographed image data, the change in brightness based on the blood flow amount on the face is smaller than the change in brightness based on the reflected light. Therefore, the RGB data obtained from the photographed image data in which the reflected light is recorded. When the blood flow volume calculated based on the analysis was analyzed, it was considered that there was a possibility that a face RGB change (so-called noise) that is not related to the brain activity might be mixed. Therefore, in order to prevent the mixing of RGB changes in the face that are not related to such brain activity, relative blood flow data is obtained from the relative RGB data with the total average value of RGB data every 30 seconds being “0”. The created blood flow data is also subjected to singular value decomposition using the MATLAB (registered trademark) SVD as an analysis tool, and a component waveform diagram and blood flow distribution diagram of each component according to the singular value S are created. Then, an analysis was performed to identify the component showing the RGB change of the face reflecting the brain activity.

  In the following, for convenience of explanation, relative blood flow data based on relative RGB data with a total average value of RGB data every predetermined time (every 30 seconds in this test) being “0” is referred to as “relative conversion. The blood flow data is referred to as “blood flow data”, and the blood flow data based on the RGB data before conversion into relative RGB data is simply referred to as “blood flow data”.

  In addition, while acquiring time-series captured image data of the face with the imaging device for 6 subjects, electrodes are connected to the scalp of each subject to measure brain waves, and brain cells at the time of waking etc. The correlation between the amplitude of the β wave (the brain wave having a frequency of 13 to 30 Hz), which is known as a waveform appearing when active, and the amplitude of the component waveform diagram were also evaluated. In the electroencephalogram measurement, the region of the scalp 19 (Fp1, Fp2, F3, F4, C3, C4, P3, P4, O1, O2, F7, F8, T3, T4, T5, based on the international method 10-20) , T6, Fz, Cz and Pz).

  Furthermore, it is conceivable that the subject's head moves up and down while the subject is given a brain function activation task. Then, the position of the subject's face with respect to the imaging device changes. In order to verify whether or not the change in the position of the face affects the RGB change in the face, a control test was performed on one subject. In the control test, time-series captured image data of the subject's face is acquired using the imaging device in the same manner as in the above test, but also during the time when the brain function activation task is not given (that is, when the brain is not activated) The subject was forced to press the keyboard at random timing. For chronological blood flow data based on RGB data obtained from time-series photographed image data of the face photographed in this control experiment, singular value decomposition was performed using the SVLAB of MATLAB (registered trademark) as an analysis tool. Then, a component waveform diagram of each component corresponding to the singular value S was created, and the presence or absence of a correlation between the amplitude of the component waveform and the brain non-activation and brain activation was analyzed. In addition, the presence or absence of a correlation between the amplitude of each component waveform and the actual facial movement was analyzed. The actual facial movement is obtained by obtaining the two-dimensional coordinates of the same part of the face from the photographed image data and calculating the movement distance of the face every 30 seconds during photographing with reference to the photographed image data at the start of the control experiment. evaluated. Furthermore, the presence / absence of a correlation between the amplitude of each component waveform and the number of keyboard inputs during shooting was also analyzed. The number of keyboard inputs during shooting was evaluated by calculating a simple moving average every 30 seconds in time-series shot image data.

(3) Analysis Result (3-1) Analysis Result of Facial Skin Temperature Data FIG. 2 is a diagram showing a part of the result of analyzing facial skin temperature data according to the temperature conversion data. FIG. 2A shows a component waveform diagram of the component 2 of the subject 1. FIG. 2B shows a temperature distribution diagram of the component 2 of the subject 1. FIG. 3A shows a component waveform diagram of the component 3 of the subject 1. FIG. 3B shows a temperature distribution diagram of the component 3 of the subject 1. 4 and 5 are diagrams showing the relationship between the amplitude of the component waveform and the electroencephalogram. FIG. 4 is a diagram illustrating the amplitude of the component waveform of the component 2 of the subject 1 and the amplitude of the β wave among the measured brain waves. FIG. 5 is a diagram illustrating the amplitude of the component waveform of the component 3 of the subject 1 and the amplitude of the β wave among the measured brain waves. FIG. 6 is a diagram showing a part of the result of analyzing the facial skin temperature data obtained in the control experiment. FIG. 6A shows a component waveform diagram of component 3. FIG. FIG. 6B shows a temperature distribution diagram of the component 3.

  Table 1 shows the analysis results of facial skin temperature data for each subject.

  Of the plurality of components obtained by decomposing time-series facial skin temperature data by singular value decomposition from the results obtained by the above analysis of facial skin temperature data, component 2 and / or component 3 and human It was confirmed that there was a significant correlation with brain activity.

  Also, as shown in FIGS. 4 and 5, a significant correlation was confirmed between the amplitude of each component waveform of component 2 and component 3 and the amplitude of the β wave of the electroencephalogram from the results of the electroencephalogram analysis.

  Furthermore, in the control experiment, there was a significant correlation between component 3 and human brain activity even when the subject was in motion while acquiring facial skin temperature data (see FIG. 6). . From this, it was recognized that among the plurality of components, the movement of the subject at the time of acquiring facial skin temperature data was not affected for component 3.

  From these results, the present inventors obtained the following knowledge.

  The time-series facial skin temperature data obtained from the test subject is decomposed into a plurality of components by singular value decomposition, and as a result of analyzing each decomposed component, component 3 of the plurality of components is a component related to brain activity. Admitted. In other words, time-series facial skin temperature data is decomposed into multiple components by singular value decomposition, components that correlate with brain activation / deactivation are extracted from the decomposed multiple components, and selective brain cooling is performed on the extracted components By analyzing using the mechanism, it was found that a component showing a change in skin temperature reflecting brain activity can be identified from a plurality of components. From this, the present inventors have obtained the knowledge that brain activity can be estimated based on the skin temperature of the human face.

(3-2) Analysis Results of Photographed Image Data of Face FIGS. 7 to 18 are diagrams of component waveforms based on photographed image data (blood flow data) or facial skin temperature data of the face, and β of the measured electroencephalogram. It is a figure which shows a part of result of having compared and analyzed the waveform figure of the wave. FIG. 7 is a diagram illustrating the amplitude of the component 2 component waveform based on the captured image data of the subject 1 and the measured β wave amplitude of the brain wave of the subject 1. FIG. 8 is a diagram showing the amplitude of the component 2 component waveform based on the facial skin temperature data of the subject 1 and the measured β wave amplitude of the brain wave of the subject 1. FIG. 9 is a diagram showing the amplitude of the component 2 component waveform based on the captured image data of the subject 2 and the measured amplitude of the β wave in the brain wave of the subject 2. FIG. 10 is a diagram illustrating the amplitude of the component 2 component waveform based on the facial skin temperature data of the subject 2 and the measured β wave amplitude of the brain wave of the subject 2. FIG. 11 is a diagram illustrating the amplitude of the component waveform of component 4 based on the captured image data of the subject 3 and the amplitude of the β wave among the measured brain waves of the subject 3. FIG. 12 is a diagram illustrating the amplitude of the component 3 component waveform based on the facial skin temperature data of the subject 3 and the measured β wave amplitude of the brain wave of the subject 3. FIG. 13 is a diagram illustrating the amplitude of the component 3 component waveform based on the captured image data of the subject 4 and the measured β-wave amplitude of the brain wave of the subject 4. FIG. 14 is a diagram showing the amplitude of the component 2 component waveform based on the facial skin temperature data of the subject 4 and the measured β wave amplitude of the brain wave of the subject 4. FIG. 15 is a diagram illustrating the amplitude of the component 2 component waveform based on the captured image data of the subject 5 and the measured β wave amplitude of the brain wave of the subject 5. FIG. 16 is a diagram showing the amplitude of the component 2 component waveform based on the facial skin temperature data of the subject 5 and the measured β wave amplitude of the brain wave of the subject 5. FIG. 17 is a diagram illustrating the amplitude of the component waveform of component 4 based on the captured image data of the subject 6 and the amplitude of the β wave among the measured brain waves of the subject 6. FIG. 18 is a diagram illustrating the amplitude of the component 3 component waveform based on the facial skin temperature data of the subject 6 and the measured β wave amplitude of the brain wave of the subject 6.

  As shown in FIG. 7 to FIG. 18, it was confirmed from the results of the component waveforms and the electroencephalogram analysis that the facial skin temperature and the facial blood flow volume are in a correlation. In the analysis based on any of the skin temperature data of the face and the blood flow data of the face, the amplitude of each component waveform and the amplitude of the β wave of the electroencephalogram measured by the electrode mounted on the top or back of the head A significant correlation was confirmed.

  Table 2 shown below shows the analysis results of the captured image data of the face for each subject.

  As shown in Table 2, a plurality of components obtained by decomposing time-series blood flow data based on the photographed image data of the face by singular value decomposition based on the results obtained by analyzing the photographed image data of the face. Of these, it was confirmed that there was a significant correlation between components 1, 2, 3, 4, 5 and human brain activity. In addition, here, not only the component in which the correlation based on the blood flow data is significant and the correlation based on the relative blood flow data is significant, but also the correlation based on the blood flow data is significant. Components that were not correlated but showed significant correlation in the correlation based on relative blood flow data were also recognized as having significant correlation with human brain activity.

  Table 3 shown below shows the results of the control experiment.

  As shown in Table 3, in the control experiment, when the subject moves while acquiring the captured image data of the face, between the amplitude of the component waveform and the brain non-activation time and brain activation time. Of the components having a significant correlation, the component 2 was not significantly correlated with the moving distance and the number of keyboard inputs. From this, among components obtained by singular value decomposition of blood flow data based on RGB data acquired from captured image data of a face, components having a significant correlation with brain activity are Even if it is affected by the movement of the subject when acquiring time-series captured image data, it is confirmed that the effect is much smaller than the effect of brain activity (effect of brain activation or non-activation). It was done.

  From these results, the present inventors obtained the following knowledge.

  Blood flow volume data obtained from facial RGB data based on time-series facial captured image data obtained from the subject was decomposed into a plurality of components by singular value decomposition, and each decomposed component was analyzed. Components 1, 2, 3, 4 and 5 were found to be components related to brain activity. That is, blood flow data obtained from facial RGB data based on time-series facial image data is decomposed into a plurality of components by singular value decomposition, and there is a correlation between activation / inactivation of the brain from the decomposed components. It was found that by extracting the components and analyzing the extracted components, it is possible to identify a component showing the RGB change of the face reflecting brain activity from a plurality of components. From this, the present inventors have found that brain activity can be estimated based on time-series captured image data of the human face.

(4) Brain Activity Visualization Device Next, brain activity visualization devices 10 and 110 according to an embodiment of the present invention that have been completed by the present inventors based on the knowledge described above will be described. In addition, the brain activity visualization apparatus according to the present invention is not limited to the following embodiments, and can be appropriately changed without departing from the gist.

  The brain activity visualization devices 10 and 110 according to an embodiment of the present invention include a brain activity estimation unit 30 that estimates brain activity based on facial skin temperature data, and / or a brain that estimates brain activity based on face image data. The activity estimation means 130 is provided. Below, before demonstrating the brain activity visualization apparatuses 10 and 110 which concern on one Embodiment of this invention, each brain activity estimation means 30 and 130 are demonstrated.

(4-1) Brain activity estimating means 30 for estimating brain activity based on facial skin temperature data
FIG. 19 is a schematic diagram of the brain activity visualization apparatus 10 according to an embodiment of the present invention. FIG. 20 is a flowchart showing the flow of processing when the brain activity visualization apparatus 10 identifies a component indicating a change in skin temperature reflecting the brain function.

  The brain activity estimating means 30 provided in the brain activity visualization device 10 estimates the brain activity of the individual from the skin temperature of the face of the individual (subject). As shown in FIG. 19, the brain activity visualization device 10 includes a facial skin temperature acquisition unit 20, a brain activity estimation unit 30, and a state visualization unit 200.

  The facial skin temperature acquisition means 20 detects the skin temperature of at least a part of the individual's face, and acquires facial skin temperature data including the detected temperature data and the position data of the detected part in time series (step S1). Here, the facial skin temperature acquisition means 20 is an infrared thermography apparatus, and includes an infrared camera 21 and a processing unit 22 as shown in FIG. The infrared camera 21 is for detecting infrared radiant energy emitted from an individual's face. Here, it is assumed that the infrared camera 21 detects infrared radiation energy from the entire face of the individual. The processing unit 22 converts the infrared radiant energy detected by the infrared camera 21 into temperature to obtain temperature data, and the temperature distribution of the facial skin temperature over the entire face using the position where the infrared radiant energy is detected as position data (coordinate data). A diagram is created, and the created temperature distribution diagram is processed as facial skin temperature data corresponding to the temperature conversion data. The facial skin temperature data corresponding to the temperature conversion data is accumulated in a storage unit (not shown) included in the processing unit 22.

  Here, the temperature distribution chart of the facial skin temperature in the entire face is created in the processing unit 22, but the present invention is not limited to this, and the temperature distribution chart of the facial skin temperature including at least the periphery of the sinuses and / or the forehead portion. May be created, and this may be the facial skin temperature data corresponding to the temperature conversion data.

  Also, here, while the facial skin temperature data corresponding to the temperature conversion data is acquired by the facial skin temperature acquisition means 20, a brain function activation task is given to the individual for a certain period. That is, the facial skin temperature data corresponding to the temperature conversion data acquired by the facial skin temperature acquisition means 20 includes data for a period during which a brain function activation task is given to an individual. The brain function activation task given to an individual is not particularly limited as long as it is estimated that the brain is activated, for example, depending on the purpose of use of the brain activity visualization device 10 You may be comprised so that the content may be determined suitably.

  The brain activity estimation unit 30 estimates human brain activity based on the facial skin temperature data corresponding to the temperature conversion data acquired by the facial skin temperature acquisition unit 20. Specifically, as shown in FIG. 19, the brain activity estimation means 30 includes a conversion unit 31, an analysis unit 32, and an estimation unit 33.

  The conversion unit 31 converts the temperature data included in the facial skin temperature data corresponding to the temperature conversion data into relative temperature data, and responds to the facial skin temperature data based on the converted relative temperature data, that is, relative temperature conversion data. Facial skin temperature data is created (step S2). Specifically, the conversion unit 31 uses the average value of the temperature data included in the facial skin temperature data corresponding to the temperature conversion data every predetermined time (for example, 30 seconds) as a reference value, and converts the temperature data into a relative value. Convert to temperature data. And the conversion part 31 produces the face skin temperature data according to relative temperature conversion data using the converted relative temperature data and position data.

  The analysis unit 32 converts the facial skin temperature data according to the time-series temperature conversion data and the facial skin temperature data according to the relative temperature conversion data into a plurality of components by singular value decomposition, principal component analysis, or independent component analysis. Decompose (step S3). Here, the analysis unit 32 analyzes the SVLAB of MATLAB (registered trademark) for each of the facial skin temperature data according to the acquired temperature conversion data and the facial skin temperature data according to the converted relative temperature conversion data. To perform singular value decomposition. The singular value decomposition is performed with the factor of time data for a predetermined period (for example, 30 seconds) for the facial skin temperature data according to the temperature conversion data acquired in time series and the facial skin temperature data according to the relative temperature conversion data, The measure is performed as facial skin temperature data corresponding to the temperature conversion data in that period and facial skin temperature data corresponding to the relative temperature conversion data. Then, by singular value decomposition, each of the facial skin temperature data according to the temperature conversion data and the facial skin temperature data according to the relative temperature conversion data is decomposed into a plurality of components, and the time distribution, spatial distribution, and A singular value indicating the magnitude is calculated.

  In addition, the analysis unit 32 determines whether or not each component satisfies the first condition and the second condition in order to identify a component indicating a change in skin temperature reflecting brain activity from a plurality of components decomposed by singular value decomposition. (Step S4a, Step S4b, Step S5a, and Step S5b). Here, the analysis unit 32 first determines whether or not the first condition is satisfied for each component based on the facial skin temperature data corresponding to the temperature conversion data (step S4a), and the first in step S4a. It is determined whether or not the second condition is satisfied for the component based on the facial skin temperature data corresponding to the temperature conversion data determined to satisfy the condition (step S4b). Of the components based on the facial skin temperature data corresponding to the relative temperature conversion data, the first condition is satisfied only for components that match the components determined to satisfy the first condition and the second condition in step S4a and step S4b. (Step S5a), and then the second condition is satisfied for the component based on the facial skin temperature data corresponding to the relative temperature conversion data determined to satisfy the first condition in Step S5a. It is determined whether or not it has been performed (step S5b). However, the order of the determination in the analysis unit 32 is not limited to this. For example, each component based on the facial skin temperature data corresponding to the temperature conversion data and the facial skin temperature data corresponding to the relative temperature conversion data are used. It may be determined whether or not each component based on the first condition and the second condition is satisfied, and a component having a matching determination result may be finally extracted.

  The first condition is a condition that the amplitude of the component waveform of the component decomposed by the singular value decomposition is correlated with a change at the time of non-activation of the brain and activation of the brain. The analysis unit 32 extracts a component that satisfies the first condition from the plurality of components as a determination component. Here, there is a certain period during which the brain function activation task is given to an individual while acquiring facial skin temperature data corresponding to the temperature conversion data. The analysis unit 32 uses the period when the brain function activation task is not given to the individual as the non-activation time of the brain, and sets the period when the brain function activation task is given to the individual as the brain activation time. The period in which the activation task is given and the period in which it is not given and the component waveform of each component are compared and analyzed. The analysis unit 32 uses the comparison analysis result based on the component waveform data to evaluate whether the component waveform of each component and the brain non-activation time and the brain activation time are correlated, and a plurality of components Among these, the component evaluated as having a correlation is extracted as a determination component that satisfies the first condition. On the other hand, the analysis unit 32 determines that a component evaluated as having no correlation among the plurality of components is not a component that does not satisfy the first condition and shows a temperature change reflecting human brain activity (step S6).

  Here, a brain function activation task is given to an individual for a certain period when acquiring facial skin temperature data corresponding to temperature conversion data, and based on this, the analysis unit 32 extracts a determination component. The contents of one condition, that is, the determination component extraction means in the analysis unit 32 is not limited to this. For example, when a component showing a component waveform correlated with the non-activation time and the activation time of the brain among a plurality of components by performing an experiment or the like in advance, the analysis unit 32 The component specified from the plurality of components is extracted as a determination component. Further, in the case where the human activity known to be related to activation / deactivation of the brain such as eye movement or tapping is detected in the present brain activity visualization device, the analysis unit 32 analyzes the detection result and each A determination component may be extracted from a plurality of components by comparative analysis and evaluation of the component waveform of the component. Note that the criterion for determining whether or not the first condition is satisfied by the analysis unit 32 is appropriately determined by simulation, experiment, desktop calculation, or the like according to the purpose of use of the brain activity visualization device 10 or the like.

  The second condition is a condition that there is a temperature change in a predetermined part of the human face in the extracted determination component. The analysis unit 32 determines a component that satisfies the second condition among the determination components as a component that is highly likely to be related to human brain activity, and extracts it as a candidate component. That is, the analysis unit 32 determines whether or not the determination component is related to human brain activity based on the presence or absence of a temperature change at a predetermined part of the human face. Specifically, the analysis unit 32 determines whether a temperature change has occurred around the sinuses and / or the forehead based on the extracted temperature distribution data of the determination component, and the temperature change has occurred. In this case, it is determined that the determination component is a component that is highly likely to be related to human brain activity that satisfies the second condition, and is extracted as a candidate component. On the other hand, when no temperature change occurs around the sinuses and / or the forehead part, the analysis unit 32 does not satisfy the second condition and shows a change in skin temperature reflecting brain activity. It is determined that it is not a component (step S6). Note that the criterion for determining whether or not the second condition is satisfied by the analysis unit 32 is appropriately determined by simulation, experiment, desktop calculation, or the like according to the purpose of use of the brain activity visualization device 10 or the like.

  And the analysis part 32 identifies the component determined to satisfy | fill 2nd conditions in step S5b as a component which shows the change of the skin temperature reflecting brain activity (step S7). That is, the component identified as the component indicating the change in skin temperature reflecting the brain activity in step S7 is the candidate component extracted by decomposing and analyzing the facial skin temperature data corresponding to the temperature conversion data by singular value decomposition And the candidate skin component extracted by analyzing the facial skin temperature data corresponding to the relative temperature conversion data by decomposing and analyzing by singular value decomposition. Note that candidate components that do not match in both analyzes are determined not to be components indicating changes in skin temperature reflecting brain activity in step S6.

  The estimation unit 33 estimates the human brain activity based on the component identified by the analysis unit 32 as the component indicating the change in skin temperature reflecting the human brain activity. Specifically, the estimation unit 33 estimates the amount of brain activity when acquiring facial skin temperature data based on the component waveform data of the component identified by the analysis unit 32.

(4-1-1) Modification 1A
The brain activity estimating means 30 has a conversion unit 31, and facial skin temperature data corresponding to the relative temperature conversion data is created by the conversion unit 31. And the analysis part 32 respond | corresponded not only the face skin temperature data according to the temperature conversion data acquired by the face skin temperature acquisition means 20, but the relative temperature data based on the temperature data converted into relative temperature data. The facial skin temperature data is also decomposed into a plurality of components by singular value decomposition, and each component is analyzed.

  Instead of this, the brain activity estimating means 30 may not have the conversion unit 31. In this case, it is possible to omit processing for creating facial skin temperature data corresponding to the relative temperature conversion data or analyzing data based on the facial skin temperature data corresponding to the relative temperature conversion data.

  However, in order to accurately identify a component related to human brain activity, the brain activity estimation means 30 has a conversion unit 31 as in the above embodiment, and the analysis unit 32 performs facial skin temperature acquisition means. A plurality of facial skin temperature data corresponding to the relative temperature data based on the temperature data converted to the relative temperature data as well as the facial skin temperature data corresponding to the temperature converted data acquired by 20 are obtained by singular value decomposition. It is desirable that each component is analyzed after being decomposed into these components.

(4-1-2) Modification 1B
The facial skin temperature acquisition means 20 is an infrared thermography device that can acquire temperature data in a non-contact state with an object.

  However, if the skin temperature of at least a part of an individual's face is detected and the facial skin temperature data including the detected temperature data and the position data of the detected part can be acquired in time series, the facial skin temperature acquisition means is infrared. It is not limited to a thermography device.

  For example, the facial skin temperature acquisition means may be a device including a temperature sensor. Specifically, a temperature sensor is attached to a predetermined part of an individual's face, and time-series facial skin temperature data is based on temperature data detected by the temperature sensor and position data of the part where the temperature sensor is attached. May be acquired. As described above, even when facial skin temperature data is acquired in a state in which the temperature sensor is in contact with the target individual, the temperature sensor does not require pre-wearing processing like an electroencephalogram electrode. Compared with conventional detection methods such as measurement, magnetic resonance imaging, and near infrared spectroscopy, data can be easily acquired. Thereby, human brain activity can be estimated simply.

(4-2) Brain activity estimating means 130 for estimating brain activity based on face image data.
FIG. 21 is a schematic diagram of the brain activity visualization apparatus 110 according to the embodiment of the present invention. FIG. 22 is a flowchart illustrating an example of a processing flow when the component indicating the RGB change of the face reflecting the brain function is identified in the brain activity visualization apparatus 110.

  The brain activity estimating means 130 included in the brain activity visualization device 110 is a device for estimating the brain activity of the individual from the captured image data of the face of the individual (subject). As shown in FIG. 21, the brain activity visualization apparatus 110 includes image data acquisition means 120, brain activity estimation means 130, and state visualization means 200.

  The image data acquisition unit 120 acquires captured image data of at least a part of an individual's face in time series (step S101). The image data acquisition means 120 is not particularly limited as long as it has at least a photographing device, and examples thereof include a portable terminal with a photographing device such as a smartphone or a tablet (for example, iPad: registered trademark). It is done. Here, the image data acquisition unit 120 includes a camera 121 as a photographing apparatus and a storage unit 122, as shown in FIG. The camera 121 is for acquiring captured image data of an individual's face in time series. Here, the camera 121 captures a moving image of the entire face of the individual and acquires the captured moving image data. The storage unit 122 accumulates time-series photographed image data photographed by the photographing apparatus. Here, the storage unit 122 accumulates moving image data acquired by the camera 121.

  Here, the moving image of the entire face is captured by the camera 121, but the present invention is not limited to this, and it is only necessary to capture a moving image including an image of at least the forehead portion and / or the sinuses around the face.

  Also, here, while the time-series captured image data of the face is acquired by the image data acquisition means 120, a brain function activation task is given to the individual for a certain period. That is, the captured image data acquired by the image data acquisition unit 120 includes data for a period in which a brain function activation task is given to an individual. The brain function activation task given to an individual is not particularly limited as long as it is estimated that the brain is activated, for example, depending on the purpose of use of the brain activity visualization device 110 You may be comprised so that the content may be determined suitably.

  The brain activity estimation unit 130 estimates human brain activity based on time-series captured image data of the face acquired by the image data acquisition unit 120. Specifically, as shown in FIG. 21, the brain activity estimation unit 130 includes an RGB processing unit 131, a conversion unit 132, a blood flow rate calculation unit 133, an analysis unit 134, and an estimation unit 135. FIG. 21 shows an aspect in which the brain activity estimation means 130 exists as one device having the RGB processing unit 131, the conversion unit 132, the blood flow calculation unit 133, the analysis unit 134, and the estimation unit 135. However, the present invention is not limited to this, and part or each of the RGB processing unit 131, the conversion unit 132, the blood flow volume calculation unit 133, the analysis unit 134, and the estimation unit 135 exists as an independent device. May be. In addition, here, the facial blood flow volume acquisition unit is configured by the image data acquisition unit 120, the RGB processing unit 131, the conversion unit 132, and the blood flow volume calculation unit 133.

  The RGB processing unit 131 performs RGB processing that decomposes the captured image data acquired by the image data acquisition unit 120 into three color components, an R component, a G component, and a B component (step S102). Here, the RGB processing may be performed on the captured image data of the entire face, but here, in order to reduce the calculation processing amount and noise, the data on the forehead part and / or the sinuses around the captured image data is reduced. It is assumed that RGB processing is performed only on the extracted data.

  The conversion unit 132 converts the RGB data of the captured image data obtained by the RGB processing into relative RGB data (step S103). Specifically, the conversion unit 132 converts the RGB data into relative RGB data using the average value of the RGB data obtained from the acquired captured image data every predetermined time (for example, 30 seconds) as a reference value. To do.

  The blood flow volume calculation unit 133 calculates time-series blood flow volume data of the face based on the RGB data of the captured image data obtained by the RGB processing (step S104).

  The analysis unit 134 decomposes the time-series relative converted blood flow data into a plurality of components by singular value decomposition, principal component analysis, or independent component analysis (step S105). Here, the analysis unit 134 performs singular value decomposition on the relative converted blood flow data using the SVLAB of MATLAB (registered trademark) as an analysis tool. Specifically, the singular value decomposition is performed on time-series relative converted blood flow data, with the factor being time data for each predetermined period (for example, 30 seconds), and the measure is calculated from relative RGB data for each period. This is performed as the calculated relative blood flow volume data for each pixel. Then, by singular value decomposition, the time-series relative converted blood flow data is decomposed into a plurality of components, and a time distribution, a spatial distribution, and a singular value indicating the size of each component are calculated.

  Further, the analysis unit 134 determines whether or not each component satisfies a predetermined condition in order to identify a component indicating the RGB change of the face reflecting the brain activity from the plurality of components decomposed by the singular value decomposition (step) S106). Here, as the predetermined condition, for example, the condition that the amplitude of the component waveform of the component decomposed by the singular value decomposition is correlated with the change at the time of non-activation of the brain and the activation of the brain (hereinafter referred to as the first condition). ), A condition that there is a change in blood flow in a predetermined part of the human face in a component decomposed by singular value decomposition (hereinafter referred to as a second condition), and the like. As the predetermined condition determined by the analysis unit 134, one or a plurality of conditions may be set. Here, it is assumed that the first condition is set as the predetermined condition.

  And the analysis part 134 extracts the component which satisfy | fills predetermined conditions among several components as a component for determination. Further, the analysis unit 134 identifies a component satisfying all the conditions included in the predetermined condition among the extracted determination components as a component indicating the RGB change of the face reflecting the brain activity (step S107). On the other hand, the analysis unit 134 determines that the component determined not to satisfy at least one condition included in the predetermined condition among the plurality of components is not a component indicating the RGB change of the face reflecting the brain activity (step S108). ).

  Here, only one condition (first condition) is set as the predetermined condition as described above, and the brain function activation task is given to the individual while acquiring time-series captured image data of the face. There is a given period of time. For this reason, the analysis unit 134 sets the period when the brain function activation task is not given to the individual as the non-activation time of the brain, and sets the period when the brain function activation task is given to the individual as the brain activation time The period in which the brain function activation task is given and the period in which the brain function activation task is not given, and the component waveform of each component are compared and analyzed. And the analysis part 134 evaluates whether the component waveform of each component and the time of non-activation of a brain and the time of activation of a brain have a correlation using the comparative analysis result based on component waveform data, Among these components, a component evaluated as having a correlation is extracted as a determination component that satisfies a predetermined condition, and is identified as a component indicating a change in RGB of the face reflecting brain activity. On the other hand, the analysis unit 134 determines that a component evaluated as having no correlation among the plurality of components is not a component that does not satisfy the predetermined condition and indicates a change in RGB of the face that reflects human brain activity.

  Here, a brain function activation task is given to an individual for a certain period when time-series captured image data of the face is acquired, and the analysis unit 134 extracts a determination component based on this, The content of the first condition, that is, the determination component extraction means in the analysis unit 134 is not limited to this. For example, when a component indicating a component waveform that is correlated with the non-activation time and the activation time of the brain among a plurality of components is identified by performing an experiment or the like in advance, the analysis unit 134 The component specified from the plurality of components is extracted as a determination component. In addition, when the human activity known to be related to activation / deactivation of the brain, such as eye movement or tapping, is also detected in the brain activity visualization device 110, the analysis unit 134 A determination component may be extracted from a plurality of components by comparative analysis and evaluation of the component waveform of each component. Note that the criterion for determining whether or not the first condition is satisfied by the analysis unit 134 is appropriately determined by simulation, experiment, desktop calculation, or the like according to the purpose of use of the brain activity visualization device 110 or the like.

  When the second condition is set as the predetermined condition, the analysis unit 134 extracts a determination component based on whether or not there is a change in the blood flow rate of the face at a predetermined part of the human face. Specifically, the analysis unit 134 determines whether or not there is a change in blood flow around the sinuses and / or the forehead based on the blood flow distribution map corresponding to a plurality of components decomposed by the singular value decomposition. If a change in blood flow occurs, it is determined that the component satisfies the second condition. On the other hand, when there is no change in the blood flow volume around the sinuses and / or the forehead, the analysis unit 134 determines that the component does not satisfy the second condition. Note that the criterion for determining whether or not the second condition is satisfied by the analysis unit 134 is appropriately determined by simulation, experiment, desktop calculation, or the like according to the purpose of use of the brain activity visualization device 110 or the like.

  Further, when time-series blood flow data based on RGB data before being converted into relative RGB data is calculated by the blood flow calculation unit 133, the blood flow data is subjected to singular value decomposition by the analysis unit 134. With respect to a plurality of components obtained by performing the same, it may be determined whether or not the first condition and / or the second condition is satisfied, and a determination component may be extracted.

  The estimation unit 135 estimates the human brain activity based on the component identified by the analysis unit 134 as the component indicating the RGB change of the face reflecting the human brain activity. Specifically, the estimation unit 135 estimates the amount of brain activity when acquiring facial image data based on the component waveform data of the component identified by the analysis unit 134.

(4-2-1) Modification 2A
As described above, as the camera 121, for example, a portable terminal with a built-in photographing apparatus such as a smartphone or a tablet (for example, iPad: registered trademark) can be used. That is, as the above-described captured image data, data that captures an image in the visible light region can be employed.

  Further, the blood flow amount calculation unit 133 may calculate the blood flow amount data of the face mainly using the R component of each pixel included in the RGB data. In addition, blood flow volume data is not necessarily limited to the erythema index as long as blood flow volume data can be calculated based on RGB data.

(4-2-2) Modification 2B
The blood flow volume calculation unit 133 calculates relative converted blood flow volume data based on the relative RGB data converted by the conversion unit 132, but instead of or in addition to this, the blood flow volume calculation unit 133 converts the blood flow volume data into relative RGB data. The blood flow volume data may be calculated based on the RGB data before the check. Here, since the blood flow volume data calculated based on the RGB data before being converted into the relative RGB data easily includes a component correlated with the brain activity (the test power is high), for example, the relative RGB data Blood flow volume data calculated based on RGB data before conversion into data may be analyzed prior to relative converted blood flow volume data calculated based on relative RGB data. In addition, for example, first, the blood flow data is analyzed to extract a component having a significant correlation, and the relative conversion blood flow data is calculated by analyzing only the component corresponding to the extracted component. Can be reduced.

(4-2-3) Modification 2C
The camera 121 is assumed to be a normal camera in the visible light region, but an infrared camera can also be used. In this case, infrared light is irradiated and the reflected wave is imaged with an infrared camera. Thereby, it is possible to obtain photographed image data such as a face change of the subject. Calculated by the present inventors mainly using the R component of the blood flow volume data calculated from the captured image data obtained by infrared reflection and each pixel included in the RGB data captured in the visible light region. It was confirmed that there was a correlation with the measured blood flow data. Therefore, human brain activity can be estimated even using such captured image data obtained from infrared reflection.

(4-2-4) Modification 2D
In the above description, the brain activity visualization device 110 is configured to include the image data acquisition unit 120 and the brain activity estimation unit 130. However, the brain activity visualization device according to the present embodiment has such a configuration. It is not limited. That is, the brain activity visualization apparatus according to the present embodiment can take any form for other configurations as long as it includes the blood flow volume calculation unit 133, the analysis unit 134, and the estimation unit 135. Specifically, the brain activity visualization device according to the present embodiment includes not only a mode in which the device itself captures image data, but also a mode in which captured image data is received from an external device and analyzed.

(4-3) State visualization means 200
The state visualizing means 200 is visualized by displaying the physiological state of the subject based on the brain activity of the subject estimated by the brain activity estimating means 30 and / or the brain activity estimating means 130. For example, the state visualization unit 200 may include an analysis unit 201 that analyzes the physiological state of the subject by analyzing changes in the amount of brain activity of the subject. Specifically, the analysis unit 201 analyzes the change in the amount of brain activity with respect to the stimulus (visual stimulus, auditory stimulus, tactile stimulus, olfactory stimulus, taste stimulus, etc.) given to the subject, so that the subject Physiological state is determined. In addition, about the kind and level of a physiological state, based on the raise degree and / or duration of brain activity amount, according to the use of the brain activity visualization apparatuses 10 and 110, you may be able to install suitably. Then, the physiological state of the subject analyzed by the analysis unit 201 is output from the display unit 202 of the state visualization means 200 to the administrator, so that the administrator can know the physiological state of the subject. As the display unit 202, any display device that displays an image or a message can be adopted as long as it can visualize information on the analyzed physiological state of the subject person to the administrator. .

  In addition, when various components of time series are acquired by the facial skin temperature acquisition unit 20 and / or the image data acquisition unit 120 after the components reflecting the brain activity are identified in the analysis units 32 and 134, In the activity visualization apparatuses 10 and 110, the acquired various data are further decomposed into a plurality of components by singular value decomposition, and only the identified components are analyzed, so that the physiological state of the subject can be known in real time. .

  Furthermore, there is a conventional technique for acquiring the subject's facial skin temperature or captured image from heartbeat information, biological information, or the like. However, various techniques obtained from the facial skin temperature acquisition means 20 and / or the image data acquisition means 120 are available. By adopting a conventional technique for a component obtained by performing singular value decomposition or the like on the data, heart rate information and biological information can be obtained with high accuracy. Therefore, the analysis unit 32 and / or the analysis unit 134 have a function of analyzing a plurality of components subjected to singular value decomposition to acquire heart rate information and biological information, and based on the acquired heart rate information and biological information, the exchange nerve / parasympathy You may give the estimation part 33,135 of the said embodiment the function which estimates the function of a nerve.

(5) Features (5-1)
In the present embodiment, human brain activity is estimated based on time-series facial skin temperature data and / or facial blood flow data acquired by the facial skin temperature acquisition means 20 and / or the image data acquisition means 120. For this reason, human brain activity can be estimated without mounting a sensor that requires processing before mounting an electroencephalogram electrode or the like. Therefore, it is possible to easily estimate human brain activity and visualize the physiological state of the subject based on the estimated brain activity.

(5-2)
Here, when time-series facial skin temperature data and / or image data is acquired, a human brain function is activated by whether or not a human brain function activation task is actually given. If the situation is activated or not activated, the component having a correlation between the component waveform of each component and the activation and non-activation of the brain is the skin temperature reflecting the brain activity and / or It can be said that it is a component with high possibility of being a component which shows the change of blood flow volume.

  In this embodiment, while the facial skin temperature acquisition means 20 and / or the image data acquisition means 120 are acquiring time-series facial skin temperature data and / or image data, the brain function activation task is performed on the individual. Given for a certain period. That is, in this embodiment, a situation is created in which the human brain is activated or not activated by actually giving or not giving a brain function activation task to an individual. The time series data acquired in this way is decomposed into a plurality of components by singular value decomposition, and the correlation between the component waveform and activation and non-activation of the brain is evaluated for each component. Are extracted from a plurality of components as determination components. For this reason, for example, compared with a case where a predetermined component specified in advance by experiments or the like is extracted from a plurality of components as an extraction component, a plurality of components that are less relevant to human brain activity are extracted as a plurality of extraction components. The possibility of being extracted from the components can be reduced.

(5-3)
Here, the brain has a mechanism that cools the brain independently of the body temperature, which is a selective brain cooling mechanism. As a selective brain cooling mechanism, it is known that heat generated by brain activity is exhausted using the forehead portion and the periphery of the sinuses. If it does so, the change of the blood flow volume of the face correlating with the facial skin temperature and facial skin temperature accompanying brain activity will appear in a forehead part and / or a sinus periphery.

  In the present embodiment, various data around the forehead and / or the sinuses are analyzed, and the determination component is extracted. For this reason, it is possible to accurately extract components related to human brain activity.

(6) Application example of brain activity visualization apparatus Next, an application example of the brain activity visualization apparatus according to the present invention will be described.

(6-1) When used for a patient An example of using the brain activity visualization devices 10 and 110 of the above-described embodiment or the modified example for a patient who has visited a hospital will be described. For example, in the case where the brain activity visualization devices 10 and 110 are used to objectively quantify the depression state, a brain function activation task such as mental arithmetic with carry-up or carry-down is given to the patient, and the brain activity activation task is given. By analyzing and visualizing changes in the amount of brain activity before and after being given, the patient's mental state can be determined. Specifically, if the amount of brain activity does not increase while the brain function activation task is given, it can be determined that the patient is in a state of lethargy, and the brain activity while the brain function activation task is given Even if the amount rises, if the rise time of the amount of brain activity is short, it can be determined that the patient's energy is reduced. Then, such an analysis is performed a plurality of times during one day, and when the decrease in the brain activity amount is recognized on average, the administrator can determine that the patient is in a depressed state.

  In addition, when the brain activity visualization devices 10 and 110 are used to determine whether or not an emergency patient is conscious or whether or not the patient has awakened, stimulation such as touching the patient's skin or calling the patient. And the change in the amount of brain activity before and after the stimulus is applied is analyzed and visualized, so that it is possible to determine whether the patient is conscious or awake. For example, it can be determined that the patient has awakened when the amount of brain activity increases while stimulating the skin or speaking to the patient into which anesthesia has been introduced. Therefore, even if the patient is in a state where he / she cannot speak, the manager can know whether or not the patient is conscious and whether or not the patient has awakened. Also, the degree of arousal can be determined by changing the intensity of the stimulus applied to the patient and analyzing the presence or absence of brain activation at this time. Examples of the low-intensity stimulus include stimuli such as holding a hand or moving the hand. Examples of the high-intensity stimulus include stimuli that change the temperature of the body, such as applying ice to the hand. And stimuli that give pain to the body.

  When the brain activity visualization devices 10 and 110 are used to determine the effect of treatment such as rehabilitation, a brain function activation task such as mental arithmetic with carry-up or carry-down is given to the patient, and the brain at this time By analyzing and visualizing changes in the amount of activity, the effect of treatment such as rehabilitation on the patient can be determined. For example, a brain function activation task having the same intensity before and after performing rehabilitation, brain training, or exercise therapy is given to a patient, and the change in the amount of brain activity at this time is analyzed. The effect of treatment such as rehabilitation can be determined from the degree of increase in activity amount and duration of increase. In addition, for example, when the brain activity when the brain function activation task is given to the patient is not increased, it can be determined that the ischemic state of the cerebral blood vessel, and the brain activity continues to increase. If the time is short, it can be determined that the cerebral blood flow is in a reduced state. Therefore, the brain activity visualization devices 10 and 110 can also be used as a monitoring device in a hyperbaric oxygen therapy device.

  Further, when the brain activity visualization devices 10 and 110 are used for quantifying patient pain, changes in the amount of brain activity when the patient feels pain (when reporting from the patient) (in particular, the brain The intensity of pain may be quantified from the degree of increase in activity and the duration thereof. By visualizing the analysis result, the administrator can know the degree of pain of the patient.

(6-2) When used for a person in a special environment that receives a shock wave or the like The brain activity visualization devices 10 and 110 according to the above-described embodiment or the modification are placed in a special environment that receives an explosion shock wave such as a fire brigade. An example in the case of using with respect to a certain person is demonstrated. When the brain activity visualization devices 10 and 110 are used for biological protection determination (for example, determination of the state of biological damage received by a shock wave) due to exposure to a shock wave or the like, a brain function activation task is given to the subject. By analyzing and visualizing the change in the amount of brain activity at this time, the administrator can estimate the blood flow state of the subject's cerebral blood vessels. For example, if the brain activity when the brain function activation task is given to the subject has not increased, it can be determined that the ischemic state of the cerebral blood vessel, and the duration of increase in brain activity Is short, it can be determined that the cerebral blood flow is in a reduced state.

(6-3) When used for determination of comfort An example of the case where the brain activity visualization devices 10 and 110 according to the embodiment or the modified example are used for determination of comfort of a subject will be described. For example, when the brain activity visualization devices 10 and 110 are used to judge the comfort of a house, the amount of brain activity when the subject in the predetermined living space feels uncomfortable (when reporting from the subject). The degree of discomfort is quantified from the change (the degree of increase in brain activity and its duration). By performing such an analysis multiple times during the day and visualizing the results of the analysis, the administrator evaluates whether or not the brain activity is on average, It is possible to determine the comfort level, that is, the feeling of pleasant or unpleasant feeling.

(6-4) When used for determination of concentration degree An example of the case where the brain activity visualization devices 10 and 110 according to the embodiment or the modification are used for determination of the concentration level during learning or surgery will be described. For example, in the case where the brain activity visualization devices 10 and 110 are used to quantify the degree of concentration of a learner in a school, a cram school, a company, e-learning, or a hospital, the learner learns (problems). By analyzing changes in the amount of brain activity during a certain period of time before and after working on (for example, the learning period) (the degree of increase during this period), it is possible to quantify the degree of concentration of the learner with respect to the learning content being worked on it can. Thereby, the administrator can evaluate the concentration degree of the learner with respect to the learning content based on the analysis result visualized.

(7) Pain determination device A pain determination device to which the brain activity visualization device according to the present invention is applied will be described.

(7-1) Configuration of Pain Determination Device FIG. 23 is a schematic diagram illustrating an example of a pain determination device according to the present embodiment. In the following description, an example of determining pain due to cancer or the like will be described.

  The pain determination apparatus 1000 includes an input unit 1010, an imaging unit 1015, an output unit 1020, a storage unit 1030, and a processing unit 1040.

  The input unit 1010 inputs various information to the pain determination apparatus 1000. For example, the input unit 1010 includes a keyboard, a mouse, and / or a touch screen. Various commands are input to the pain determination apparatus 1000 via the input unit 1010, and processing according to the commands is executed in the processing unit 1040.

  The imaging unit 1015 captures a “face image” including the face of the subject 300. For example, the imaging unit 1015 includes a solid-state imaging device such as a CCD and a CMOS that acquires RGB images, an infrared camera that acquires a thermogram, and the like. It is desirable that the infrared camera or the like can detect 29.0 ° C. to 37.0 ° C. with high sensitivity under normal room temperature conditions. In addition, the imaging unit 1015 can perform continuous imaging at a predetermined interval. When capturing a facial image, it is desirable to perform the illumination under constant lighting conditions from the front. When the front image cannot be obtained due to the posture variation, the perturbation space method is used, and the three-dimensional shape of the face is estimated for the posture variation image, and the face image is obtained by rendering the front image. As for the illumination variation image, a facial image under a constant illumination condition is obtained by using a facial illumination base model constructed based on the diffuse reflection model. Then, the facial image continuously captured by the imaging unit 1015 is sent to the processing unit 1040.

  The output unit 1020 outputs various information from the pain determination apparatus 1000. For example, the output unit 1020 includes a display, a speaker, and the like. Here, the brain function activation information described later is provided to the target person 300 via the output unit 1020. In addition, when auditory stimulus information is used as brain function activation information, the headphone 1020a can be used as the output unit 1020.

  The storage unit 1030 stores information input to the pain determination apparatus 1000, information calculated by the pain determination apparatus 1000, and the like. For example, the storage unit 1030 includes a memory and a hard disk device. The storage unit 1030 stores a program for realizing each function of the processing unit 1040 described later. Here, the storage unit 1030 includes a brain function activation information database 1031 and a determination information database 1032.

  The brain function activation information database 1031 stores brain function activation information that activates human brain functions. Here, the “brain function activation information” includes auditory stimulation information that stimulates hearing. The “auditory stimulus information” corresponds to, for example, acoustic information that causes the subject 300 to recognize text, music, or other sounds. The brain function activation information is not limited to this, and any information that activates the brain function of the subject can be used. For example, as other brain function activation information, “visual stimulus information” including images that allow the subject to recognize images, moving images, and schematic diagrams, or memory of symbols, characters, or languages, etc. Arbitrary stimulus information or the like that causes the subject to perform the psychological work.

  As shown in FIG. 24, the determination information database 1032 is a reference determination component extracted when the target person 300 is in a normal state of the correlation value r2 of the determination component extracted in response to provision of brain function activation information. The change amount Δr (= r2−r1) in a predetermined range from the “reference correlation value” r1 is stored in advance as “determination information” in association with the “pain state level”. The change amount Δr is expressed as an absolute value. Here, the “normal state” in the subject 300 is a state in which the subject 300 has no pain, and is a state in which the emotion of the subject 300 is considered stable. In addition, the “reference determination component” includes, in addition to the determination component data extracted when the target person 300 is in a normal state, the previously extracted determination component data, the determination component data provided from the outside, or the like It is also possible to set by. The reference determination component data is stored in the determination information database 1032.

  In the example illustrated in FIG. 24, the determination information database 1032 indicates that “severe pain” is less than Δr = Δra, “mild pain” is less than Δra to Δrb, and is Δrb or more, depending on the range of the amount of change Δr as the pain state level. Is stored as “normal (no pain)”. Here, the values increase in the order of Δra and Δrb. Note that when the subject 300 feels severe pain, the subject 300 cannot respond to the auditory stimulus information or the like even if the auditory stimulus information or the like is provided. Therefore, in this case, the amount of change Δr is small. On the other hand, when the subject 300 does not feel pain, the subject 300 responds to the auditory stimulus information and the like, and therefore the value of the change amount Δr becomes large.

  The processing unit 1040 executes information processing in the pain determination apparatus 1000. Specifically, the processing unit 1040 includes a CPU and a cache memory. The processing unit 1040 executes a program incorporated in the storage unit 1030, whereby a brain function activation information providing unit 1041, a face change information acquisition unit 1042, a face change information decomposition unit 1043, a determination component extraction unit 1044, and Functions as the pain determination unit 1045.

  The brain function activation information providing unit 1041 provides brain function activation information. For example, the brain function activation information providing unit 1041 reads out auditory stimulation information (brain function activation information) from the brain function activation information database 1031 according to an operation of the input unit 1010 and outputs the information to the output unit 1020.

  The face change information acquisition unit 1042 acquires “face data” and “face change information” indicating a time-series change of face data from the face image captured by the image capturing unit 1015. Specifically, the face change information acquisition unit 1042 acquires face data via the imaging unit 1015 in synchronization with the timing when the brain function activation information providing unit 1041 provides the brain function activation information. Then, the face change information acquisition unit 1042 acquires face change information indicating a time-series change of the face data of the target person 300 from the continuously acquired face data. For example, the face change information is a set of 4,608,000 data when 60 points of 240 × 320 pixel face data are acquired at predetermined intervals. The acquired face change information is sent to the face change information decomposing unit 1043. When the imaging unit 1015 is an infrared camera, the face change information acquisition unit 1042 acquires facial skin temperature data indicating the skin temperature of the face of the subject 300 as the facial data. When the imaging unit 1015 is a solid-state imaging device such as a CCD or CMOS, the facial change information acquisition unit 1042 acquires facial blood flow data based on RGB data of the face of the subject 300 as facial data. Note that the face change information acquisition unit 1042 may acquire only data on the periphery of the sinus and / or the forehead part of the subject 300 as the face data.

  The face change information decomposition unit 1043 decomposes face change information, which is a collection of a large number of data, into a plurality of components 1, 2, 3,... By singular value decomposition, principal component analysis, or independent component analysis. Information of each decomposed component is sent to the determination component extraction unit 1044. Here, when face change information is subjected to singular value decomposition, components 1, 2, 3,... Are set in descending order of singular values. In addition, the higher the singular value, the more easily the influence of the fluctuation is reflected. For this reason, the effect of noise or the like of the external environment is often reflected in the component 1 rather than the effect of providing brain function activation information.

  The determination component extraction unit 1044 extracts a component related to brain function activation information as a “determination component” from a plurality of components 1, 2, 3,. Further, the determination component extraction unit 1044 calculates a correlation value r for the brain function activation information of the extracted determination component. Specifically, the determination component extraction unit 1044 includes a plurality of components 1, 2, 3,... Obtained by the face change information decomposition unit 1043 and a “determination waveform” corresponding to the brain function activation information. A correlation value r is calculated. Next, when the calculated correlation value r is greater than or equal to a predetermined value, the determination component extraction unit 1044 sets the component corresponding to the correlation value r as related to the brain function activation information. Then, the determination component extraction unit 1044 extracts a determination component based on the risk factor value. That is, the determination component extraction unit 1044 extracts a component with a low risk factor as a determination component. The extracted determination component and the calculated correlation value r are sent to the storage unit 1030 or the pain determination unit 1045. As the above-described “determination waveform”, a deformed wave considering a human physiological reaction is used. Further, the determination waveform is displaced after a predetermined time has elapsed since the brain function activation information is provided. Specifically, a rectangular wave can be adopted as the determination waveform. In addition, a waveform obtained by convolving a rectangular wave and a red spot dynamic response function may be employed as the determination waveform. The red spot dynamic response function is a component in which a correlation is recognized among a plurality of components 1, 2, 3,... Decomposed by the face change information decomposition unit 1043 when the brain function activation information is given for a moment. Is calculated a plurality of times, and is generated from the average value of the calculated plurality of components. At this time, since the amplitude (height direction) is an arbitrary unit and an absolute value cannot be defined, a signal when the subject 300 is in a normal state is used as a baseline value, and the waveform height is determined based on that value. The Then, the average value of the superposition of data obtained from a plurality of subjects is calculated to generate a red spot dynamic response function. The initial value of the red spot dynamic response function has a waveform as shown in FIG. 25 when the brain function activation information is given for a moment. When brain function activation information is given for a certain period of time, it is created by convolution with a rectangular wave. The red spot dynamic response function has a waveform in which the peak value is extended in the horizontal axis direction from the peak point as the displacement increases. The red spot dynamic response function has a waveform in which the displacement decreases with the phase being delayed from the time when provision (stimulation) of brain function activation information is completed. When such a red spot dynamic response function has a significant correlation with the component obtained from facial change information, the correlation waveform is close in shape, so a higher correlation value than a rectangular wave or the like is obtained. Show. Therefore, the extraction accuracy of the determination component can be increased.

  The pain determination unit 1045 provides brain function activation information when performing pain determination and the reference correlation value r1 for the reference determination component extracted by providing brain function activation information when the subject 300 is in a normal state. A difference Δr with respect to the correlation value r2 for the determination component extracted in this way is calculated. Then, the pain determination unit 1050 determines a pain state level corresponding to the difference Δr between the reference correlation value r1 and the correlation value r2 based on the determination information stored in the determination information database 1032. The determined pain state level is output to a display device or the like via the output unit 1020.

(7-2) Operation of Pain Determination Device FIG. 26 is a flowchart showing the operation of the pain determination device 1000. In the following description, an example of determining pain due to cancer or the like will be described.

  First, when the target person 300 is in the normal state, the “reference setting mode” is selected, and the reference determination component is extracted (U1). Specifically, an instruction to output brain function activation information is input to the pain determination apparatus 1000 via the input unit 1010. Subsequently, brain function activation information is read from the brain function activation information database 1031 and output to the output unit 1020 (U2). For example, “auditory stimulus information” is output as the brain function activation information through the headphones 1020a. As a premise, the subject 300 is required to carefully listen to auditory stimulus information (sound) flowing from the headphones 1020a and the like.

  Next, at the same time as the output of the brain function activation information or at a predetermined timing, a face image including the face of the subject 300 is captured at predetermined intervals by the imaging unit 1015. The captured face image is sent to the face change information acquisition unit 1042.

  Subsequently, the face change information acquisition unit 1042 acquires face change information indicating a time-series change of the face data of the target person 300 from the captured face image (U3). Then, the face change information decomposition unit 1043 decomposes the face change information into a plurality of components 1, 2, 3,... By singular value decomposition, principal component analysis, or independent component analysis (U4).

  Next, the determination component extraction unit 1044 calculates a correlation value between the plurality of components 1, 2, 3,... Decomposed by the face change information decomposition unit 1043 and the determination waveform corresponding to the brain function activation information. Is done. Then, the determination component extraction unit 1044 determines whether or not the correlation value is greater than or equal to a predetermined value (U5). When it is determined that the value is equal to or greater than the predetermined value, it is determined that there is a correlation between the brain function activation information and the component (U5-Yes). Then, the determination component extraction unit 1044 extracts a component having a low risk ratio as a “reference determination component” among the correlated components (U6). Further, the determination component extraction unit 1044 sets a correlation value between the reference determination component and the brain function activation information as the reference correlation value r1. Information on these reference determination components is stored in the storage unit 1030 (U7). On the other hand, if the correlation value between the brain function activation information and each component 1, 2, 3,... Is less than a predetermined value, it is determined that there is no correlation between the two, and the information is stored in the storage unit 1030. (U5-No, U7).

  Thereafter, the “determination mode” is selected at a predetermined timing, and the subject's pain determination is executed (U8).

  First, the same processing as the above steps U2 to U5 is executed, and the correlation value r2 between the determination component extracted from the face change information and the brain function activation information is calculated (U9 to U12). Here, in step U12, the correlation between the determination component and the brain function activation information may not be detected (U12-No). In this case, it is considered that the activation of the brain function based on the brain function activation information is inhibited (masked) by pain, and the subject 300 is in a state of feeling “pain (severe pain)”. (U12-No, U16).

  On the other hand, when the correlation value r2 is calculated in step S12, the pain determination unit 1045 extracts the reference correlation value r1 of the brain function activation information for the reference determination component extracted in the reference setting mode and the determination mode. A change amount Δr that is a difference from the correlation value r2 of the brain function activation information with respect to the determination component is calculated (U12−Yes, U13). Subsequently, the pain determination unit 1045 determines whether or not the change amount Δr of the correlation value r2 with respect to the reference correlation value r1 exceeds a predetermined range (U14). Whether or not the predetermined range is exceeded is determined based on the determination information stored in the determination information database 1032. When the change amount Δr of the correlation value r2 with respect to the reference correlation value r1 exceeds a predetermined range (when Δr is equal to or greater than Δrb), the pain determination unit 1045 causes the subject 300 to be “normal (a state in which no pain is felt)”. (U14-Yes, U15). On the other hand, when the change amount Δr of the correlation value r2 with respect to the reference correlation value r1 does not exceed the predetermined range (when Δr is less than Δrb), the pain determination unit 1045 determines that the subject 300 is “pain (light pain or severe pain). "Is felt (U14-No, U16). And these determination results are output to a display apparatus etc. via the output part 1020 (U17).

(7-3) Features of pain determination device (7-3-1)
As described above, the pain determination apparatus 1000 according to the present embodiment includes the brain function activation information providing unit 1041, the face change information acquisition unit 1042, the face change information decomposition unit 1043, the determination component extraction unit 1044, A pain determination unit 1045. The brain function activation information providing unit 1041 provides the target person 300 with “brain function activation information” that activates the human brain function. The face change information acquisition unit 1042 acquires “face change information” indicating a time-series change of the face data of the subject 300. The face change information decomposition unit 1043 decomposes the face change information into a plurality of components 1, 2, 3,... By singular value decomposition, principal component analysis, or independent component analysis. The determination component extraction unit 1044 extracts a component related to the brain function activation information from the plurality of components 1, 2, 3,. The pain determination unit 1045 determines the pain state of the subject 300 based on the extraction result by the determination component extraction unit.

  Therefore, in the pain determination apparatus 1000 according to the present embodiment, the face change information is obtained from a plurality of components 1, 2, 3,... Obtained by performing singular value decomposition, principal component analysis, and independent component analysis. Since the determination component related to the function activation information is extracted, it is possible to easily estimate the presence or absence of the brain activity of the subject 300 without using an electrode or the like that requires pretreatment before wearing. Thereby, based on the component for determination corresponding to the brain function of the subject person 300, the pain state in the subject person 300 can be easily judged.

(7-3-2)
Further, in the pain determination apparatus 1000 according to the present embodiment, the facial change information acquisition unit 1042 acquires data on the periphery of the sinuses and / or the forehead part of the subject 300 as facial data, and thus is related to brain activity. The determination component to be extracted can be extracted with high accuracy. Here, the brain has a mechanism called a selective brain cooling system that cools the brain independently of the body temperature. The selective brain cooling mechanism exhausts heat generated by brain activity using the sinuses and the forehead area. Therefore, by analyzing the data of these parts, components related to brain activity can be extracted with high accuracy. As a result, the pain determination apparatus 1000 according to the present embodiment can execute the determination of the pain state with high accuracy.

(7-3-3)
In the pain determination apparatus 1000 according to the present embodiment, the facial change information acquisition unit 1042 acquires facial skin temperature data indicating the skin temperature of the face of the subject 300 as facial data. In other words, the pain determination apparatus 1000 can determine the pain state using an infrared camera or the like.

(7-3-4)
In the pain determination apparatus 1000 according to the present embodiment, the facial change information acquisition unit 1042 acquires facial blood flow data based on RGB data of the face of the subject 300 as facial data. That is, the pain determination apparatus 1000 can determine a pain state using a solid-state imaging device (CCD, CMOS). Thereby, determination of a pain state can be performed with a simple structure.

(7-3-5)
Further, in the pain determination apparatus 1000 according to the present embodiment, the determination component extraction unit 1044 extracts a determination component based on the risk factor value. In the pain determination apparatus 1000, since the determination component related to the brain function activation information is extracted based on the value of the risk rate, the reliability of the determination of the pain state can be improved.

(7-3-6)
In the pain determination apparatus 1000 according to the present embodiment, the brain function activation information providing unit 1041 provides the target person 300 with auditory stimulation information that stimulates hearing as brain function activation information. By using auditory stimulus information, there is no correlation between the auditory stimulus information and the component indicating the activation state of the brain function when there is pain, and the component indicating the activation state of the auditory stimulus information and the brain function when there is no pain Can be realized. Thereby, the pain state can be determined based on whether or not the activation of the brain function based on the brain function activation information is inhibited (masked) by the pain. And in such a structure, since pain determination can be performed only by providing auditory stimulus information at arbitrary timing, a pain state can be determined with a simple structure.

(7-3-7)
In the pain determination apparatus 1000 according to the present embodiment, the pain determination unit 1045 determines a pain state according to the amount of change in the determination component. Thereby, according to the variation | change_quantity from the reference | standard acquired previously, a pain state level can be determined easily.

  Specifically, the pain determination apparatus 1000 further includes a determination information database (determination information storage unit) 1032. The determination information database 1032 is a change in a predetermined range of the correlation value r2 of the determination component calculated for the brain function activation information from the reference correlation value r1 of the reference determination component calculated for the brain function activation information. The amount Δr is stored as “determination information” in association with the pain state level. In addition, the pain determination unit 1045 calculates the correlation value r2 of the determination component for the brain function activation information, and determines the pain state level of the target person 300 based on the calculated correlation value r2 and the determination information.

  With such a configuration, the pain determination apparatus 1000 can easily determine the pain state level using a reference determination component acquired in advance. In short, the pain determination apparatus 1000 can determine and output the pain state level as well as the presence or absence of pain.

(7-3-8)
The pain state determination method according to the present embodiment does not necessarily require the pain determination apparatus 1000. That is, the pain state determination method according to the present embodiment provides a brain function activation information providing step for providing brain function activation information for activating a human brain function to the target person 300 regardless of the presence or absence of the pain determination apparatus 1000; After providing the brain function activation information, a face change information acquisition step for acquiring “face change information” indicating a time-series change of the face data of the target person 300, and the face change information, singular value decomposition, principal component analysis or independent Based on a facial change information decomposition step that decomposes into a plurality of components by component analysis, a determination component extraction step that extracts a component related to brain function activation information from a plurality of components as a determination component, and a determination component The pain state determination step for determining the pain state of the subject person 300 may be used.

  According to such a pain state determination method, after providing brain function activation information, brain function activation is performed from a plurality of components obtained by performing singular value decomposition, principal component analysis, or independent component analysis on facial change information. Since the determination component related to information is extracted to determine the pain state, the pain state of the subject 300 can be easily determined.

(7-3-9)
Moreover, the pain determination apparatus 1000 according to the present embodiment may be one in which the brain function activation information providing unit 1041 and the face change information acquiring unit 1042 are incorporated in a helmet-type device. With such a configuration, it is possible to acquire a face image with little movement of the face, and thus it is possible to acquire highly reliable face change information. Moreover, convenience can be improved by using a helmet type. As a result, highly reliable pain determination can be easily realized.

(7-3-10)
In addition, the pain determination apparatus 1000 not only determines the pain state using the amount of change in a predetermined range from the reference value of the correlation value for the determination component as described above, but also performs multiple regression on the determination component. The pain state may be determined based on any one or any combination of a value obtained by analysis, an area generated by the determination waveform, an average value of the determination waveform, and a centroid value of the determination waveform. Supplementally, when “multiple regression analysis” is used, the correlation values of the responses of multiple stimuli can be easily quantified. When “correlation value” is used, the correlation value of the response by a single stimulus can be easily quantified. By using “area”, the absolute value of the reaction can be easily quantified. Using the “average value”, the absolute value of the reaction can be easily quantified. Further, noise can be reduced compared to the area. Using the “centroid value” makes it possible to easily quantify the absolute value of the reaction. In addition, it is possible to easily determine at which timing the reaction has occurred compared to the area.

(7-3-11)
In the pain determination apparatus 1000 according to the present embodiment, the determination component extraction unit 1044 extracts a determination component based on correlation values between a determination waveform corresponding to brain function activation information and a plurality of components. With such a configuration, the determination component corresponding to the activation state of the brain function of the subject 300 can be specified.

  In addition, here, a deformed wave taking into account a human physiological reaction can be employed as the determination waveform. Further, the determination waveform is displaced after a predetermined time has elapsed since the brain function activation information is provided. Such a determination waveform exhibits a high correlation value when a significant correlation is recognized with the component obtained from the face change information, so that the determination component extraction accuracy can be increased. In addition, a highly accurate correlation can be obtained by slightly delaying the phase with respect to the reaction of the brain. Specifically, a red spot dynamic response function can be employed as the determination waveform. The red spot dynamic response function is a component in which a correlation is recognized among a plurality of components 1, 2, 3,... Decomposed by the face change information decomposition unit 1043 when the brain function activation information is given for a moment. Is calculated a plurality of times, and is generated from the average value of the calculated plurality of components. Since the red spot dynamic response function is optimized based on the past history, it shows a high correlation value when a significant correlation is recognized with the component obtained from the face change information. Thereby, the extraction accuracy of the determination component can be increased.

  A rectangular wave can also be adopted as the determination waveform. Since the rectangular wave can easily correspond to the provision timing of brain function activation information, the determination component can be easily extracted.

(7-4) Modified Example of Pain Determination Device As shown in FIG. 27, the pain determination device 1000 according to the present embodiment may use a determination information providing device 1100 provided on a network.

  Here, the determination information providing apparatus 1100 includes a storage unit 1130 and a processing unit 1140.

  The storage unit 1130 includes a determination information database 1132. The determination information database 1132 has the same configuration as the determination information database 1032 described above. That is, the determination information database 1132 includes the reference correlation value r1 of the reference determination component extracted when the target person 300 is in the normal state of the correlation value r2 of the determination component calculated in response to the provision of the brain function activation information. Is stored as determination information in association with the pain state level.

  The processing unit 1140 transmits the determination information stored in the determination information database 1132 in response to a request from the pain determination apparatus 1000. Note that the processing unit 1140 may have a function of generating determination information as big data based on predetermined information independently of the determination component extracted by the pain determination apparatus 1000. Further, when the reference correlation value r1 is calculated by the pain determination apparatus 1000, the processing unit 1140 executes a process of updating the reference correlation value r1 stored in the determination information database 1032 as needed.

  In this modification, the pain determination unit 1045 requests the determination information providing apparatus 1100 described above to provide determination information. Specifically, in the pain determination apparatus 1000 according to this modification, the determination information database 1132 is stored in the determination information providing apparatus 1100 on the network, and the determination is performed when the pain determination unit 1045 determines the pain state level. The information providing apparatus 1100 is accessed. And the pain determination part 1045 determines the pain state level of the subject 300 based on the calculated correlation value r2 and determination information.

  Therefore, if it is the pain determination apparatus 1000 of this modification, the pain determination part 1045 can determine the pain state level of the subject person 300 using the determination information provision apparatus 1100 on a network.

  Furthermore, according to the pain state determination method of the present modification, it is possible to determine the pain state using big data. That is, the reference correlation value r1 and the predetermined change amount Δr can be obtained from the big data. Thereby, determination information can be optimized at any time.

(8) Drug Effect Determination Device A drug effect determination device to which the brain activity visualization device according to the present invention is applied will be described.

(8-1) Configuration of Drug Effect Determination Device FIG. 28 is a schematic diagram illustrating an example of a drug effect determination device according to this embodiment.

  The drug effect determination apparatus 1200 includes an input unit 1210, an imaging unit 1215, an acquisition unit 1216, an output unit 1220, a storage unit 1230, and a processing unit 1240. Examples of the “drug 305” determined by the drug effect determination apparatus 1200 include opinoid-based analgesics including morphine and analgesics such as NSAIDs.

  The input unit 1210 is used to input various types of information to the drug effect determination device 1200. For example, the input unit 1210 includes a keyboard, a mouse, and / or a touch screen. Various commands are input to the drug effect determination apparatus 1200 via the input unit 1210, and processing according to the commands is executed in the processing unit 1240.

  The imaging unit 1215 captures a “face image” including the face of the subject 300. For example, the imaging unit 1215 includes a solid-state imaging device such as a CCD and a CMOS that acquires RGB images, an infrared camera that acquires a thermogram, and the like. It is desirable that the infrared camera or the like can detect 29.0 ° C. to 37.0 ° C. with high sensitivity under normal room temperature conditions. Further, the imaging unit 1215 can perform continuous imaging at a predetermined interval. When capturing a facial image, it is desirable to perform the illumination under constant lighting conditions from the front. When the front image cannot be obtained due to the posture variation, the perturbation space method is used, and the three-dimensional shape of the face is estimated for the posture variation image, and the face image is obtained by rendering the front image. As for the illumination variation image, a facial image under a constant illumination condition is obtained by using a facial illumination base model constructed based on the diffuse reflection model. Then, the face image continuously captured by the imaging unit 1215 is sent to the processing unit 1240.

  The acquisition unit 1216 acquires administration information indicating that the medicine 305 has been administered to the subject 300. Specifically, the “administration information” may be any one of voices generated by the subject 300 or the like, key input by the subject 300 or the like, switch input via a button pressed by the subject 300 or the like, or any arbitrary Generated from the combination. Note that the target person 300 or the like is used to include not only the target person 300 but also an arbitrary person existing around the target person 300, for example, a meaning including a doctor, a nurse, or an assistant. The administration information acquired by the acquisition unit 1216 is sent to the administration information detection unit 1241 described later.

  The output unit 1220 outputs various types of information from the drug effect determination apparatus 1200. For example, the output unit 1220 includes a display and a speaker. Here, brain function activation information, which will be described later, is provided to the target person 300 via the output unit 1220.

  The storage unit 1230 stores information input to the drug effect determination apparatus 1200, information calculated by the drug effect determination apparatus 1200, and the like. For example, the storage unit 1230 includes a memory and a hard disk device. The storage unit 1230 stores a program for realizing each function of the processing unit 1240 described later. Here, the storage unit 1230 includes a brain function activation information database 1231 and a determination information database 1232.

  The brain function activation information database 1231 stores “brain function activation information” for activating human brain functions. Here, as the brain function activation information, “administration information” indicating that the medicine 305 has been administered can be mentioned. The brain function activation information database 1231 stores administration information in association with the type of the medicine 305.

  As shown in FIG. 29, the determination information database 1232 stores in advance the correlation value q2 of the determination component extracted when the administration information of the drug 305 is detected after a predetermined time has elapsed since the drug 305 was administered. The change amount Δq (= q2−q1) within a predetermined range from the stored “reference value” q1 is stored in advance as “determination information” in association with the “drug effect level”. The change amount Δq is expressed as an absolute value. Here, the “predetermined time” means a time when the effect of the medicine 305 is assumed to occur. The “reference value” is set based on, for example, a numerical rating scale (NRS) that is a subjective evaluation of pain when the drug 305 is an analgesic. Specifically, the determination information database 1232 stores “less effective” for less than Δq = Δqa, “small effect” for Δqa to Δqb, and “no effect” for Δqb or more, depending on the range of the amount of change Δq. To do. Here, the values increase in the order of Δqa and Δqb.

  Further, the determination information database 1232 stores information on the dosage of a drug that needs to be additionally administered in association with the drug effect level. For example, in the determination information database 1232, when the drug effect level is “high effect”, “small effect”, and “no effect”, the dose of the drug that needs to be additionally administered does not need to be additionally administered ( “Dosing not required)”, “half the previous amount (small dose)”, “same amount as previous (large dose)”, etc.

  Supplementally, when the subject 300 feels severe pain, an analgesic is administered as the drug 305, and when the effect of the analgesic is effectively produced, the analgesic is administered after a predetermined time has elapsed. On the other hand, the correlation value q of the component having a correlation decreases. This means that the value of Δq becomes smaller. In this case, since the effect of the analgesic has occurred effectively, it is not necessary to administer an additional analgesic. On the other hand, if the effect of the analgesic is not effectively produced, the correlation value q of the component having a correlation with the administration of the analgesic does not decrease even after a predetermined time has elapsed. This means that the value of Δq remains large. In this case, since the effect of the analgesic has not been effectively produced, it becomes necessary to administer the analgesic again. In short, there is a certain relationship between the drug effect level and the dose of the drug to be administered additionally, and information indicating such a relationship is stored in the determination information database 1232.

  The processing unit 1240 executes information processing in the drug effect determination apparatus 1200. Specifically, the processing unit 1240 includes a CPU and a cache memory. The processing unit 1240 executes a program incorporated in the storage unit 1230, whereby a dosing information detection unit 1241, a face change information acquisition unit 1242, a face change information decomposition unit 1243, a determination component extraction unit 1244, and a medicine It functions as the effect determination unit 1245.

  The administration information detection unit 1241 detects administration information sent from the acquisition unit 1216 as brain function activation information. When administration information is detected, it is considered that the brain of the subject 300 is in an activated state. Note that the administration information detection unit 1241 collates with information stored in the brain function activation information database 1231 when detecting administration information.

  The face change information acquisition unit 1242 acquires “face data” and “face change information” indicating time-series changes of face data from the face image captured by the image capturing unit 1215. Specifically, the facial change information acquisition unit 1242 acquires facial data via the imaging unit 1215 in synchronization with the timing at which the administration information detection unit 1241 detects administration information (brain function activation information). Then, the face change information acquisition unit 1242 acquires face change information indicating time-series changes in the face data of the target person 300 from the continuously acquired face data. For example, the face change information is a set of 4,608,000 data when 60 points of 240 × 320 pixel face data are acquired at predetermined intervals. The acquired face change information is sent to the face change information decomposing unit 1243. When the imaging unit 1215 is an infrared camera, the face change information acquisition unit 1242 acquires face skin temperature data indicating the skin temperature of the face of the subject 300 as the face data. When the imaging unit 1215 is a solid-state imaging device such as a CCD or CMOS, the face change information acquisition unit 1242 acquires facial blood flow data based on RGB data of the face of the subject 300 as facial data. Note that the face change information acquisition unit 1242 may acquire only data on the periphery of the sinus and / or the forehead part of the subject 300 as the face data.

  The face change information decomposition unit 1243 decomposes face change information, which is a collection of a large number of data, into a plurality of components 1, 2, 3,... By singular value decomposition, principal component analysis, or independent component analysis. The information of each decomposed component is sent to the determination component extraction unit 1244. Here, when face change information is subjected to singular value decomposition, components 1, 2, 3,... Are set in descending order of singular values. In addition, the higher the singular value, the more easily the influence of the fluctuation is reflected. For this reason, the effect of noise or the like of the external environment is often reflected in the component 1 rather than the effect of providing brain function activation information.

  The determination component extraction unit 1244 extracts a component related to administration information (brain function activation information) as a “determination component” from the plurality of components 1, 2, 3. Further, the determination component extraction unit 1244 calculates a correlation value q with respect to the brain function activation information of the extracted determination component. Specifically, the determination component extraction unit 1244 correlates a plurality of components 1, 2, 3,... Obtained by the face change information decomposition unit 1243 and the “determination waveform” corresponding to the administration information. q is calculated. Next, when the calculated correlation value q is greater than or equal to a predetermined value, the determination component extraction unit 1244 sets the component corresponding to the correlation value q as related to the administration information. Then, the determination component extraction unit 1244 extracts the determination component based on the risk factor value. That is, the determination component extraction unit 1244 extracts a component having a low risk rate as a determination component. The extracted determination component and the calculated correlation value q are sent to the storage unit 1230 or the drug effect determination unit 1245. As the above-described “determination waveform”, a deformed wave considering a human physiological reaction is used. The determination waveform is displaced after a predetermined time has elapsed since the administration information was detected. Specifically, a rectangular wave can be adopted as the determination waveform. Also, a redspot-dynamic response function can be adopted as the determination waveform.

  The drug effect determination unit 1245 determines the effect of the drug 305 according to the time-series change of the determination component. Specifically, the drug effect determination unit 1245 includes a correlation value q2 for the determination component extracted when administration information is detected after a predetermined time has elapsed since the drug 305 was administered, and a preset reference value A difference Δq from q1 is calculated. Then, the drug effect determination unit 1250 determines the drug effect level corresponding to the difference Δq between the reference value q1 and the correlation value q2 based on the determination information stored in the determination information database 1232. The determined drug effect level is output to a display device or the like via the output unit 1220.

  The dose calculation unit 1246 calculates the dose of the drug 305 to be administered to the subject 300 according to the drug effect level. Specifically, the dose calculation unit 1246 calculates the dose of the medicine 305 that needs to be additionally administered based on the information stored in the determination information database 1232.

(8-2) Operation of Drug Effect Determination Device FIG. 30 is a flowchart showing the operation of the drug effect determination device 1200.

  First, a test start instruction is input to the drug effect determination apparatus 1200 via the input unit 1210 or the like. Subsequently, a guidance screen is displayed on the output unit 1220 and guidance is performed so that the face of the target person 300 is located at the center of the guidance screen. Then, the imaging unit 1215 starts capturing the facial image of the target person 300 (V1). The captured face image is sent to the face change information acquisition unit 1242.

  The medicine 305 is administered to the operator 301 after the start of facial image capturing. At this time, an operation such as button pressing by the target person 300 or the like is performed. In response to this, the drug effect determination apparatus 1200 detects administration information indicating that the drug 305 has been administered (V2).

  Further, in the drug effect determination apparatus 1200, an analysis of the captured facial image is executed. Specifically, the face change information acquisition unit 1242 acquires face change information indicating a time-series change in the face data of the target person 300 from the captured face image. Then, the face change information decomposition unit 1243 decomposes the face change information into a plurality of components 1, 2, 3,... By performing singular value decomposition, principal component analysis, or independent component analysis. Further, the determination component extraction unit 1244 calculates a correlation value between the plurality of components 1, 2, 3,... Decomposed by the face change information decomposition unit 1243 and the determination waveform corresponding to the administration information. Then, the determination component extraction unit 1244 determines whether or not the correlation value is greater than or equal to a predetermined value (V4). When it is determined that the value is equal to or greater than the predetermined value, it is determined that there is a “correlation” between the administration information and the component (V4-Yes). Then, the determination component extraction unit 1244 extracts a component having a low risk ratio from the correlated components as a “determination component” (V5). Information on these extracted determination components is stored in the storage unit 1230 (V6). On the other hand, if the correlation value between the determination waveform corresponding to the administration information and each component 1, 2, 3,... Is less than a predetermined value, it is determined that there is no correlation between the two and the information is It is stored in the storage unit 1230 (V4-No, V6). In addition, the operation | movement of the said step V4-V6 can be abbreviate | omitted by specifying in advance the "correlated component" in the administration information from the past measurement values.

  Next, a correlation value q2 for the determination component extracted when administration information is detected after a predetermined time has elapsed since the drug 305 was administered by the drug effect determination unit 1245, and a preset reference value q1 A change amount Δq, which is a difference from the above, is calculated (V7).

  Subsequently, the drug effect determination unit 1245 determines whether or not the change amount Δq of the correlation value q2 with respect to the reference value q1 is within a predetermined range (V8). Whether or not it is within the predetermined range is determined by collating with the determination information stored in the determination information database 1232. When the change amount Δq of the correlation value q2 with respect to the reference value q1 is within a predetermined range (Δq is smaller than Δqa), the drug effect determination unit 1245 determines that the drug is “high effect” (V8-Yes, V9). In this case, it is determined that the additional administration of the medicine 305 is unnecessary for the subject 300. On the other hand, when the change amount Δq of the correlation value q2 with respect to the reference value q1 is not within the predetermined range (when Δq is equal to or greater than Δqa), the drug effect determination unit 1245 determines “no effect” or the like (V8-No, V10). . In this case, it is determined that additional administration of the medicine 305 is necessary for the subject 300. Then, based on the information stored in the determination information database 1232, the dose calculation unit 1246 calculates the dose of the medicine 305 that needs to be additionally administered (V11).

  Thereafter, the drug effect determination apparatus 1200 stores data in accordance with an input instruction from the apparatus user. Specifically, the drug effect determination device 1200 stores determination result data, analysis waveforms, measurement results, image display conditions, and the like in the storage unit 1230 for each subject.

(8-3) Features of the drug effect determination device (8-3-1)
As described above, the drug effect determination apparatus 1200 according to the present embodiment includes the face change information acquisition unit 1242, the face change information decomposition unit 1243, the determination component extraction unit 1244, and the drug effect determination unit 1245. Prepare. The face change information acquisition unit 1242 acquires “face change information” indicating a time-series change of face data of the subject 300 to which the medicine 305 has been administered. The face change information decomposition unit 1243 decomposes the face change information into a plurality of components 1, 2, 3,... By singular value decomposition, principal component analysis, or independent component analysis. The determination component extraction unit 1244 extracts a component related to the administration of the medicine 305 as a “determination component” from the plurality of components 1, 2, 3,. The drug effect determination unit 1245 determines the effect of the drug 305 on the target person 300 based on the determination component.

  Therefore, in the drug effect determination apparatus 1200 according to the present embodiment, the face change information is determined from a plurality of components 1, 2, 3,... Obtained by performing singular value decomposition, principal component analysis, and independent component analysis. Since the components for use are extracted, the presence or absence of the brain activity of the subject 300 can be easily estimated without using electrodes or the like that require pretreatment before wearing. Thereby, based on the component for determination corresponding to the brain function of the subject person 300, the effect of the medicine 305 on the subject person 300 can be easily judged.

(8-3-2)
Further, in the drug effect determination apparatus 1200 according to the present embodiment, the facial change information acquisition unit 1242 acquires data on the periphery of the sinuses and / or the forehead part of the subject 300 as the facial data. Related determination components can be extracted with high accuracy. Here, the brain has a mechanism called a selective brain cooling system that cools the brain independently of the body temperature. The selective brain cooling mechanism exhausts heat generated by brain activity using the sinuses and the forehead area. Therefore, by analyzing the data of these parts, components related to brain activity can be extracted with high accuracy. As a result, the drug effect determination apparatus 1200 according to the present embodiment can execute a drug effect determination with high accuracy.

(8-3-3)
Further, in the drug effect determination apparatus 1200 according to the present embodiment, the face change information acquisition unit 1242 acquires face skin temperature data indicating the skin temperature of the face of the subject 300 as the face data. In other words, the drug effect determination apparatus 1200 can determine the drug effect using an infrared camera or the like.

(8-3-4)
In the drug effect determination apparatus 1200 according to the present embodiment, the facial change information acquisition unit 1242 acquires facial blood flow data based on RGB data of the face of the subject 300 as facial data. That is, the drug effect determination apparatus 1200 can determine the drug effect using a solid-state imaging device (CCD, CMOS). Thereby, the determination of a medicine effect can be performed with a simple configuration.

(8-3-5)
Further, in the drug effect determination apparatus 1200 according to the present embodiment, the determination component extraction unit 1244 extracts a determination component based on the risk factor value. Since the drug effect determination apparatus 1200 extracts the determination component related to the administration information based on the risk factor value, the reliability of the drug effect determination can be improved.

(8-3-6)
In the drug effect determination apparatus 1200 according to the present embodiment, the drug effect determination unit 1245 determines the effect of the drug 305 according to the time-series change of the determination component. Thereby, the drug effect level can be easily determined according to the amount of change from the reference value acquired in advance.

(8-3-7)
The drug effect determination apparatus 1200 according to the present embodiment further includes a determination information database (determination information storage unit) 1032. The determination information database 1032 shows the amount of change Δq with respect to the reference value q1 of the correlation value q2 of the determination component calculated for the administration of the drug 305 after a predetermined time has elapsed since the drug 305 was administered. It is stored as “determination information” in association with the drug effect level. In addition, the drug effect determination unit 1245 calculates the correlation value q2 of the determination component for the administration of the drug 305, and determines the drug effect level for the target person 300 based on the calculated correlation value and determination information. With such a configuration, the drug effect level can be easily determined using the reference value q1 acquired in advance. In short, the drug effect determination apparatus 1200 can determine and output the drug effect level as well as the presence or absence of the drug effect.

(8-3-8)
The drug effect determination method according to the present embodiment does not necessarily require the drug effect determination device 1200. That is, the determination method of the drug effect according to the present embodiment is that the drug 305 is administered to the subject 300 as brain function activation information that activates the human brain function regardless of the presence or absence of the drug effect determination device 1200. A brain function activation information detection step for detecting the administration information to be shown; a face change information acquisition step for acquiring “face change information” indicating a time-series change in the face data of the subject after detecting the administration information; and a face change information. Is a facial change information decomposition step that decomposes into multiple components by performing singular value decomposition, principal component analysis, or independent component analysis, and components for determining administration information (brain function activation information) from multiple components A determination component extraction step that is extracted as follows, and a drug effect determination step that determines a drug effect on the subject based on the determination component. There.

  According to such a method for determining a drug effect, after detection of administration information, administration information (brain function) is obtained from a plurality of components obtained by performing singular value decomposition, principal component analysis, or independent component analysis on facial change information. Since the determination component related to the activation information) is extracted and the drug effect is determined, the drug effect of the subject 300 can be easily determined.

(8-3-9)
The drug effect determination apparatus 1200 according to the present embodiment further includes a dose calculation unit 1246 that calculates the dose of the drug 305 to be administered to the subject 300 according to the drug effect level. With such a configuration, an appropriate drug 305 can be administered to the subject 300.

(8-3-10)
Further, as described above, the drug effect determination apparatus 1200 not only determines the drug effect using the amount of change in a predetermined range from the reference value of the correlation value with respect to the determination component, but also overlaps with the determination component. The drug effect may be determined based on any one or any combination of a value obtained by regression analysis, an area generated by the determination waveform, an average value of the determination waveform, and a centroid value of the determination waveform.

(8-3-11)
Note that in the drug effect determination apparatus 1200 according to the present embodiment, the determination component extraction unit 1244 extracts a determination component based on the correlation value between the determination waveform corresponding to the administration information and a plurality of components. With such a configuration, the determination component corresponding to the activation state of the brain function of the subject 300 can be specified.

  In addition, here, a deformed wave taking into account a human physiological reaction can be employed as the determination waveform. The determination waveform is displaced after a predetermined time has elapsed since the administration information was detected. Such a determination waveform exhibits a high correlation value when a significant correlation is recognized with the component obtained from the face change information, so that the determination component extraction accuracy can be increased. In addition, a highly accurate correlation can be obtained by slightly delaying the phase with respect to the reaction of the brain. Specifically, a red spot dynamic response function can be employed as the determination waveform. The red spot dynamic response function has a correlation among a plurality of components 1, 2, 3,... Decomposed by the face change information decomposition unit 1243 corresponding to the administration information (brain function activation information). The component is calculated a plurality of times and is generated based on the calculated plurality of components. Since the red spot dynamic response function is optimized based on the past history, it shows a high correlation value when a significant correlation is recognized with the component obtained from the face change information. Thereby, the extraction accuracy of the determination component can be increased.

  A rectangular wave can also be adopted as the determination waveform. Since the rectangular wave can easily correspond to the timing of detecting the administration information, the determination component can be easily extracted.

(8-4) Modification Example of Drug Effect Determination Device (8-4-1)
As shown in FIG. 31, the drug effect determination apparatus 1200 according to the present embodiment may use a determination information providing apparatus 1300 provided on a network.

  Here, the determination information providing apparatus 1300 includes a storage unit 1330 and a processing unit 1340.

  The storage unit 1330 includes a determination information database 1332. The determination information database 1332 has the same configuration as the determination information database 1232 described above. That is, the determination information database 1332 indicates the amount of change from the reference value q1 of the correlation value q2 of the determination component calculated for the administration of the drug 305 after a predetermined time has elapsed since the drug 305 was administered. Δq is stored as determination information in association with the drug effect level.

  The processing unit 1340 transmits the determination information stored in the determination information database 1332 in response to a request from the drug effect determination apparatus 1200. Note that the processing unit 1340 may have a function of generating determination information as big data based on predetermined information independently of the determination component extracted by the drug effect determination apparatus 1200. In addition, when the reference value q1 is calculated by the drug effect determination device 1200, the processing unit 1340 executes a process for updating the reference value q1 stored in the determination information database 1232 as needed.

  In this modification, the drug effect determination unit 1245 requests the determination information providing apparatus 1300 described above to provide determination information. Specifically, in the drug effect determination device 1200 according to this modification, the determination information database 1332 is stored in the determination information providing device 1300 on the network, and the drug effect determination unit 1245 determines when the drug effect level is determined. The determination information providing apparatus 1300 is accessed. Then, the drug effect determination unit 1345 determines the drug effect level of the target person 300 based on the calculated correlation value q2 and the determination information.

  Therefore, in the case of the drug effect determination apparatus 1200 according to the present modification, the drug effect determination unit 1245 can determine the drug effect level of the target person 300 using the determination information providing apparatus 1300 on the network.

  Furthermore, according to the drug effect determination method of the present modification, determination of drug effect using big data can be realized. That is, the reference value q1 and the predetermined change amount Δq can be obtained from big data. Thereby, determination information can be optimized at any time.

  Further, the determination information providing apparatus 1300 according to the present modification further stores the dose of the drug 305 as determination information in association with the drug effect level. Further, the drug effect determination device 1200 accesses the determination information providing device 1300 and acquires the determination information. Then, the drug effect determination unit 1245 of the drug effect determination apparatus 1200 determines the drug effect level and the dose of the drug based on the determination information. Since such a configuration is provided, the drug effect determination apparatus 1200 according to the present modification can administer an appropriate drug 305 to the subject 300. In addition, by using an external device, the configuration of the drug effect determination device 1200 can be simplified.

(8-4-2)
In addition, in order to confirm the effect of the drug effect determination apparatus 1200, the pain determination apparatus 1000 described above may be combined. For example, the presence or absence of the drug effect may be confirmed by administering the drug 305 to the subject 300 feeling pain and determining the pain state of the subject 300 using the pain determination apparatus 1000.

(8-5) Verification Test of Drug Effect Determination Device A verification test of the drug effect determination device according to the present embodiment was performed as follows. In general, an opioid analgesic is sometimes administered orally as an effective treatment for cancer pain. Therefore, in this study, whether or not pain is alleviated with clear consciousness when an opioid analgesic is additionally administered to a subject (subject 300) who has already been treated with opioid analgesic during pain augmentation. Verified. For comparison, the pain relieving effect of opioid analgesics was determined from a numerical rating scale (NRS), which is a subjective evaluation of pain.

  First, in this test, facial images were obtained from subjects. Specifically, the subject was seated on a chair installed in the laboratory, and the subject was put on headphones. Then, an HD webcam c270 made by Logitech was fixed to the arm of this headphone, and this photographing apparatus was installed at a point 0.2 m away on the front side of the subject. And using this imaging device, photographic image data around the nose of the entire face of the subject was acquired in time series. Specifically, the moving image data of the face was obtained by continuously shooting the shooting image data of 1260 seconds along the time axis with a shooting period of 30 frames / second by the shooting device. Image data was stored as color moving image data in MacBook Air (registered trademark) manufactured by Apple Inc.

  And in this test, the opioid analgesic which is a chemical | medical agent was administered to the test subject, while acquiring the face image. The administration of an opioid analgesic was detected by pressing a button by an assistant present around the subject.

  For analysis of facial images, blood flow volume data is calculated based on RGB data obtained from the captured facial moving image data, and singular value decomposition is performed using MATLAB® SVD (Singular Value Decomposition) as an analysis tool. went. Here, according to the CIE-L * a * b * color system, an erythema index “a *” correlated with the redness and hemoglobin amount calculated from the RGB data of the image is obtained, and this is used as blood flow data. . In the singular value decomposition, blood flow volume data (here, erythema index) based on RGB data obtained from all moving image data (data for 1260 seconds) acquired in time series is targeted, and the factor is determined every second. Erythema index calculated from RGB data in the period (every 1 second) with time data (1260 time points in 1260 seconds), calculated from the average value of RGB values obtained from the frame data Erythema index, adjusted according to the video data). Then, by singular value decomposition, time-series blood flow data based on RGB data obtained from facial moving image data is decomposed into a plurality of components, and each component's time distribution V, spatial distribution U, and each component And a singular value S indicating the magnitude of. These relationships are expressed by the following equations. V ′ is a matrix in which the rows and columns of V are exchanged.

  Then, the time distribution V and the spatial distribution U of each component obtained by singular value decomposition were plotted on a graph, and a component waveform diagram and a blood flow distribution diagram of each component were created.

  Further, analysis was performed on the component waveform diagram and blood flow distribution diagram of each component created to identify a change in facial blood flow that reflects brain function activation activity, that is, a component indicating a change in facial RGB. About the component waveform figure of each component, it analyzed about the presence or absence of the correlation with the amplitude of the component waveform, and administration of an opioid type analgesic. Under the above conditions, a result as shown in FIG. 32 was obtained.

  In FIG. 32, the horizontal axis indicates the elapsed time, and the vertical axis indicates the time-series change of the determination component obtained from the facial image information. This determination component is a component that correlates with the administration of an opioid analgesic. From the graph of FIG. 32, a result indicating that the reaction of the determination component decreases with time.

  FIG. 33 is a diagram showing an estimated pain change obtained from a numerical evaluation scale which is a subjective evaluation of pain. In FIG. 33, the horizontal axis represents elapsed time, and the vertical axis represents a numerical evaluation scale of pain. From the graph of FIG. 33, the result which shows the waveform similar to FIG. 32 was obtained.

  From such a result, it has been recognized that the determination component corresponding to the administration of the opioid analgesic can be extracted. It was also recognized that the pain relieving effect of opioid analgesics can be objectively evaluated and determined.

<Appendix>
In addition, this invention is not limited to the said embodiment as it is. The present invention can be embodied by modifying the components without departing from the scope of the invention in the implementation stage. In addition, the present invention can form various inventions by appropriately combining a plurality of constituent elements disclosed in the above embodiments. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements may be appropriately combined in different embodiments.

  According to the present invention, since brain activity can be easily estimated, application to a brain activity visualization device that visualizes the physiological state of a subject based on brain activity is effective.

DESCRIPTION OF SYMBOLS 10 Brain activity visualization apparatus 20 Facial skin temperature acquisition means 30 Brain activity estimation means 110 Brain activity visualization apparatus 120 Image data acquisition means 130 Brain activity estimation means 200 State visualization means 201 Analysis part 300 Target person 305 Drug 1000 Pain determination apparatus 1010 Input part 1015 Imaging unit 1020 Output unit 1030 Storage unit 1031 Brain function activation information database 1032 Determination information database (determination information storage unit)
1040 Processing unit 1041 Brain function activation information providing unit 1042 Face change information acquiring unit 1043 Face change information decomposing unit 1044 Determination component extracting unit 1045 Pain determination unit 1100 Determination information providing device 1130 Storage unit 1132 Determination information database (determination information storage unit)
1140 processing unit 1200 drug effect determination device 1210 input unit 1215 imaging unit 1216 acquisition unit 1220 output unit 1230 storage unit 1231 brain function activation information database 1232 determination information database (determination information storage unit)
1240 Processing Unit 1241 Administration Information Detection Unit 1242 Face Change Information Acquisition Unit 1243 Face Change Information Decomposition Unit 1244 Determination Component Extraction Unit 1245 Drug Effect Determination Unit 1246 Dose Determination Unit 1300 Determination Information Providing Device 1330 Storage Unit 1332 Determination Information Database (determination Information storage unit)
1340 processing unit

JP 2013-176406 A

Claims (14)

A brain function activation information providing unit (1041) for providing brain function activation information for activating human brain function to the target person;
A face change information acquisition unit (1042) for acquiring face change information indicating a time-series change in the face data of the subject;
A face change information decomposition unit (1043) that decomposes the face change information into a plurality of components by singular value decomposition, principal component analysis, or independent component analysis;
A determination component extraction unit (1044) that extracts a component related to the brain function activation information from the plurality of components as a determination component;
A pain determination unit (1045) for determining the presence or absence of pain in the subject based on the extraction result by the determination component extraction unit;
A pain determination apparatus (1000) comprising: The determination component extraction unit extracts the determination component based on a correlation value between the determination waveform corresponding to the brain function activation information and the plurality of components.
The pain determination apparatus according to claim 1. The brain function activation information providing unit provides the target person with auditory stimulation information for stimulating hearing or visual stimulation information for stimulating vision as the brain function activation information.
The pain determination apparatus according to claim 1 or 2. The pain determination unit determines the pain state according to a change amount of the determination component;
The pain determination apparatus according to any one of claims 1 to 3. The amount of change in a predetermined range of the correlation value of the determination component calculated for the brain function activation information from the reference correlation value of the reference determination component calculated for the brain function activation information is set to the pain state level. A determination information storage unit (1032) that stores the determination information in association with each other;
The pain determination unit calculates a correlation value of the determination component with respect to the brain function activation information, and determines the pain state level of the subject based on the calculated correlation value and the determination information;
The pain determination apparatus according to any one of claims 1 to 4. From the reference correlation value of the reference determination component calculated for the brain function activation information, the determination value providing apparatus (1100) on the network calculates the correlation value of the determination component calculated for the brain function activation information. A determination information storage unit (1132) that stores the amount of change in the predetermined range as determination information in association with the pain state level;
The pain determination unit calculates a correlation value of the determination component with respect to the brain function activation information, and determines the pain state level of the subject based on the calculated correlation value and the determination information;
The pain determination apparatus according to any one of claims 1 to 5. The function activation information providing unit and the face change information acquiring unit are incorporated in a helmet-type device.
The pain determination apparatus according to any one of claims 1 to 6. A face change information acquisition unit (1242) that acquires face change information indicating a time-series change in face data of the subject to whom the medicine (305) has been administered;
A face change information decomposition unit (1243) that decomposes the face change information into a plurality of components by singular value decomposition, principal component analysis, or independent component analysis;
A determination component extraction unit (1244) that extracts, as a determination component, a component related to the administration of the medicine from the plurality of components;
A drug effect determination unit (1245) for determining the effect of the drug based on the extraction result by the determination component extraction unit;
A drug effect determination device (1200). The determination component extraction unit extracts the determination component based on a correlation value between the determination waveform corresponding to the administration of the medicine and the plurality of components.
The drug effect determination device according to claim 8. The drug effect determination unit determines the effect of the drug according to a time-series change of the determination component,
The drug effect determination device according to claim 8 or 9. The change amount of the correlation value of the determination component calculated with respect to the administration of the drug after a predetermined time has elapsed from the administration of the drug with respect to the reference value is stored as determination information in association with the drug effect level. A judgment information storage unit (1232);
The drug effect determination unit calculates a correlation value of the determination component with respect to the administration of the drug, and determines a drug effect level for the subject based on the calculated correlation value and the determination information.
The analgesic effect determination device according to any one of claims 8 to 10. A dose calculation unit (1246) for calculating a dose of a drug to be administered to the subject according to the drug effect level;
The drug effect determination device according to claim 11. The amount of change from the reference value of the correlation value of the determination component calculated for the administration of the drug after a predetermined time has elapsed since the determination information providing apparatus (1300) on the network has administered the drug And a determination information storage unit (1332) that stores the determination information in association with the drug effect level,
The drug effect determination unit calculates a correlation value of the determination component with respect to the administration of the drug, and determines a drug effect level for the subject based on the calculated correlation value and the determination information.
The drug effect determination device according to any one of claims 8 to 12. The determination information providing device further stores a dose of the drug in association with the drug effect level as the determination information,
The drug effect determination unit determines the drug effect level and the dose of the drug based on the determination information.
The drug effect determination device according to claim 13.