JP6233776B2 - Psychosomatic state determination apparatus and psychosomatic state determination program - Google Patents

Description

  The present invention relates to a psychosomatic state determination apparatus and a psychosomatic state determination program capable of physically and psychologically determining whether or not there is a tendency to depression.

Mental illness represented by depression has become a major social problem in recent years with the sophistication and sophistication of society, and the importance of promptly detecting these mental illnesses and promoting treatment has been screamed.
Conventionally, it is often done to determine whether or not there is a tendency to depression on the questionnaire, but going to the hospital itself is a burden, and there are cases where accurate information can not be written on the questionnaire The limits of certain isoforms have been reported.
Thus, an apparatus and method for physically determining whether or not there is a tendency to depression using physical information has been proposed. For example, Patent Document 1 measures a heart rate and determines a biophysical parameter of a subject. There has been proposed a method of measuring a pattern of the above and further diagnosing a mental disorder using the pattern.
Patent Document 2 discloses a method for determining depression by a biomarker blood test for diagnosing depression by discovering that the concentration of ethanolamine phosphate is low in depressed patients by performing metabolomic analysis of blood. Proposed.

JP-T-2001-518823 WO / 2011/019072

However, in the method described in Patent Document 1, it is indispensable to obtain an accurate pattern, and it takes 90 minutes to measure the heart rate alone, and requires measurement in a state without external factors. It was something that forced the burden of.
Further, the method of Patent Document 2 is effective in determining whether or not the patient is seriously depressed, but is not suitable for the purpose of simply knowing the tendency of depression because blood must be collected. It is a thing.
In short, none of the conventionally proposed methods for determining depression tendency can be measured easily, and there is a demand for the development of an apparatus and method that can determine whether there is a tendency to depression more easily and simply. The current situation is.
Accordingly, an object of the present invention is to provide a psychosomatic state determination apparatus capable of physically determining a psychosomatic state such as a mental state whether or not there is a tendency to depression easily and simply.

As a result of intensive studies to solve the above-mentioned problems, the present inventors applied a fact that blood flow can be measured by image processing of a part of the body, and pulse waves can be measured by calculating such blood flow. It has been found that the tendency of depression can be determined by measuring, and the present invention has been completed by making ingenuity so that it can be applied to a portable terminal based on such knowledge.
That is, the present invention provides the following inventions.
1. A light emitting part for irradiating the subject's body with light;
An image acquisition unit that continuously acquires an image of a portion irradiated with light by the light emitting unit;
A computer unit for calculating the pulse wave by computing the acquired image and calculating the state of the pulse wave over time;
An audio output unit for listening to environmental music to the subject;
A video output unit for showing the environmental video to the subject;
A determination program for determining a psychosomatic state by operating a psychosomatic state determination device comprising:
Operation input means for displaying an instruction on the video output unit and receiving an operation by a subject;
When the subject responds to the operation input means, an instruction means for displaying and instructing the action to be obtained from the subject,
Image acquisition means for performing light irradiation and image acquisition for a predetermined time according to the action when the subject behaves according to the instruction means;
The image acquired by the image acquisition means is arithmetically processed and converted into a pulse wave, continuously displayed with the measurement time on the video output unit, and the data obtained by the arithmetic processing is integrated to obtain an LF (Low Frequency: 0). .04-0.15 Hz) and HF (High Frequency: 0.15-0.4 Hz)
Video music means for displaying environmental video on the video output unit and flowing environmental music to the audio output unit for a predetermined time after the measurement is completed;
After the display of the video and the music performance, the recalculation means for measuring the pulse wave again by operating the instruction means, the image acquisition means, the calculation display means,
And a psychosomatic state determination program comprising determination means for comparing the LF and HF data obtained by the computing means with the LF and HF data obtained by the recalculating means to determine the psychosomatic state.
2. An apparatus for executing the mental determination program according to claim 1,
A light emitting part for irradiating the subject's body with light;
An image acquisition unit that continuously acquires an image of a portion irradiated with light by the light emitting unit;
A computer unit for calculating the pulse wave by computing the acquired image and calculating the state of the pulse wave over time;
An audio output unit for listening to environmental music to the subject;
A video output unit for showing the environmental video to the subject;
Comprising
The computer part is
A memory unit that temporarily stores the acquired image data and temporarily stores output data after the arithmetic processing, and an arithmetic processing unit that performs arithmetic processing;
A psychosomatic state determination apparatus comprising a drive unit for storing a program for operating a computer.

The psychosomatic state determination apparatus of the present invention can easily and simply determine a mental state such as whether or not there is a tendency to depression and a physical and mental state such as a physical state such as diabetes.
In addition, the psychosomatic state determination program of the present invention can be applied to an existing portable terminal and can easily determine the mental and physical states such as the mental state and the physical state.

FIG. 1A is a perspective view showing a state when one embodiment of a mind-body state determination device of the present invention is seen from one side, and FIG. 1B is a perspective view showing a state seen from multiple sides. FIG. 2 is an explanatory diagram showing an outline of the apparatus shown in FIG. FIG. 3A is a photograph (drawing substitute photograph) showing a screen when an application configured by the program of the present invention is started, and FIG. 3B is a photograph (drawing substitute photograph) showing a screen at the time of measurement. Yes, (c) is a photograph (drawing substitute photograph) showing a state where an environmental video is displayed, (d) is a photograph (drawing substitute photograph) showing a state where re-measurement is performed, ( e) is a photograph (drawing substitute photograph) showing the determination result. FIG. 4 is a chart showing a change rate graph created by substituting the pixel sum of white pixels at an arbitrary time t as SW (t) into a predetermined equation. FIG. 5 is a graph in which HF and LF are calculated from the pulse wave, and a graph comparing how the healthy person and the person with depression tend to change before and after the video music means show the video and listen to the music. It is.

DESCRIPTION OF SYMBOLS 1 Mind and body state determination apparatus: 10 light sources: 20 Cameras: 30 Computer part: 40 Display: 50 Earphone: 31 Memory part: 32 CPU: 33 Storage medium

Embodiments of the present invention will be described below with reference to the accompanying drawings.
<Psychosomatic state determination device>
First, a mental state determination device as a mental and physical state determination device of the present invention will be described with reference to FIGS.
The mental state determination apparatus 1 shown in FIGS.
A light source 10 as a light emitting part for irradiating the subject's body with light;
A camera 20 as an image acquisition unit that continuously acquires images of a portion irradiated with light by the light source 10;
A computer unit 30 that calculates the pulse wave by computing the acquired image and calculates the state of the pulse wave over time;
An earphone 50 as an audio output unit for letting the subject hear environmental music;
And a display 40 as a video output unit for showing an environmental video to the subject.
The computer unit 30 also temporarily stores the acquired image data and temporarily stores output data after the arithmetic processing, and a central processing unit (CPU) as an arithmetic processing unit that performs arithmetic processing. 32 and a storage medium (in this embodiment, a solid state drive: SSD) 33 that stores a program for operating the computer.

The mental state determination device 1 of the present embodiment configured as described above is configured in the same manner as a normal mobile phone or mobile terminal as is apparent from FIG. 1, but each member will be described below.
The light source 10 is configured by an LED light in this embodiment, and is connected to a computer unit via a wiring (not shown) as shown in FIG. It can be done.
The camera 20 is a CCD camera that is usually used for this type of portable terminal, and is juxtaposed with the light source 10. It is connected to a CPU via a memory or directly so that execution and stop of imaging can be appropriately performed. The acquired data is transferred to the CPU via a memory or directly so that data processing can be performed.
The computer unit 30 is arranged inside the apparatus shown in FIG. 1, and includes a CPU 32 coupled to a memory 31 and a storage medium 33. The CPU 32 is wired (not shown) to each part via the memory or directly. It is connected through.
The earphone 50 is connected to an earphone jack (not shown), and in addition to this, a speaker is also provided in this apparatus (not shown). However, when the mental state is determined using this apparatus, it is preferable to use an earphone instead of a speaker.
The display 40 is installed on the surface opposite to the light source 10 and the camera 20, and is configured by a known touch-type liquid crystal panel in this embodiment, and can be operated by directly touching the liquid crystal panel. Yes.

<Mental condition determination program>
In the apparatus 1 of the present embodiment, a mental state determination program as a psychosomatic state determination program is accommodated in the storage medium 33, and the mental state is determined by the mental state determination program.
Determining program of the present embodiment, Ru der which determines the mental condition by operating the computer unit in psychiatric state determining apparatus. The computer unit 30 in which the determination program is stored is an operation input means for displaying an instruction on the display 40 and receiving an operation by the subject,
When the subject responds to the operation input means, an instruction means for displaying and instructing the action to be obtained from the subject,
Image acquisition means for performing light irradiation and image acquisition for a predetermined time according to the action when the subject behaves according to the instruction means;
The image acquired by the image acquisition means is arithmetically processed and converted into a pulse wave, continuously displayed on the display 40 together with the measurement time, and the data obtained by the arithmetic processing is integrated to obtain an LF (Low Frequency: 0.04). -0.15 Hz) and HF (High Frequency: 0.15-0.4 Hz)
Video music means for displaying environmental video on the display 40 and flowing environmental music to the earphone 50 for a predetermined time after the measurement is completed;
After the display of the video and the music performance, the recalculation means for measuring the pulse wave again by operating the instruction means, the image acquisition means, the calculation display means,
And determining means for comparing the LF and HF data obtained by the computing means with the LF and HF data obtained by the recalculating means to determine the mental state . Determination program, these means, is executed by issuing the command to the computer unit 30.
Hereinafter, each means will be described in detail.

(Operation input means)
First, although not shown, the program is configured such that an application button is displayed on the display 40 and the application is activated when the application button is touched.
(Instruction means)
When the subject (user) touches the application button, the application is activated and an initial screen is displayed. Here, as shown in FIG. 3A, the subject places a finger as a part of the body at a predetermined position. In this manner, an explanatory text for instructing and an operation instruction photo are displayed. Also, the display screen shown in FIG. 3A is configured to display a pulse wave waveform, a graph indicating the passage of measurement time, and a pulse rate.

(Image acquisition means)
In accordance with this instruction, the subject places his finger, irradiates the finger with light from the light source, takes an image of the irradiated part with a camera, and performs a binarization process on the obtained image to determine whether or not measurement is possible. Configured to check.
This confirmation is performed as follows.
First, binarization processing is performed on an image acquired from a camera based on an arbitrary standard, specifically, Expression (3) described later.
The black pixel sum and the white pixel sum obtained as a result of the binarization processing are obtained by the formulas (1) and (2), and until these ratios are equal, specifically, the following formula (3) is satisfied, 2 Continue to automatically adjust the threshold value. Specifically, when the value is outside the range of the expression (3), the binarization value is adjusted by increasing or decreasing by one.
Expressions (1) and (2) are expressions that perform the following processing.
When the horizontal pixel number of the camera image is h and the vertical pixel number is w, the resolution is h_w, and the image pixel is Pi; (where i indicates the pixel position). The total value SW and the total black pixel value SB can be obtained.
Next, when the ratio of SW; SB satisfies the following equation (3), automatic threshold adjustment is stopped and it is determined that an appropriate threshold has been found.
As a result, image adjustment necessary for pulse wave acquisition can be automatically performed.
For the appropriate threshold search, the method by Otsu et al. (Otsu: Automatic threshold selection method based on discrimination and least squares criterion, IEICE Transactions D, Vol.63, No.4, pp.349-356 ( 1989).) Enables faster threshold search. However, even in this case, if the threshold value is constantly adjusted, the pulse wave cannot be measured. Therefore, when an appropriate value is found within a certain range, specifically, within the range of the following formula (3), the threshold value is not reset. Is required. However, if an appropriate threshold value is not actually found within the range of equation (3), the threshold value is first determined with a temporary setting equation of 0.46 ≤ Sw / (Sw + Sb) ≤ 0.56, and the pulse wave with this threshold value During the pulse detection, the threshold value is not reset as long as the condition of the expression (3) is satisfied. In this case, once the threshold value is set using the provisional setting formula, the adjustment is performed again so that it falls within the range of the formula (3). If the pulse cannot be detected again, the threshold value is set again using the temporary setting formula, Perform wave measurement again.

(Calculation display means)
If the above equation (3) is satisfied, it is determined that the binarization process is possible and the pulse wave can be measured, the screen of FIG. 3B is displayed, and the pulse wave measurement is started. In the during-measurement screen shown in FIG. 3B, a graph displaying the pulse wave being measured, a pie chart indicating the elapsed time, and a pulse are displayed.
The pulse wave measurement algorithm is as follows.
The binarization process is performed as described above, and attention is paid to the SW value. SW is a pixel sum of white pixels, and the value is updated for each acquisition frame rate of the camera.
Therefore, the pixel sum of white pixels at an arbitrary time t is SW (t), and a change rate graph is created by the following equation (4). A graph acquired in an actual prototype application is shown in FIG. However, this graph is not shown to the subject and is processed in the computer section.

As shown in the lower chart of FIG. 4, the pulse wave position can be estimated. Therefore, next, this pulse wave position is specified. Various methods are conceivable for this, but it is known from experiments that a sufficiently accurate pulse wave position can be specified even with a simple threshold setting. Therefore, in the program of this embodiment, the pulse wave position is simply specified by the threshold setting. . Note that the threshold value in the present embodiment is specifically 0.121 [px / MS].
Next, in order to obtain HRV, first, a spectral function of frequency is obtained. Heart rate variability index (Heart Rate), which is an index of autonomic nervous activity evaluation usually obtained from electrocardiogram
Variability (HRV) is an index representing the activity of the autonomic nerve, and is an index calculated by dividing the fluctuation of the heart rate RR interval into a low frequency region and a high frequency region. LF (Low Frequency: 0.04-0.15 Hz) affected by both sympathetic and parasympathetic nerves and HF (High Frequency: 0.15-0.4 Hz) affected by parasympathetic nerves are widely used. . The ratio between them, LF / HF, is an index of sympathetic nerve activity, but is also used as a stress evaluation technique.
In this case, first, a graph is created by taking each pulse wave measurement time on the horizontal axis and the time (beat interval) between each pulse wave on the vertical axis. The measurement time required for one measurement is 64 seconds (this is due to the FFT calculation limit), and the graph is resampled every 0.25 seconds, and then frequency analysis is performed. Note that linear interpolation is used for the resampling process, and the frequency analysis is performed by performing a Hanning window function process and then calculating by FFT. The spectrum function thus obtained is defined as F (ω) (where “ω” is a frequency).
As a method for obtaining the frequency spectrum, there is the following method. A heart rate peak is obtained from the measured pulse wave, and a time series of intervals (PP Interval: corresponding to the RR interval of the electrocardiogram) is obtained. Next, in order to obtain HRV using Fast Fourier Transform (FFT), it is necessary to sample at regular time intervals, but since PP Interval fluctuates with autonomic nerve activity, the time series of PP Interval is Smooth using a polynomial and resample at regular time intervals. This is a technique for obtaining a frequency spectrum by performing FFT on the PP Interval resampled at a constant time interval.

Finally, HF and LF are calculated from the following formula, and the calculation result is stored in the memory.

(Video music means)
When the measurement is completed, the environmental video is displayed on the display 40 shown in FIG. 3C and the environmental music is played from the earphone 50.
Here, the environmental video is a photograph of nature or an animal that makes you feel calm by watching, and is set so that the video is switched every 2 to 5 seconds in a slide show format. Environmental music is a general term for music that is calmed by listening and sounds in nature.
The display and broadcasting of the environmental video and environmental music are preferably performed for 3 minutes to 7 minutes, and more preferably for 4 minutes to 6 minutes.

(Recalculation means)
Then, after the end of the video and music means, the screen of FIG. 3 (a) is displayed again to instruct the subject to urge the finger to be applied to the camera. Then, the image acquisition means is set so that the pulse wave can be measured, and then the screen shown in FIG. 3 (d) is displayed, the calculation display means is performed, the pulse wave is measured, and LF and HF are set. obtain.

(Judgment means)
HF obtained by the calculation display means stored in the memory; LF is set to HF1; LF1, the environmental video obtained by the recalculation means is shown, and HF after listening to the environmental music; LF is set to HF2; LF2. .
Here, the present inventors show that the HF decreasing tendency is shown with a significant difference by the research, and if the healthy person shows the environmental video, after showing and letting the HF appear before the environmental music is heard. As a result of further examination based on such knowledge, it is possible to easily determine whether or not there is a tendency to depression if determined according to the following formulas (7) to (9) Is heading. That is, it has been found that LF can be ignored. That is, as shown in FIG. 5, as a result of measuring an electrocardiogram as to how HF and LF fluctuate before and after environmental music is shown to a healthy person and a depressed patient while environmental music is heard. In contrast, there was a difference between healthy subjects and depressed patients for HF, but no difference for LF. Based on this result, it was decided to determine the depression tendency by examining only HF.
In view of this, the present means is configured so as to make the determination according to the following equations (7) to (9).
Then, as shown in FIG. 3 (e), the result of dividing the case and the change in the value of HF are shown by the result explanation and the determination result, and a simple as to whether or not the subject has a current mental state, particularly whether or not there is a depression tendency. The result of the judgment is shown.

<Usage method and mental state determination method>
The determination of the mental state as a psychosomatic state using the mental state determination device and the mental state determination program of the present embodiment,
An image acquisition step of continuously acquiring the partial image by applying light to a part of the subject's body;
A measurement process for analyzing the blood flow state from the acquired image and measuring the pulse wave;
A stabilization process in which the subject is listened to environmental music and an environmental video is shown and relaxed;
After completion of the stabilization process, re-measurement process in which light is applied to a part of the subject's body again to continuously acquire the partial image, and the blood flow state is analyzed by the acquired image to measure the pulse wave; ,
It can implement by performing the mental state determination process which determines a test subject's mental state based on the result obtained by the said determination process and the said re-measurement process.
Hereinafter, each step will be described.

(Image acquisition process)
First, as shown in FIG. 3A, the fingertip is arranged so as to hit the camera of the mobile phone. Then, by turning on the light source (white LED light) beside the camera, an image through which the fingertip is seen is obtained. Then, blood is transported in time with the heartbeat, which causes a difference in blood concentration in the blood vessel, which is continuously imaged by the camera and analyzed by the above calculation display means, thereby measuring the heartbeat interval. can do.
The graph at the center of the screen in FIG. 3 (a) displays the displacement of SW and shows the pulse wave waveform. In FIG. 3 (a), the circular graph below shows the progress, and the gauge accumulates by continuing to measure the pulse, and when the measurement continues for an arbitrary time, it is shown in FIG. 3 (b). Displays a screen for measuring the calculation process. As a result, the subject can easily measure the pulse wave without performing calibration work. If the pulse wave cannot be measured in the middle, the stored gauge value becomes 0 and the measurement is performed again. In short, when it can be confirmed that the pulse wave can be measured for a certain time, the program determines that the pulse wave can be stably measured, and the screen shown in FIG. indicate.

(Judgment process)
Then, the heartbeat of the subject is actually measured for 64 seconds on the screen shown in FIG. 3B, and HF and LF are calculated using the above-described calculation display means. Similar to the screen shown in FIG. 3 (a), a pulse wave waveform indicating Sw and a graph showing the progress of a circular gauge are displayed on the screen, and the measurement ends when the gauge is accumulated. However, if the heartbeat cannot be measured even once as in the image acquisition process, the gauge returns to 0 and remeasurement is performed.
(Stabilization process)
When the measurement is completed, the screen shifts to the screen shown in FIG. 3 (c), where the subject is shown a video and listens to music to be relaxed. This video presentation time is approximately 5 minutes, and the user's pulse wave is not measured during this time.
(Re-measurement process)
After completion of the stabilization process, the subject is prompted to measure again. The procedure at this time is the same as the image acquisition process and the determination process.
(Mental state judgment process)
After completion of the remeasurement process, the degree of depression tendency of the subject using the above-described determination means using the HF and LF values obtained in the determination process and the HF and LF values obtained in the remeasurement process is shown in FIG. Displayed in three stages on the screen shown in (e).

Although the psychosomatic state determination apparatus and the psychosomatic state determination program of the present invention have been described above, the present invention is not limited to these, and various modifications can be made without departing from the spirit of the present invention.
For example, in the above-described embodiment, the apparatus and program for determining the mental state have been described. However, the present invention is not limited to this. A Simple screening for diabetes B) Survival assessment of patients after myocardial infarction (if LF / HF <0.7, the 5-year survival rate tends to decrease, so this can be used to determine) C, Septic patients Of various physical conditions such as prediction of whether healing or transition to multiple organ failure can be determined (if there is a tendency to shift to multiple organ failure when HRV decreases in septic patients) It can be used as a device and a program.

Claims (2)

A light emitting part for irradiating the subject's body with light;
An image acquisition unit that continuously acquires an image of a portion irradiated with light by the light emitting unit;
A computer unit for calculating the pulse wave by computing the acquired image and calculating the state of the pulse wave over time;
An audio output unit for listening to environmental music to the subject;
A video output unit for showing the environmental video to the subject;
A psychosomatic state determination program for determining a psychosomatic state by operating the computer unit in a psychosomatic state determining apparatus comprising:
The computer part
Operation input means for displaying an instruction on the video output unit and receiving an operation by a subject;
When the subject responds to the operation input means, an instruction means for displaying and instructing the action to be obtained from the subject,
Image acquisition means for performing light irradiation and image acquisition for a predetermined time according to the action when the subject behaves according to the instruction means;
The image acquired by the image acquisition means is arithmetically processed and converted into a pulse wave, continuously displayed with the measurement time on the video output unit, and the data obtained by the arithmetic processing is integrated to obtain an LF (Low Frequency: 0). .04-0.15 Hz) and HF (High Frequency: 0.15-0.4 Hz)
Video music means for displaying environmental video on the video output unit and flowing environmental music to the audio output unit for a predetermined time after the measurement is completed;
After the display of the video and the music performance, the recalculation means for measuring the pulse wave again by operating the instruction means, the image acquisition means, the calculation display means,
And a psychosomatic state determination program for comparing the LF and HF data obtained by the calculation display means with the LF and HF data obtained by the recalculation means to function as a determination means for determining a psychosomatic state. An apparatus for executing the psychosomatic state determination program according to claim 1,
A light emitting part for irradiating the subject's body with light;
An image acquisition unit that continuously acquires an image of a portion irradiated with light by the light emitting unit;
A computer unit for calculating the pulse wave by computing the acquired image and calculating the state of the pulse wave over time;
An audio output unit for listening to environmental music to the subject;
A video output unit for showing the environmental video to the subject;
Comprising
The computer part is
A memory unit that temporarily stores the acquired image data and temporarily stores output data after the arithmetic processing, and an arithmetic processing unit that performs arithmetic processing;
A psychosomatic state determining apparatus comprising: a drive unit that stores the above psychosomatic state determining program for operating a computer.