JP7162254B2 - Biological information monitoring method and biological information monitoring system - Google Patents

Description

The present disclosure relates to a biological information monitoring method and a biological information monitoring system.

In Patent Document 1, when diagnosing fatigue stress, a reference value for each age is held as master data, and the measurement data obtained by measuring the electrocardiographic pulse wave of the subject is compared with the reference value to classify a plurality of classifications. Disclosed is a fatigue stress medical checkup system that outputs determination results divided into . In addition, this fatigue stress health checkup system can calculate the autonomic nerve function age, and judge not only the strength of the subject's autonomic nerves but also the balance of the autonomic nerves.

Japanese Unexamined Patent Application Publication No. 2015-54002

However, in Patent Document 1, the electrocardiogram and pulse wave data is measured while the subject is in direct contact with the electrocardiogram and pulse wave measuring device, which imposes a considerable burden on the subject. On the other hand, there is known a technique of detecting vital information (so-called biological information) such as a heartbeat of a subject in a non-contact manner and quantitatively deriving stress of the subject using the detected vital information. However, when the vital information is detected in an environment where there is a noise signal that has a periodicity similar to the vital information and has an amplitude that converges within a certain width, the subject is not actually present. Due to the existence of such a noise signal, it was sometimes mistakenly detected as the patient's vital information. For this reason, the subject's stress may be erroneously derived, and further improvements are desired from the viewpoint of improving the detection accuracy of vital information.

The present disclosure has been devised in view of the above-described conventional situation, even in an environment where a noise signal having characteristics similar to those of the subject's vital information detected contactlessly is detected contactlessly It is an object of the present invention to provide a biological information monitoring method and a biological information monitoring system that appropriately determine whether or not the vital information of a subject is valid, and accurately ensure the reliability of the stress of the subject.

The present disclosure includes a step of extracting biological information of a subject , calculating a first parameter indicating the reliability of the extracted biological information of the subject, and calculating the first parameter above a predetermined first threshold. a step of calculating a stress index corresponding to the biological information based on the extracted biological information; and calculating the calculated stress index based on the biological information extracted within a predetermined period. and outputting the calculated stress index according to the result of determination that the stress index is effective, wherein the determination of the effectiveness includes the step of calculating a second parameter indicating a ratio between the number of the biometric information items for which one parameter is equal to or greater than the first threshold value and the number of the biometric information items extracted within the predetermined period; If it is equal to or greater than the threshold, determine that the calculated stress index is valid, and measure the environmental noise within the angle of view of the camera that captures the subject within the angle of view in setting the second threshold; A biological information monitoring method is provided in which the second threshold is changed to be higher or lower than a default value according to the measured noise .

Further, the present disclosure calculates a biological information extraction unit that extracts biological information of a subject , and calculates a first parameter indicating the reliability of the extracted biological information of the subject, and the calculated first parameter is a predetermined A stress calculation unit that calculates a stress index corresponding to the biological information based on the extracted biological information when the stress index is equal to or greater than the first threshold value, and based on the biological information extracted within a predetermined period, a validity determination unit that determines whether or not the calculated stress index is valid; and an output unit that outputs the calculated stress index according to a determination result that the stress index is valid, The validity determination unit calculates a second parameter indicating a ratio between the number of the biometric information whose first parameter is equal to or greater than the first threshold value and the number of the biometric information extracted within the predetermined period, If the second parameter is equal to or greater than a predetermined second threshold, determine that the calculated stress index is valid, measure the environmental noise within the angle of view of the camera that captures the subject within the angle of view, A biological information monitoring system is provided in which the second threshold is changed to be higher or lower than a default value according to the measured noise .

According to the present disclosure, even in an environment where a noise signal having characteristics similar to those of the subject's vital information detected without contact exists, the validity of the subject's vital information detected without contact can be determined. Appropriate determination can be made, and the reliability of the subject's stress can be ensured accurately.

BRIEF DESCRIPTION OF THE DRAWINGS The figure which shows the example of an outline|summary of the biological information monitoring system which concerns on embodiment. Block diagram showing a hardware configuration example of a biological information monitoring system Flowchart showing an example of a stress estimation procedure Flowchart showing an example of a stress effectiveness determination procedure A table showing an example of the contents of a log file in which the stress index is recorded Graph showing an example of time variation of the reliability ratio A diagram showing an example of a stress index list screen displaying a list of stress indexes per day for each of a plurality of subjects. A diagram showing an example of a stress index transition screen showing changes in the stress index of the selected subject over time per day. A diagram showing an example of a stress index transition screen showing changes in the stress index of the selected subject on a daily basis within one month. A diagram showing an example of a setting screen for setting the e-mail address of the alert destination and the upper and lower limits of the stress index.

Hereinafter, embodiments specifically disclosing the configuration and operation of the biological information monitoring method and the biological information monitoring system according to the present disclosure will be described in detail with reference to the drawings as appropriate. However, more detailed description than necessary may be omitted. For example, detailed descriptions of well-known matters and redundant descriptions of substantially the same configurations may be omitted. This is to avoid unnecessary verbosity in the following description and to facilitate understanding by those skilled in the art. It should be noted that the accompanying drawings and the following description are provided for a thorough understanding of the present disclosure by those skilled in the art and are not intended to limit the claimed subject matter.

FIG. 1 is a diagram showing a schematic example of a biological information monitoring system 5 according to an embodiment. The biological information monitoring system 5 estimates the pulse of the human body from the captured image of the camera 2 obtained without contact with the human body (for example, the subject hm), and calculates the stress index of the human body based on the fluctuation of the pulse. do. A pulse is captured as a pulse rate or a pulse wave. The biological information monitoring system 5 is described as being installed in the office 100 , for example, but the installation location is not limited to the office 100 . In the office 100, a chair 130 on which the subject hm sits and a desk 150 are prepared. For example, on the desk 150 are placed the camera 2, the pulse estimation device 3, and the monitor 8 that constitute the biological information monitoring system 5. FIG. Note that the monitor 8 may be replaced by a display unit 28 such as a display included in the pulse estimation device 3 .

The subject hm undergoes a stress test while sitting on the chair 130 facing the camera 2 placed on the desk 150, for example. The camera 2 captures the subject hm's face and the like to obtain a captured image. A pulse estimation device 3 estimates the pulse of the subject hm based on the captured image obtained by the camera 2 . The pulse estimation device 3 is connected to the cloud server 40 via the network NW, and transmits stress information including both the estimated pulse value and the calculated stress index to the cloud server 40 . The monitor 8 displays the results of stress information statistically processed by the cloud server 40 .

FIG. 2 is a block diagram showing a hardware configuration example of the biological information monitoring system 5. As shown in FIG. The biological information monitoring system 5 includes a camera 2 , a pulse estimation device 3 , a monitor 8 and a cloud server 40 . The pulse estimation device 3 is communicably connected to the cloud server 40 via the network NW, and is configured using a communication device such as a personal computer (PC), a tablet terminal, or a smart phone, for example. Note that the pulse estimation device 3 may be directly connected to the cloud server 40 by a communication cable without going through the network NW so as to be able to communicate therewith.

The camera 2 includes an image capturing unit 11 for capturing an image of the subject hm who is a subject. Here, a case where the imaging unit 11 images a part of the subject hm's body, particularly the face that is exposed on a daily basis, is shown. In addition, the imaging unit 11 may image not only the face of the subject hm, but also the neck, palms, arms, legs, and the like. The imaging unit 11 has a solid-state imaging device (image sensor) such as a CCD (charged-coupled device) or CMOS (complementary metal-oxide-semiconductor), forms an image of light from a subject, and produces an optical image. It converts it into an electrical signal and outputs a video signal. A video signal output from the imaging unit 11 is input to the pulse estimation device 3 . Note that the number of cameras 2 is not limited to one, and may be plural.

The pulse estimation device 3 includes an image input unit 21, a region extraction unit 22, a pulse wave detection unit 23, a pulse estimation unit 24, a pulse wave storage unit 25, a pulse wave test unit 26, and an output determination unit 27. , a display unit 28 , a stress estimation unit 29 , a stress effectiveness determination unit 30 , and a communication unit 31 .

An image input unit 21, an area extraction unit 22, a pulse wave detection unit 23, a pulse estimation unit 24, a pulse wave storage unit 25, a pulse wave test unit 26, an output determination unit 27, and a stress estimation unit 29. , the stress effectiveness determination unit 30 is a memory (not shown) in which a processor such as a CPU (Central Processing Unit), a DSP (Digital Signal Processor), or an FPGA (Field Programmable Gate Array) is incorporated in the pulse estimation device 3. It is functionally implemented by loading an operating program stored in advance in the .

The memory (not shown) is composed of a RAM (Random Access Memory) and a ROM (Read Only Memory), and includes a program necessary for executing the operation of the pulse estimation device 3 and a program generated during operation. Temporarily store data or information. RAM is a work memory used, for example, during operation of the processor (see above). The ROM prestores, for example, a program for controlling the processor (see above).

The image input unit 21 inputs data of temporally continuous captured images (frame images) including the face of the subject hm captured by the imaging unit 11 of the camera 2 as video signals from the camera 2 . The image input unit 21 may be composed of input interface circuits such as short-range wireless communication, USB (Universal Serial Bus), and HDMI (High-Definition Multimedia Interface). Further, when the camera 2 is a web camera, the image input unit 21 may be configured by a communication interface circuit or the like with a LAN (Local Area Network) to which the web camera is connected.

The region extracting unit 22 extracts a flesh-colored region (here, referred to as a face region) of the subject hm's face included in the input captured image. The flesh-colored area extracted by the area extracting unit 22 is an area where the skin is exposed in the human body, and is an area in which the pulse can be estimated from the captured image data of the area.

The pulse wave detector 23 detects the pulse wave signal of the subject hm based on the information obtained from the skin color area of the face of the subject hm extracted by the area extractor 22 .

A pulse estimator 24 as an example of a biological information extractor estimates the pulse of the subject hm based on the pulse wave signal detected by the pulse wave detector 23 .

The pulse wave storage unit 25 is configured using, for example, the memory (not shown) described above, and stores the pulse wave signal detected by the pulse wave detection unit 23 .

Pulse wave tester 26 tests the pulse wave signal stored in pulse wave storage 25 .

The output determination unit 27 determines whether or not the estimation result (pulse) of the pulse estimation unit 24 is output based on the test result of the pulse wave test unit 26 .

The display unit 28 is configured using a known display, and displays various information including the estimation result (pulse) in the pulse estimation unit 24 to the user of the pulse estimation device 3 including the subject hm.

The stress estimator 29, which is an example of a stress calculator, acquires the pulse estimated by the pulse estimator 24 and the reliability of the pulse wave signal verified by the pulse wave tester 26, and calculates the pulse and the pulse wave signal. A stress index is calculated based on the reliability.

Based on the stress index calculated by the stress estimation unit 29, the stress effectiveness determination unit 30 calculates a reliability ratio (see below) of the stress index. A stress effectiveness determination unit 30, which is an example of the effectiveness determination unit, determines stress effectiveness based on this reliability ratio, and determines presence/vacancy (in other words, presence/absence) of the subject hm. .

The communication unit 31 communicates with the cloud server 40 connected to the network NW and the monitor 8 connected to the pulse estimation device 3 . The communication unit 31 has a network interface connectable to the network NW, a short-range wireless communication interface connectable to the monitor 8, and the like. The communication unit 31 as an example of an output unit transmits stress information including pulse, stress index, etc. to the cloud server 40 . The communication unit 31 receives data of various screens displayed on the monitor 8 from the cloud server 40 .

The monitor 8 is a device capable of displaying various screens, and may be a liquid crystal display device, an organic EL (Electroluminescence) device, or any other display device.

The cloud server 40 performs various statistical processes on the stress index based on the stress information obtained from the pulse estimating device 3 . The cloud server 40 has a configuration including a processor 41 , a memory 42 , a storage 43 and a communication section 44 .

The processor 41 is configured using one of a CPU (Central Processing Unit), DSP (Digital Signal Processor), FPGA (Field Programmable Gate Array), and the like. The processor 41 functions as a controller that governs the operation of the cloud server 40, and performs control processing for overall control of the operation of each unit of the cloud server 40, data input/output processing with each unit of the cloud server 40, and data processing. calculation (calculation) processing and data storage processing. Processor 41 operates according to a program stored in memory 42 . Processor 41 uses memory 42 in operation and stores data generated by processor 41 in memory 42 . The processor 41 realizes the functions of the alert determination unit 412 and the report creation unit 411 by executing an operation program stored in the memory 42, for example.

The alert determination unit 412 performs alert determination for each subject hm based on the stress information of the subject hm obtained from the pulse rate estimation device 3 . For alert determination, for example, when the stress index of the subject hm exceeds a predetermined upper limit or an upper limit set for each subject hm, a predetermined alert is issued (for example, when the stress index exceeds the upper limit This is a determination for sending a message to that effect by e-mail or the like).

Based on the stress information of the subject hm obtained from the pulse estimating device 3, the report creating unit 411 creates various screens serving as reports to be displayed on the monitor 8. FIG. Note that the report creation unit 411 may include the determination result of the alert determination unit 412 when creating various screens as reports.

The memory 42 is configured using RAM (Random Access Memory) and ROM (Read Only Memory), and temporarily stores the programs necessary for executing the operation of the cloud server 40, as well as data or information generated during operation. Save to The RAM is a work memory used when the processor 41 operates, for example. The ROM stores in advance a program for controlling the processor 41, for example.

The storage 43 is configured using, for example, a HDD (Hard Disk Drive) or an SSD (Solid State Drive), and stores various screens created by the processor 41 and data of various statistical results executed by the processor 41, respectively. save.

The communication unit 44 is communicably connected to the pulse estimation device 3 via the network NW. The communication unit 44 receives stress information from the pulse estimation device 3 . The communication unit 44 transmits screen data of various display screens to the pulse estimation device 3 . The communication unit 44 has a network interface connectable to the network NW.

The network NW may be a local area network (LAN), a wide area network (WAN), a mobile network, a power line communication network, a wide area communication network such as the Internet.

Although the camera 2 and the pulse estimation device 3 are configured as separate devices here, they may be configured as a single device housed in the same housing. When the camera 2 and the pulse estimation device 3 are configured as a single device, the camera 2 is installed in the upper center of the display of the pulse estimation device 3, for example.

Next, each part of the pulse estimation device 3 according to Embodiment 1 will be described in more detail.

The region extracting unit 22 executes known face detection processing for recognizing face feature amounts for each captured image (frame image), extracts the detected face region as the skin color region of the subject hm, and tracks the detected face region. . By extracting the face area as the flesh-colored area, the area extracting section 22 can easily extract the flesh-colored area. The method for extracting the skin area is not limited to the above method, and pixels having preset skin color components may be extracted from the captured image, and the area from which the pixels are extracted may be used as the skin color area. The skin color component has, for example, a preset ratio based on each pixel value of RGB (Red Green Blue), and has different values depending on individual differences. In this case, parts other than the face where skin is exposed (for example, hands, arms, etc.) can also be extracted as flesh-colored areas. FIG. 1 shows a case where there is only one subject hm, but when a plurality of persons are included in the captured image, the region extracting unit 22 extracts a plurality of face regions using a plurality of persons as subjects. may

The pulse wave detection unit 23 calculates, for example, the pixel value (0 to 255 gradations) of each component of RGB for each pixel constituting the skin color region extracted in the temporally continuous captured images, and calculates the representative value ( Here, the time-series data of the average value of each pixel) is generated as a pulse wave signal. In this case, the pulse wave detection unit 23 may generate time-series data based on the pixel values of only the green component (G), which has a particularly large fluctuation due to pulsation, or the components of each pixel may be Y, Cb, Cr , and time-series data may be generated from color difference components (Cb, Cr) that are less affected by environmental brightness fluctuations.

The generated time-series data of pixel values (average values) accompanies minute fluctuations (for example, fluctuations of less than one gradation of pixel values) based on changes in hemoglobin concentration in blood, for example. For this reason, the pulse wave detection unit 23 performs filter processing (for example, processing using a band-pass filter in which a predetermined passband is set, etc.) on the time-series data based on the pixel values, so that high-frequency noise components and low-frequency The pulse wave is detected (extracted) from which the fluctuation component is removed.

The pulse estimator 24 calculates the interval between adjacent peaks in the pulse wave signal detected by the pulse wave detector 23, that is, the interval of one pulse cycle of the pulse wave (pulse wave interval). Estimate pulse rate based on intervals. Since the pulse wave interval is caused by the beating of the heart, it can be regarded as equivalent to the heart rate interval (RRI) in an electrocardiograph. Therefore, it is possible to estimate the pulse rate based on the pulse interval.

For example, the pulse estimator 24 calculates the time corresponding to one cycle (pulse wave interval, PWI: Pulse Wave Interval), and based on this extracted PWI, the pulse rate of the subject hm may be calculated according to Equation (1).

Here, it is assumed that the subject is one person, but if the image captured by the camera 2 includes a plurality of subjects, the image range containing each subject is cut out, and based on each image data , may estimate the pulse rate of each subject. In addition, the pulse estimation unit 24 may estimate the pulse in time series in real time, or may perform it every M frames. M frames are the number of frames corresponding to, for example, 2 to 3 seconds. In this embodiment, pulse estimation is performed in real time in chronological order. When estimating the pulse every M frames, the pulse estimator 24 may use the pulse wave signals for M frames stored in the pulse wave storage 25 to estimate the pulse.

The pulse wave signal detected by the pulse wave detector 23 is stored in the pulse wave memory 25 . Specifically, a pulse wave signal for each predetermined frame (that is, for each predetermined period) is stored. In this embodiment, pulse wave signals for at least M frames are stored.

The pulse wave tester 26 tests pulse wave signals for M frames stored in the pulse wave storage 25 . Specifically, the feature of the waveform of the pulse wave signal is extracted, and it is determined whether or not the feature of the extracted waveform falls within a predetermined reference range. The timing for testing the pulse wave signal may be every M frames or the timing at which the pulse is estimated by the pulse estimator 24 . In this embodiment, the pulse wave signal is tested every M frames. Based on the characteristics of the waveform of the pulse wave signal, the pulse wave test unit 26 determines whether the characteristics of the waveform fall within a reference range. The reference range is defined by the waveform amplitude and interval of the pulse wave signal.

Here, various methods can be used for the test by the pulse wave test section 26 . An example of the testing method by the pulse wave testing unit 26 will be described below.

(a) Examination based on the degree of fluctuation of the interval L of the waveform Based on the interval between adjacent peaks, a statistical method is used to calculate the degree of fluctuation of the waveform interval as an evaluation value. Specifically, the pulse wave test unit 26 calculates statistical values such as average deviation, variance, and standard deviation for these intervals, and uses the statistical values as evaluation values (wavelength evaluation values) of interval fluctuations. This wavelength evaluation value becomes the feature of the waveform. The pulse wave test unit 26 compares the calculated wavelength evaluation value with a predetermined evaluation threshold value, and determines whether the wavelength evaluation value, which is a feature of the waveform, falls within a predetermined reference range. The evaluation threshold is obtained in advance through experiments, simulations, or the like.

(b) Test based on degree of fluctuation of amplitude A of waveform
The pulse wave tester 26 uses a statistical method to calculate the degree of amplitude fluctuation of the waveform as an evaluation value based on the amplitudes between the peaks and bottoms of the waveform that are adjacent to each other in the vertical direction. Specifically, the pulse wave test unit 26 calculates statistical values such as the mean deviation, variance, and standard deviation of the amplitude, and uses the statistical values as evaluation values of amplitude fluctuations (amplitude evaluation value). This amplitude evaluation value becomes a feature of the waveform. The pulse wave test unit 26 compares the calculated amplitude evaluation value with a predetermined evaluation threshold, and determines whether or not the amplitude evaluation value, which is a feature of the waveform, falls within a predetermined reference range. The evaluation threshold is obtained in advance through experiments, simulations, or the like.

(c) Test based on the degree of fluctuation of the waveform interval and the degree of fluctuation of the amplitude of the waveform It is also possible to use both the wavelength evaluation value of the waveform interval and the amplitude evaluation value of the amplitude of the waveform as the characteristics of the waveform. can. Specifically, the pulse wave testing unit 26 uses an evaluation value (composite evaluation value) calculated by combining the wavelength evaluation value and the amplitude evaluation value as the waveform feature. The combined evaluation value is calculated using Equation (2). w1 and w2 are weighting factors.

The pulse wave test unit 26 compares the calculated combined evaluation value with a predetermined evaluation threshold value in the same manner as in the case of the waveform interval and amplitude, and determines that the combined evaluation value, which is a feature of the waveform, falls within a predetermined reference range. determines whether it fits in The evaluation threshold is obtained in advance through experiments, simulations, or the like.

The pulse estimator 3 uses the wavelength evaluation value, amplitude evaluation value, or composite evaluation value obtained by any one of the above tests (a), (b), and (c) as the reliability of the pulse wave signal.

Details of the stress estimation unit 29 and the stress effectiveness determination unit 30 will be described in the operation of the biological information monitoring system 5 .

Next, the operation of the biological information monitoring system 5 according to Embodiment 1 will be described.

FIG. 3 is a flow chart showing an example of a stress estimation procedure. Each process of the stress estimation procedure shown in FIG. 3 is executed by the stress estimation unit 29 of the pulse estimation device 3, for example.

In FIG. 3, the stress estimator 29 acquires the reliability of the pulse estimated by the pulse estimator 24 and the pulse wave signal verified by the pulse wave tester 26 (S1). The stress estimator 29 determines whether or not the reliability of the acquired pulse wave signal exceeds a predetermined threshold Th0 (S2). The threshold Th0 is a value for determining that the signal is a pulse wave signal without depending on, for example, high-frequency noise components or low-frequency shaking components. If the reliability of the pulse wave signal does not exceed the threshold Th0 (S2, NO), the process of the stress estimator 29 returns to step S1.

On the other hand, when the reliability of the pulse wave signal exceeds the threshold Th0 (S2, YES), the stress estimator 29 calculates a stress index (S3). The stress index is obtained as time-series data of heart rate variability based on the pulse wave interval, that is, the heart rate interval (RRI) caused by heart beats. A stress index based on heart rate variability includes, for example, LF/HF, which represents the overall balance between sympathetic nerves and parasympathetic nerves. LF (low frequency) represents a low frequency of 0.004 to 0.15 Hz of time-series data of heart rate variability, and is a fluctuating wave called a Mayer wave whose signal source is blood pressure changes with a cycle of about 10 seconds. HF (high frequency) represents a high frequency of 0.15 to 0.4 Hz, and is a fluctuating wave having a period of about 3 to 4 seconds and whose signal source is respiration. When the stress index LF/HF is high, the subject hm is in a state in which the sympathetic nerve is dominant, and the stress is relatively higher than in the relaxed state. On the other hand, when the stress index LF/HF is low, the subject hm is in a state in which the parasympathetic nervous system is dominant, and is in a state of being relatively more relaxed than in a state of high stress. In this embodiment, the stress index is not limited to LF/HF, and other indexes may be used.

FIG. 4 is a flow chart showing an example of the stress validity determination procedure. Each process of the stress effectiveness determination procedure shown in FIG. 4 is executed by the stress effectiveness determination unit 30 of the pulse rate estimation device 3, for example.

In FIG. 4, the stress effectiveness determination unit 30 acquires the stress index obtained as a result of the processing shown in FIG. 3 (S11). The stress effectiveness determination unit 30 calculates a reliability ratio based on the stress index obtained over a predetermined period (for example, 5 minutes) (S12). The reliability ratio is extracted within the same predetermined period (for example, 5 minutes) as the pulse rate (that is, the number of biological information) at which the reliability of the pulse wave signal obtained in the predetermined period (for example, 5 minutes) is equal to or greater than the threshold Th0. It is the ratio to the total number of pulse rates measured.

The stress effectiveness determination unit 30 determines whether or not the reliability ratio is less than the threshold Th1 (S13). The threshold Th1 is a value for determining whether or not the stress index is effective. Here, the threshold Th1 may be set to a predetermined fixed value. Also, the threshold Th1 may be set to a different value for each subject. In this case, an authorized administrator may set the threshold Th1 for each subject, or the subject may set his or her own threshold Th1. Also, the threshold Th1 may be adaptively set according to the measurement environment. For example, under noisy environments, the threshold Th1 may be set to a large value. The amount of noise is represented by the ratio of the peak value of the band (for example, 0.5 to 2 Hz) containing the heart rate in the frequency domain and the average value of other bands. Specifically, the stress effectiveness determination unit 30 measures the environmental noise within the angle of view of the camera 2, and depending on the measured noise, the threshold Th1 may be changed to be higher or lower than the default value. good.

When the reliability ratio is less than the threshold Th1 (S13, YES), the stress effectiveness determination unit 30 initializes the stress index to a value of 0 (S14). On the other hand, when the reliability ratio is equal to or greater than the threshold Th1 (S13, NO), the stress effectiveness determination unit 30 determines that the stress index is effective as the stress effectiveness determination result based on the stress index (S15). The communication unit 31 transmits stress information including the stress index obtained in step S14 or step S15 to the cloud server 40 (S16). After that, the stress effectiveness determination unit 30 terminates the processing shown in FIG.

Further, in step S14, when the calculated reliability ratio is less than the threshold Th2, the stress effectiveness determination unit 30 determines that the subject is absent (vacant seat), and forcibly sets the reliability ratio to 0. good too. The threshold Th2 is a value for determining whether a subject is present or not. The threshold Th2 may be set to any value greater than 0(%) and less than 100(%), for example, set to 10%. Typically, the threshold Th2 is set to a value smaller than the threshold Th1 for determining stress effectiveness. Note that when the reliability ratio is not 0, the stress effectiveness determination unit 30 determines that the subject is present. Therefore, the stress effectiveness determination unit 30 can prevent erroneous determination that the subject is present due to the reliability ratio becoming a non-zero value less than the threshold Th2 during the subject's absence period.

In the above description, the point of determining the effectiveness of the stress index calculated by comparing the reliability ratio and the threshold Th1 was described. The effectiveness of the stress index may be determined based on whether or not the number of pieces of information exceeds a predetermined threshold.

FIG. 5 is a table showing an example of recorded contents of a log file fg1 in which stress indexes are recorded. When the stress estimator 29 estimates the stress index, the processor (see above) of the pulse estimating device 3 generates a log file fg1, adds it to a memory (not shown) built into the pulse estimating device 3, and saves it. do. In the log file fg1, records including the user ID, test date/time, LF/HF, reliability, and additional information for each subject are recorded in chronological order. For example, the first row record includes user ID: 0011, test date: 2019/1/25, LF/HF: 3, reliability: 2, additional information: PC-20180. The user ID is, for example, the employee ID of the subject hm. The test date and time is the date and time when the stress index of the subject hm was calculated (in other words, the date and time when the pulse of the subject hm was tested). LH/HF is an example of a parameter that indicates a stress index. The reliability is the reliability of the pulse wave signal that is the basis for calculating the stress index (for example, LH/HF), and is the wavelength evaluation value and amplitude evaluation value calculated by the pulse wave tester 26 of the pulse estimating device 3. or the reliability of the pulse wave signal corresponding to one of the combined evaluation values. The additional information is, for example, the model number of the pulse estimating device 3 used. Note that the log file fg1 may be managed by the communication unit 31 transmitting the stress information to the cloud server 40 and the cloud server 40 storing it in the storage 43 .

FIG. 6 is a graph showing an example of temporal change in the reliability ratio. The vertical axis of the graph shown in FIG. 6 represents the reliability rate (%) calculated by the stress effectiveness determination unit 30. As shown in FIG. The horizontal axis of the graph shown in FIG. 6 represents time. FIG. 6 also shows a bar indicator sc1 corresponding to the time axis of the graph and indicating whether the subject is in front of the desk 150 or is not in front of the desk 150. . For example, a line graph gh1 of the reliability ratio is shown in the time zone 10:30 to 17:30 corresponding to the office core time. In the time period from 12:30 to 12:50, the reliability ratio line graph gh1 has a value of 0, indicating that the subject is absent. Similarly, in the time period from 13:20 to 14:25 and the time period from 15:00 to 15:15, the reliability ratio line graph gh1 has a value of 0, indicating that the subject is absent. Here, the threshold Th2 for determining presence/vacancy is set to 10%, for example. Further, when the subject is present during the time slot of 11:30 to 16:00, the line graph gh1 of the reliability ratio shows a high value in the range of approximately 60 to 90%.

Note that the line graph gh1 has a period of time after 17:00 when the threshold Th2 is temporarily below the threshold Th2, but since the period of time below the threshold Th2 is short (that is, below the predetermined threshold), the bar indicator It does not mark sc1 as absent.

FIG. 7 is a diagram showing an example of a stress index list screen GM1 that displays a list of stress indexes per day for each of a plurality of subjects. The stress index list screen GM<b>1 may be created by the report creation unit 411 and displayed on the monitor 8 or may be displayed on the display unit 28 of the pulse estimation device 3 . On the stress index list screen GM1, for example, for each department of an organization within a company, a user name (that is, an employee name), a status indicating presence/absence, a current stress index, and a daily working hours are displayed. A heat map of the stress index is displayed in association with it. For example, user name: Mr. b, status: present, current stress index: 20, heat map of stress index: heat map of stress index from 7:00 to the present time (16:30). Also, the stress index (for example, "80" in FIG. 7) at the position indicated by the cursor Ks is displayed.

The stress index in FIG. 7 is a value that is intuitively understandable to the subject by the report creation unit 411 based on the LF/HF value (for example, possible values 0 to 5) calculated by the stress estimation unit 29. It is a value converted from 0 to 100. For example, the higher the value of LF/HF, the higher the value after conversion. For example, when the LF/HF value is the maximum value of 5, the stress index is 100, and when the LF/HF value is the minimum value of 0, the stress index is 0. Note that the conversion described above may be performed by the stress estimation unit 29 .

Also, the user name: Mr. a has a status of "communication interrupted," and for example, it is considered that the power of the pulse estimation device 3 is not turned on. In other words, user name: Mr. a is considered to be absent from the office on this day (that is, February 20, 2018) due to a holiday, going out, or the like.

FIG. 8 is a diagram showing an example of the stress index transition screen GM2 showing the time change of the stress index of the selected subject per day. The stress index transition screen GM2 may be created by the report creation unit 411 and displayed on the monitor 8 or may be displayed on the display unit 28 of the pulse estimation device 3 . On the stress index transition screen GM2, a line graph gh2 is displayed that indicates the change over time of the stress index per day of the subject (for example, user name of section XX: Mr. j). The vertical axis of the graph represents the stress index (value converted to the value 0 to 100 described above), and the horizontal axis represents time. The line graph gh2 of the stress index shows that the stress index is substantially within the upper limit UL1 and the lower limit DL1 in the range of working hours from 9:00 to 20:00 in a day, for example. Therefore, it is presumed that this subject (user name: Mr. j) does not feel much stress. These upper limit value UL1 and lower limit value DL1 may be uniformly fixed values regardless of subjects, or may be different values for each organization or for each individual in consideration of differences in organizational work, for example.

In addition, the department and user name for selecting subjects can be selected from pull-down menus m1 and m2 on the stress index transition screen GM2. Also, the date on which the stress index was tested can be selected from the pull-down menu m3.

FIG. 9 is a diagram showing an example of the stress index transition screen GM3 showing daily changes in the stress index of the selected subject within one month. The stress index transition screen GM3 may be created by the report creating unit 411 and displayed on the monitor 8 or may be displayed on the display unit 28 of the pulse estimating device 3 . On the stress index transition screen GM3, the average value of the stress index (the above-mentioned value converted to 0 to 100) of the subject (for example, the user name of XX section: Mr. j) for a certain period (for example, one month) is displayed. A line graph gh3 showing the stress index (the value converted into the value 0 to 100 described above), a bar graph gh4 showing the time when the stress index (the value converted to the value 0 to 100 described above) exceeds the lower limit DL1, and the stress index (the value converted to the value 0 to 100 described above) is the upper limit A bar graph gh5 is displayed showing the time over which the value UL1 is exceeded. The vertical axis on the left side of the graph represents the stress index (value converted to the above value 0 to 100), and the right vertical axis represents the stress index (value converted to the above value 0 to 100) is the threshold (upper limit value It represents the time exceeding UL1 or the lower limit DL1). The horizontal axis of the graph represents dates.

Further, in the stress index transition screen GM3, similarly to the stress index transition screen GM2, the department and user name for selecting subjects can be selected from pull-down menus m1 and m2, respectively. Also, the month in which the stress index was tested can be selected from the pull-down menu m3.

FIG. 10 is a diagram showing an example of the setting screen GM4 for setting the e-mail address of the alert transmission destination and the upper and lower limits of the stress index. The setting screen GM<b>4 may be created by the report creation unit 411 and displayed on the monitor 8 or may be displayed on the display unit 28 of the pulse estimation device 3 . The setting screen GM4 displays, for each user (in other words, subject), a user ID, department, display name, a table Tb1 for setting lower and upper alert threshold values, and an email address of an alert destination. Here, an alert is a message for pointing out that the stress index (for example, the value converted into the value 0 to 100 described above) has exceeded the upper limit value. For example, the table Tb1 registers user name: user1, affiliation: XX section, display name: Mr. a, alert threshold (lower limit): 15, and alert threshold (upper limit): 85. In addition, the e-mail address of the alert transmission destination: OOO@jp.ΔΔΔ.com is set, and for example, the e-mail address of a person corresponding to a managerial position or higher in each organization is set.

As described above, in the biological information monitoring system 5 or the biological information monitoring method according to Embodiment 1, the pulse estimation unit 24 estimates the pulse of the subject hm based on the image captured by the camera 2 capturing the subject within the angle of view. (That is, biometric information is extracted without contact). The stress estimator 29 calculates a stress index based on the pulse estimated by the pulse estimator 24 . The stress effectiveness determination unit 30 determines whether or not the calculated stress index is effective based on the pulse estimated within a predetermined period (for example, 5 minutes). The communication unit 31 transmits (that is, outputs) the stress information including the calculated stress index to the cloud server 40 according to the determination result that the stress index is valid.

As a result, even in an environment where a noise signal having characteristics similar to those of the subject's vital information detected in a non-contact manner exists, the biological information monitoring system 5 or the biological information monitoring method can detect the vital information in a non-contact manner. It is possible to appropriately determine whether or not the vital information of the subject is valid. Therefore, according to the biological information monitoring system 5 or the biological information monitoring method, the reliability of the subject's stress can be ensured accurately.

Further, in the biological information monitoring method, the pulse wave tester 26 calculates the reliability of the pulse wave signal (an example of the first parameter). In calculating the stress index, the stress estimator 29 calculates the stress index when the reliability of the calculated pulse wave signal is equal to or greater than a threshold Th0 (an example of a predetermined first threshold). Thus, according to the biological information monitoring system 5 or the biological information monitoring method, the pulse can be estimated when the reliability of the pulse wave signal is high, and the accuracy of the calculated stress index can be improved.

In the biological information monitoring method, the stress validity determination unit 30 determines the validity of the stress index by determining the pulse rate (an example of the number of biological information) at which the reliability of the pulse wave signal is equal to or higher than the threshold Th0 and the predetermined period of time. A reliability ratio (an example of a second parameter) indicating a ratio to the pulse rate extracted in the second parameter is calculated. The stress effectiveness determination unit 30 determines that the calculated stress index is effective when the reliability ratio is equal to or greater than a threshold Th1 (an example of a predetermined second threshold). Thus, according to the biological information monitoring system 5 or the biological information monitoring method, the stress index can be accurately calculated even in an environment where noise signals having characteristics similar to vital information exist. Therefore, according to the biological information monitoring system 5 or the biological information monitoring method, it is possible to suppress the calculation of the stress index assuming that the subject is present even though the subject is absent.

Moreover, in the biological information monitoring method, the threshold Th1 is a different value for each subject. Thus, according to the biological information monitoring system 5 or the biological information monitoring method, a more accurate stress index can be calculated according to the aptitude of the subject.

Further, in the biological information monitoring method, the stress validity determination unit 30 measures the environmental noise within the angle of view of the camera 2, and changes the threshold Th1 to be higher or lower than the default value according to the measured noise. set. Thereby, the stress effectiveness determining section 30 can adaptively set the threshold Th1 according to the measurement environment.

Further, in the biological information monitoring method, the report creation unit 411 of the cloud server 40 creates a screen (for example, the stress index transition screen GM3) including the result of performing predetermined statistical processing based on the output stress index for a certain period of time. to generate A monitor 8 displays the generated screen. Thereby, a user such as a subject can visually confirm the presence or absence of stress, and can respond according to the amount of the stress index.

Further, in the biological information monitoring method, the stress effectiveness determination unit 30 forces the reliability ratio when the reliability ratio is less than the threshold Th2 (an example of the predetermined third threshold) in the evaluation of the effectiveness of the stress index. It is determined that the subject is absent by setting the value to zero. Thereby, the biological information monitoring system 5 can suppress erroneous determination that the subject is present even though the seat is vacant.

In addition, in the biological information monitoring method, the report creation unit 411 of the cloud server 40 generates the stress index list screen GM1 including the stress index for each subject. Thereby, a user such as a subject can visually grasp the degree of stress of many subjects in a list.

In addition, in the biological information monitoring method, the report creation unit 411 of the cloud server 40 generates the stress index transition screen GM2 including the time change characteristic of the stress index. This allows a user such as a subject to check changes in the stress index over time.

Further, in the biological information monitoring method, the report creation unit 411 of the cloud server 40 creates a setting screen GM4 that includes the upper limit of the stress index and the e-mail address of the subject to be notified when the stress index exceeds the upper limit. to generate Thereby, the biological information monitoring system 5 can alert a user such as a subject when the stress index frequently exceeds the upper limit.

Although the embodiments have been described above with reference to the accompanying drawings, the present disclosure is not limited to such examples. It is obvious that a person skilled in the art can conceive of various modifications, modifications, substitutions, additions, deletions, and equivalents within the scope of the claims. It is understood that it belongs to the technical scope of the present disclosure. Also, the components in the above-described embodiments may be combined arbitrarily without departing from the spirit of the invention.

For example, in the above embodiment, the pulse estimator 24, the pulse wave storage 25, the pulse wave detector 26, the output determiner 27, the display 28, the stress estimator 29, and the stress effectiveness The determination unit 30 and the communication unit 31 may be omitted from the pulse estimation device 3 and included in the cloud server 40 . In this case, the cloud server 40 includes the above units (that is, the pulse rate estimation unit 24, the pulse wave storage unit 25, the pulse wave test unit 26, the output determination unit 27, the display unit 28, the stress estimation unit 29, the stress effectiveness determination unit 30 and the operation of the communication unit 31).

In the above embodiment, the threshold Th1 for determining the effectiveness of stress and the threshold Th2 for determining the presence/vacancy of the subject are different values, but they may be the same value.

The present disclosure appropriately determines the validity of non-contact detected vital information of a subject even in an environment where a noise signal having characteristics similar to those of the non-contact detected vital information of the subject exists. It is useful as a biological information monitoring method and a biological information monitoring system that discriminate and accurately secure the reliability of the subject's stress.

2 Camera 3 Pulse estimation device 5 Biological information monitoring system 11 Imaging unit 21 Image input unit 22 Region extraction unit 23 Pulse wave detection unit 24 Pulse estimation unit 25 Pulse wave storage unit 26 Pulse wave test unit 27 Output determination unit 28 Display unit 29 Stress Estimation unit 30 Stress effectiveness determination unit 31 Communication unit 40 Cloud server 41 Processor 411 Report creation unit 412 Alert determination unit 42 Memory 43 Storage

Claims (8)

extracting biometric information of the subject;
A first parameter indicating the reliability of the extracted biological information of the subject is calculated, and when the calculated first parameter is equal to or greater than a predetermined first threshold, calculating a stress index corresponding to the biological information based on the extracted biological information;
determining whether or not the calculated stress index is effective based on the biological information extracted within a predetermined period;
and outputting the calculated stress index according to a determination result that the stress index is valid.death,
Calculate a second parameter indicating a ratio between the number of the biometric information whose first parameter is equal to or greater than the first threshold value and the number of the biometric information extracted within the predetermined period in determining whether or not the validity is present. and determining that the calculated stress index is valid when the second parameter is equal to or greater than a predetermined second threshold,
In setting the second threshold, measure the environmental noise within the angle of view of the camera that captures the subject within the angle of view, and change the second threshold higher or lower than the default value according to the measured noise. and set do,
A biological information monitoring method. The second threshold is a different value for each subject,
The biological information monitoring method according to claim 1 . a step of generating a screen including a result of performing a predetermined statistical processing based on the output stress index for a certain period of time;
and displaying the generated screen.
The biological information monitoring method according to claim 1. In determining the presence or absence of the validity,
determining that the subject is absent if the second parameter is less than a predetermined third threshold;
The biological information monitoring method according to claim 1 . wherein the screen includes the stress index for each subject;
The biological information monitoring method according to claim 3 . wherein the screen includes a time-varying profile of the stress index;
The biological information monitoring method according to claim 3 . In generating the screen,
a setting screen for setting the upper limit of the stress index and the contact information of the subject to whom, when the stress index exceeds the upper limit, a message to the effect that the stress index exceeds the upper limit is notified; generate,
The biological information monitoring method according to claim 3 . a biometric information extraction unit for extracting biometric information of a subject;
A first parameter indicating the reliability of the extracted biological information of the subject is calculated, and when the calculated first parameter is equal to or greater than a predetermined first threshold, a stress calculator that calculates a stress index corresponding to the biological information based on the extracted biological information;
an effectiveness determination unit that determines whether or not the calculated stress index is effective based on the biological information extracted within a predetermined period;
and an output unit that outputs the calculated stress index according to a determination result that the stress index is valid.,
The validity determination unit
calculating a second parameter indicating a ratio between the number of the biometric information whose first parameter is equal to or greater than the first threshold and the number of the biometric information extracted within the predetermined period; determining that the calculated stress index is valid when it is equal to or greater than the second threshold;
Measure the environmental noise within the angle of view of the camera that captures the subject within the angle of view, and set the second threshold higher or lower than the default value according to the measured noise. Ru
Biological information monitoring system.

Images