JP7152727B2 - Biological information calculation device and biological information calculation method - Google Patents

Description

The present invention relates to a biological information calculation device and a biological information calculation method.

For example, the device disclosed in Patent Document 1 includes a camera, two linear LED lighting devices (hereinafter, this may be referred to as a lighting system), a control unit, and a processing unit. _The arterial blood oxygen saturation SpO2 is measured without the device in contact with the mammal of interest.

US2017/0127988A1

However, in the conventional method as described in Patent Document 1, there is a problem that a dedicated lighting system is required for preparing the shooting environment for measuring biological information.

An object of one aspect of the embodiments of the present invention is to provide a biometric information calculation device that calculates biometric information with a general-purpose configuration by using a device used for a general camera function such as a smartphone.

A biometric information calculation device for calculating one or more biometric information of an object to be measured from image information of an object to be measured, which is a living organism, comprising an acquisition unit for acquiring image information in RGB format, and a color specification of the acquired image information from RGB format to XYZ or Lab format; a fast Fourier transform unit for calculating at least one power spectral density of oxygenated blood pigment concentration and deoxygenated blood pigment concentration; and a biological information extraction unit for extracting one or more pieces of biological information from the peak of the power spectral density. An object of the present invention is to provide a biological information calculation device comprising:

A biological information calculation method performed by a biological information calculation device for calculating one or more pieces of biological information of an object to be measured from image information of an object to be measured, which is a living body, comprising a first step of acquiring image information in RGB format. a second step of converting the color specification of the acquired image information from the RGB format to the XYZ format or the Lab format; a third step of estimating; a fourth step of calculating power spectral densities of at least one of oxygenated hemoglobin concentration and deoxygenated hemoglobin concentration; and extracting one or more biometric information from the power spectral density peaks. and a fifth step to provide a biometric information calculation method.

According to one aspect of the embodiments of the present invention, it is possible to realize a biometric information calculation device that calculates biometric information with a general-purpose configuration by using a device used for a general camera function such as a smartphone.

FIG. 1 is a block diagram showing the configuration of the biological information calculation device according to this embodiment. FIG. 2 is a diagram showing information stored in the auxiliary storage device according to this embodiment. FIG. 3 is a graph showing oxygenated hemoglobin concentration over time according to the present invention. FIG. 4 is a graph obtained by performing a fast Fourier transform on the graph of FIG. FIG. 5 is a graph showing deoxygenated hemoglobin concentration over time according to the present embodiment. FIG. 6 is a graph obtained by performing a fast Fourier transform on the graph of FIG. FIG. 7 is a flowchart showing the procedure of biometric information calculation processing according to the present embodiment.

One aspect of the embodiment of the present invention will be described in detail below with reference to the drawings.

(this embodiment)
First, the biological information calculation device 1 in the stage of use will be described with reference to FIG. FIG. 1 is a block diagram showing the configuration of a biological information calculation device 1 according to this embodiment. The biometric information calculation device 1 calculates one or a plurality of biometric information of a measurement target from image information of a measurement target, which is a living body.

The biological information calculation device 1 includes a central processing unit 2 , a main storage device 3 and an auxiliary storage device 4 . For example, a smart phone includes a biometric information calculation device 1 and an imaging device 10 . The central processing unit 2 is, for example, a CPU (Central Processing Unit), and executes processing by calling programs stored in the main storage device 3 . The main storage device 3 is, for example, a RAM (Random Access Memory), and stores programs such as an acquisition unit 5, a color system conversion unit 6, a pigment concentration estimation unit 7, a fast Fourier transform unit 8, and a biological information extraction unit 9, which will be described later. Remember.

The auxiliary storage device 4 is, for example, an SSD (Solid State Drive) or HDD (Hard Disk Drive), and stores tables such as a conversion coefficient table TB1, an estimation formula coefficient table TB2, and a function coefficient table TB3.

The acquiring unit 5 acquires a plurality of time-series continuous image information in which a person's face is imaged for 10 seconds, for example, from an imaging device 10 such as a camera in RGB format. The color system converter 6 converts the color system of image information from RGB format to XYZ format or Lab format using a conversion formula.

The pigment concentration estimation unit 7 estimates the oxygenated blood pigment concentration _CHbO and the deoxygenated blood pigment concentration _CHbR from the XYZ format or Lab format color specification using an estimation formula. The fast Fourier transform unit 8 calculates the power spectrum density of at least one of the oxygenated blood pigment concentration _CHbO and the deoxygenated blood pigment concentration _CHbR by fast Fourier transform.

The biometric information extraction unit 9 extracts one or a plurality of biometric information from the power spectral density peaks PSD _HbO and PSD _HbR , and/or oxygenated hemoglobin concentration C _HbO and oxygenated hemoglobin concentration C _HbO and deoxygenated Biological information is extracted from the ratio to the sum of blood pigment concentration _CHbR . Here, the peak means that the power spectral density is local and maximum.

For example, the biometric information includes the percutaneous arterial oxygen saturation SpO2, which is the peak PSD _HbO of the heartbeat _- derived oscillatory component in the power spectral density of the oxygenated _hemoglobin concentration _CHbO . and the peak PSD _HbR of the vibration component derived from the heartbeat in the power spectral density of the deoxygenated hemoglobin concentration C _HbR .

Further, for example, the biological information includes a pulse rate HR, and the pulse rate HR is _{a peak PSD HbO , PSD} It is calculated from the frequencies f _pHbO and f _pHbR when it becomes _HbR .

Further, for example, the biological information includes tissue oxygen saturation _StO2 , and tissue oxygen saturation _StO2 is the sum of oxygenated hemoglobin concentration _CHbO and deoxygenated _hemoglobin concentration _CHbR . It is calculated by dividing by

Further, for example, the biological information includes a respiratory rate BR, and the respiratory rate BR is the time when the respiration-derived oscillation component in the power spectral density of the oxygenated blood pigment concentration _CHbO or the deoxygenated blood pigment concentration _CHbR peaks. is calculated from the frequencies f _bHbO and f _bHbR of .

Next, referring to FIG. 2, the conversion formula used by the color system conversion unit 6 for conversion, the estimation formula used by the pigment concentration estimation unit 7 for estimation, and the percutaneous arterial blood oxygen saturation Details of the predetermined function used when calculating the degree SpO ₂ will be described.

FIG. 2 shows information stored in the auxiliary storage device 4 according to this embodiment. The auxiliary storage device 4 stores a conversion coefficient table TB1, an estimation formula coefficient table TB2, and a function coefficient table TB3 that hold conversion formulas, estimation formulas, and predetermined function coefficients as information.

………（１）
パラメータＴ_１１，Ｔ_１２，Ｔ_１３，Ｔ_２１，Ｔ_２２，Ｔ_２３，Ｔ_３１，Ｔ_３２，Ｔ_３３は、変換係数テーブルＴＢ１に記憶されており、例えば国際照明委員会が定義する数値とする。 The color system conversion unit 6 converts the color system from the RGB format to the XYZ format using the conversion formula shown in formula (1) below. Conversion from the RGB format to the Lab format is performed using a conversion formula from the XYZ format to the Lab format defined by the International Commission on Illumination. Parameters required for conversion from RGB format to Lab format may be stored in the conversion coefficient table TB1.

…………(1)
The parameters T ₁₁ , T ₁₂ , T ₁₃ , T ₂₁ , T ₂₂ , T ₂₃ , T ₃₁ , T ₃₂ , T ₃₃ are stored in the conversion coefficient table TB1, and are defined by the International Commission on Illumination, for example. .

………（２） The pigment concentration estimating unit 7 estimates the oxygenated blood pigment concentration _CHbO and the deoxygenated blood pigment concentration _CHbR from the XYZ color specification using the estimation formula shown in Equation (2) below.

…………(2)

………（３）
なおメラニン色素濃度Ｃ_ｍは、酸素化血液色素濃度Ｃ_ＨｂＯ及び脱酸素化血液色素濃度Ｃ_ＨｂＲをＸＹＺ形式又はＬａｂ形式の表色から推定する際に算出される。 The pigment concentration estimating unit 7 may estimate the oxygenated blood pigment concentration _CHbO and the deoxygenated blood pigment concentration _CHbR from the Lab colorimetry using the estimation formula shown in the following equation (3).

…………(3)
Note that the melanin pigment concentration _Cm is calculated when estimating the oxygenated blood pigment concentration C _HbO and the deoxygenated blood pigment concentration C _HbR from the XYZ format or Lab format color specification.

Parameters _L11 , _L12 , _L13 , _L10 , _L21 , _L22 , _L23 , _L20 , _L31 , _L32 , _L33 , _L30 , _L41 , _L42 , _L40 , _L51 , L ₅₂ , L ₅₀ , L ₆₁ , L ₆₂ , and L ₆₀ are determined using light propagation Monte Carlo simulation.

Specifically, for example, 300 sets of combinations of explanatory variables (X, Y, Z) and objective variables (C _m , C _HbO , C _HbR ) are prepared, and multiple regression is performed using the multiple regression equation shown in Equation (2) Parameters _L11 , _L12 , _L13 , _L10 , _L21 , _L22 , _L23 , _L20 , _L31 , _L32 , _L33 , and _L30 are calculated by performing the analysis.

Specifically, for example, 300 sets of combinations of explanatory variables (a, b) and objective variables (C _m , C _HbO , C _HbR ) are prepared, and multiple regression analysis is performed using the multiple regression equation shown in equation (3). Parameters L ₄₁ , L ₄₂ , L ₄₀ , L ₅₁ , L ₅₂ , L ₅₀ , L ₆₁ , L ₆₂ , and L ₆₀ are calculated by performing

………（４）
パラメータＡ，Ｂ，Ｃは、予め測定したＳｐＯ_２とＰＳＤ_ＨｂＯ／ＰＳＤ_ＨｂＲとの組み合わせから決定されるものとする。なお（４）式で示したように、所定の関数は、適切な曲線で２つの値を関係づけるようなフィッテイング関数であって、例えば指数関数とする。 The biological information extraction unit ₉ calculates the percutaneous arterial blood oxygen saturation SpO2 using a predetermined function represented by the following equation (4). Note that % is vol. Equivalent to %.

…………(4)
Parameters A, B, and C shall be determined from a combination of previously measured SpO ₂ and PSD _HbO /PSD _HbR . As indicated by the formula (4), the predetermined function is a fitting function that relates two values with an appropriate curve, such as an exponential function.

………（５） The biological information extraction unit 9 calculates the tissue oxygen saturation _StO2 using the following equation (5).

…………(5)

Next, details of the fast Fourier transform performed by the fast Fourier transform unit 8 and biometric information extraction performed by the biometric information extraction unit 9 will be described with reference to FIGS. 4 to 6. FIG. FIG. 3 is a graph 15 showing oxygenated hemoglobin concentration over time according to the present invention. FIG. 4 is a graph 20 obtained by performing a fast Fourier transform on the graph 15 of FIG.

FIG. 5 is a graph 25 showing deoxygenated hemoglobin concentration over time according to the present invention. FIG. 6 is a graph 30 obtained by performing a fast Fourier transform on the graph 25 of FIG.

As shown in FIGS. 3 and 5, the oxygenated blood pigment concentration C _HbO and the deoxygenated blood pigment concentration C _HbR are, for example, XYZ format or Lab format color specifications within a certain range in chronologically continuous images. It is estimated from the average using equation (2) or (3).

The certain range in the image is, for example, the forehead portion of a person's face. Since the forehead portion is targeted, in the present embodiment, the pigment concentration estimation unit 7 estimates the oxygenated blood pigment concentration _CHbO and the deoxygenated blood pigment concentration _CHbR in the brain.

Also, as shown in FIGS. 4 and 6, the vibration component of the power spectrum density can be divided into a respiration-derived component and a heartbeat-derived component. A frequency band of less than 5 Hz in the power spectral density is defined as a respiration-derived band, and a vibration component in the respiration-derived band is called a respiration-derived component.

A frequency band of 0.7 to 4 Hz in the power spectral densities shown in FIGS. 4 and 6 is defined as a heartbeat-derived band, and a vibration component in the heartbeat-derived band is called a heartbeat-derived component. The respiration-derived band is particularly preferably a band of 1 Hz or less, which is a band in which the influence of noise is removed.

………（６）

………（７） Using the frequencies f _pHbO and f _pHbR shown in FIGS. 4 and 6, the pulse rate HR can be expressed by the following equation (6) or (7).

…………(6)

…………(7)

………（８）

………（９） The respiratory rate BR can be expressed by the following equation (8) or (9) using the frequencies f _bHbO and f _bHbR shown in FIGS.

…………(8)

…………(9)

Next, a biometric information calculation method according to the present embodiment will be described with reference to FIG. FIG. 7 is a flowchart showing the procedure of biometric information calculation processing according to the present embodiment. First, the acquiring unit 5 acquires image information in RGB format (S1).

Next, the color system converter 6 converts the color system of the acquired image information from RGB format to XYZ format or Lab format using a conversion formula (S2). Next, the pigment concentration estimation unit 7 estimates the oxygenated blood pigment concentration _CHbO and the deoxygenated blood pigment concentration _CHbR from the XYZ format or Lab format color specification using an estimation formula (S3).

Next, the fast Fourier transform unit 8 calculates the power spectrum density of at least one of the oxygenated blood pigment concentration _CHbO and the deoxygenated blood pigment concentration _CHbR (S4). Next, the biometric information extraction unit 9 extracts one or a plurality of biometric information from the power spectral density peaks PSD _HbO and PSD _HbR , or/and the oxygenated hemoglobin concentration C _HbO and the oxygenated hemoglobin concentration C _HbO and Biological information is extracted from the ratio to the sum of the deoxygenated blood pigment concentration _CHbR (S5).

As described above, according to the present embodiment, the color system is converted from the RGB format to the XYZ format or the Lab format color system, the dye density is estimated from the XYZ format or the Lab format color system, and the hue density is calculated. On the other hand, by performing fast Fourier transform, it becomes possible to calculate biometric information with a general-purpose configuration by diverting devices used for general camera functions such as smartphones.

Further, in the present embodiment, a plurality of biological information such as percutaneous arterial blood oxygen saturation _SpO2 , tissue oxygen saturation _StO2 , and respiratory rate BR can be calculated. By referring to a plurality of pieces of biological information, the user of the biological information calculation device 1 can distinguish the disease state.

For example, as a specific pathological condition, when organs such as the brain _are hypoxic, the respiratory rate BR increases and the percutaneous arterial blood oxygen saturation SpO2 and tissue oxygen saturation _StO2 decrease. In addition, when infected with coronavirus, if the infected person is young, the respiratory rate BR increases before the percutaneous arterial blood oxygen saturation _SpO2 decreases, and if the infected person is elderly, before the respiratory rate increases Percutaneous arterial oxygen saturation SpO ₂ increases.

_In addition, percutaneous arterial oxygen saturation SpO2 cannot be measured or is inaccurate when peripheral perfusion is greatly reduced, such as in hypotensive shock or when sympathetic nerve activity is rapidly increased. However, tissue oxygen saturation StO ₂ is accurate. Therefore, the user of the biological information calculation apparatus 1 can distinguish the disease state even when any value of the biological information becomes unmeasurable.

In the present embodiment, the acquiring unit 5, the color system converting unit 6, the pigment concentration estimating unit 7, the fast Fourier transforming unit 8, and the biological information extracting unit 9 are programs, but are not limited to these and may be logic circuits. .

The acquisition unit 5, color system conversion unit 6, dye density estimation unit 7, fast Fourier transform unit 8, biological information extraction unit 9, conversion coefficient table TB1, estimation formula coefficient table TB2, and function coefficient table TB3 are integrated into one device. It may not be implemented and may be implemented in a distributed manner in a plurality of devices connected via a network.

In the present embodiment, a person's face is acquired as an image, but the present invention is not limited to this, and exposed skin may be acquired as an image. An image of a person's palm may be acquired in addition to the person's face, or two images of the same person's face and palm may be acquired. Examples of exposed skin other than the face and palms include the backs of hands and feet.

In addition, the target does not have to be limited to people. When a palm image is acquired, the biometric information calculation device ₁ calculates percutaneous arterial blood oxygen saturation SpO2 and tissue oxygen saturation _StO2 of peripheral tissues.

When biometric information is calculated for a plurality of locations on the same person, for example, the percutaneous arterial blood oxygen saturation _SpO2 calculated based on the image information of the palm is the same normal value as normal, and the image information of the forehead is the normal value. It is possible to discriminate a case where the tissue oxygen saturation _StO2 calculated based on the above is an abnormal value different from the normal value.

If the percutaneous arterial oxygen saturation SpO ₂ in one part is normal and the tissue oxygen saturation StO ₂ in another part is abnormal in the same person, the decrease in blood circulation to another part is differentiated. It becomes possible to Possible causes of decreased blood circulation include vascular stenosis, vascular occlusion, bleeding, and pressure disorders.

……（１０）

……（１１） In the present embodiment, the order of the estimation formula is 1 as in formula (2) or (3), but the estimation formula is not limited to this. The above polynomial may be used. Note that the order of the estimation formula may be 3.

......(10)

......(11)

Even when polynomials of degree 2 or higher are used, parameters _N10 , _N11 , _N12 , _N13 , _N14 , _N15 , _N16 , _N17 , _N18 , _N19 , _N20 , _N21 , _N22 , _N23 , _N24 , _N25 , _N26 , _N27 , _N28 , _N29 , _N30 , _N31 , _N32 , _N33 , _N34 , _N35 , _N36 , _N37 , _N38 , _N39 , _N40 , _N41 , _N42 , _N43 , _N44 , _N45 , _N50 , _N51 , _N52 , _N53 , _N54 , _N55 , _N60 , _N61 , _N62 , _N63 , N ₆₄ and N ₆₅ are determined using light propagation Monte Carlo simulations. By increasing the order of the estimation formula, it becomes possible to more accurately estimate the oxygenated blood pigment concentration _CHbO and the deoxygenated blood pigment concentration _CHbR from the XYZ format or Lab format color specification.

In the present embodiment, the case where the imaging device 10 is included in the smart phone is taken as an example, but the imaging device 10 is not limited to this, and may be a web camera or the like. Note that the web camera may be equipped with a ring LED or the like.

………（１２）
パラメータＤ，Ｅ，Ｆ，Ｇは、予め測定したＳｐＯ_２とＰＳＤ_ＨｂＯ／ＰＳＤ_ＨｂＲとの組み合わせから決定されるものとする。 In the present embodiment, the fitting function for calculating the percutaneous arterial blood oxygen saturation SpO ₂ is an exponential function such as the equation (4) as an example, but is not limited to this, the following equation (12) It may be a quadratic function such as

…………(12)
Parameters D, E, F, G shall be determined from a combination of previously measured SpO ₂ and PSD _HbO /PSD _HbR .

1... Biological information calculation device, 2... Central processing unit, 3... Main storage device, 4... Auxiliary storage device, 5... Acquisition unit, 6... Color system conversion unit, 7... Pigment concentration estimation Section 8: Fast Fourier transform section, 9: Biological information extraction section, 10: Imaging device.

Claims (5)

one or more of the target to be measured from the image information of the target to be measured which is a living bodyIncluding percutaneous arterial oxygen saturationA biological information calculation device for calculating biological information,
an acquisition unit that acquires the image information in RGB format;
a color system conversion unit that converts the color system of the acquired image information from RGB format to XYZ format or Lab format using a conversion formula;
a pigment concentration estimating unit that estimates the oxygenated blood pigment concentration and the deoxygenated blood pigment concentration from the color specification in the XYZ format or the Lab format using an estimation formula;
a fast Fourier transform unit for calculating a power spectral density of at least one of the oxygenated hemoglobin concentration and the deoxygenated hemoglobin concentration;
Extracting the one or more biometric information from the power spectral density peakrawa body information extraction unit;
equipped with,
The percutaneous arterial blood oxygen saturation is the peak of the heartbeat-derived oscillating component in the power spectral density of the oxygenated hemoglobin concentration and the heartbeat-derived oscillating component in the power spectral density of the deoxygenated hemoglobin concentration. The output is obtained by inputting the ratio of the peak and the
Biological information calculation device. The biological information further includes a pulse rate,
2. The living body according to claim 1 , wherein the pulse rate is calculated from a frequency at which a heartbeat-derived vibration component in the power spectral density of the oxygenated hemoglobin concentration or the deoxygenated hemoglobin concentration peaks. Information calculation device. the biological information further includes tissue oxygen saturation;
3. The biological information calculation apparatus according to claim 2 , wherein said tissue oxygen saturation is calculated by dividing said oxygenated hemoglobin concentration by the sum of said oxygenated hemoglobin concentration and said deoxygenated hemoglobin concentration. The biological information further includes respiratory rate,
4. The living body according to claim 3 , wherein the respiratory rate is calculated from a frequency at which a respiration-derived vibration component in the power spectral density of the oxygenated hemoglobin concentration or the deoxygenated hemoglobin concentration peaks. Information calculation device. one or more of the target to be measured from the image information of the target to be measured which is a living bodyIncluding percutaneous arterial oxygen saturationA biological information calculation method performed by a biological information calculation device for calculating biological information,
a first step of obtaining the image information in RGB format;
a second step of converting the color specification of the acquired image information from RGB format to XYZ format or Lab format using a conversion formula;
a third step of estimating the oxygenated blood pigment concentration and the deoxygenated blood pigment concentration from the color specification in the XYZ format or the Lab format using an estimation formula;
a fourth step of calculating a power spectral density of at least one of said oxygenated hemoglobin concentration and said deoxygenated hemoglobin concentration;
Extracting the one or more biometric information from the power spectral density peakfirst5 steps;
equipped with,
The percutaneous arterial blood oxygen saturation is the peak of the heartbeat-derived oscillating component in the power spectral density of the oxygenated hemoglobin concentration and the heartbeat-derived oscillating component in the power spectral density of the deoxygenated hemoglobin concentration. The output is obtained by inputting the ratio of the peak and the
Biometric information calculation method.

Images