KR102411622B1 - Method and apparatus for non-contact oxygen saturation measurement - Google Patents

Abstract

The present invention relates to a method for measuring oxygen saturation, and more particularly, to a method and apparatus for measuring coral saturation in an image captured by an RGB camera. According to an embodiment of the present invention, the non-contact oxygen saturation measurement method and apparatus can measure oxygen saturation through a single basic RGB camera without contact with the body.

Description

Method and apparatus for non-contact oxygen saturation measurement

The present invention relates to a method for measuring oxygen saturation, and more particularly, to a method and apparatus for measuring oxygen saturation in an image captured by an RGB camera.

Due to changes in lifestyle, numerous diseases that were not common in the past have begun to appear. Among them, heart disease can be said to be one of the representative diseases of modern society, and the representative cause of heart disease is stress. Tension and anger caused by stress excite the sympathetic nerves of the brain to increase the heart rate or intensity, and increase the amount of oxygen consumed by the heart, while the coronary arteries are rather constricted, resulting in inadequate blood circulation. Eventually, the heart may develop anemia, which exacerbates high blood pressure. Therefore, stress management is essential.

However, for such stress management, it is not possible to measure the level of stress in any place, and it is inconvenient to visit a place equipped with professional stress measuring equipment. In addition, to measure stress, it is necessary to pay a lot of money every time it is measured, and there are restrictions on movement by attaching several measuring sensors to the body.

In the conventional technique for measuring oxygen saturation, a sensor is used to measure oxygen saturation, or a light source of a measurement wavelength and a camera for measuring it are used to measure the oxygen saturation, so that various equipments are used.

1. Republic of Korea Patent Registration No. 10-1752560 “A method of measuring oxygen saturation using an image and a computer-readable recording medium recording a method of measuring oxygen saturation using an image” (Registration date: June 23, 2017)

The present invention provides a non-contact oxygen saturation measurement method and apparatus for measuring oxygen saturation in an image obtained by measuring a user's face using an RGB camera.

According to one aspect of the present invention, there is provided a non-contact oxygen saturation measuring apparatus.

A non-contact oxygen saturation measuring apparatus according to an embodiment of the present invention includes an image acquisition unit that acquires a face image through an RGB camera, a data converter that converts RGB color of a skin region from the acquired face image to YCgCr color, and the converted data It may include a data processing unit for filtering the data in a preset band, extracting respective data from the data of the filtered Cg and Cr color regions, and an oxygen saturation measuring unit measuring oxygen saturation based on the extracted data.

According to one aspect of the present invention, a non-contact oxygen saturation measurement method and a computer program for executing the same are provided.

The non-contact oxygen saturation measurement method according to an embodiment of the present invention includes acquiring a face image through an RGB camera, converting RGB color of a skin region into YCgCr color in the acquired face image, and converting the converted data into a preset band It may include the steps of filtering as shown, extracting respective data from the data of the filtered Cg and Cr color regions, and measuring oxygen saturation based on the extracted data.

According to an embodiment of the present invention, oxygen saturation can be measured through a single basic RGB camera without contact with the body.

In addition, according to an embodiment of the present invention, , can be obtained in a natural environment in a non-contact manner, regardless of the surrounding situation and space, so that measurement can be performed without feeling of rejection.

1 is a view for explaining a non-contact oxygen saturation measuring apparatus according to an embodiment of the present invention.
2 is a view for explaining a non-contact oxygen saturation measurement method according to an embodiment of the present invention.

Since the present invention can have various changes and can have various embodiments, specific embodiments are illustrated in the drawings and will be described in detail through detailed description. However, this is not intended to limit the present invention to specific embodiments, it should be understood to include all modifications, equivalents and substitutes included in the spirit and scope of the present invention. In describing the present invention, if it is determined that a detailed description of a related known technology may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted. Also, the expressions "a" and "a", "a" and "a", as used in this specification and claims, should generally be construed to mean "one or more" unless stated otherwise.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. do it with

1 is a view for explaining a non-contact oxygen saturation measuring apparatus according to an embodiment of the present invention.

Referring to FIG. 1 , the non-contact oxygen saturation measurement apparatus 100 includes an image acquisition unit 110 , a data conversion unit 120 , a data processing unit 130 , and an oxygen saturation measurement unit 140 .

The image acquisition unit 110 may include a camera for photographing a body part to be observed, such as the user's face and fingers. The image acquisition unit 110 may acquire a face image through the RGB camera at a location distant from the user.

The data converter 120 may obtain a face image of the subject using the RGB camera and extract a skin color region from the face image obtained through the RGB camera.

The data converter 120 may acquire a PPG signal in real time through a camera using a remote-PPG (rPPG) technology. The PPG may be a heartbeat signal measured through the amount of light reflected by irradiating light on the skin surface after contact with the sensor and the skin.

The data converter 120 may detect a face in an image frame of RGB image data at a rate of 30 frames per second.

The data converter 120 may detect a region of interest (ROI) from a face region for each frame by using skin segmentation in the detected face image.

The data converter 120 may detect a skin region in the detected face as an ROI in order to see a change in skin color by tracking the detected face in successive image frames using a Kernelized Correlation Filter (KCF) tracker. To this end, the data converter 120 may detect only the skin region using the YCbCr color region in order to detect only the skin color in the face region.

That is, when extracting the skin region as an ROI, the data converter 120 may convert the RGB color region into the YCbCr color region. Thereafter, the data converter 120 may detect a skin region corresponding to the skin color selected from the face image as an ROI.

According to an embodiment, the data converter 120 may separate the RGB data of the face region into the YCbCr color region and then detect a corresponding range on the Cb-Cr plane as the skin region. The data converter 120 may convert an image frame in the RGB color space into the YCbCr color space and then perform skin pixel clustering in the Cb-Cr plane to use skin pixel characteristics. At this time, the data conversion unit 120 detects the skin region in the acquired face image using the characteristic of forming a close cluster near the center of the plane when expressed in the Cb-Cr plane excluding the illumination component (Y) in the YCbCr color region. and skin color information may be detected from pixel values of the corresponding region.

The data converter 120 may convert the RGB color region of the detected ROI region into a YCgCr color region to obtain oxygen saturation.

After determining the skin region, the data converter 120 may extract a PPG signal for measuring oxygen saturation by using a color value of the skin region. To this end, the data converter 120 may extract the PPG signal by using the R color value, the G color value, and the B color value of the ROI.

The data converter 120 may convert the RGB color region into the YCgCr color region in the extracted PPG signal. That is, the data converter 120 may convert the RGB color region of the detected ROI region into the YCgCr color region.

The data converter 120 may convert the RGB color region into the YCgCr color region in the skin region because the absorbances of the two hemoglobins are opposite to each other at the green wavelength and the red wavelength.

The data conversion unit 120 converts the RGB color area of the PPG signal extracted from the face image into the YCgCr color area to use the different extinction coefficients of deoxygenated hemoglobin and oxygenated hemoglobin at two different wavelengths to measure oxygen saturation, and Cg, The Cr signal can be extracted.

To this end, the data conversion unit 120 may convert the RGB color area into the YCgCr color area through Equations 1 and 2 below.

In this case, R’,G’,B’ is a value converted from the R,G,B value to a value between 0 and 1.

는

파장에서 매질의 흡광계수, C는 매질의 농도, l은 매질 내에서 빛의 이동거리를 의미한다.In this case, I is the reflected light,

Is

At wavelength, the extinction coefficient of the medium, C is the concentration of the medium, and l is the movement distance of light in the medium.

In this case, the data processing unit 130 may remove the effect of light brightness values from the Cg and Cr signals in which green and red wavelength light are selected and the RGB color space is extracted from the YCgCr color space.

The data processing unit 130 may filter the Cg and Cr signals from which the light intensity has been removed in a preset band band to obtain only a pulsation waveform (AC) signal related to a heartbeat.

The data processing unit 130 may extract a signal of a frequency band from which noise has been removed by performing band pass filtering on the skin color information. Bandpass filtering can be performed in a band of 0.7Hz to 3Hz.

The data processing unit 130 may apply a sliding window to the signal extracted by band pass filtering. According to an embodiment, the sliding window used in the data processing unit 130 may be a 10-second sliding window.

The data processing unit 130 may calculate a peak value and a valley value by extracting each peak and valley from the pulsation waveform (AC) signal of the Cg and Cr signals to which the sliding window is applied.

The data processing unit 130 may obtain a peak value and a valley value from the Cg and Cr signals, respectively, through Equation 3 below.

[Equation 3]

는 Cr 신호에서 밸리 값이고,

는 Cr 신호에서 피크 값이고,

는 Cg 신호에서 밸리 값이고,

는 Cr 신호에서 피크 값이다.

와

는 Cg, Cr 파장에서의 탈산소화 헤모글로빈의 흡광계수와 농도로 탈산소화 헤모글로빈으로 인한 흡광도이고,

와

는 Cg, Cr 파장에서의 산소화 헤모글로빈의 흡광계수와 농도로 산소화 헤모글로빈으로 인한 흡광도이다.At this time,

is the valley value in the Cr signal,

is the peak value in the Cr signal,

is the valley value in the Cg signal,

is the peak value in the Cr signal.

Wow

is the extinction coefficient and concentration of deoxygenated hemoglobin at Cg and Cr wavelengths, and is the absorbance due to deoxygenated hemoglobin,

Wow

is the extinction coefficient and concentration of oxygenated hemoglobin at Cg and Cr wavelengths, and is the absorbance due to oxygenated hemoglobin.

만큼 많아져 빛의 흡수가 많아서 반사가 적게 일어나는 부분이다. 피크 부분은 맥동파형(AC) 부분의 진폭이 최대가 되는 점으로 심장의 이완으로 인해 혈액량이 기본 l만큼만 있는 부분이다. Here, the valley part is the point where the amplitude of the pulsation waveform (AC) part is the minimum, and the blood volume is changed from the basic l due to the contraction of the heart.

It is the part where light is absorbed more and less reflection occurs. The peak part is the point where the amplitude of the pulsation waveform (AC) part is the maximum, and it is the part where the blood volume is only as basic as l due to the relaxation of the heart.

The data processing unit 130 may calculate a ratio of values due to changes in blood flow by dividing the valley values obtained from Cg and Cr by the peak values.

The data processing unit 130 divides the valley value by the peak value to leave only the effect when the blood fluctuates. Since oxygen is transported when the heart contracts and blood circulates, the data processing unit 130 may calculate the oxygen saturation by considering only this part.

The data processing unit 130 may obtain a ratio of a valley to a peak for two wavelengths of Cg and Cr in the signals of Cg and Cr through Equation 4 below.

The data processing unit 130 may perform low pass filtering on the ratio of values resulting from the change in blood flow. Low-pass filtering may be performed with a bandwidth of 0.05 Hz or less.

The data processing unit 130 may calculate a value obtained by dividing the ratio of Cr filtered by the low-pass filtering by the ratio of Cg.

The data processing unit 130 may remove the dependence on the blood volume by dividing the valley and peak ratios of Cg and Cr obtained in Equation (4).

The oxygen saturation measurement unit 140 may measure the oxygen saturation level based on the data extracted from the data processing unit 130 .

Since the oxygen saturation measuring unit 140 indicates the ratio of oxygenated hemoglobin to deoxygenated hemoglobin at the wavelengths of Cg and Cr, oxygen saturation can be calculated according to Equation 5 below.

The oxygen saturation measurement unit 140 converts the R value obtained here and the reference oxygen saturation value measured by the sensor to the R value through linear regression.

)를 측정할 수 있다.That is, the oxygen saturation measurement unit 140 divides the filtered Cr ratio by the Cg ratio, and then divides this value by the remote oxygen saturation (

) can be measured.

2 is a view for explaining a non-contact oxygen saturation measurement method according to an embodiment of the present invention.

Referring to FIG. 2 , in the non-contact oxygen saturation measurement method, a 30Hz RGB image may be input through a camera in step S10.

In step S20, the non-contact oxygen saturation measuring apparatus may detect the ROI of the face region from the 30 Hz RGB image.

In step S30, the non-contact oxygen saturation measuring apparatus may calculate an average frame of each RGB of the detected ROI.

In step S40, the non-contact oxygen saturation measuring apparatus may convert the RGB color region of the ROI into the YCgCr color region.

In step S50, the non-contact oxygen saturation measuring apparatus may perform band-pass filtering on the converted Cg and Cr signals with a frequency between 0.7 Hz and 3 Hz.

In step S60 , the non-contact oxygen saturation measuring apparatus may apply a 10-second sliding window to the band pass filtering result.

In step S70 , the non-contact oxygen saturation measuring apparatus may extract peak values and valley values of Cg and Cr wavelengths in the 10-second sliding window section, respectively.

In step S80 , the non-contact oxygen saturation measuring apparatus may calculate a ratio of values due to a change in blood flow by dividing each valley value of the extracted Cg and Cr wavelengths by a peak value.

In step S90 , the non-contact oxygen saturation measuring apparatus may perform low-pass filtering on each change rate of Cg and Cr with a frequency of 0.05 Hz.

In step S100 , the non-contact oxygen saturation measuring apparatus may divide the low-pass filtered Cr ratio by the Cg ratio.

In step S110 , the non-contact oxygen saturation measuring apparatus may perform linear regression based on a value obtained by dividing the Cr ratio by the Cg ratio and a reference value measured by the oxygen saturation sensor.

In step S120 , the non-contact oxygen saturation measuring apparatus may measure oxygen saturation through linear regression.

The above-described non-contact oxygen saturation measurement method may be implemented as a computer-readable code on a computer-readable medium. The computer-readable recording medium may be, for example, a removable recording medium (CD, DVD, Blu-ray disk, USB storage device, removable hard disk) or a fixed recording medium (ROM, RAM, computer-equipped hard disk). can The computer program recorded on the computer-readable recording medium may be transmitted to another computing device through a network such as the Internet and installed in the other computing device, thereby being used in the other computing device.

In the above, even though it has been described that all components constituting the embodiment of the present invention are combined or operated as one, the present invention is not necessarily limited to this embodiment. That is, within the scope of the object of the present invention, all the components may operate by selectively combining one or more.

Although acts are shown in a particular order in the drawings, it should not be understood that the acts must be performed in the specific order or sequential order shown, or that all illustrated acts must be performed to obtain a desired result. In certain circumstances, multitasking and parallel processing may be advantageous. Moreover, the separation of the various components in the embodiments described above should not be construed as necessarily requiring such separation, and the described program components and systems may generally be integrated together into a single software product or packaged into multiple software products. It should be understood that there is

So far, the present invention has been focused on the embodiments thereof. Those of ordinary skill in the art to which the present invention pertains will understand that the present invention can be implemented in a modified form without departing from the essential characteristics of the present invention. Therefore, the disclosed embodiments should be considered in an illustrative rather than a restrictive sense. The scope of the present invention is indicated in the claims rather than the foregoing description, and all differences within the scope equivalent thereto should be construed as being included in the present invention.

100: non-contact oxygen saturation measurement device
110: image acquisition unit
120: data conversion unit
130: data processing unit
140: oxygen saturation measurement unit

Claims (11)

an image acquisition unit for acquiring a face image through an RGB camera;
a data conversion unit that converts the RGB color area of the skin area into the YCgCr color area in the acquired face image;
a data processing unit that filters the converted data in a preset band and extracts a ratio of values due to changes in blood flow volume from the filtered Cg and Cr color region data; and
and an oxygen saturation measuring unit for measuring oxygen saturation based on the extracted ratio
Non-contact oxygen saturation measurement device.
According to claim 1,
The image acquisition unit
Acquiring 30Hz RGB image through RGB camera
Non-contact oxygen saturation measurement device.
3. The method of claim 2,
The data conversion unit
The skin color is detected by converting the RGB color gamut into the YCbCr color gamut in the acquired RGB image,
detecting a skin region corresponding to the skin color selected in the face image as an ROI;
Converting the RGB color area of the detected ROI area to the YCgCr color area
Non-contact oxygen saturation measurement device.
4. The method of claim 3,
The data processing unit
performing band pass filtering on the converted Cg and Cr signals,
Extracting the peak values and valley values of the band-pass filtered Cg and Cr signals in the sliding window section,
Calculate the ratio of values due to changes in blood flow by dividing the extracted Cg and Cr signals by the peak value, respectively,
Low-pass filtering of each change ratio of Cg and Cr
Non-contact oxygen saturation measurement device.
5. The method of claim 4,
The oxygen saturation measuring unit
Oxygen saturation is measured through linear regression based on the value obtained by dividing the Cr ratio by the Cg ratio and the reference value measured by the oxygen saturation sensor.
Non-contact oxygen saturation measurement device.

acquiring, by an image acquisition unit, a face image through an RGB camera;
converting the RGB color region of the skin region into the YCgCr color region in the face image obtained by the data converter;
filtering, by the data processing unit, the converted data according to a preset band, and extracting a ratio of values due to changes in blood flow from the filtered data of the Cg and Cr color regions; and
comprising the step of measuring oxygen saturation based on the extracted ratio by an oxygen saturation measuring unit
Non-contact oxygen saturation measurement method
7. The method of claim 6,
The step of acquiring the face image is
Acquiring an RGB image of 30 Hz through an RGB camera.
Non-contact oxygen saturation measurement method.
8. The method of claim 7,
The step of converting the RGB color gamut to the YCgCr color gamut includes:
detecting a skin color by converting the RGB color gamut into a YCbCr color gamut in the obtained RGB image;
detecting a skin region corresponding to a skin color selected from the face image as an ROI; and
converting the RGB color region of the detected ROI region into a YCgCr color region
Non-contact oxygen saturation measurement method.
9. The method of claim 8,
The step of extracting the data is
performing band pass filtering on the converted Cg and Cr signals;
extracting a peak value and a valley value of each of the band-pass filtered Cg and Cr signals in the sliding window section;
calculating a ratio of values due to a change in blood flow by dividing each of the extracted Cg and Cr signals by a peak value; and
Including the step of low-pass filtering each change rate of Cg and Cr
Non-contact oxygen saturation measurement method.
9. The method of claim 8,
Measuring the oxygen saturation
Measuring oxygen saturation through linear regression based on a value obtained by dividing the Cr ratio by the Cg ratio and a reference value measured by an oxygen saturation sensor
Non-contact oxygen saturation measurement method.
A computer program for executing the non-contact oxygen saturation measurement method according to any one of claims 6 to 10 and recorded in a computer-readable recording medium.

Images