KR102123121B1 - Blood pressure monitoring method that can identify the user and blood pressure monitoring system that can identify the user - Google Patents

Abstract

The present invention relates to a blood pressure monitoring method capable of identifying a user's identity, and more specifically, as a blood pressure monitoring method, (1) recognizing a user's face in an image sequence collected through a camera; (2) authenticating the user's identity by comparing the face recognized in step (1) with a server database; (3) generating coordinates for measuring heart rate based on the feature points of the face from the facial data recognized in step (1); (4) tracking the coordinates generated in step (3), and measuring the heart rate in the user's face; (5) measuring a user's heart rate at the body terminal; And (6) calculating blood pressure using a time difference between the facial heart rate measured in step (4) and the body-end heart rate measured in step (5).
In addition, the present invention relates to a blood pressure monitoring system capable of identifying a user's identity, and more specifically, as a blood pressure monitoring system, a facial recognition unit for recognizing a user's face in an image sequence collected through a camera; An identity authentication unit authenticating the user's identity by comparing the face recognized by the facial recognition unit with a database of a server; A coordinate generator for generating coordinates for measuring a heart rate based on the facial feature points from the facial data recognized by the facial recognition unit; A facial heart rate measuring unit tracking the coordinates generated by the coordinate generating unit and measuring a heart rate in the user's facial unit; A body-end heart rate measuring unit measuring a user's heart rate at the body end; And a blood pressure measuring unit that calculates blood pressure using a time difference between the facial heart rate measured by the facial heart rate measuring unit and the body terminal heart rate measured by the body terminal heart rate measuring unit.
According to the blood pressure monitoring method and system capable of identifying the user's identity proposed in the present invention, the user's identity can be identified, and blood pressure is measured through the camera without a cuff or ECG sensor, making it easier and simpler to measure the user's blood pressure. Collect and process them.

Description

BLOOD PRESSURE MONITORING METHOD THAT CAN IDENTIFY THE USER AND BLOOD PRESSURE MONITORING SYSTEM THAT CAN IDENTIFY THE USER}

The present invention relates to a blood pressure monitoring method and system, and more particularly, to a real-time blood pressure monitoring method and system capable of identifying a user's identity.

The patient monitoring system is a system for continuous and intensive monitoring of a patient's health condition, and a patient side monitor is used as a main medical device for this purpose. The patient monitoring system has the effect of reducing the manpower, effort, and burden involved in monitoring the patient, and enables quick response of medical personnel through quick patient status identification. The basic function of the patient monitoring and monitoring system is to collect, process, and analyze the bio signals of the patient from various sensors attached to the patient.

1 is a view showing a conventional patient monitoring system. As illustrated in FIG. 1, in general, in the case of a patient monitoring system, it is targeted at patients hospitalized. In the case of such a monitoring system, it is not suitable for health care for a discharged patient or a system for a general public who has never visited a hospital.

The current medical system focuses on prevention rather than treatment of diseases. To prevent diseases, accurate biodata measurement of patients and rapid care based on the biodata are required. In accordance with the changes in the medical system, medical devices are increasingly miniaturized and simplified for accurate biodata measurement of patients, and wearable devices or implantable devices are being developed.

2 is a view showing a miniaturized and simplified wearable biosignal monitoring system. As shown in FIG. 2, the miniaturized and simplified monitoring devices have the advantage of easily measuring patient data without the patient having to visit the hospital, and have the advantage of measuring data for almost 24 hours. However, there is also a need to supplement wearable devices or implantable devices currently being developed.

Wearable biosignal monitoring devices are said to have been simplified and miniaturized, but they still have difficulty in daily life while wearing them, and also have the disadvantage that it is difficult to accurately identify the identity of the data measured by wearing the devices. In the case of an implantable device in the body, it is difficult to select a material as it needs to be inserted into the body, and there is a problem in consideration of compatibility with the human body. In addition, there is a hassle that needs to be replaced periodically due to battery problems.

Due to these problems, research is currently being conducted on a method of measuring a biosignal without any manipulation so that a patient feels comfortable rather than attaching or inserting it on the body. Typical examples are a heart rate measurement using a smartphone camera or an oxygen saturation measurement. Biometric data measurement using a camera image allows a patient to measure biometric data without any manipulation. Using this, the patient can be monitored in real time and can respond immediately to the patient's acute disease. In addition, it is possible to continuously and continuously measure biometric data, which enables a more accurate understanding of the patient's condition and can be a great help in preventing diseases that may occur to the patient.

In order to represent such a situation, there is a system that provides information about health to users without limitation in time and place by fusion with communication technology such as the Internet or mobile under the name of U-healthcare system. In the past, the healthcare system was expanded to the area of managing the health of users on a daily basis depending on the individual's condition, if it was intended to treat a specific patient. As such, the U-Healthcare market has grown rapidly every year as the number of users expands, and the healthcare system penetrates deeply into real life with IoT technology that exchanges information by connecting the Internet between objects together with a smartphone.

On the other hand, as prior art related to the present invention, Republic of Korea Patent Publication No. 10-2013-0118512 (invention name: patient condition monitoring system through face recognition and a method for providing a patient condition monitoring service using the same) and Republic of Korea Patent Publication 10-2015-0039113 (invention name: a method for processing content based on a biosignal, and a device accordingly) has been disclosed.

The present invention has been proposed to solve the above-described problems of the previously proposed methods, so that the user's identity can be identified, and by measuring blood pressure through the camera without a cuff or ECG sensor, the user's blood pressure value is more easily and conveniently It is an object of the present invention to provide a blood pressure monitoring method and system capable of identifying a user's identity to collect and process data.

A blood pressure monitoring method capable of identifying the user's identity according to the features of the present invention for achieving the above object,

As a blood pressure monitoring method,

(1) recognizing a user's face in an image sequence collected through a camera;

(2) authenticating the user's identity by comparing the face recognized in step (1) with a server database;

(3) generating coordinates for measuring heart rate based on the feature points of the face from the facial data recognized in step (1);

(4) tracking the coordinates generated in step (3), and measuring the heart rate in the user's face;

(5) measuring a user's heart rate at the body terminal; And

And (6) calculating the blood pressure using the time difference between the facial heart rate measured in step (4) and the body-end heart rate measured in step (5).

Preferably, in step (1),

A webcam (Webcam) can be used as the camera for collecting an image sequence.

Preferably, the step (1),

(1-1) collecting images through a camera; And

(1-2) may include the step of selecting an image of the face portion from the image collected in step (1-1).

More preferably, in step (1-2),

The face common shape information (Eigen face) is formed using PCA (Principal Component Analysis), and among the images collected in step (1-1), a portion having a shape similar to the common face shape information is formed. It can be selected as an image of the human face.

Preferably, the step (2),

(2-1) extracting the feature points of the face recognized in step (1);

(2-2) generating a bunch graph from geometric information of the feature points extracted in step (2-1) through an EBGM (Elastic Bunch graph matching) method; And

(2-3) Using the bunch graph generated in step (2-2), it may include the step of authenticating the user's identity.

More preferably, in step (2-2),

The Gabor coefficients of the feature points are obtained through a Gabor wavelet function, the Gabor coefficients of each feature point are grouped to generate a bunch, and the bunch graphs of the entire facial feature points are collected to generate the bunch graph. have.

More preferably, in step (2-3),

A user's identity can be authenticated as a person corresponding to the most similar batch graph by comparing the similarity between the recognized batch graph of the user's face and the batch graphs existing in the database (DB).

Even more preferably, in step (2-3),

Similarity can be compared using non-rigid matching among point matching techniques.

Even more preferably, in step (2-3) above,

To use the non-rigid matching technique, a Topology Preserving Relaxation Labeling (TPRL) algorithm can be used.

Preferably, in step (3),

Based on the extracted feature points, coordinates for measuring heart rate may be generated in a portion corresponding to a user's cheek.

Preferably, the step (4),

(4-1) extracting independent factors using an independent component analysis (ICA) algorithm from the image of the facial part selected in step (1) using BSS (Blind Source Separation);

(4-2) From the independent factor extracted in the step (4-1), the optical volume pulse wave (Photoplethysmography: PPG) which is a signal to measure the heart rate state by measuring the blood flow rate in the blood vessel using the optical properties of the biological tissue Extracting a signal; And

(4-3) measuring the heart rate of the user's face from the light volume pulse wave (PPG) signal extracted in step (4-2).

More preferably, in step (4-1),

Three independent factors can be extracted from the RGB channel of the camera.

More preferably, in step (4-1),

Three independent factors can be extracted from the camera's COG (Cyan, Orange, Green) channels.

Preferably,

The step (4),

It is possible to measure the heart rate at the user's face using the color change according to the optical characteristics of the coordinates for measuring the heart rate generated in step (3).

Preferably, in step (5),

The user's heart rate may be measured at the body terminal through a wearable device worn on the user's body terminal.

In addition, according to the features of the present invention for achieving the above object, the blood pressure monitoring system capable of identifying the user's identity,

Blood pressure monitoring system,

A facial recognition unit that recognizes a user's face from the image sequence collected through the camera;

An identity authentication unit authenticating the user's identity by comparing the face recognized by the facial recognition unit with a database of a server;

A coordinate generator for generating coordinates for measuring heart rate based on facial feature points recognized from the facial data recognized by the facial recognition unit;

A facial heart rate measuring unit tracking the coordinates generated by the coordinate generating unit and measuring a heart rate in the user's facial unit;

A body-end heart rate measuring unit measuring a user's heart rate at the body end; And

It comprises a blood pressure measuring unit for calculating blood pressure by using the time difference between the facial heart rate measured by the facial heart rate measuring unit and the body terminal heart rate measured by the body terminal heart rate measuring unit.

Preferably, the facial recognition unit,

An image collection module for collecting images through a camera; And

It may include a facial image sorting module for sorting the image of the face portion from the image collected in the image collection module.

Preferably, the identity authentication unit,

A feature extraction module that extracts feature points of the face recognized by the face recognition unit;

A bunch graph generation module that generates a bunch graph from geometric information of feature points extracted from the feature point extraction module through an EBGM (Elastic Bunch graph matching) method; And

It may include an authentication module for authenticating the user's identity using the batch graph generated by the batch graph generation module.

More preferably, the authentication module,

By comparing the similarity between the recognized batch graph of the user's face and the batch graphs existing in the database (DB), the user's identity is authenticated as the person corresponding to the highest similarity batch graph.

Similarity can be compared using non-rigid matching among point matching techniques.

Even more preferably, the authentication module,

To use the non-rigid matching technique, a Topology Preserving Relaxation Labeling (TPRL) algorithm can be used.

According to the blood pressure monitoring method and system capable of identifying the user's identity proposed in the present invention, the user's identity can be identified, and blood pressure is measured through the camera without a cuff or ECG sensor, making it easier and simpler to measure the user's blood pressure. Collect and process them.

1 is a view showing a conventional patient monitoring system.
2 is a view showing a miniaturized and simplified wearable biosignal monitoring system.
3 is a diagram illustrating a flow of a blood pressure monitoring method capable of identifying a user's identity according to an embodiment of the present invention.
4 is a diagram illustrating a blood pressure monitoring method capable of identifying a user's identity according to an embodiment of the present invention.
5 is a diagram showing the detailed flow of step S100 in a blood pressure monitoring method capable of identifying a user's identity according to an embodiment of the present invention.
6 is a view for explaining a step of selecting an image of the face, in the blood pressure monitoring method capable of identifying the user's identity according to an embodiment of the present invention.
7 is a view showing a detailed flow of step S200 in a blood pressure monitoring method capable of identifying a user's identity according to an embodiment of the present invention.
8 is a view illustrating a step S200 in a blood pressure monitoring method capable of identifying a user's identity according to an embodiment of the present invention.
9 is a view illustrating a process of extracting three independent factors from an RGB channel of a camera in a blood pressure monitoring method capable of identifying a user's identity according to an embodiment of the present invention.
10 is a view illustrating a process of measuring a heart rate in a face part according to a face part in a blood pressure monitoring method capable of identifying a user's identity according to an embodiment of the present invention.
11 is a view illustrating a step S600 of a blood pressure monitoring method capable of identifying a user's identity according to an embodiment of the present invention.
12 is a block diagram of a blood pressure monitoring system capable of identifying a user's identity according to an embodiment of the present invention.
13 is a block diagram of a facial recognition unit in a blood pressure monitoring system capable of identifying a user's identity according to an embodiment of the present invention.
14 is a block diagram of an identity authentication unit in a blood pressure monitoring system capable of identifying a user's identity according to an embodiment of the present invention.

Hereinafter, preferred embodiments will be described in detail with reference to the accompanying drawings so that those skilled in the art to which the present invention pertains can easily implement the present invention. However, in the detailed description of a preferred embodiment of the present invention, when it is determined that a detailed description of related known functions or configurations may unnecessarily obscure the subject matter of the present invention, the detailed description will be omitted. In addition, the same or similar reference numerals are used throughout the drawings for parts having similar functions and functions.

In addition, throughout the specification, when a part is said to be'connected' to another part, it is not only'directly connected', but also'indirectly connected' with another element in between. Includes. In addition, "including" a component means that other components may be further included instead of excluding other components, unless otherwise stated.

3 is a diagram illustrating a flow of a blood pressure monitoring method capable of identifying a user's identity according to an embodiment of the present invention, and FIG. 4 is a diagram illustrating a blood pressure monitoring method capable of identifying a user's identity according to an embodiment of the present invention It is one drawing. 3 and 4, a blood pressure monitoring method capable of identifying a user's identity according to an embodiment of the present invention comprises: recognizing a user's face in an image sequence collected through a camera (S100); Authenticating the user's identity by comparing the face recognized in S100 with the database of the server (S200), and generating coordinates for measuring the heart rate based on the feature points of the face from the face data recognized in step S100 ( S300), tracking the coordinates generated in step S300, measuring the heart rate at the user's face using the color change of the same part (S400), and measuring the user's heart rate at the body end (S500) ), and calculating the blood pressure using the time difference between the facial heart rate measured in step S400 and the body-end heart rate measured in step S500 (S600 ).

According to an embodiment, in step S100, a webcam (Webcam) may be used as a camera for collecting an image sequence. In the case of using a webcam, the heart rate can be measured after defining the face area from the recognized facial data, and thus it is possible to measure the heart rate through the camera not only when the user is alone, but also when there are a large number of people.

5 is a view showing a detailed flow of step S100 in the blood pressure monitoring method capable of identifying the user's identity according to an embodiment of the present invention. As shown in FIG. 5, step S100 of a blood pressure monitoring method capable of identifying a user's identity according to an embodiment of the present invention includes collecting an image through a camera (S110 ), and from the image collected in step S110 It may be implemented by including the step of selecting the image of the face (S120).

In step S120, the face common shape information is formed using PCA (Principal Component Analysis), and among the images collected through the camera in step S110, the face common shape information is similar. It may be a step of selecting a part as an image of the face of a person. In more detail, in step S120, an Eigen face (common face shape information) is formed using PCA to designate and trace a portion of the input image having a similar shape as a human face.

In step S120, in the step S110, among the images collected through the camera, selecting a part having a shape similar to the common face shape information formed as a face image of a person, more specifically, the eye from the face common shape information After determining the shape of the characteristic parts such as, nose and mouth, the part corresponding to the characteristic part among the collected images may be determined, and a corresponding high similarity area may be selected as a part image of the face of the person.

On the other hand, PCA is a secondary statistical technique that uses statistical properties up to average and variance. The PCA finds a set of orthogonal normalized series of axes pointing in each direction of the maximum covariance to the input data. This has the advantage of finding the most important axes of the input data and efficiently reducing the dimension of the data. However, since PCA only uses secondary statistical data, it is difficult to display the most basic features in the image.

Let x be the given data, and if there are n observed samples, x is x=[x ₁ , x ₂ ,… , x _n ]. In this case, each sample x _i of x is x _i =[x _i (1), x _i (2), ... , x _i (m)] is composed of m data, such as ^T , where T represents the transpose of the matrix, and in the case of a face image, the m value is the number of pixels of the face and can be represented as a one-dimensional vector.

The method of expressing data in PCA is as follows. If R is the one where the data is represented by PCA, then each row matches a sample of the original data. When V is a matrix containing eigenvectors as columns, R can be obtained by R=X ^T V. Since the eigenvector V is symmetric and orthogonally normalized, it has the property of VV ^T =1, and inversely transforming data can be obtained as X ^T =RV ^T.

The feature vector space obtained by applying PCA includes features such as changes in the lighting of the image and changes in the expression of the face, and thus the recognition rate decreases. Taking this into consideration, only the approximate location of the face is brought through the PCA, and the recognition rate can be increased by extracting the feature points within the location and performing facial recognition using feature point comparison.

6 is a view for explaining a step (S120) of screening the image of the face portion in the blood pressure monitoring method capable of identifying the user's identity according to an embodiment of the present invention. As shown in FIG. 6, the Eigen face (common face shape information) is formed from various face images, and among the collected images, a face having a shape similar to the previously formed face common shape information is recognized as the face part. You can select by image.

7 is a view showing a detailed flow of step S200 in a blood pressure monitoring method capable of identifying a user's identity according to an embodiment of the present invention. As shown in FIG. 7, step S200 of the blood pressure monitoring method capable of identifying the user's identity according to an embodiment of the present invention can authenticate the user's identity by comparing the face recognized in step S100 with the database of the server. have. According to an embodiment, step S200 extracts feature points of the face recognized in step S100 (S210 ), and generates a bunch graph from geometric information of feature points extracted in step S210 through an EBGM (Elastic Bunch graph matching) method. Step (S220), and using the batch graph generated in step S220 may be implemented, including the step of authenticating the user's identity (S230).

In step S210, the feature points of the face recognized in step S100 may be extracted. Since the face recognition method usually uses the entire (pixel) information of the image, even small lighting, posture, and facial expression changes of one local part of the face image affect the recognition algorithm and are less robust to lighting, posture, and facial expression changes. On the other hand, the model-based face recognition method can construct a model in consideration of lighting, posture, and facial expression changes, so that the influence of these factors upon recognition can be reduced. As a feature vector used in constructing the model, a Gabor feature vector (coefficient obtained by convolution of the Gabor wavelet kernel with respect to the face image feature point) is less affected by lighting, posture, and facial expression changes.

A representative face recognition method using a Gabor feature vector is EBGM (Elastic Bunch graph matching). In the face recognition method by EBGM, after finding the feature points of the face, Gabor coefficients are obtained from these feature points and face recognition is performed using the feature points. The image-based face recognition technique has a disadvantage of not being able to cope with the changing environment because it uses the entire image information. To compensate for this, EBGM, a feature vector-based technique that uses local information, can be used. EBGM is achieved by obtaining Gabor coefficients from feature points that are geometric information of the face such as eyes, nose, and mouth, and comparing the similarities between the Gabor coefficients to authenticate the person with the highest similarity.

The EBGM can detect only the local frequency characteristics in the image through the Gabor wavelet function. The Gabor wavelet filter is used to divide each into real parts and imaginary parts to discretize each, to make a Gabor wavelet mask, and to obtain coefficients obtained by ascending and integrating pixel values of regions of feature points in the face image.

In step S220, a bunch graph may be generated from geometric information of feature points extracted in step S210 through an EBGM (Elastic Bunch graph matching) method. Here, the coefficients obtained at each feature point are called a Gabor jet, and a bundle of Gabor jets is called a bunch. The batch of each bunch of facial feature points is a bunch graph. More specifically, in step S220, after finding the part corresponding to the face for the M model images, and performing a normalization to make the face size the same size by straightening the face posture, each feature point in the normalized face It is possible to detect and obtain a Gabor jet for each of these feature points. At this time, the M Gabor jet obtained from the corresponding feature points in each of the M model images, the coordinates of the corresponding feature points in each of the M model images, and the average position of the corresponding feature points are combined to obtain Gabor Bunch at the corresponding feature points. The set of all Gabor bunches at v feature points is called the Model Bunch Graph. The concept of a model bunch graph was introduced in EBGM.

The model bunch graph may be used in step S230 of authenticating the user's identity. Feature point extraction used in EBGM can be greatly influenced by the number and type of model batch graphs. For this reason, the model bunch graph should be made considering various genders and ages, lighting, poses, and facial expressions. That is, the model image used to create the model bunch graph must be selected evenly (randomly distributed) by reflecting various poses, expressions, and lighting, so that facial feature points can be well detected for various various face images.

In step S230, the identity of the user may be authenticated using the batch graph generated in step S220. More specifically, a user's identity may be authenticated as a person corresponding to the most similar batch graph by comparing the similarity between the recognized batch graph of the user's face and the batch graphs existing in the database DB.

8 is a diagram illustrating a step S200 in a blood pressure monitoring method capable of identifying a user's identity according to an embodiment of the present invention. As illustrated in FIG. 8, when the feature points are extracted from the recognized face, Gabor coefficients of the feature points are obtained through a Gabor wavelet function, bundles of coefficients obtained at each feature point are generated, and a feature point of the entire face You can create a bunch graph by collecting bunches of them. Furthermore, the user's identity can be authenticated as a person corresponding to the most similar batch graph by comparing the user's batch graph and the batch graphs existing in the database DB.

Point matching is the most widely used comparison technique in computer vision and pattern recognition. Point matching can be classified into two types, rigid matching and non-rigid matching depending on the degree of deformation of the target object extracted from the image sequence. Non-rigid matching is much more complicated than rigid matching.

In the case of non-rigid matching, it is difficult to compare the relationship between each feature point due to the large variation depending on the degree of deformation of the target object. In the case of facial feature points, it is very severe non-rigid matching due to a large variation depending on the expression or angle, and an effective non-rigid matching algorithm is needed to solve this.

In a blood pressure monitoring method capable of identifying a user's identity according to an embodiment of the present invention, Topology Preserving Relaxation Labeling (TPRL) is a modified Robust Point Matching-preserving Local Neighborhood Structure (RPM-LNS) algorithm for feature point comparison for effective identification. ) You can do point matching using an algorithm.

Step S300 is a step of generating coordinates for measuring a heart rate based on the feature points of the face from the facial data recognized in step S100. The feature point of the face, which is used as the basis for generating coordinates for measuring the heart rate in step S300, may be a feature point extracted in the step of authenticating the user's identity in step S200. A blood pressure monitoring method capable of identifying a user's identity according to an embodiment of the present invention may generate coordinates for measuring a heart rate on a portion corresponding to a user's cheek based on the extracted feature points.

On the face, dense areas of capillaries include the forehead and cheeks. Accordingly, in the blood pressure monitoring method capable of identifying the user's identity according to an embodiment of the present invention, two areas of the forehead and cheeks are preferable as the measurement site, but according to a person, hair may cover the forehead. It is desirable to generate coordinates for measuring the heart rate at the face in the portion. That is, the blood pressure monitoring method capable of identifying the user's identity according to an embodiment of the present invention may generate coordinates for measuring the heart rate at the facial part in a portion corresponding to the user's cheek based on the extracted feature points.

Looking at the step of generating the coordinates for measuring the heart rate on the part corresponding to the user's cheek in more detail, when extracting the feature points of the face, n(t) for the nostrils, e(t) for the ears, m(t) for the mouth ). The center of gravity of a triangle consisting of three points, n(t), e(t), and m(t) is called G(x,y).At this point, the value can be extracted from 10×10 pixels and averaged. have.

Step S400 may be a step of tracking the coordinates generated in step S300 and measuring the heart rate in the user's face using color change of the same area. According to an embodiment of the present invention, a blood pressure monitoring method capable of identifying a user's identity is measured through a PPG (Photoplethysmography) that measures a heart rate by measuring blood flow through a blood vessel using optical characteristics of biological tissue, and the user's facial part. Heart rate in can be measured. That is, according to the embodiment, step S400, using a BSS (Blind Source Separation) using the independent factor analysis (Independent Component Analysis (ICA)) algorithm from the image of the face portion selected in step S100 to extract the independent factor , Extracting a photoplethysmography (PPG) signal, which is a signal to measure a heartbeat state by measuring the blood flow rate in blood vessels using the optical characteristics of biological tissues from the extracted independent factors, and the extracted optical volume pulse wave It may be implemented by including the step of measuring the heart rate in the face of the user from the (PPG) signal.

More specifically, in the step of extracting the independent factor using an independent component analysis (ICA) algorithm from the image of the face portion selected in step S100 using BSS (Blind Source Separation), more specifically , Independent factors can be extracted from at least one of a camera's RGB channel or COG channel.

Biological signals such as heart rate and breathing rate can be measured only with a normal camera and proper lighting. In addition, in biosignal measurement through a camera, the face is a good site for extracting a photoplethysmography (PPG) signal. At this time, when measuring through the RGB channel, the green channel may be more suitable for measuring the heart rate and respiratory rate than the red or blue channel. This is because the light absorption wavelengths of hemoglobin and oxidized hemoglobin, which are factors for the development of blood, are 520-580 nm, and this frequency belongs to the frequency bandwidth of the green filter of the camera.

In a blood pressure monitoring method capable of identifying a user's identity according to an embodiment of the present invention, three independent factors can be extracted from the RGB channel of the camera using Blind Source Separation (BSS). 9 is a diagram illustrating a process of extracting three independent factors from an RGB channel of a camera in a blood pressure monitoring method capable of identifying a user's identity according to an embodiment of the present invention. As illustrated in FIG. 9, three independent factors can be extracted from the RGB channel of the camera using Blind Source Separation (BSS). In addition, an independent component analysis (ICA) algorithm can be used in this process. PPG (Photoplethysmography) can be extracted from one of the three independent factors.

Independent Component Analysis (ICA) is a computational method for separating multivariate signals into statistically independent subcomponents. Each component is a non-Gaussianity signal and is composed of components that are statistically independent from each other. Independent component analysis can be a special method of isolating blind signals.

If the assumption of independence is correct, independent component analysis of the mixed signal can yield good results. Even for non-mixed signals, the same method can be used for analysis. As a typical independent component analysis application, there is a separation of sound sources that subtract the voice of a specific person from conversations of several people recorded indoors. In general, assuming there is no delay or response, the problem is simplified. An important point to consider is that if there are N resources, at least N observation devices (such as microphones) are needed to separate the individuals. This statistical technique can find the independent components (elements, potential variables, sources, etc.) to maximize the statistical independence of the predicted components. According to the central limit theorem, non-Gaussianity is one way to measure component independence. In addition, the amount of mutual information is also a measure of independence between signals. A typical algorithm for independent component analysis is a whole step to reduce complexity, and requires processes such as centering, whitening, and dimensionality reduction. Whitening and dimensionality reduction are performed by principal component analysis and singular value decomposition. Independent component analysis algorithms include Infomax, FastICA, and JADE. Independent component analysis is important for blind signal separation and may have any specific application.

For bio data measurement using a camera, it may be more suitable to use a cyan, orange and green (COG) channel than an RGB channel. In the case of COG channels, each channel is Cyan (470-570 nm), Green (520-620 nm), and Orange (530-630 nm) because it overlaps more with the light absorption wavelengths of hemoglobin and oxidized hemoglobin.

Although several studies have been conducted in recent years, there are various problems in measuring bio-data, and the most frequently occurring problem is that the measured value is sensitively changed according to the lighting condition and the movement of the measurement object. In particular, in the case of rotation of the measurement object, since the coordinates for measuring the color are greatly changed, there is difficulty in continuous measurement. Therefore, in developing the system, it is necessary to develop a system that can compensate for changes in illuminance and continuously measure even if there is movement of the measurement object.

In a blood pressure monitoring method capable of identifying a user's identity according to an embodiment of the present invention, step S400 measures a user's heart rate using color changes according to optical characteristics of coordinates for measuring the heart rate generated in step S300. It may be. At this time, the coordinates for measuring the heart rate may correspond to the coordinates generated in the portion corresponding to the user's cheek in the same manner as described above.

Measuring the heart rate using color change according to the optical characteristics is basically to measure the heart rate using a camera. The method of measuring the heart rate using the camera in the detected face area is basically the same as the PPG measurement principle.

Generally, a sensor used for PPG measurement is an optical sensor, that is, an element that converts light into electric charge, and a photoelectric conversion element may be referred to as a sensor, and a photodiode may be mainly used. When light entering through the lens is transmitted to the photoelectric conversion element, the light is converted into electric charge, and the converted charge transfers the intensity of light at each pixel position to the central processing unit through the DSP through the charge-coupled element to realize an image. In the past, it was a method of sensing the light of an object, but now it is possible to use a method of obtaining information about the object based on the light being hit and reflected after the light source is provided by the machine. The light volume pulse wave using this principle is a signal that irradiates light of a specific wavelength band to the human body and detects reflected or transmitted light, and is a signal representing a pulsating component generated according to a heartbeat.

PPG measurement may be a method of estimating the heart rate by measuring the blood flow rate in a blood vessel using optical properties of biological tissue. PPG slows the flow of blood between the diastolic and systolic stages of the heart when blood flows to the peripheral blood vessels, which causes a change in the transparency of the blood vessels and is based on detecting this change in transparency. These shapes can be measured reliably on the face of the body's peripheral tissues or fingers, and in the case of general PPG measurement, they are measured using fingers.

When this principle is applied to the face, the blood flow in the blood vessels of the facial skin will also change according to the heartbeat state, and the heartbeat state can be estimated by measuring and analyzing it through a camera.

In the heart rate measurement using a camera, a PPG signal p(t) is extracted by recording and analyzing a person's face toward the camera. The recorded image is represented in the form of V(x,y,t) for each frame according to the signal strength. Each video frame stores the light reflected from the face as pixel values in each x and y coordinates, and when the camera sensor measures several colors (eg red, green, blue), it is separated into color channels in one frame. You can get multiple data values (eg V _r (x,y,t), V _g (x,y,t), V _b (x,y,t)). In addition, V(x,y,t) of the recorded image may also be composed of two elements: the intensity of illumination and the reflectance of the skin (V(x,y,t)=I(x,y,t)R) (x,y,t)).

The intensity of illumination I(x,y,t) represents all the light shining on the face. When measuring the heart rate using a camera, a constant intensity of illumination is required at the measurement site. The reflectance of the skin R(x,y,t) represents the intensity of light reflected from the skin, which consists of two types of reflection: reflection of the skin surface and reflection of the subcutaneous tissue.

At the intensity V(x,y,t) of the extracted signal, it is divided into regions (regions of interests: ROIs) that are small enough to have a constant blood flow, and is represented by R. Here, y _i (t) is an average value of pixel values of ROI R _{i over} time t, and i is an index value of R. Also, the light intensity I(x,y) of ROI R _i is constant, and when expressed as I _i , y _i (t) is y _i (t)=I _i (a _i ×p(t)+b _i )+q _i (t).

In the above equation, I _i represents the intensity of light of ROI R _i , and a _i represents the intensity of blood flow. Also, b _i refers to the reflectance at the skin surface of the face, and q _i (t) represents the camera quantization noise. When the light of I _i shines on the ROI R _i , a large amount of reflection (b _i ) occurs on the skin surface. In this case, the change according to blood flow has no effect at all. However, a portion of light penetrates the skin surface and the amount of reflection varies depending on the change in blood flow. This value is represented by p(t). In addition, a _i shows a change depending on the absorption of light of the hemoglobin and hemoglobin oxidized, which are the main color development factors of blood, and the wavelength of light irradiated. Therefore, the subcutaneous reflectance can be obtained from two factors a _i and p(t) that contribute to the reflection of light below the skin surface. In a blood pressure monitoring method capable of identifying a user's identity according to an embodiment of the present invention, it may be desirable to use a green channel having a wavelength having the highest light absorption of two color factors, hemoglobin and hemoglobin oxide.

10 is a view illustrating a step of measuring a heart rate in a face part according to a face part in a blood pressure monitoring method capable of identifying a user's identity according to an embodiment of the present invention. As shown in FIG. 10, it can be seen that the graph is clearly displayed in the portion corresponding to the cheek (y ₁ (t)), so that the facial heart rate measurement can be effectively performed.

Step S500 may be a step of measuring the user's heart rate at the body terminal. According to an embodiment, in step S500, the heart rate of the user at the body terminal may be measured through a wearable device worn on the user's body terminal. Conventionally, a separate cuff has to be installed for blood pressure measurement. In the case of the blood pressure measurement method through the cuff, continuous blood pressure measurement is impossible, and there is a disadvantage in that the cuff must be mounted cumbersomely. Accordingly, recently, instead of the cuff-based blood pressure measurement method, the time at which the pulse generated from the heart reaches the finger, which is the body end, is measured using an ECG sensor (electrocardiogram sensor) and a light volume pulse wave. Research is being conducted on a method of predicting blood pressure using the time difference. In the blood pressure monitoring method capable of identifying the user's identity according to an embodiment of the present invention, the ECG R Peak time is estimated using the camera and the light volume pulse wave as well as measuring the blood pressure using the ECG sensor and the light volume pulse wave. By measuring the blood pressure, the user's blood pressure can be measured and monitored.

The most commonly used non-invasive blood pressure measurement method is the cuff-based blood pressure measurement method. However, this cuff-based blood pressure measurement method, in order to reduce the error of the measured blood pressure, requires an interval of 1 to 2 minutes to measure the blood pressure, there is a disadvantage that can not measure continuous blood pressure data. For this reason, it is not possible to measure the change in blood pressure within a short time, and an immediate response to the change in blood pressure is not possible. To solve this, a method of measuring blood pressure using a time difference between a fingertip pulse and a heartbeat has been developed.

11 is a view illustrating a step S600 of a blood pressure monitoring method capable of identifying a user's identity according to an embodiment of the present invention. Continuous and non-invasive blood pressure measurement is based on pulse wave velocity (PWV). PWV represents the speed of the pressure pulse and can be calculated using the pulse transit time (PTT, pulse wave transfer time) shown in FIG. 11. Pulse wave propagation time (PTT) is the time it takes for a pulsating pressure wave from the heart to pass from the aortic valve to the peripheral part of the body, PTT is the opposite of PWV, proportional to the purity of the blood vessel, and the length of the blood vessel, It is affected by the cross-sectional area and the characteristics of the blood vessel wall. As shown in FIG. 11, PTT is obtained by measuring the time between the maximum value or the maximum value of the R wave of the ECG and the maximum value of the PPG, and when the time difference between the RR is shortened due to a faster heart rate, the blood pressure is increased to increase the blood pressure. Can be measured. However, there are cases where it is necessary to measure an accurate blood pressure level, not a change in blood pressure.

The exact blood pressure value can be calculated based on the distance between the heart and the body part heart rate measurement site and the blood flow rate calculated through the time difference of the R peak. PWV can be expressed as follows.

Here, BDC is a body correlation factor and height represents height. BDC is 0.5 for adults. In the blood pressure monitoring method capable of identifying the user's identity according to an embodiment of the present invention, blood pressure may be measured using PTT, which is a time difference between the R peak at the face and the R peak at the body. However, instead of the hassle of attaching an ECG sensor, blood pressure can be measured using other existing methods.

First, a wearable device such as a smart watch may include a function of measuring a pulse at a body terminal through the principle of a light volume pulse wave. Using this, it is possible to measure the pulse without an ECG sensor at the body end. The user's blood pressure can be calculated by using a time difference between the heart rate at the body end measured by the light volume pulse wave and the heart rate at the face measured by the camera.

Looking at the blood pressure calculation method in more detail, if the ECG R peak occurrence time is t _ECG , based on this, the peak time of the pulse at the body end is t _h , and the peak time of the pulse at the face is t _f , The delivery time can be expressed as PTT=t _h -t _ECG .

However, since the ECG is not used in the blood pressure monitoring method capable of identifying the user's identity according to an embodiment of the present invention, the time at which the ECG R peak occurs is unknown, (t _h -t _f ):(t _h -t _ECG ) PTT can be measured in the same way as =(s _h -s _f ):s _h .

Here, s _h represents the distance from the heart to the body end, and s _f represents the distance from the heart to the face. As described above, it is possible to estimate the occurrence time of the ECG R peak without the ECG sensor using the distance ratio from the heart that has a great influence on the arrival time of the pulse, and use the estimated R Peak time to measure the pulse propagation time as follows. Can be.

In the blood pressure monitoring method capable of identifying the user's identity according to an embodiment of the present invention, blood pressure may be measured using the estimated pulse wave propagation time.

12 is a block diagram of a blood pressure monitoring system 10 capable of identifying a user's identity according to an embodiment of the present invention. 12, the blood pressure monitoring system 10 capable of identifying a user's identity according to an embodiment of the present invention includes a facial recognition unit 100, an identity authentication unit 200, and a coordinate generation unit 300. , It may be configured to include a facial heart rate measuring unit 400, a body end heart rate measuring unit 500 and a blood pressure measuring unit 600.

More specifically, the blood pressure monitoring system 10 capable of identifying a user's identity according to an embodiment of the present invention is a blood pressure monitoring system, a facial recognition unit that recognizes a user's face in an image sequence collected through a camera ( 100), the facial recognition unit 100 compares the facial recognition with the database of the server, the identity authentication unit 200 for authenticating the user's identity, the facial recognition data from the facial recognition unit 100, facial feature points Based on this, the coordinate generator 300 that generates coordinates for measuring the heart rate in the facial part, and tracks the coordinates generated in the coordinate generator 300 to measure the facial heart rate in the user's facial part to measure the heart rate Part 400, body heart rate measurement unit 500 that measures the user's heart rate at the body end portion, and facial heart rate and body rate heart rate measurement unit 500 measured by the facial heart rate measurement unit 400 It may include a blood pressure measurement unit 600 for calculating the blood pressure using the time difference of the body end heart rate measured in.

The facial recognition unit 100 may recognize the user's face in the image sequence collected through the camera. 13 is a block diagram of a facial recognition unit 100 in a blood pressure monitoring system 10 capable of identifying a user's identity according to an embodiment of the present invention. 13, the facial recognition unit 100 of the blood pressure monitoring system 10 capable of identifying the user's identity according to an embodiment of the present invention, the image collection module 110 for collecting images through the camera And, it may be configured to include a facial image sorting module 120 for sorting the image of the face portion from the image collected by the image collection module 110.

The identity authentication unit 200 may authenticate the user's identity by comparing the face recognized by the facial recognition unit 100 with a database of the server. 14 is a block diagram of an identity authentication unit 200 in a blood pressure monitoring system 10 capable of identifying a user's identity according to an embodiment of the present invention. As illustrated in FIG. 14, the identity authentication unit 200 of the blood pressure monitoring system 10 capable of identifying the user's identity according to an embodiment of the present invention may display facial features recognized by the facial recognition unit 100. Extracting feature point extraction module 210, batch graph generation module 220 for generating a bunch graph from geometric information of feature points extracted from feature point extraction module 210 through EBGM (Elastic Bunch graph matching) method, and generating bunch graph It may include an authentication module 230 for authenticating the user's identity by using the batch graph generated by the module 220.

According to an embodiment, the authentication module 230 compares the similarity between the recognized batch graph of the user's face and the batch graphs existing in the database DB to authenticate the user's identity as a person corresponding to the highest similarity batch graph. can do.

Since the detailed contents related to the respective components have been sufficiently described in connection with the blood pressure monitoring method capable of identifying the user's identity according to an embodiment of the present invention, a detailed description will be omitted.

As described above, according to the blood pressure monitoring method and system capable of identifying the user's identity proposed in the present invention, the user's identity can be identified, and blood pressure is measured through the camera without a cuff or ECG sensor, making it easier and more convenient. Allow users to collect and process blood pressure readings.

The present invention described above can be variously modified or applied by a person having ordinary knowledge in the technical field to which the present invention belongs, and the scope of the technical idea according to the present invention should be defined by the following claims.

10: blood pressure monitoring system capable of identifying the user's identity according to an embodiment of the present invention
100: facial recognition
110: image acquisition module
120: facial image sorting module
200: identity authentication department
210: feature point extraction module
220: Bunch graph generation module
230: authentication module
300: coordinate generation unit
400: facial heart rate measurement unit
500: body end heart rate measurement unit
600: blood pressure measuring unit
S100: Recognizing the user's face in the image sequence collected through the camera
S110: collecting images through the camera
S120: Step of selecting an image of the face portion from the image collected in step S110
S200: Step of authenticating the user's identity by comparing the face recognized in step S100 with the database of the server
S210: Extracting the feature points of the face recognized in step S100
S220: generating a bunch graph from geometric information of feature points extracted in step S210 through an EBGM (Elastic Bunch graph matching) method
S230: authenticating the user's identity using the bunch graph generated in step S220
S300: generating coordinates for measuring heart rate based on facial feature points recognized from the step S100.
S400: Tracking the coordinates generated in step S300, and measuring the heart rate in the user's face using color change of the same area
S500: Step of measuring the user's heart rate at the body end
S600: Calculating blood pressure using the time difference between the facial heart rate measured in step S400 and the body-end heart rate measured in step S500

Claims (20)

Blood pressure including facial recognition unit 100, identity authentication unit 200, coordinate generation unit 300, facial heart rate measurement unit 400, body-end heart rate measurement unit 500, and blood pressure measurement unit 600 As a blood pressure monitoring method using the monitoring system 10,
(1) the face recognition unit 100 recognizes the user's face in the image sequence collected through the camera;
(2) the identity authentication unit 200 authenticating the user's identity by comparing the face recognized in step (1) with the database of the server;
(3) the coordinate generating unit 300 generating coordinates for measuring a heart rate based on the facial feature points from the facial data recognized in the step (1);
(4) measuring the heart rate in the facial portion of the user by tracking the coordinates generated in the step (3) by the facial heart rate measuring unit 400;
(5) measuring the heart rate of the user at the body end heart rate measuring unit 500 at the body end; And
(6) the blood pressure measuring unit 600 includes the step of calculating the blood pressure using the time difference between the facial heart rate measured in step (4) and the body-end heart rate measured in step (5),
The step (2),
(2-1) extracting the feature points of the face recognized in step (1);
(2-2) generating a bunch graph from geometric information of the feature points extracted in step (2-1) through an EBGM (Elastic Bunch graph matching) method; And
(2-3) A blood pressure monitoring method capable of identifying the user's identity, comprising the step of authenticating the user's identity using the batch graph generated in step (2-2).
According to claim 1, In the step (1),
Blood pressure monitoring method, characterized in that using a webcam (Webcam) as the camera for collecting the image sequence.
The method of claim 1, wherein the step (1),
(1-1) collecting images through a camera; And
(1-2) characterized in that it comprises the step of selecting the image of the face portion from the image collected in the step (1-1), blood pressure monitoring method capable of identifying the user.
According to claim 3, In the step (1-2),
The face common shape information (Eigen face) is formed using PCA (Principal Component Analysis), and among the images collected in step (1-1), a portion having a shape similar to the common face shape information is formed. A blood pressure monitoring method capable of identifying a user's identity, characterized in that it is screened by a facial image of a person.
delete According to claim 1, In the step (2-2),
The Gabor coefficient of the feature points is obtained through a Gabor wavelet function, the Gabor coefficients of each feature point are grouped to generate a bunch, and the bunch graphs of the whole face feature points are collected to generate the bunch graph. Characterized, blood pressure monitoring method capable of identifying the user's identity.
According to claim 1, In the step (2-3),
Comparing the similarity between the recognized batch graph of the user's face and the batch graphs existing in the database (DB), it is possible to identify the user's identity by authenticating the user's identity as the person corresponding to the most similar batch graph. How to monitor blood pressure.
The method according to claim 7, wherein in step (2-3),
A blood pressure monitoring method capable of identifying a user's identity, characterized by comparing similarities using a non-rigid matching technique among point matching techniques.
The method of claim 8, wherein in step (2-3),
A blood pressure monitoring method capable of identifying a user, characterized in that a Topology Preserving Relaxation Labeling (TPRL) algorithm is used to use the non-rigid matching technique.
According to claim 1, In the step (3),
Blood pressure monitoring method that can identify the user, characterized in that the coordinates for measuring the heart rate on the part corresponding to the user's cheeks based on the extracted feature points.
The method of claim 1, wherein the step (4),
(4-1) extracting independent factors using an independent component analysis (ICA) algorithm from the image of the facial part selected in step (1) using BSS (Blind Source Separation);
(4-2) Photoplethysmography (PPG), which is a signal to measure the heart rate by measuring the blood flow rate in blood vessels using the optical properties of biological tissues from the independent factors extracted in step (4-1). Extracting a signal; And
(4-3) characterized in that it comprises the step of measuring the heart rate in the user's face from the light volume pulse wave (PPG) signal extracted in the step (4-2), blood pressure monitoring capable of identifying the user Way.
The method of claim 11, wherein in step (4-1),
A blood pressure monitoring method capable of identifying a user's identity, characterized in that three independent factors are extracted from the RGB channel of the camera.
The method of claim 11, wherein in step (4-1),
A blood pressure monitoring method capable of identifying the user's identity, characterized in that three independent factors are extracted from the camera's COG (Cyan, Orange, Green) channels.
The method of claim 1, wherein the step (4),
A blood pressure monitoring method capable of identifying a user, characterized in that the heart rate in the user's face is measured using a color change according to optical characteristics of coordinates for measuring the heart rate generated in the step (3).
The method of claim 1, wherein in step (5),
Blood pressure monitoring method capable of identifying the user's identity, characterized in that the user's heart rate is measured at the body terminal through a wearable device worn on the user's body terminal.
Blood pressure monitoring system,
A facial recognition unit 100 for recognizing a user's face in the image sequence collected through the camera;
An identity authentication unit 200 for authenticating the user's identity by comparing the face recognized by the facial recognition unit 100 with a database of a server;
A coordinate generator 300 for generating coordinates for measuring a heart rate based on the facial feature points from the facial data recognized by the facial recognition unit 100;
A facial heart rate measuring unit 400 tracking the coordinates generated by the coordinate generating unit 300 and measuring the heart rate in the user's facial unit;
A body-end heart rate measuring unit 500 which measures a user's heart rate at the body end unit; And
Includes a blood pressure measuring unit 600 for calculating blood pressure using a time difference between the facial heart rate measured by the facial heart rate measuring unit 400 and the body terminal heart rate measured by the body-end heart rate measuring unit 500, ,
The identity authentication unit 200,
A feature point extraction module 210 for extracting the feature points of the face recognized by the face recognition unit 100;
A bunch graph generation module 220 that generates a bunch graph from geometric information of feature points extracted from the feature point extraction module 210 through an EBGM (Elastic Bunch graph matching) method; And
And an authentication module 230 for authenticating the user's identity using the batch graph generated by the batch graph generation module 220.
The method of claim 16, wherein the facial recognition unit 100,
An image collection module 110 for collecting images through a camera; And
Blood pressure monitoring system (10), characterized in that it comprises a facial image sorting module (120) for sorting the image of the facial part from the image collected from the image collecting module (110).
delete The method of claim 16, wherein the authentication module 230,
By comparing the similarity between the recognized batch graph of the user's face and the batch graphs existing in the database (DB), the user's identity is authenticated as the person corresponding to the highest similarity batch graph.
A blood pressure monitoring system capable of identifying a user's identity, characterized in that the similarity is compared using a non-rigid matching technique among point matching techniques.
The authentication module of claim 19,
A blood pressure monitoring system capable of identifying a user's identity (10), characterized by using a Topology Preserving Relaxation Labeling (TPRL) algorithm to use the non-rigid matching technique.

Images