KR20230156868A - Machine learning-based non-contact user health anomaly detection method and system - Google Patents

Abstract

The present invention relates to a machine learning-based non-contact user health abnormality detection system.
The present invention includes a data acquisition unit that acquires user's biometric detection data from an image captured through a non-contact biometric signal sensor; a data conversion unit that converts the biometric detection data acquired through the data acquisition unit into biosignal data; and a user risk detection unit that detects the user's health risk state using the user's biosignal data converted through the data conversion unit.

Description

Machine learning-based non-contact user health anomaly detection method and system}

The present invention relates to a non-contact user health abnormality detection system and method with autonomous data collection function, which detects a patient's bio signals using a non-contact sensing system such as a wearable device bio signal measurement sensor, camera, and IR-UWB lidar. It is about technology that automatically measures, collects, and uses deep learning models to detect the user's health risk status.

With the development of biosignal measurement and signal processing technology, many electronic technologies are being developed to determine a person's health status. In particular, there have been recent advances in deep learning-based signal processing methods that surpass human performance. It is also being applied to the medical and healthcare fields to improve performance.

Additionally, non-contact sensor-based biological signal measurement methods are being proposed to improve user convenience.

Typically, IR-UWB LiDAR is being studied to measure the user's pulse or breathing rate, but there are limitations in signal processing techniques to ensure high accuracy. Deep learning-based technologies can be used to solve these limitations.

In constructing deep learning-based biosignal processing techniques, there are many difficulties in acquiring data and adding annotations/labels to the acquired data.

To solve this problem, an approach other than building a system using existing supervised learning methods is needed.

Unsupervised learning is a technology that analyzes data by learning the distribution of data or learning features without annotations/labels of the input data.

Therefore, there is an advantage that there is no need to add annotations/labels for existing supervised learning.

Information on vital signs such as body temperature and blood pressure is used to determine health status. Changes in vital signs may reflect changes in health status.

Therefore, by observing vital signs, one can infer a person's health status and prepare for danger.

However, collecting human data is a difficult problem to solve in the medical and healthcare fields, making it difficult to develop autonomous patient monitoring systems based on artificial intelligence.

The proposed method utilizes a wearable device to automatically build learning data for a non-contact biosignal sensing system.

Most wearable devices are provided with PPG (photoplethysmogram) sensors and ECG (electrocardiogram) sensors, allowing data such as pulse rate, blood pressure, and respiration to be acquired. Learning data can be constructed by synchronizing these data with data from non-contact sensors.

The present invention was developed to solve conventional problems, and includes a method for autonomously acquiring a user's biosignals using non-contact sensors and a system configuration method for detecting a user's health risk using the acquired data. It's about.

The purpose of the present invention is to provide a method for automatically acquiring user biosignal data using a non-contact sensor and a machine learning-based non-contact user health abnormality detection system that detects the user's health risk status from the acquired biosignal. .

The object of the present invention is not limited to the object mentioned above, and other objects not mentioned will be clearly understood by those skilled in the art from the description below.

To achieve the above object, a non-contact user health abnormality detection system based on machine learning according to an embodiment of the present invention includes a data acquisition unit that acquires biometric detection data of the user from an image captured through a non-contact biometric signal sensor; a data conversion unit that converts the biometric detection data acquired through the data acquisition unit into biosignal data; and a user risk detection unit that detects the user's health risk state using the user's biosignal data converted through the data conversion unit.

A machine learning-based non-contact user health abnormality detection method according to an embodiment of the present invention includes the steps of acquiring user's biometric detection data from a non-contact biometric signal sensor; converting the acquired biometric detection data into biometric signal data; and detecting the user's health risk state using the converted biosignal data.

According to one embodiment of the present invention, the user's health status can be conveniently monitored using a contact sensor and a non-contact sensor. Additionally, the proposed method has the effect of easily collecting data.

1 is a block diagram illustrating a non-contact user health abnormality detection system based on machine learning according to an embodiment of the present invention.
FIG. 2 is a block diagram for explaining the configuration of the data acquisition unit of FIG. 1.
FIG. 3 is a diagram illustrating the data conversion process of the deep learning-based biological signal calculation model used in the data conversion unit shown in FIG. 1.
FIG. 4 is a diagram illustrating the learning process of a deep learning-based biosignal calculation model used in the data conversion unit shown in FIG. 1.
FIG. 5 is a diagram illustrating a deep learning-based restoration model used in the user risk detection unit shown in FIG. 1.
Figure 6 is a block diagram illustrating a machine learning-based non-contact user health abnormality detection system according to another embodiment of the present invention.
Figure 7 is a diagram for explaining a deep learning-based restoration model used in the user risk detection unit in another embodiment of the present invention.
Figure 8 is a diagram for explaining the determination method used in the user risk detection unit in another embodiment of the present invention.
Figure 9 is a flowchart illustrating a machine learning-based non-contact user health abnormality detection method according to an embodiment of the present invention.

The advantages and features of the present invention and methods for achieving them will become clear by referring to the embodiments described in detail below along with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below and will be implemented in various different forms. The present embodiments only serve to ensure that the disclosure of the present invention is complete and that common knowledge in the technical field to which the present invention pertains is not limited. It is provided to fully inform those who have the scope of the invention, and the present invention is only defined by the scope of the claims. Meanwhile, the terms used in this specification are for describing embodiments and are not intended to limit the present invention. As used herein, singular forms also include plural forms, unless specifically stated otherwise in the context. As used in the specification, “comprises” and/or “comprising” means that a referenced element, step, operation and/or element precludes the presence of one or more other elements, steps, operations and/or elements. or does not rule out addition.

Figure 1 is a block diagram illustrating a non-contact user health abnormality detection system based on machine learning according to the present invention.

As shown in Figure 1, the machine learning-based non-contact user health abnormality detection system according to an embodiment of the present invention includes a data acquisition unit 100, a user risk detection unit 300, and a detection result notification and management unit 400. Includes.

The data acquisition unit 100 is a non-contact biometric signal sensor 110 that acquires the user's biometric information. It photographs the user whose biometric information is to be acquired and detects the user's biometric detection data from the captured image. Here, the non-contact biosignal sensor may be one or a combination of a camera, a depth camera, a thermal imaging camera, and an IR-UWB lidar. And, as shown in FIG. 2, the data acquisition unit 100 may further include a contact-type biometric signal sensor 120 that is worn directly by the user and measures the user's biometric signal for model learning.

Accordingly, the data acquisition unit 100 acquires biometric detection data through the captured image, and the acquired biometric measurement data is stored according to the order in which it was measured and generated to have sequential information. Here, biometric detection data may include user identification information, measurement state detection information, and biometric measurement information. As an example, the data acquisition unit 100 may detect the user's blood vessel in a captured image, detect the movement of the blood vessel, and detect the user's pulse rate among the biometric detection data. Additionally, when using an infrared camera, the user's body temperature can be detected among biometric detection data.

As shown in FIG. 3, the data conversion unit 200 converts the biometric detection data acquired through the data acquisition unit 100 into biosignal data using a deep learning-based biosignal calculation model. And deep learning-based biosignal calculation models can be used such as convolutional networks, long short-term memory, and autoencoder.

Here, the biometric signal data uses biometric detection data acquired through the non-contact biosignal sensor 110 as an input value and biometric measurement data measured through the contact type biosignal sensor 120 as a result value, and input and output This is data converted from biometric detection data using a deep learning-based biosignal calculation model, which is a deep learning model learned to be the same.

As shown in FIG. 4, the deep learning-based biometric signal calculation model uses biometric detection data acquired through the non-contact biometric signal sensor 100 of the data acquisition unit 100 as input, and the contact biosignal sensor 120 ) is learned using the sensor measurement data measured through ) as output.

Here, the contact-type biological signal sensor 120 is a wearable device such as a smart watch, and is equipped with a photoplethysmogram (PPG) sensor, an electrocardiogram (ECG) sensor, etc. Accordingly, the contact-type biosignal sensor 120 can acquire biometric data such as pulse rate, respiratory rate, body temperature, etc. through a photoplethysmogram (PPG) sensor and an electrocardiogram (ECG) sensor.

Accordingly, the user risk detection unit 300 converts the acquired biometric detection data into biometric signal data corresponding to the biometric measurement data measured through the contact-type biosignal sensor 110 through a deep learning-based biosignal calculation model. can do. And, the biosignal data converted through the data conversion unit 200 can be stored in the device or transmitted to an external terminal. Here, the biosignal data may be pulse rate, respiratory rate, body temperature, and blood pressure.

Meanwhile, the user risk detection unit 300 determines the user's health risk status using a deep learning-based restoration model. Here, the deep learning-based restoration model is a model that receives normally input biometric signal data and restores it to usual biometric measurement data, as shown in FIG. 5. These deep learning-based restoration models can be used as deep learning models such as convolutional networks, long short-term memory, autoencoder, and generative adversarial networks.

Therefore, it is desirable that the deep learning-based restoration model used by the user risk detection unit 300 is fixed to the user's biosignal data in a normal state for the output value, even if the input value changes.

Therefore, when user biosignal data in an abnormal state comes in, a deep learning-based restoration model performs deep learning to restore user biosignal data in an abnormal state to user biosignal data in a normal state. At this time, the deep learning-based restoration model can detect the user's health risk status using input and errors generated during data restoration.

The detection result notification and management unit 400 can deliver the user health risk results determined through a deep learning-based restoration model to the user terminal or the terminal of a designated person other than the user. The detection result notification and management unit 400 may store the user health risk results in the user's terminal or server.

According to an embodiment of the present invention, it is possible to detect a user's biosignal through a captured image and detect a user's health risk using the user's biosignal.

According to one embodiment of the present invention, it is possible to detect the user's health risk through the captured image, so that it can be installed in a location where the user faces or passes by, such as an elevator, restroom, mirror, refrigerator, etc. there is.

In addition, according to an embodiment of the present invention, risk detection results and used data can be confirmed and observed through a monitoring device, and in the step of configuring learning data for a model that converts non-contact sensor data into biometric information values, a wearable device is used. This has the effect of allowing data to be collected gradually by setting the data collection mode when the wearable device is worn and the use mode when the wearable device is not worn.

And according to an embodiment of the present invention, there is an effect of checking whether additional learning is necessary by comparing the biometric information value of the wearable device after learning with the biometric information value estimated from the non-contact sensor.

And, according to an embodiment of the present invention, contact biological signal measurement sensors can be used in the data collection process to convert data from a non-contact sensor into biometric information, and the values of the biometric signal measurement sensors can be inputted using the values of the non-contact sensor. There is an effect of learning by setting learning goals.

Additionally, the present invention can be used together with diagnosis results as input to the user's risk detection algorithm.

As shown in FIG. 6, the machine learning-based non-contact user health abnormality detection system according to an embodiment of the present invention uses biometric detection data (e.g., face recognition, iris recognition) acquired through the data acquisition unit 100. etc.) may further include a user identification unit 500 that authenticates the user. Through this, the present invention can store and manage data separately for each user, and has the effect of detecting the health risk status of each user even if the health risk status of multiple users is determined through one image.

And, as shown in FIG. 6, the machine learning-based non-contact user health abnormality detection system according to an embodiment of the present invention uses the camera image and depth camera, which are the data acquisition unit 100, to collect biometric detection data. It may further include a measurement state detection unit 600 that estimates the user's posture and checks whether the user is currently wearing the biosignal measurement sensor correctly or is in a normal posture.

This measurement state detection unit 600 has the effect of enabling more accurate judgment by determining the user's health abnormality using only biosignal information measured by the user in a normal state.

Meanwhile, the machine learning-based non-contact user health abnormality detection system according to an embodiment of the present invention may further include an event data acquisition unit 700 that receives event data, as shown in FIG. 6. Here, the event data acquisition unit 700 collects information directly input by the user or external medical information. As an example of use, data can be collected by checking whether the user went to the hospital today or not through payment records by linking with mobile phone and credit card companies, or linking with the hospital's medical record system to provide medical records when the user visits the hospital. You can receive it and use it as input. As another example, if a user thinks there is something wrong with their body, symptoms can be received directly from the user through a mobile device and used as input. Event data may be included in the preprocessed data in the next step.

Accordingly, the user risk detection unit 300 may integrate the event data collected through the event data acquisition unit 700 and the user's biosignal data acquired through the data acquisition unit 100 into one biosignal data.

In addition, since the structure of the data input to the deep learning-based restoration model is a multidimensional array with a size equal to the type of data and the measured time length, the function of filtering and synchronizing contaminated data among the input data is performed. You can.

In addition, in another embodiment of the present invention, as shown in FIG. 7, the user risk detection unit 300 measures biometric detection data detected through the non-contact biometric signal sensor 110 and the contact biosignal sensor 120. The biometric data can also be used as input to a deep learning-based restoration model.

As shown in FIG. 8, the user risk detection unit 300 according to another embodiment of the present invention uses a preset risk threshold (each The user's risk status can also be determined by comparing it with the threshold value of the biosignal.

A machine learning-based non-contact user health abnormality detection method according to an embodiment of the present invention will be described with reference to FIG. 9.

The data acquisition unit 100 detects the user's biometric detection data from the image captured through the non-contact biometric signal sensor (S100). Here, biometric detection data can be detected from an image captured by the non-contact biometric signal sensor 110. As an example, the data acquisition unit 100 detects the user's blood vessels in an image captured through a non-contact biosignal sensor and detects the pulse rate, which is one of the user's biosignal data, among the biometric detection data by detecting the movement of the corresponding blood vessels. You can. Additionally, when using an infrared camera, the user's body temperature can be detected among biometric detection data.

The data conversion unit 200 converts the biometric detection data detected by the data acquisition unit 100 into biometric detection data using a deep learning model. In other words, the detected biometric detection data is converted into the same structure and type as biometric data measured through an actual contact-type biometric signal sensor.

For this purpose, the data conversion unit 200 uses a deep learning-based biosignal calculation model. Here, the deep learning-based biosignal calculation model uses biometric detection data acquired through the non-contact biosignal sensor 110 as input and biometric measurement data measured through the contact biosignal sensor 120 as the result. Therefore, it is a deep learning model that has been trained so that the input and output are the same. And deep learning-based biosignal calculation models can be used such as convolutional networks, long short-term memory, and autoencoder.

Next, the user risk detection unit 300 detects the user's health risk status, which is the user risk level, by using the converted biosignal as an input to a deep learning-based restoration model (S300). Deep learning-based restoration models are learned so that input and output are the same value. At this time, the structure of the input data is a multidimensional array with a size equal to the type of data and the measured time length. In addition, deep learning-based restoration models such as convolutional networks, long short-term memory, autoencoder, and generative adversarial networks can be used.

Afterwards, the detection result notification and management unit 400 provides a result of the user's health risk status according to a deep learning-based restoration model (S400). That is, the detection result notification and management unit 400 notifies the user or a designated person other than the user of the user's health risk status according to the risk detection result. The input data used and the user's health risk status results can be stored on the device or server.

Meanwhile, the data acquisition unit 100 may be further equipped with a contact-type biometric signal sensor 120 that is worn directly by the user and measures the user's biometric signal in order to learn a deep learning-based biometric signal calculation model.

In another embodiment of the present invention, the method for detecting the user's health risk status is to compare the pulse rate, respiratory rate, body temperature, blood pressure, etc. calculated from deep learning-based biosignals with a predetermined risk threshold to determine the user's health risk status. You can also sense it.

Deep learning-based restoration models are learned so that input and output are the same value. At this time, the structure of the input data is a multidimensional array with a size equal to the type of data and the measured time length. Deep learning models such as convolutional networks, long short-term memory, autoencoder, and generative adversarial networks can be used as deep learning-based restoration models.

The detection result notification and management unit 400 processes calculation results and data of a deep learning-based restoration model. Depending on the risk detection results, the results are notified to the user or a designated person other than the user. The input data used and detection results are stored on the device or server.

In one embodiment of the present invention, as a method of detecting a user's health risk state, the user risk detection unit 300 does not use a deep learning-based restoration model, but uses deep learning-based biosignal data (pulse rate, respiratory rate, The user's risk status can also be detected by comparing body temperature, blood pressure, etc.) with a predetermined risk threshold.

Above, the configuration of the present invention has been described in detail with reference to the accompanying drawings, but this is merely an example, and those skilled in the art will be able to make various modifications and changes within the scope of the technical idea of the present invention. Of course this is possible. Therefore, the scope of protection of the present invention should not be limited to the above-described embodiments, but should be determined by the description of the claims below.

Claims (20)

a data acquisition unit that acquires user's biometric detection data from an image captured through a non-contact biometric signal sensor;
a data conversion unit that converts the biometric detection data acquired through the data acquisition unit into biosignal data; and
A machine learning-based non-contact user health abnormality detection system including a user risk detection unit that detects the user's health risk status using the user's biosignal data converted through the data conversion unit.
According to paragraph 1,
The data acquisition unit,
A machine learning-based non-contact user health abnormality detection system further comprising a contact-type biosignal sensor that is worn directly by the user and measures the user's biometric data.
According to paragraph 1,
A machine learning-based non-contact user health abnormality detection system that further includes an event data acquisition unit that collects information directly entered by the user or external medical information.
According to paragraph 1,
The user risk detection unit,
A preprocessor that filters contaminated data,
A machine learning-based non-contact user health abnormality detection system further comprising synchronization to integrate the event data and data acquired from sensors into one biosignal data.
According to paragraph 4,
The biometric detection data is,
A machine learning-based non-contact user health abnormality detection system characterized by one or more of the user's body temperature, pulse rate, and respiratory rate.
According to paragraph 2,
The data conversion unit,
Calculate biometric signals based on deep learning, using biometric detection data acquired through a non-contact biosignal sensor as input and biometric measurement data measured through a contact biosignal sensor as result, so that the input and output are the same. A machine learning-based non-contact user health abnormality detection system characterized by using a model.
According to paragraph 1,
The user risk state detection unit,
A machine learning-based non-contact user health abnormality detection system characterized by using an unsupervised learning-based deep learning-based restoration model that receives user biosignal data input on a daily basis and restores the usual biometric data.
According to paragraph 1,
The user risk state detection unit,
A machine learning-based non-contact user health abnormality detection system characterized in that the user's health risk status is detected by comparing the bio-signal data provided through the data conversion unit with a preset threshold.
According to paragraph 1,
A machine learning-based non-contact user health abnormality detection system further comprising a user identification unit that authenticates the user using biometric detection data obtained through the data acquisition unit.
According to paragraph 1,
A machine learning-based non-contact user health abnormality detection system further comprising a measurement state detection unit that detects the user's posture through the biometric detection data.
Acquiring biometric detection data of a user from a non-contact biometric signal sensor;
converting the acquired biometric detection data into biometric signal data; and
A machine learning-based non-contact user health abnormality detection method comprising detecting a user's health risk using the converted biosignal data.
According to clause 11,
A machine learning-based non-contact user health abnormality detection method further comprising the step of having the user wear a contact biometric signal sensor and measuring the user's biometric data.
According to clause 11,
It further includes a step of collecting information directly entered by the user or external medical information by an event data acquisition unit,
A machine learning-based non-contact user health abnormality state further comprising integrating the event data and bio-signal data acquired from sensors into one bio-signal data in the step of detecting the user's health risk state by a synchronization unit. Detection method.
According to clause 11,
The step of detecting the user's health risk condition is,
A machine learning-based non-contact user health abnormality detection method further comprising filtering contaminated data by a preprocessor.
According to clause 11,
The biometric detection data is,
A machine learning-based non-contact user health abnormality detection method characterized by one or more of body temperature, pulse rate, and respiratory rate.
According to clause 12,
The step of converting into biosignal data is,
Calculate biometric signals based on deep learning, using biometric detection data acquired through a non-contact biosignal sensor as input and biometric measurement data measured through a contact biosignal sensor as result, so that the input and output are the same. A machine learning-based non-contact user health abnormality detection method characterized by using a model.
According to clause 11,
The step of detecting the user's health risk condition is,
A machine learning-based non-contact user health abnormality detection method characterized by using an unsupervised learning-based deep learning-based restoration model that receives user biosignal data input on a daily basis and restores the user biometric data to usual biometric measurement data.
According to clause 17,
The step of detecting the user's health risk condition is,
A non-contact user health abnormality detection method based on machine learning, characterized in that the user's health risk status is detected by comparing the provided bio-signal data with a preset threshold.
According to clause 11,
A machine learning-based non-contact user health abnormality detection method further comprising authenticating the user using the obtained biometric detection data by a user identification unit.
According to clause 11,
A machine learning-based non-contact user health abnormality detection method further comprising the step of a measurement state detection unit detecting the user's posture using the biometric detection data.