KR20220003887A - A method and an apparatus for estimating blood pressure - Google Patents

Abstract

A method for measuring blood pressure of an examinee according to one aspect of the present invention comprises the steps of: obtaining a first pulse wave signal corresponding to a first human body portion based on at least one image expressing the first human body portion of the examinee photographed through a first camera; obtaining a second pulse wave signal corresponding to a second human body portion based on at least one image expressing the second human body portion of the examinee photographed through a second camera which is disposed at a position separated from the first camera; correcting the second pulse wave signal based on the first pulse wave signal; and estimating the blood pressure of the examinee from the corrected second pulse wave signal and the first pulse wave signal by using a machine learning technology. The present invention enables the examinee to measure his or her blood pressure easily and conveniently.

Description

A method and apparatus for estimating blood pressure {A method and an apparatus for estimating blood pressure}

It relates to a method and apparatus for estimating blood pressure.

Biosignals contain various information that can be used to determine health status. Accordingly, the biosignal of the examinee is measured, and the current health state of the subject can be known from the measured biosignal. One of the biosignals widely measured for this purpose is blood pressure.

Accordingly, various studies have been conducted on blood pressure monitors that allow subjects to easily measure their own blood pressure. In particular, with the development of the electronics industry, an automated blood pressure monitor capable of measuring blood pressure in an indirect way has been developed. For example, a blood pressure measuring device and a blood pressure measuring method capable of quickly and with high accuracy finding a radial artery have been disclosed.

However, such a conventional blood pressure monitor has a relatively large volume, so it is difficult to carry, and it is cumbersome to wear a guff every time blood pressure is measured.

Pulse wave, one of the representative biosignals showing physiological responses through the central nervous system and autonomic nervous system, is one of the effective methods to detect the heartbeat cycle while minimizing discomfort to the subject. It is a biosignal that is being studied a lot in the field.

The heart rate cycle is used as an important signal in clinical practice through disease monitoring and measurement of fetuses, patients, and the elderly. In particular, heart rate variability (hereafter referred to as “HRV”) is a signal created as a series of data by sequentially connecting successive changes in the heart rate cycle. ) is used as a tool for In addition, HRV is being used as a very useful indicator in studies such as stress, sleep analysis, and emotion analysis.

As the pulse wave propagates as a pulsating waveform due to the contraction and relaxation of the heart, a change in the volume of the blood vessel appears. Photoplethysmograph (hereinafter referred to as “PPG”) is a method of measuring pulse waves using light, and changes in optical properties such as reflection and absorption/transmission ratio of living tissue that occur when the volume of blood vessels change are measured by an optical sensor. detect and measure Unlike electrocardiogram, changes in blood flow delivered to peripheral blood vessels are measured, but pulse waves can be measured conveniently by applying light sources and optical sensors to fingers, forehead, and earlobes without using electrocardiogram electrodes. Also, although the change in the pulse wave using the PPG is not the change in the electrocardiogram, the HRV can be estimated by the variability of the pulse wave measured from the PPG. Although PPG is a suitable method for wearable measuring devices, various studies have been conducted to deal with it because it reacts very sensitively to motion noise due to the characteristics of the measurement method.

The commonality of the methods presented in various studies to measure pulse waves so far is that a separate light source and optical sensor are used to measure the pulse waves, and the method of transmitting the results to a mobile device such as a host PC, server, or smartphone is mostly it was Most of the pulse wave measurement methods are simple and convenient, but because they require a separate measurement device, the user's portability is reduced and the usage time according to the power supply of the measurement device is reduced. Therefore, there is a problem in that the advantages of u-Healthcare equipment are lowered and the restrictions on use are increased.

In addition, recently, various biosignal recognition sensors mounted on mobile terminals, for example, PPG sensors, or technologies using a camera provided on the rear of the mobile terminal are being studied.

[Prior art literature number]

Prior Document 1: Korean Patent Publication No. 10-136680

Prior Document 2: "Development of a photoplethysmography method using a CMOS image sensor of a smart phone" Journal of the Korean Society of Industrial-Academic Technology, Vol. 7, No. 6, 2015

The present invention was created to solve the above-described problems, and an object of the present invention is to provide a method and an apparatus for estimating blood pressure. Another object of the present invention is to provide a computer-readable recording medium in which a program for executing the method in a computer is recorded. The technical problem to be solved is not limited to the technical problems as described above, and other technical problems may exist.

According to one aspect, a method for measuring blood pressure of a subject acquires a first pulse wave signal corresponding to a first body part based on at least one image representing a first body part of the subject captured through a first camera to do; acquiring a second pulse wave signal corresponding to the second body part based on at least one image representing a second body part of the examinee captured through a second camera disposed at a location spaced apart from the first camera ; correcting the second pulse wave signal based on the first pulse wave signal; and predicting the blood pressure of the subject from the corrected second pulse wave signal and the first pulse wave signal using a machine learning technique.

In the above-described method, the first pulse wave signal includes a signal having a relatively high signal to noise ratio compared to the second pulse wave signal.

In the above-described method, the correcting includes removing noise included in the second pulse wave signal based on the first pulse wave signal through an adaptive filter to correct the second pulse wave signal. Correct.

In the above method, the adaptive filter includes at least one of a least mean square (LMS) filter, a normalized least mean square (NLMS) filter, and a recursive least square (RLS) filter.

In the above-described method, the acquiring of the first pulse wave signal includes generating a first image signal by averaging a G value among the RGB values of the at least one image for each image frame, and the generated first image The pulse wave signal is extracted by applying an IIR filter to the signal.

In the above method, the machine learning technique includes a machine learning technique based on a regression model.

In the above method, the first body part includes a finger of the subject, and the second body part includes a face of the examinee.

A computer-readable recording medium according to another aspect includes a recording medium recording a program for executing the above-described method in a computer.

An apparatus for measuring blood pressure of a subject according to another aspect includes: a first camera; a second camera disposed at a position spaced apart from the first camera; Memory; and a processor, wherein the processor acquires a first pulse wave signal corresponding to the first body part based on at least one image corresponding to the first body part of the examinee captured through the first camera and acquiring a second pulse wave signal corresponding to the second body part based on at least one image representing a second body part of the examinee captured by the second camera, and based on the first pulse wave signal The second pulse wave signal is corrected, and the blood pressure of the examinee is predicted from the corrected second pulse wave signal and the first pulse wave signal using a machine learning technique.

In the above-described apparatus, the first pulse wave signal includes a signal having a relatively high signal to noise ratio compared to the second pulse wave signal.

In the above-described apparatus, the processor corrects the second pulse wave signal by removing noise included in the second pulse wave signal based on the first pulse wave signal through an adaptive filter. .

In the above-described apparatus, the adaptive filter includes at least one of a Least Mean Square (LMS) filter, a Normalized Least Mean Square (NLMS) filter, and a Recursive Least Square (RLS) filter.

In the above-described apparatus, the processor generates a first image signal by averaging the G values among the RGB values of the at least one image for each image frame, and applies an IIR filter to the generated first image signal. The pulse wave signal is extracted.

In the above apparatus, the machine learning technique includes a machine learning technique based on a regression model.

In the device described above, the first body part includes a finger of the examinee, and the second body part includes a face of the examinee.

Since the method and apparatus for estimating blood pressure according to some embodiments of the present invention can accurately estimate the blood pressure of a subject from an image obtained by photographing the subject's face, a separate tool (eg, Guff etc) is not required.

In addition, since the method and apparatus for estimating blood pressure according to some embodiments of the present invention can be implemented by a mobile terminal (eg, a smart phone, etc.), the examinee can measure his/her own blood pressure very easily and conveniently. have.

Effects of the present invention are not limited to the above-described effects, and potential effects expected by the technical features of the present invention will be clearly understood from the following description.

1 is a diagram for describing an example of a method for estimating blood pressure according to some embodiments.
2 is a configuration diagram illustrating an example of an apparatus for estimating blood pressure according to some embodiments.
3 is a configuration diagram illustrating an example of a mobile terminal according to some embodiments.
4 is a flowchart illustrating an example of a method for estimating blood pressure according to some embodiments.
5 is a view for explaining an example in which a first camera captures an image representing a first body part, according to some embodiments.
6 is a view for explaining an example in which a second camera captures an image representing a second body part, according to some embodiments.
7A to 7D are diagrams for explaining an example in which a processor extracts a first pulse wave signal and a second pulse wave signal according to some embodiments.
8 is a diagram for describing an example in which a processor corrects a second pulse wave signal according to some embodiments.
9 is a diagram for describing an example of a corrected second pulse wave signal according to some embodiments.
10 is a diagram for describing an example in which an apparatus for estimating blood pressure according to some exemplary embodiments operates.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art to which the present disclosure pertains can easily implement them. However, the present disclosure may be implemented in several different forms and is not limited to the embodiments described herein. In addition, in order to clearly explain the present disclosure in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

The terms used in the present disclosure have been described as general terms currently used in consideration of the functions mentioned in the present disclosure, but may mean various other terms depending on the intention or precedent of a person skilled in the art, the emergence of new technology, etc. can Therefore, the terms used in the present disclosure should not be construed only as the names of the terms, but should be interpreted based on the meaning of the terms and the contents of the present disclosure.

In addition, the terms used in the present disclosure are only used to describe specific embodiments, and are not intended to limit the present disclosure. Expressions in the singular include the plural meaning unless the context clearly implies the singular. In addition, throughout the specification, when a part is "connected" with another part, it is not only "directly connected" but also "electrically connected" with another element interposed therebetween. include Also, when a part "includes" a component, it means that other components may be further included, rather than excluding other components, unless otherwise stated.

The above and similar referents used in this specification, particularly in the claims, may refer to both the singular and the plural. Moreover, unless there is a description explicitly designating the order of steps describing a method according to the present disclosure, the described steps may be performed in an appropriate order. The present disclosure is not limited according to the description order of the described steps.

Phrases such as “in some embodiments” or “in one embodiment” appearing in various places in this specification are not necessarily all referring to the same embodiment.

Some embodiments of the present disclosure may be represented by functional block configurations and various processing steps. Some or all of these functional blocks may be implemented in various numbers of hardware and/or software configurations that perform specific functions. For example, the functional blocks of the present disclosure may be implemented by one or more microprocessors, or may be implemented by circuit configurations for a given function. Also, for example, the functional blocks of the present disclosure may be implemented in various programming or scripting languages. The functional blocks may be implemented as an algorithm running on one or more processors. In addition, the present disclosure may employ prior art for electronic configuration, signal processing, and/or data processing, and the like. Terms such as mechanism, element, means and configuration may be used broadly and are not limited to mechanical and physical configurations.

In addition, the connecting lines or connecting members between the components shown in the drawings only exemplify functional connections and/or physical or circuit connections. In an actual device, a connection between components may be represented by various functional connections, physical connections, or circuit connections that are replaceable or added.

Hereinafter, the present embodiments will be described in detail with reference to the accompanying drawings.

1 is a diagram for describing an example of a method for estimating blood pressure according to some embodiments.

Referring to FIG. 1 , a subject 10 may measure their own blood pressure using the mobile terminal 20 . In other words, the mobile terminal 20 may measure the blood pressure of the subject 10 and display an image representing the measured blood pressure on the screen. Here, the mobile terminal 20 refers to a device including at least one processor having data processing and arithmetic capabilities. For example, the mobile terminal 20 may be a smartphone, a wearable device, a tablet PC, or the like, but is not limited thereto.

In general, in order to measure blood pressure, a separate blood pressure measuring device including a guff is required. Therefore, in order for the subject 10 to measure blood pressure, it is necessary to visit a place (eg, a hospital) equipped with a separate blood pressure measuring device, so that the subject 10 can freely measure blood pressure at a desired time and place. is subject to restrictions.

Meanwhile, in recent years, a technology for measuring blood pressure through a camera included in a mobile terminal has been developed. Specifically, when the subject 10 touches the camera with a finger, a method in which the blood pressure of the subject 10 is measured has become popular. In this case, the subject 10 must precisely contact the finger with the position of the camera, and there is an inconvenience of maintaining the finger contact while the blood pressure is measured.

The method for estimating blood pressure according to some embodiments can be implemented to operate in the mobile terminal 20 , and accordingly, the subject 10 can easily use the mobile terminal 20 to carry the blood pressure measurement result. can be checked. In addition, in the case of the method of estimating blood pressure according to some embodiments, it is possible to estimate (measure) blood pressure based on an image obtained by photographing the subject's 10 face. No unnecessary motion (eg, keeping a finger in contact with the camera) is required.

Hereinafter, examples of a method for estimating blood pressure and an apparatus for estimating blood pressure according to an exemplary embodiment will be described with reference to FIGS. 2 to 13 .

2 is a block diagram illustrating an example of an apparatus for estimating blood pressure according to some embodiments.

Referring to FIG. 2 , the blood pressure estimation apparatus 100 includes a first camera 110 , a second camera 120 , a memory 130 , and a processor 140 . In the blood pressure estimating apparatus 100 shown in FIG. 2, only the components related to the present embodiment are shown. Accordingly, it can be understood by those of ordinary skill in the art related to the present embodiment that other general-purpose components may be further included in addition to the components shown in FIG. 2 .

The blood pressure estimating apparatus 100 illustrated in FIG. 2 may be implemented as the mobile terminal 20 of FIG. 1 , and in more detail, may be implemented as a mobile terminal 300 , which will be described later with reference to FIG. 3 .

The first camera 110 captures at least one image corresponding to the first body part of the subject 10 . For example, the first body part may correspond to at least a portion of a finger of the subject 10 , and preferably, the first body part may correspond to a tip of a finger of the subject 10 . However, the first body part is not limited to the finger part, and any body part that can be photographed by the camera may be applied without limitation. An example in which the first camera 110 captures at least one image corresponding to the first body part will be described later with reference to FIG. 5 .

The second camera 120 captures at least one image representing the second body part of the subject 10 . For example, the second body part may correspond to the face of the subject 10 . However, the second body part is not limited to the face, and any body part that can be photographed by the camera may be applied without limitation. An example in which the second camera 120 captures at least one image corresponding to the second body part will be described later with reference to FIG. 6 .

The memory 130 stores images captured by the first camera 110 and the second camera 120 . In addition, the memory 130 stores at least one program necessary for the processor 140 to operate. Also, the memory 130 may store various data generated by the operation of the blood pressure estimating apparatus 100 or required for the blood pressure estimating apparatus 100 to operate.

The processor 140 generally controls the components included in the blood pressure estimation apparatus 100 . Also, the processor 140 predicts the blood pressure of the subject 10 based on the image captured by the first camera 110 and the image captured by the second camera 120 . For example, the processor 140 acquires a first pulse wave signal corresponding to the first body part based on at least one image captured by the first camera 110 . Also, the processor 140 acquires a second pulse wave signal corresponding to the second body part based on at least one image captured by the second camera 120 . Then, the processor 140 corrects the second pulse wave signal based on the first pulse wave signal, and the blood pressure of the subject 10 from the corrected second pulse wave signal and the first pulse wave signal using machine learning technology. predict

For example, the processor 140 may be implemented as an array of a plurality of logic gates, or may be implemented as a combination of a general-purpose microprocessor and a memory in which a program executable in the microprocessor is stored. In addition, it can be understood by those skilled in the art that the present embodiment may be implemented in other types of hardware.

An example in which the processor 140 predicts the blood pressure of the subject 10 will be described later with reference to FIGS. 4 to 10 .

Meanwhile, as described above with reference to FIG. 1 , the blood pressure estimating apparatus 100 may be implemented by the mobile terminal 20 . Also, the operation of the blood pressure estimating apparatus 100 (specifically, the function of the processor 140 ) may be implemented by a server (not shown) separated from the mobile terminal 20 . Hereinafter, an example in which the blood pressure estimating apparatus 100 is implemented in the mobile terminal 20 will be described with reference to FIG. 3 .

3 is a configuration diagram illustrating an example of a mobile terminal according to some embodiments.

The blood pressure estimating apparatus 100 described above with reference to FIG. 2 may be the mobile terminal 300 illustrated in FIG. 3 or the processor 320 of the mobile terminal, or may be a separate device.

Referring to FIG. 3 , the mobile terminal 300 may include a processor 310 , a communication unit 320 , and an output unit 330 . However, not all illustrated components are essential components. The mobile terminal 300 may be implemented by more elements than the illustrated elements, and the mobile terminal 300 may be implemented by fewer elements than that.

For example, the mobile terminal 300 according to an embodiment includes a user input unit 340 , a sensing unit 350 , and an A/V input unit 360 in addition to the processor 310 , the communication unit 320 , and the output unit 330 . and a memory 370 .

Hereinafter, the components will be described in turn.

The processor 310 typically controls the overall operation of the mobile terminal 300 . For example, the processor 310 executes programs stored in the memory 370 , and thus the communication unit 320 , the output unit 330 , the user input unit 340 , the sensing unit 350 , and the A/V input unit 360 . ) and the memory 370 can be controlled in general.

The processor 310 may extract a pulse wave signal from the plurality of images while acquiring a plurality of images of the subject 10 through the camera 361 and the memory 370 . In addition, the processor 310 may correct the pulse wave signal and predict the blood pressure of the subject 10 from the corrected pulse wave signal using a machine learning technique.

Meanwhile, although it has been described that the processor 310 of the mobile terminal 300 predicts the blood pressure of the subject 10 , the present invention is not limited thereto. For example, the mobile terminal 300 acquires only images of the first body part and the second body part of the subject 10 , and transmits the image data to an external server (not shown) to measure the blood pressure of the subject 10 . Of course, it can be predicted.

The communication unit 320 may include one or more components that allow communication between the mobile terminal 300 and a server (not shown). For example, the communication unit 320 may include a short-distance communication unit 321 , a mobile communication unit 322 , and a broadcast receiving unit 323 .

Short-range wireless communication unit 321 is a Bluetooth communication unit, BLE (Bluetooth Low Energy) communication unit, short-range wireless communication unit (Near Field Communication unit), WLAN (Wi-Fi) communication unit, Zigbee (Zigbee) communication unit, infrared ( It may include an IrDA, infrared Data Association) communication unit, a Wi-Fi Direct (WFD) communication unit, an ultra wideband (UWB) communication unit, an Ant+ communication unit, and the like, but is not limited thereto. The mobile communication unit 322 transmits/receives a radio signal to and from at least one of a base station, an external terminal, and a server on a mobile communication network. Here, the wireless signal may include various types of data according to transmission and reception of a voice call signal, a video call signal, or a text/multimedia message. The broadcast receiving unit 323 receives a broadcast signal and/or broadcast-related information from the outside through a broadcast channel. The broadcast channel may include a satellite channel and a terrestrial channel. Depending on the embodiment, the mobile terminal 300 may not include the broadcast receiver 323 .

The output unit 330 outputs blood pressure information of the subject 10 .

The output unit 330 is for outputting an audio signal, a video signal, or a vibration signal, and may include a display unit 331 , a sound output unit 332 , a vibration motor 333 , and the like. The display unit 331 includes a key pad, a dome switch, a touch pad (contact capacitive method, pressure resistance film method, infrared sensing method, surface ultrasonic conduction method, integral tension measurement method, piezo effect method, etc.), a jog wheel, a jog switch, etc. may be included, but is not limited thereto.

Meanwhile, the display unit 331 may be configured as a touch screen by forming a layer structure of a touch pad. The display unit 331 includes a liquid crystal display, a thin film transistor-liquid crystal display, an organic light-emitting diode, a flexible display, a three-dimensional display ( 3D display) and electrophoretic display (electrophoretic display) may include at least one. Also, depending on the implementation form of the mobile terminal 300 , the mobile terminal 300 may include two or more display units 331 .

The sound output unit 332 outputs audio data received from the communication unit 320 or stored in the memory 370 . Also, the sound output unit 332 outputs a sound signal related to a function (eg, a call signal reception sound, a message reception sound, and a notification sound) performed in the mobile terminal 300 . The sound output unit 332 may include a speaker, a buzzer, and the like.

The vibration motor 333 may output a vibration signal. For example, the vibration motor 333 may output a vibration signal corresponding to the output of audio data or video data (eg, a call signal reception sound, a message reception sound, etc.). Also, the vibration motor 733 may output a vibration signal when a touch is input to the touch screen.

The user input unit 340 means a means for a user to input data for controlling the mobile terminal 300 . For example, the user input unit 340 includes a key pad, a dome switch, and a touch pad (contact capacitive method, pressure resistance film method, infrared sensing method, surface ultrasonic conduction method, integral type). There may be a tension measurement method, a piezo effect method, etc.), a jog wheel, a jog switch, and the like, but is not limited thereto.

The sensing unit 350 may detect a state of the mobile terminal 300 or a state around the mobile terminal 300 , and transmit the sensed information to the processor 310 .

On the other hand, the sensing unit 350, a geomagnetic sensor (Magnetic sensor) 351, acceleration sensor (Acceleration sensor) 352, temperature / humidity sensor 353, infrared sensor 354, gyroscope sensor 355, It may include, but is not limited to, at least one of a location sensor (eg, GPS) 356 , a barometric pressure sensor 357 , a proximity sensor 358 , and an RGB sensor (illuminance sensor) 359 . Since a function of each sensor can be intuitively inferred from the name of a person skilled in the art, a detailed description thereof will be omitted.

The A/V (Audio/Video) input unit 360 is for inputting an audio signal or a video signal, and may include a camera 361 , a microphone 362 , and the like.

The camera 361 may obtain an image frame such as a still image or a moving image through an image sensor in a video call mode or a shooting mode. The image captured through the image sensor may be processed through the processor 310 or a separate image processing unit (not shown).

The image frame processed by the camera 361 may be stored in the memory 370 or transmitted to the outside through the communication unit 320 . Two or more cameras 361 may be provided according to the configuration of the mobile terminal. The camera 361 according to the embodiment may be provided on the rear side of the mobile terminal 300 as shown in FIG. 5 and/or on the front side of the mobile terminal 300 as shown in FIG. 6 .

The microphone 362 receives an external sound signal and processes it as electrical voice data. For example, the microphone 362 may receive an acoustic signal from an external device or a speaker. The microphone 362 may use various noise removal algorithms for removing noise generated in the process of receiving an external sound signal.

The memory 370 may store a program for processing and control of the processor 310 or may store input/output data.

The memory 370 may include a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (eg, SD or XD memory), and a RAM. (RAM, Random Access Memory) SRAM (Static Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), magnetic memory, magnetic disk , may include at least one type of storage medium among optical disks. Also, the mobile terminal 300 may operate a web storage or a cloud server that performs a storage function of the memory 370 on the Internet.

Programs stored in the memory 370 may be classified into a plurality of modules according to their functions, for example, may be classified into a UI module 371 , a touch screen module 372 , a notification module 373 , and the like. .

The UI module 371 may provide a specialized UI, GUI, etc. that are interlocked with the mobile terminal 300 for each application. The touch screen module 372 may detect a touch gesture on the user's touch screen and transmit information about the touch gesture to the processor 310 . The touch screen module 372 according to an embodiment of the present invention may recognize and analyze a touch code. The touch screen module 372 may be configured as separate hardware including a controller.

Various sensors may be provided inside or near the touch screen to detect a touch or a proximity touch of the touch screen. As an example of a sensor for detecting a touch of a touch screen, there is a tactile sensor. A tactile sensor refers to a sensor that senses a touch of a specific object to the extent or higher than that of a human being. The tactile sensor may sense various information such as the roughness of the contact surface, the hardness of the contact object, and the temperature of the contact point.

In addition, as an example of a sensor for detecting a touch of a touch screen, there is a proximity sensor.

The proximity sensor refers to a sensor that detects the presence or absence of an object approaching a predetermined detection surface or an object existing in the vicinity without mechanical contact using the force of an electromagnetic field or infrared rays. Examples of the proximity sensor include a transmission type photoelectric sensor, a direct reflection type photoelectric sensor, a mirror reflection type photoelectric sensor, a high frequency oscillation type proximity sensor, a capacitive type proximity sensor, a magnetic type proximity sensor, and an infrared proximity sensor. The user's touch gesture may include tap, touch & hold, double tap, drag, pan, flick, drag and drop, swipe, and the like.

The notification module 373 may generate a signal for notifying the occurrence of an event in the mobile terminal 300 . Examples of events generated in the mobile terminal 300 include call signal reception, message reception, key signal input, schedule notification, and the like. In addition, as an example of an event occurring in the mobile terminal 300 , a signal indicating that a user input is received may be generated based on a haptic signal generated based on a user input received from the display unit 331 .

The notification module 373 may output a notification signal in the form of a video signal through the display unit 331 , may output a notification signal in the form of an audio signal through the sound output unit 332 , and the vibration motor 333 . It is also possible to output a notification signal in the form of a vibration signal through

4 is a flowchart illustrating an example of a method for estimating blood pressure according to some embodiments.

Referring to FIG. 4 , the method for estimating blood pressure includes steps that are time-series processed by the blood pressure estimating apparatus 100 or the mobile terminals 20 and 300 illustrated in FIGS. 1 to 3 . Therefore, even if omitted below, it should be noted that the contents described above with respect to the blood pressure estimating apparatus 100 or the mobile terminals 20 and 300 illustrated in FIGS. 1 to 3 are also applied to the method of estimating the blood pressure of FIG. 4 . Able to know.

In step 410 , the processors 140 and 310 perform a first operation corresponding to the first body part based on at least one image representing the first body part of the subject 10 photographed through the first cameras 110 and 361 . Acquire a pulse wave signal. Here, the first pulse wave signal means a signal having a relatively high signal to noise ratio compared to a second pulse wave signal to be described later. For example, the first pulse wave signal may be a PPG signal corresponding to the first body part, but is not limited thereto.

For example, the first body part may correspond to at least a portion of a finger of the subject 10 , and preferably, the first body part may correspond to a tip of a finger of the subject 10 . However, the first body part is not limited to the finger part, and any body part that can be photographed by the camera may be applied without limitation.

For example, when the subject 10 touches the tip of a finger to a point where the first cameras 110 and 361 are located, the first cameras 110 and 361 may capture at least one image of the touched portion. have. Hereinafter, an example in which the first cameras 110 and 361 operate will be described with reference to FIG. 5 .

5 is a view for explaining an example in which a first camera captures an image representing a first body part, according to some embodiments.

Referring to FIG. 5 , an example in which a finger of the subject 10 is in contact with the rear camera 510 of the mobile terminal 300 and an image of the contacted finger is captured is illustrated. An image of a finger may be acquired through a rear camera 510 and a memory 370 that are generally built in the mobile terminal 300 .

For example, the rear camera 510 may acquire video data of approximately 30 seconds, and in the case of 30 fps, an image of approximately 900 frames may be obtained. Here, although it is an exemplary number of frames for detecting a pulse wave signal, it is not limited to the above-described number of frames.

As will be described later with reference to FIGS. 7A to 7D , the processors 140 and 310 may extract the first pulse wave signal from the captured image using various signal processing techniques (eg, IIR filter, etc.).

Referring back to FIG. 4 , in step 420 , the processors 140 and 310 perform a second operation based on at least one image representing a second body part of the subject 10 captured through the second cameras 120 and 361 . A second pulse wave signal corresponding to the body part is acquired. Here, the first cameras 110 and 361 and the second cameras 120 and 361 are disposed at positions spaced apart from each other. For example, the second pulse wave signal may be a PPG signal corresponding to the second body part, but is not limited thereto.

For example, the second body part may correspond to the face of the subject 10 . However, the second body part is not limited to the face, and any body part that can be photographed by the camera may be applied without limitation.

For example, by the operation of the subject 10 , the second cameras 120 and 361 may capture at least one image representing the face of the subject 10 . Hereinafter, an example in which the second cameras 120 and 361 operate will be described with reference to FIG. 6 .

6 is a view for explaining an example in which a second camera captures an image representing a second body part, according to some embodiments.

Referring to FIG. 6 , an example in which the face of the subject 10 is positioned adjacent to the front camera 610 of the mobile terminal 300 and an image of the face is captured is illustrated. An image of the face may be acquired through the front camera 610 and the memory 370 that are generally built in the mobile terminal 300 .

For example, the front camera 610 may acquire video data of approximately 30 seconds, and in the case of 30 fps, an image of approximately 900 frames may be obtained. Here, although it is an exemplary number of frames for detecting a pulse wave signal, it is not limited to the above-described number of frames.

As will be described later with reference to FIGS. 7A to 7D , the processors 140 and 310 may extract the second pulse wave signal from the captured image using various signal processing techniques (eg, IIR filter, etc.).

7A to 7D are diagrams for explaining an example in which a processor extracts a first pulse wave signal and a second pulse wave signal.

The processors 140 and 310 extract a pulse wave signal from the images using various signal processing techniques (eg, an IIR filter, etc.). Here, the pulse wave signal means a graph showing the pulsation of the peripheral vascular system that occurs through contraction and relaxation of the heart. The technique of detecting the pulse wave period on such a graph is possible in the time domain and the frequency domain, and the methods in the time domain include peak picking, autocorrelation function, AMDF (Average Magnitude Difference Function), and the like. The method may use high-frequency peak detection, spectral similarity analysis, or the like.

For example, as shown in FIG. 7A , the processors 140 and 310 may separately record time information for each frame acquired by the first camera and the second camera. Here, the time information may be, for example, in milliseconds (ms) units.

And, as shown in FIG. 7B , the processors 140 and 310 may average the G value among the RGB values of each frame for each frame. And, as shown in FIG. 7C , the processors 140 and 310 may extract the first pulse wave signal and the second pulse wave signal through the IIR filter.

Then, the processors 140 and 310 may correct the first pulse wave signal and the second pulse wave signal as shown in FIG. 7D after applying the cubic spline interpolation method based on the time information for each frame. . Here, although it has been described that the pulse wave signal is corrected by applying the cubic spline interpolation method, the present invention is not limited thereto, and various interpolation methods may be applied.

Referring back to FIG. 4 , in step 430 , the processors 140 and 310 correct the second pulse wave signal based on the first pulse wave signal. Here, the first pulse wave signal means a signal having a relatively high signal-to-noise ratio compared to the second pulse wave signal.

For example, the processors 140 and 310 may correct the second pulse wave signal by removing noise included in the second pulse wave signal based on the first pulse wave signal through an adaptive filter. can Here, the adaptive filter may correspond to at least one of a least mean square (LMS) filter, a normalized least mean square (NLMS) filter, and a recursive least square (RLS) filter.

For example, if it is assumed that the first body part is a finger and the second body part is the face, the processors 140 and 310 may extract a pulse wave signal by tracking changes in skin color based on an image of the face. . Specifically, in the case of the face, although it is invisible to the human eye, the color of the skin is slightly changed according to the flow of blood, so the processors 140 and 310 display images captured by the second cameras 120 and 361 . Based on the , the second pulse wave signal may be extracted.

Meanwhile, the second pulse wave signal may have a lower signal-to-noise ratio than the first pulse wave signal. Accordingly, the processors 140 and 310 may obtain a more accurate pulse wave signal by correcting the second pulse wave signal based on the first pulse wave signal. Accordingly, the processors 140 and 310 may accurately predict the blood pressure of the subject 10 .

Hereinafter, an example in which the processors 140 and 310 correct the second pulse wave signal will be described with reference to FIGS. 8 and 9 .

8 is a diagram for describing an example in which a processor corrects a second pulse wave signal according to some embodiments.

8 illustrates an example of an adaptive filter used by the processors 140 and 310 to correct the second pulse wave signal. In FIG. 8 , d(n) denotes a second pulse wave signal, and the second pulse wave signal includes a sum of a noise component N(n) and a signal component X′(n). Also, X(n) denotes the first pulse wave signal. Also, y(n) denotes a corrected second pulse wave signal. Therefore, when y(n) and X'(n) are the same, it means that all noise components are removed from the second pulse wave signal. Also, e(n) denotes an error component generated according to a comparison operation between the first pulse wave signal and the second pulse wave signal.

Since the first pulse wave signal and the second pulse wave signal are signals obtained from the same subject 10 , they have high correlation. Accordingly, a noise component may be removed from the second pulse wave signal based on the first pulse wave signal.

Referring to FIG. 8 , the first pulse wave signal is calculated with the second pulse wave signal through a digital filter. Here, the digital filter corresponds to a general technique in the field of signal processing, and detailed description thereof will be omitted.

The error component derived through the comparison operation of the digitally filtered first pulse wave signal and the second pulse wave signal is input to the adaptive algorithm, so that the difference between the first pulse wave signal and the second pulse wave signal is minimized (that is, the error component is minimized) . Accordingly, since the noise component of the second pulse wave signal is removed, the corrected second pulse wave signal may be output as a final signal.

9 is a diagram for describing an example of a corrected second pulse wave signal according to some embodiments.

9 shows an example of the second pulse wave signal corrected through the adaptive filter of FIG. 8 . Specifically, FIG. 9 shows a first pulse wave signal 940 , a second pulse wave signal 920 , a corrected second pulse wave signal 930 , and an ideal second pulse wave signal 910 .

Referring to FIG. 9 , as the processors 140 and 310 correct the second pulse wave signal 920 through the adaptive filter, the corrected second pulse wave signal 930 may be output. At this time, since the corrected second pulse wave signal 930 can be derived very similarly to the ideal second pulse wave signal 910 , the blood pressure of the subject 10 can be accurately measured.

Referring back to FIG. 4 , in step 440 , the processors 140 and 310 predict the blood pressure of the subject 10 from the corrected second pulse wave signal and the first pulse wave signal using a machine learning technique.

For example, the processors 140 and 310 may calculate the blood pressure of the subject 10 based on the pulse transit time (PTT) of the first pulse wave signal and the corrected second pulse wave signal. Specifically, since the PTT is inversely proportional to the blood pressure, the processors 140 and 310 may calculate the blood pressure of the subject 10 based on the PTT. For example, the processors 140 and 310 may check the PTT from a time when a peak is confirmed in the first pulse wave signal to a time when a peak is confirmed in the second pulse wave signal.

Here, the machine learning technique may be a machine learning technique based on a regression model, but is not limited thereto. Machine learning technology refers to an algorithm or technology that enables an appropriate operation to be performed on new data based on what has been learned using multiple data, and can be implemented as a neural network.

The machine learning technology may be a set of algorithms for learning a method of predicting blood pressure from the corrected second pulse wave signal input to the neural network based on artificial intelligence. For example, machine learning may learn a method of predicting blood pressure from the corrected second pulse wave signal based on supervised learning and/or unsupervised learning. Also, for example, the neural network predicts blood pressure from the corrected second pulse wave signal and the first pulse wave signal using reinforcement learning using feedback on whether a result of predicting blood pressure according to learning is correct. can learn how to

In addition, the neural network performs calculations for inference and prediction according to artificial intelligence (AI) technology. Specifically, the neural network may be a deep neural network (DNN) that performs an operation through a plurality of layers. A neural network is classified as a deep neural network (DNN) when the number of layers is plural according to the number of internal layers performing the operation, that is, when the depth of the neural network performing the operation increases. can be In addition, the deep neural network (DNN) operation may include a convolutional neural network (CNN) operation and the like.

That is, the processors 140 and 310 may implement a model for predicting blood pressure through the exemplified neural network, and train the implemented model using training data. Then, the blood pressure of the subject 10 may be predicted by analyzing or classifying the corrected second pulse wave signal and the first pulse wave signal input using the learned model.

10 is a diagram for describing an example in which an apparatus for estimating blood pressure according to some exemplary embodiments operates.

FIG. 10 illustrates the method for estimating blood pressure described above with reference to FIGS. 1 to 9 and steps processed in time series in the mobile terminal. Accordingly, it can be seen that the above-described contents with reference to FIGS. 1 to 9 are also applied to the contents shown in FIG. 10 even if the contents are omitted below.

The processors 140 and 310 capture a first image representing a first body part in contact with the first camera, and capture a second image representing a second body part through the second camera. Here, the first body part may mean a finger, and the second body part may mean a face, but is not limited thereto.

Thereafter, the processors 140 and 310 obtain a first pulse wave signal through the first image and a second pulse wave signal through the second image. For example, the processors 140 and 310 may recognize a second body part from the second image and acquire a second pulse wave signal from the recognized second body part.

Thereafter, the processors 140 and 310 correct the second pulse wave signal through the adaptive filter. Specifically, since the first pulse wave signal is a signal having a relatively high signal-to-noise ratio compared to the second pulse wave signal, the processors 140 and 310 use the adaptive filter to perform the second pulse wave signal based on the first pulse wave signal. can be corrected.

Thereafter, the processors 140 and 310 predict the blood pressure of the subject from the corrected second pulse wave signal and the first pulse wave signal using a machine learning technique. For example, the processors 140 and 310 may calculate the blood pressure of the subject 10 based on the PTT of the first pulse wave signal and the corrected second pulse wave signal.

In addition, the processors 140 and 310 may predict the subject's blood pressure by further considering the subject's height, weight, gender, age, and the like, but is not limited thereto.

The method for estimating blood pressure according to some embodiments may also be implemented in the form of a recording medium including instructions executable by a computer, such as a program module executed by a computer. Computer-readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. In addition, computer-readable media may include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Communication media typically includes computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave, or other transport mechanism, and includes any information delivery media.

Also, in this specification, a unit may be a hardware component such as a processor or circuit, and/or a software component executed by a hardware component such as a processor.

The description of the present invention described above is for illustration, and those of ordinary skill in the art to which the present invention pertains can understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a dispersed form, and likewise components described as distributed may also be implemented in a combined form.

The scope of the present invention is indicated by the following claims rather than the above detailed description, and all changes or modifications derived from the meaning and scope of the claims and their equivalents should be interpreted as being included in the scope of the present invention. do.

10: Subject
20: mobile terminal

Claims (15)

In the method of measuring the blood pressure of a subject,
acquiring a first pulse wave signal corresponding to the first body part based on at least one image representing a first body part of the examinee captured through a first camera;
acquiring a second pulse wave signal corresponding to the second body part based on at least one image representing a second body part of the examinee captured through a second camera disposed at a location spaced apart from the first camera ;
correcting the second pulse wave signal based on the first pulse wave signal; and
Predicting the blood pressure of the subject from the corrected second pulse wave signal and the first pulse wave signal using a machine learning technique. The method of claim 1,
The first pulse wave signal includes a signal having a relatively high signal to noise ratio compared to the second pulse wave signal. The method of claim 1,
The correcting step is
A method of correcting the second pulse wave signal by removing noise included in the second pulse wave signal based on the first pulse wave signal through an adaptive filter. 4. The method of claim 3,
The adaptive filter includes at least one of a Least Mean Square (LMS) filter, a Normalized Least Mean Square (NLMS) filter, and a Recursive Least Square (RLS) filter. The method of claim 1,
Acquiring the first pulse wave signal comprises:
A method of generating a first image signal by averaging a G value among RGB values of the at least one image for each image frame, and extracting the pulse wave signal by applying an IIR filter to the generated first image signal. The method of claim 1,
The machine learning technique includes a machine learning technique based on a regression model. The method of claim 1,
The first body part comprises a finger of the subject, and the second body part comprises a face of the subject. A computer-readable recording medium in which a program for executing the method of any one of claims 1 to 7 on a computer is recorded. In the device for measuring the blood pressure of a subject,
a first camera;
a second camera disposed at a position spaced apart from the first camera;
Memory; and
processor; including;
The processor is
A first pulse wave signal corresponding to the first body part is acquired based on at least one image corresponding to the first body part of the examinee captured by the first camera, and the second pulse wave signal captured by the second camera is obtained. Obtaining a second pulse wave signal corresponding to the second body part based on at least one image representing a second body part of the subject, correcting the second pulse wave signal based on the first pulse wave signal, and machine learning An apparatus for predicting the blood pressure of the subject from the corrected second pulse wave signal and the first pulse wave signal using a machine learning technique. 10. The method of claim 9,
The first pulse wave signal includes a signal having a relatively high signal to noise ratio compared to the second pulse wave signal. 10. The method of claim 9,
The processor is
An apparatus for correcting the second pulse wave signal by removing noise included in the second pulse wave signal based on the first pulse wave signal through an adaptive filter. 12. The method of claim 11,
The adaptive filter includes at least one of a Least Mean Square (LMS) filter, a Normalized Least Mean Square (NLMS) filter, and a Recursive Least Square (RLS) filter. 10. The method of claim 9,
The processor is
An apparatus for generating a first image signal by averaging a G value among RGB values of the at least one image for each image frame, and extracting the pulse wave signal by applying an IIR filter to the generated first image signal. 10. The method of claim 9,
The machine learning technique is an apparatus including a machine learning technique based on a regression model. 10. The method of claim 9,
The first body part comprises a finger of the subject, and the second body part comprises a face of the subject.

Images