KR20200064744A - Non contact blood pressure measuring method and apparatus using video - Google Patents

Abstract

Disclosed are a non-contact type blood pressure measuring device and method using an image. According to the present invention, the non-contact type blood pressure measuring method using an image, performed by the non-contact type blood pressure measuring device using an image, comprises the steps of: receiving image information including the face and a palm of a user; generating facial skin area information and hand skin area information of the user from the image information; calculating heart rate information of the user using the facial skin area information and the hand skin area information; and calculating blood pressure information of the user by applying the heart rate information to a personalized blood pressure measurement model.

Description

Non-contact blood pressure measuring device and method using video{NON CONTACT BLOOD PRESSURE MEASURING METHOD AND APPARATUS USING VIDEO}

The present invention relates to a non-contact blood pressure measurement technology using an image, and more particularly, to a technology for measuring a user's blood pressure by extracting a heartbeat signal of a user's face region and hand region included in the image information.

Recently, healthcare-related IT devices can measure an individual's exercise or heart rate through a wearable device in the form of a watch or bracelet worn on the wrist. Attempts to acquire the user's biosignal information through the wearable device have continued.

The technology for measuring the heart rate in the ear ball or ear by embedding the heart rate extraction sensor in the earphone or headphone device was introduced, and the chest belt type heart rate measuring device worn around the user's chest for use in a more dynamic environment has already been introduced. It is sold. In addition, wearable devices interworking with smartphones are also being released these days.

However, it is still common to measure blood pressure with a sphygmomanometer sensor using pressure in the upper arm, the upper arm of the arm. There is also a wrist blood pressure monitor with a reduced volume so that it can be worn on the wrist for increased portability, but still requires some uncomfortable measurement using a physical device.

Therefore, there is a need to develop a technology that can easily measure blood pressure, which is an important signal among biometric information signals, in a non-contact manner.

Korean Registered Patent No. 10-1631925, published on 04/04/2016 (Name: Marine multi-band network selection system)

An object of the present invention is to allow the user's blood pressure to be measured in a non-contact manner.

In addition, an object of the present invention is to extract the user's heart rate signal from the image information, and analyze the extracted heart rate signal to measure the user's blood pressure.

In addition, an object of the present invention is to make it easier to measure blood pressure than a method for measuring blood pressure in the upper arm of the arm.

In addition, an object of the present invention is to make it possible to easily measure blood pressure using a user terminal with a built-in imaging module without a blood pressure monitor.

A method of measuring a non-contact blood pressure using an image performed by a non-contact blood pressure measuring apparatus using an image according to the present invention for achieving the above object comprises: receiving image information including a user's face and palm, from the image information Generating face skin area information and hand skin area information of the user, calculating heart rate information of the user using the face skin area information and the hand skin area information, and the personalized blood pressure measurement model. And applying the heart rate information to calculate blood pressure information of the user.

In this case, the personalized blood pressure measurement model may be a blood pressure measurement model of the user generated based on the image data of the user received while measuring the blood pressure of the user with a blood pressure monitor and blood pressure numerical information of the blood pressure monitor.

At this time, while measuring the blood pressure of the user using the blood pressure monitor, receiving the image data of the user, receiving the blood pressure numerical information measured using the blood pressure monitor, and the user obtained through multiple measurements The method may further include generating the personalized blood pressure measurement model of the user based on the plurality of the image data and the blood pressure numerical information corresponding to.

At this time, the step of generating the face skin area information and the hand skin area information comprises converting the image information, which is an RGB model, to an HSV color model, setting a mask, and applying the set mask to the RGB model to apply the face skin area. At least one of information and the hand skin region information may be generated.

In this case, in the generating of the facial skin area information and the hand skin area information, the mask corresponding to the hand skin area information may be set by using the reference skin threshold value for the mask corresponding to the facial skin area information. have.

In addition, a non-contact blood pressure measuring apparatus using an image according to an embodiment of the present invention, an image information input unit for receiving image information including a user's face and palm, from the image information, the user's face skin area information and hand skin A skin area information generating unit generating area information, a heart rate information calculating unit calculating the heart rate information of the user using the face skin area information and the hand skin area information, and applying the heart rate information to a personalized blood pressure measurement model Thus, it includes a blood pressure measuring unit for calculating the blood pressure information of the user.

In this case, the personalized blood pressure measurement model may be a blood pressure measurement model of the user generated based on the image data of the user received while measuring the blood pressure of the user with a blood pressure monitor and blood pressure numerical information of the blood pressure monitor.

At this time, while measuring the blood pressure of the user using the blood pressure monitor, the user's image data is input, the blood pressure numerical information measured using the blood pressure monitor is input, and corresponding to the user obtained through multiple measurements The blood pressure measurement model generation unit may further include a blood pressure measurement model generator that generates the personalized blood pressure measurement model of the user based on the plurality of image data and the blood pressure numeric information.

At this time, the skin region information generating unit converts the image information, which is an RGB model, into an HSV color model, sets a mask, and applies the set mask to the RGB model to at least one of the face skin region information and the hand skin region information. You can create either.

In this case, the skin area information generating unit may set a mask corresponding to the hand skin area information by using the reference skin threshold value for the mask corresponding to the face skin area information.

According to the present invention, the user's blood pressure can be measured in a non-contact manner.

In addition, according to the present invention, the user's heart rate signal can be extracted from the image information, and the extracted heart rate signal can be analyzed to measure the user's blood pressure.

In addition, according to the present invention, blood pressure can be measured more easily than a method of measuring blood pressure in the upper arm of the arm.

In addition, according to the present invention, a blood pressure can be easily measured using a user terminal with a built-in imaging module without a blood pressure monitor.

1 is a view schematically showing an environment in which a non-contact blood pressure measuring apparatus using an image according to an embodiment of the present invention is applied.
Figure 2 is a block diagram showing the configuration of a non-contact blood pressure measuring apparatus using an image according to an embodiment of the present invention.
3 is a flow chart for explaining a non-contact blood pressure measurement method using an image according to an embodiment of the present invention.
4 is an exemplary view showing image information according to an embodiment of the present invention.
5 is a graph showing heart rate signals of a facial skin region and a hand skin region according to an embodiment of the present invention.
6 is a flowchart illustrating a process of calculating heart rate information according to an embodiment of the present invention.
7 is an exemplary view showing a user's face region and face skin region according to an embodiment of the present invention.
8 is a flowchart illustrating a process of generating a personalized blood pressure measurement model according to an embodiment of the present invention.
9 is a block diagram showing a computer system according to an embodiment of the present invention.

The present invention can be applied to various changes and can have various embodiments, and specific embodiments will be illustrated in the drawings and described in detail.

However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention.

The terms used in this application are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, terms such as “include” or “have” are intended to indicate the presence of features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, one or more other features. It should be understood that the existence or addition possibilities of fields or numbers, steps, operations, components, parts or combinations thereof are not excluded in advance.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person skilled in the art to which the present invention pertains. Terms such as those defined in a commonly used dictionary should be interpreted as having meanings consistent with meanings in the context of related technologies, and should not be interpreted as ideal or excessively formal meanings unless explicitly defined in the present application. Does not.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In order to facilitate the overall understanding in describing the present invention, the same reference numerals are used for the same components in the drawings, and duplicate descriptions for the same components are omitted.

1 is a view schematically showing an environment in which a non-contact blood pressure measuring apparatus using an image according to an embodiment of the present invention is applied.

As shown in FIG. 1, a non-contact blood pressure measurement system using an image may include at least one of a photographing device 100, a non-contact blood pressure measurement device 200 using an image, and a user terminal 300.

The photographing apparatus 100 includes a photographing module capable of photographing image information, and may generate image information by photographing a user. At this time, the photographing apparatus 100 may photograph an image including the upper body of the user, particularly the user's face and palms. In addition, the photographing apparatus 100 may transmit image information to the non-contact blood pressure measuring apparatus 200 using an image.

Here, the image information may be an image captured when the user takes the basic posture for blood pressure measurement, and the basic posture for blood pressure measurement is shown by the user as shown in FIG. 4 to be described later. It may be a holding position facing toward 100).

For convenience of description, the photographing apparatus 100 is described as photographing image information to be used for estimating blood pressure information of a user, but is not limited thereto, and the photographing apparatus 100 is an image for generating a personalized blood pressure measurement model. Data may be photographed, and image data may be transmitted to the non-contact blood pressure measuring apparatus 200 using the image.

The photographing apparatus 100 may be any one of a smartphone camera, a laptop camera, a web camera for a PC, and a camera capable of taking a general digital video, and the type of the camera is not limited thereto.

When the user terminal 300 to be described later includes a photographing module, the non-contact blood pressure measurement system using an image may not include the photographing apparatus 100.

Next, the non-contact blood pressure measurement apparatus 200 using the image may generate a personalized blood pressure measurement model for the user. In addition, the non-contact blood pressure measurement apparatus 200 using an image may measure a user's blood pressure information in a non-contact manner by using the generated personalized blood pressure measurement model.

The non-contact blood pressure measurement apparatus 200 using an image may receive image data of a user from at least one of the imaging device 100 and the user terminal 300, and a personalized blood pressure measurement model based on the received image data. Can be created.

When creating a personalized blood pressure measurement model, the non-contact blood pressure measurement apparatus 200 using an image may receive at least one of image data for a user from the user terminal 300, blood pressure numerical information measured using a blood pressure monitor, and user information. Can be.

In particular, the non-contact blood pressure measuring apparatus 200 using an image may receive image data when a user measures blood pressure using a blood pressure monitor along with blood pressure numerical information, and the non-contact blood pressure measuring apparatus 200 using an image may be a user. Regression analysis may be performed using a plurality of image data and a plurality of blood pressure numerical values for, and through this, a personalized blood pressure measurement model of the user may be generated.

At this time, the non-contact blood pressure measuring apparatus 200 using an image extracts a heartbeat signal from a user's skin region included in the image data, and performs regression analysis based on the extracted heartbeat signal and actual blood pressure numerical information corresponding to the image data. Thus, a personalized blood pressure measurement model can be generated.

In addition, the non-contact blood pressure measurement apparatus 200 using the image may generate a personalized blood pressure measurement model, receive the user's image information, and extract the heart rate signal from the skin region included in the image information to generate heart rate information. . In addition, the non-contact blood pressure measuring apparatus 200 using the image may estimate the current blood pressure information of the user by applying the generated heart rate information to the personalized blood pressure measurement model.

At this time, the non-contact blood pressure measuring apparatus 200 using the image generates the user's face skin area information and the hand skin area information from the image information, and calculates the user's heart rate information using the face skin area information and the hand skin area information. Can be.

The non-contact blood pressure measurement apparatus 200 using an image may receive image information from the imaging device 100 or the user terminal 300 equipped with the imaging module, and estimate the user's blood pressure information based on the received image information. Can be. Here, the image information may be image information photographed with the palm of the hand while the user looks at the front of the photographing apparatus 100 or the photographing module.

The non-contact blood pressure measuring apparatus 200 using an image may be implemented as an application (App) driven on the user terminal 300, an application program for a PC, or as a separate device from the user terminal 300.

Finally, the user terminal 300 is equipped with a shooting module to take at least one of the image information and image data for the user, and transmits the captured image information or image data to the non-contact blood pressure measurement device 200 using the image Can be.

In addition, when the user measures the blood pressure using the blood pressure monitor in the process of calibrating the blood pressure measurement model, the user terminal 300 may receive the user's blood pressure numerical information from the user or receive it from a communicable blood pressure monitor, and display the blood pressure numerical information. It can be transmitted to the non-contact blood pressure measurement device 200 using.

In addition, when communication with the blood pressure monitor is possible, the user terminal 300 may control the blood pressure monitor to measure the blood pressure of the user at a time corresponding to the shooting time of the image data. In addition, the user terminal 300 may control the imaging module or the imaging device 100 such that the imaging module or the imaging device 100 is driven while the blood pressure monitor measures the user's blood pressure.

Here, the user terminal 300 is a mobile terminal such as a smartphone (Smart Phone), a smart pad (Smart Pad), a tablet computer (Tablet PC), a laptop (Note book), a netbook (Net book) with a built-in camera or a desktop (Desktop).

The type of the user terminal 300 is not limited thereto, and can be applied to various terminals such as all information and communication devices, multimedia terminals, wired terminals, fixed terminals, and IP (Internet Protocol) terminals, and mobile phones and PMPs (Portable Multimedia Players). ), a mobile Internet device (MID), a personal digital assistant (PDA), a smart TV and a terminal having various mobile communication specifications such as information and communication devices.

As described above, the non-contact blood pressure measuring apparatus 200 using an image according to an embodiment of the present invention can easily measure the blood pressure of a user who needs to follow blood pressure, such as a hypertensive patient, without a blood pressure monitor. In particular, since the user's blood pressure can be measured by photographing the user's face and hands through the camera built in the user terminal, the user can easily measure the blood pressure anytime, anywhere.

Hereinafter, a configuration of a non-contact blood pressure measuring apparatus using an image according to an embodiment of the present invention will be described in more detail with reference to FIG. 2.

Figure 2 is a block diagram showing the configuration of a non-contact blood pressure measuring apparatus using an image according to an embodiment of the present invention.

As shown in FIG. 2, the non-contact blood pressure measurement apparatus 200 using an image includes an image information input unit 210, a skin region information generation unit 220, a heart rate information calculation unit 230, a blood pressure measurement unit 240, and blood pressure Measurement model generation unit 250 may be included.

The image information input unit 210 receives image information including a user's face and palms. In this case, the image information input unit 210 may receive image information from at least one of the imaging device 100 and the user terminal 300 of FIG. 1.

For convenience of description, the non-contact blood pressure measuring device 200 using an image is described as receiving image information from an external device, but is not limited thereto, and the non-contact blood pressure measuring device 200 using an image is provided with an imaging module. In addition, image information may be generated using the provided imaging module.

In addition, the image information input unit 210 may receive image data for performing a blood pressure measurement model correction process. For convenience of explanation, the image information input unit 210 is named, but the image information input unit 210 may receive at least one of image data, blood pressure numerical information, and user information to perform a blood pressure measurement model correction process. .

Next, the skin region information generating unit 220 generates skin region information including at least one of the user's face skin region information and hand skin region information from the image information.

The skin area information generation unit 220 may detect a face area and a hand area included in image information using a face and hand tracker, and the face skin area information and the hand skin area information from the detected face area and hand area. Can generate

The skin region information generation unit 220 may convert image information, which is an RGB model, to an HSV color model, in order to extract only pure skin information from the image information. In addition, the skin region information generation unit 220 designates values in which the value of the Hue channel (H channel) is smaller than the reference skin threshold (th_skin) in the converted HSV model as the skin region, and determines that the other regions are not skin To set the mask. After setting the mask, the skin region information generating unit 220 may set the mask in the region of the G channel of the RGB model to generate skin region information specific to the G channel.

At this time, the skin area information generating unit 220 according to an embodiment of the present invention extracts the skin area from the face and hand areas as large as possible to generate wide skin area information, and thus derives skin area information based on a reference skin threshold. Can generate

In particular, the skin area information generating unit 220 may generate skin area information based on a reference skin threshold to minimize the difference between users and extract the skin area as large as possible, and the skin area according to the user's unique skin color. In order to secure as wide as possible, skin area information may be generated based on a plurality of correction threshold values.

When generating skin area information from image information multiple times, the skin area information generation unit 220 designates a skin area based on a reference skin threshold value (th_skin), and corrects and generates the reference skin threshold value. The skin region can be designated based on the correction threshold values. Here, the correction threshold values may be values generated by gradually changing the reference skin threshold value, or may be generated by modifying the reference skin threshold value within a preset range.

In addition, the skin area information generating unit 220 finds a combination in which the cross-sectional area ratio with the face area and the skin area area are the largest among the skin areas specified based on the reference skin threshold and the skin areas specified based on the correction threshold values. , You can create a mask with that combination. In this case, the skin region information generating unit 220 may generate a face mask with the combination, and then apply a skin threshold value corresponding to the combination to the hand region mask.

Next, the heart rate information calculating unit 230 calculates the heart rate information of the user by using facial skin area information and hand skin area information.

The heart rate information calculating unit 230 compares the heart rate signal extracted from the facial skin area information and the heart rate signal extracted from the hand skin area information, and the remote pulse transit time (rPTT), which is the information that is the basis of blood pressure estimation. Can be measured.

At this time, the heart rate information calculating unit 230 detects a peak of the heart rate signal extracted from the facial skin area information, and detects a peak of the heart rate signal extracted from the hand skin area information. In addition, the heart rate information calculating unit 230 may calculate a moving average of a peak of a heart rate signal corresponding to a face and a peak of a heart rate signal corresponding to a hand, and calculate a user's rPTT.

Here, the user's rPTT may be user's heart rate information, and is named as a remote pulse wave delivery time (rPTT) so as to be distinguished from a pulse wave delivery time (PTT) calculated from a heartbeat signal at an adjacent site.

In addition, the blood pressure measurement unit 240 may estimate the blood pressure information of the user by applying heart rate information to the personalized blood pressure measurement model. In particular, the blood pressure measurement unit 240 may apply rPTT to the personalized blood pressure measurement model and calculate blood pressure information based on the user's rPTT.

Finally, the blood pressure measurement model generation unit 250 generates a personalized blood pressure measurement model for the user.

The blood pressure measurement model generation unit 250 may generate a personalized blood pressure measurement model for the user by using blood pressure measurement information of the user and a plurality of image data corresponding to the blood pressure measurement information.

The blood pressure measurement model generation unit 250 may receive user image data while measuring a user's blood pressure using a blood pressure monitor, receive blood pressure numerical information of a user measured using a blood pressure monitor, and generate a blood pressure measurement model The 250 may generate a personalized blood pressure measurement model of the user based on the user's image data and blood pressure numerical information and user information received multiple times.

The personalized blood pressure measurement model generated by the blood pressure measurement model generation unit 250 is a personalized model for the user, and the blood pressure measurement model generation unit 250 may perform a correction process to correct the user's personalized blood pressure measurement model. .

In particular, the blood pressure measurement model generation unit 250 may perform regression analysis using rPTT values and blood pressure numerical information repeatedly measured from the image data, and blood pressure information and blood pressure numerical information of the blood pressure monitor, which are the results of the regression analysis, may be performed. Based on the difference value, it is possible to determine whether to perform the calibration process of the blood pressure measurement model.

Hereinafter, a non-contact blood pressure measuring method using an image performed by a non-contact blood pressure measuring apparatus using an image according to an embodiment of the present invention will be described in more detail with reference to FIGS. 3 to 5.

At this time, the process of FIG. 3 may be performed when a personalized blood pressure measurement model is generated for the user.

3 is a flow chart for explaining a non-contact blood pressure measurement method using an image according to an embodiment of the present invention.

First, the non-contact blood pressure measuring apparatus 200 using an image receives image information of a user (S310).

The non-contact blood pressure measuring apparatus 200 using the image may receive image information including the face and palm of the user from the imaging apparatus 100 or the user terminal 300. In addition, when the non-contact blood pressure measuring apparatus 200 using an image includes a photographing module, the non-contact blood pressure measuring apparatus 200 using the image may photograph an image including a user's face and a palm of the user.

4 is an exemplary view showing image information according to an embodiment of the present invention.

As shown in FIG. 4, the non-contact blood pressure measuring apparatus 200 using an image may include a face image 410 and a hand image 420, and the non-contact blood pressure measuring apparatus 200 using an image is fronted by a user. Looking at the shooting module of the, it is possible to output a voice guidance or guide information to the user, so that the palm of the hand toward the shooting module can be taken while holding the hand.

In addition, the non-contact blood pressure measuring apparatus 200 using an image may recognize a user's posture and perform step S320, which will be described later, only when it is determined that the posture is suitable for non-contact blood pressure measurement. For example, when the user's posture is in a state where the user looks at the front-facing imaging module and the user's palm is held toward the imaging module, the non-contact blood pressure measuring apparatus 200 using an image is a posture suitable for non-contact blood pressure measurement It can be judged as being.

Next, the non-contact blood pressure measuring apparatus 200 using the image generates skin region information of the user's face from the image information (S320), and generates skin region information of the user's hand (S330).

The non-contact blood pressure measuring apparatus 200 using an image may detect a user's face region and hand region every frame of image information using an image-based face and hand tracker. In addition, the non-contact blood pressure measuring apparatus 200 using the image may generate facial skin region information and hand skin region information from the detected face region and hand region.

The apparatus 200 for non-contact blood pressure measurement using an image converts image information, which is an RGB model, into an HSV color model to extract skin region information from the image information, and the Hue channel value in the HSV color model is smaller than a reference skin threshold value. You can designate them as skin areas. In addition, the non-contact blood pressure measurement apparatus 200 using an image may determine that an area other than the designated skin area is not skin and set it as a mask, and may generate skin area information by setting a corresponding mask in the G channel of the RGB model.

At this time, the non-contact blood pressure measuring apparatus 200 using the image may generate a plurality of correction threshold values by modifying the reference skin threshold value, and may generate skin region information corresponding to each of the correction threshold values.

In addition, the non-contact blood pressure measuring apparatus 200 using the image may generate skin area information that has a maximum cross-sectional area ratio and a skin area area of a plurality of skin area information as facial skin area information.

At this time, the non-contact blood pressure measuring apparatus 200 using the image may generate facial skin area information, and then generate hand skin area information using the skin threshold applied when generating the facial skin area information. The process in which the non-contact blood pressure measuring apparatus 200 using the image generates hand skin area information may be substantially the same as the process of generating face skin area information.

Next, the non-contact blood pressure measuring apparatus 200 using the image calculates the heart rate information of the user based on the face skin area information and the hand skin area information (S340).

The non-contact blood pressure measuring apparatus 200 using an image may calculate the user's heart rate information based on a heart rate signal extracted from facial skin area information and a heart rate signal extracted from hand skin area information.

5 is a graph showing heart rate signals of a facial skin region and a hand skin region according to an embodiment of the present invention.

As shown in FIG. 5, the non-contact blood pressure measuring apparatus 200 using an image finds a peak of a heart rate signal of a facial skin area and a peak of a heart rate signal of a hand skin area, and a peak of a heart rate signal of the face skin area and a heart rate signal of the hand skin area By calculating the moving average of the peaks of, the user's rPTT (remote pulse transit time) can be calculated. At this time, the user's rPTT may be one of the user's heart rate information.

Finally, the non-contact blood pressure measuring apparatus 200 using the image estimates blood pressure information of the user (S350).

The non-contact blood pressure measurement apparatus 200 using the image may estimate the blood pressure information of the user by applying the heart rate information of the user calculated in step S340 to the personalized blood pressure measurement model.

The non-contact blood pressure measuring apparatus 200 using the image estimates the user's blood pressure information by applying the user's current rPTT to the blood pressure measurement model generated by performing regression analysis using the user's rPTT value and the user's blood pressure numerical information. can do. At this time, the non-contact blood pressure measurement apparatus 200 using the image may estimate the user's blood pressure information by additionally considering the user's information, current time information, and the like.

Hereinafter, a process of calculating heart rate information through steps S320 to S340 of FIG. 3 through FIGS. 6 and 7 will be described in more detail.

6 is a flowchart illustrating a process of calculating heart rate information according to an embodiment of the present invention.

The non-contact blood pressure measuring apparatus 200 using the image converts image information of the RGB model to an HSV model (S610).

)을 추출할 수 있다. 그리고 영상을 이용한 비접촉식 혈압측정 장치(200)는 얼굴 영역(

)에서 피부 정보만을 추출하기 위하여, RGB 모델인 얼굴 영역(

)을 HSV 모델로 변환할 수 있다. The non-contact blood pressure measuring apparatus 200 using an image may include a user's face area (

) Can be extracted. And the non-contact blood pressure measuring apparatus 200 using the image is a face area (

), in order to extract only skin information, the face region, which is an RGB model (

) To the HSV model.

)의 Hue 채널 값과 기준 피부 문턱값(th_skin)을 비교한다(S620). And the non-contact blood pressure measuring apparatus 200 using the image is a face area (

), and a reference skin threshold (th_skin) is compared (S620 ).

The non-contact blood pressure measuring apparatus 200 using the image compares the value of the Hue channel (H channel) and the reference skin threshold (th_skin) in the HSV model.

The non-contact blood pressure measuring apparatus 200 using the image designates values in which the Hue channel value in the face region is smaller than the reference skin threshold (th_skin) as the skin region (S630), and the Hue channel value in the face region is equal to or greater than the reference skin threshold value. A mask is set by determining values as areas other than the skin (S640).

)을 추출한다(S650). And the non-contact blood pressure measuring apparatus 200 using the image is the final skin area (

) Is extracted (S650 ).

The non-contact blood pressure measuring apparatus 200 using an image may set a mask in a region other than the skin, and then mask the region of the G channel of the RGB model to generate a skin region unique to the G channel.

At this time, the non-contact blood pressure measuring apparatus 200 using the image may generate skin region information by masking the region of the G channel determined to contain the most heart rate change information through a plurality of experiments.

Since the non-contact blood pressure measurement apparatus 200 using an image according to an embodiment of the present invention measures a user's blood pressure using a personalized blood pressure measurement model, a correction procedure may be performed for each user.

When performing the correction procedure, the non-contact blood pressure measuring apparatus 200 using an image may repeatedly photograph a user's image information multiple times and set a mask based on a reference skin threshold (th_skin).

In addition, the non-contact blood pressure measuring apparatus 200 using the image may generate correction threshold values by changing the reference skin threshold little by little, and may set a mask based on the correction threshold values. At this time, the non-contact blood pressure measuring apparatus 200 using the image may generate correction threshold values by changing a reference skin threshold value by a preset ratio or a preset value.

In addition, the non-contact blood pressure measuring apparatus 200 using an image may calculate an intersection ratio between the set masks and the face region. The non-contact blood pressure measuring apparatus 200 using an image may select a mask having the largest area of the mask itself among the masks having the largest cross-sectional area ratio and generate facial skin area information using the selected mask.

The non-contact blood pressure measuring apparatus 200 using the image may generate hand skin area information by performing substantially the same process as the process of generating face skin area information. At this time, the non-contact blood pressure measurement apparatus 200 using the image may generate hand skin area information by setting a threshold value corresponding to the selected mask as the reference threshold value when generating the hand skin area when generating face skin area information.

)에서 피부 이외의 영역을 제거한 최종 피부영역(

)을 추출할 수 있다. Referring again to FIG. 6, the non-contact blood pressure measuring apparatus 200 using an image includes a face region (

), the final skin area (excluding areas other than skin)

) Can be extracted.

7 is an exemplary view showing a user's face region and face skin region according to an embodiment of the present invention.

)(710)에서 피부 이외의 영역을 제거한 최종 피부영역(

)(720)을 추출할 수 있다. As shown in FIG. 7, the non-contact blood pressure measuring apparatus 200 using an image includes a face region (

) (710) the final skin area (excluding areas other than skin)

) 720.

)을 1차원 신호로 변환한다(S660). And the non-contact blood pressure measuring apparatus 200 using the image is the final skin area (

) To a one-dimensional signal (S660).

)이 추출된 프레임이 결정되면, 최종 피부영역(

)으로부터 1차원 신호인 심박 신호를 추출하기 위하여, 영역 전체에 대한 누적 평균을 수행하여 최종 1차원 신호(

)로 변환할 수 있다. Non-contact blood pressure measuring apparatus 200 using the image is the final skin area (

When the extracted frame is determined, the final skin area (

In order to extract the heart rate signal, which is a one-dimensional signal, a cumulative average of the entire region is performed to obtain a final one-dimensional signal (

).

)을 최종 1차원 신호(

)로 변환하여, 특정 프레임의 하나의 점으로 구할 수 있다. That is, the non-contact blood pressure measuring apparatus 200 using the image is the final skin area (

) To the final one-dimensional signal (

) To obtain a single point of a specific frame.

[Equation 1]

)는 1초당 30Hz를 갖는 1차원 신호로 변환될 수 있다. Since the image is processed in real time, usually 10 to 30 frames per minute, if the frame of image information is processed at 30 frames per second, the final 1D signal (

) Can be converted into a one-dimensional signal having 30 Hz per second.

The non-contact blood pressure measuring apparatus 200 using the image performs noise reduction (S670).

)에는 잡음이 포함되어 있다. 따라서, 영상을 이용한 비접촉식 혈압측정 장치(200)는 필터를 이용하여 최종 1차원 신호(

)에 포함된 잡음을 제거할 수 있다. Final one-dimensional signal calculated in step S660 (

) Contains noise. Therefore, the non-contact blood pressure measuring apparatus 200 using an image uses a filter to obtain a final one-dimensional signal (

) Can be removed.

At this time, the non-contact blood pressure measurement apparatus 200 using the image uses the band pass filter to preserve basic information of the heart rate, and removes the final noise using an adaptive filter such as a Kalman filter so as to be less sensitive to lighting changes or motion noise. can do.

Next, the non-contact blood pressure measuring apparatus 200 using the image may calculate a moving average of the peak (S680), and calculate a user's heart rate information based on the moving average (S690).

)에 대한 최종 1차원 신호(

)를 생성하는 것으로 설명하였으나, 영상을 이용한 비접촉식 혈압측정 장치(200)는 S610 단계 내지 S670 단계를 통하여 사용자의 손 영역(

)에 대한 최종 1차원 신호(

)도 생성할 수 있다. For convenience of description, the non-contact blood pressure measuring apparatus 200 using an image performs steps S610 to S670 to perform a user's facial area (

Final one-dimensional signal for)

), but the non-contact blood pressure measuring apparatus 200 using an image is a user's hand area (S610 through S670 ).

Final one-dimensional signal for)

) Can also be generated.

After generating the final one-dimensional signal for the face region and the final one-dimensional signal for the hand region, the non-contact blood pressure measuring apparatus 200 using the image is the final one-dimensional signal peak for the face region and the final one for the hand region. The peak of the dimensional signal can be detected. In addition, the non-contact blood pressure measuring apparatus 200 using an image calculates a moving average of two peaks and estimates a user's heart rate based on the moving average.

At this time, the non-contact blood pressure measurement apparatus 200 using the image calculates the moving average of the peak as the remote pulse wave transmission time (rPTT) and applies the calculated remote pulse wave transmission time (rPTT) to the personalized blood pressure measurement model corresponding to the user. By calculating the user's heart rate information.

Hereinafter, a process of generating a personalized blood pressure measurement model according to an embodiment of the present invention will be described in more detail with reference to FIG. 8.

8 is a flowchart illustrating a process of generating a personalized blood pressure measurement model according to an embodiment of the present invention.

The non-contact blood pressure measuring apparatus 200 using an image receives user information (S810).

The non-contact blood pressure measurement apparatus 200 using an image receives user information of a user who wants to create a personalized blood pressure measurement model. At this time, the user information may include at least one of a user's gender, age, and weight. In addition, the non-contact blood pressure measuring apparatus 200 using the image may receive user information from the user only once for the first time, and may store the received user information.

For convenience of description, the non-contact blood pressure measuring apparatus 200 using the image is described as receiving user information before capturing the user's image information, but is not limited thereto, and the non-contact blood pressure measuring apparatus 200 using the image In step S850, user information may be input at any point before performing the step of generating a personalized blood pressure measurement model.

Then, the non-contact blood pressure measuring apparatus 200 using the image captures the image information of the user (S820), and receives blood pressure numerical information of the blood pressure monitor measured during the image information shooting (S830).

The non-contact blood pressure measurement apparatus 200 using an image may start recording and photograph a user when the user's blood pressure measurement preparation is completed. At this time, it may be taken to include the upper body of the user of the non-contact blood pressure measuring apparatus 200 using the image, recognize the user's posture, start recording when the user takes the basic posture of blood pressure measurement, or take the basic posture of blood pressure measurement You can start recording by receiving a button to start recording from the user.

The basic posture for blood pressure measurement may refer to a state in which the user is facing the imaging module in front and the palm of the user is facing the imaging module. In addition, the non-contact blood pressure measurement apparatus 200 using an image may provide a voice guidance for a user to take a basic posture for blood pressure measurement or output a posture guide line on a display.

For convenience of description, the non-contact blood pressure measuring device 200 using the image is described as directly photographing the user's image information, but is not limited thereto, and the non-contact blood pressure measuring device 200 using the image is the imaging device 100 or The user terminal 300 may control the user to photograph the image data of the user, or receive the image data from the imaging device 100 or the user terminal 300 that photographed the image data, and use it as image information.

In addition, the non-contact blood pressure measuring apparatus 200 using an image may receive blood pressure numerical information of a user measured using a blood pressure monitor while the user's image is being photographed.

At this time, the non-contact blood pressure measuring apparatus 200 using an image may control blood pressure measurement of the blood pressure monitor and receive blood pressure numerical information from the blood pressure monitor. In addition, when the blood pressure monitor does not support the communication function, the non-contact blood pressure measuring apparatus 200 using an image may receive blood pressure numerical information displayed on the blood pressure monitor from a user.

When the blood pressure measurement of the sphygmomanometer is completed, the non-contact blood pressure measurement device 200 using the image may stop capturing image information and store the user's blood pressure numerical information.

The apparatus 200 for non-contact blood pressure measurement using an image may repeatedly perform steps S810 to S840 to obtain a plurality of blood pressure numerical information and image information for generating a personalized blood pressure measurement model.

At this time, the non-contact blood pressure measuring apparatus 200 using the image may acquire image information and blood pressure numerical information by performing steps S820 and S830 daily or periodically, so that the user performs steps S820 and S830 at a predetermined time. A blood pressure measurement time alarm can be output.

When the number of stored blood pressure numerical information is greater than the threshold (S840 No), the non-contact blood pressure measuring apparatus 200 using the image generates a personalized blood pressure measurement model (S850).

On the other hand, if the number of blood pressure numerical information is less than the threshold (S840 Yes), the non-contact blood pressure measuring apparatus 200 using the image may perform step S810 of FIG. 8 again to obtain the image information and the blood pressure numerical information. .

The non-contact blood pressure measurement apparatus 200 using an image may perform regression analysis using a plurality of image data and a plurality of blood pressure numerical information for a user, thereby generating a user's personalized blood pressure measurement model.

At this time, the non-contact blood pressure measuring apparatus 200 using an image extracts a heartbeat signal from a user's skin region included in the image data, and performs regression analysis based on the extracted heartbeat signal and actual blood pressure numerical information corresponding to the image data. Thus, a personalized blood pressure measurement model can be generated.

In addition, the apparatus 200 for non-contact blood pressure measurement using an image may estimate a user's blood pressure information by generating a personalized blood pressure measurement model in step S850 and applying a heart rate signal extracted from the image data to the personalized blood pressure measurement model. . At this time, the non-contact blood pressure measuring apparatus 200 using the image may perform regression analysis as shown in Equation 2 below to estimate the user's blood pressure information.

[Equation 2]

Here, Y is a target variable for systolic blood pressure and diastolic blood pressure.

In addition, the non-contact blood pressure measuring apparatus 200 using the image may calculate the error information by comparing the estimated blood pressure information of the user with actual blood pressure value information of the user input when the corresponding image data is captured.

If the error information is greater than a preset value, the non-contact blood pressure measurement apparatus 200 using the image may correct the personalized blood pressure measurement model by performing the process of FIG. 8 again.

On the other hand, if the error information is smaller than a preset value and the degree of error is included in an allowable range, the non-contact blood pressure measurement apparatus 200 using the image does not perform a correction process and ends the process of generating a personalized blood pressure measurement model. You can. In addition, the non-contact blood pressure measurement apparatus 200 using an image may measure a user's blood pressure information using a user's image information without a blood pressure monitor by using the generated personalized blood pressure measurement model.

9 is a block diagram showing a computer system according to an embodiment of the present invention.

9, an embodiment of the present invention may be implemented in a computer system 900, such as a computer-readable recording medium. As shown in FIG. 9, computer system 900 includes one or more processors 910, memory 930, user interface input device 940, and user interface output device 950 that communicate with each other via bus 920. And storage 960. In addition, the computer system 900 may further include a network interface 970 connected to the network 980. The processor 910 may be a central processing unit or a semiconductor device that executes processing instructions stored in the memory 930 or the storage 960. The memory 930 and the storage 960 may be various types of volatile or nonvolatile storage media. For example, the memory may include ROM 931 or RAM 932.

Accordingly, an embodiment of the present invention may be implemented as a computer-implemented method or a non-transitory computer-readable medium having computer-executable instructions recorded thereon. When computer readable instructions are executed by a processor, computer readable instructions may perform the method according to at least one aspect of the present invention.

As described above, a non-contact blood pressure measuring apparatus using an image according to an embodiment of the present invention is a technology that enables a user to measure an approximate blood pressure of a user using image information of a user without a blood pressure monitor. The non-contact blood pressure measurement device using an image is a user terminal with a built-in camera that can take a picture of the user's face and hands and measure the user's blood pressure information. , Non-contact type can conveniently measure blood pressure.

As described above, the non-contact blood pressure measuring apparatus and method using the image according to the present invention is not limited to the configuration and method of the embodiments described as described above, and the above embodiments are provided so that various modifications can be made. All or some of the embodiments may be configured by selectively combining.

100: photographing device
200: Non-contact blood pressure measurement device using images
210: image information input unit 220: skin area information generation unit
230: heart rate information calculation unit 240: blood pressure measurement unit
250: blood pressure measurement model generation unit 300: user terminal
410: face image 420: hand image
710: face area 720: final skin area
900: computer system 910: processor
920: bus 930: memory
931: Roman 932: Ram
940: user interface input device
950: user interface output device
960: storage 970: network interface
980: network

Claims (1)

In the non-contact blood pressure measurement method using an image performed by a non-contact blood pressure measuring device using an image,
Receiving image information including the user's face and palm,
Generating facial skin region information and hand skin region information of the user from the image information,
Calculating heart rate information of the user using the facial skin area information and the hand skin area information, and
A method of measuring non-contact blood pressure using an image, comprising applying the heart rate information to a personalized blood pressure measurement model and calculating blood pressure information of the user.

Images