KR102305309B1 - Method for measuringing skin and health indicators of an user using a video image and apparatus using the same - Google Patents

Abstract

According to the present invention, there is provided a method for measuring a user's health index, comprising: photographing, by a skin and health diagnosis apparatus, a part of the user's skin, and detecting a skin ROI or coordinates of the skin part from the acquired image image ; The skin and health diagnosis apparatus calculates the average of the color data in the skin ROI, corrects the detected coordinates of the skin part through a moving average algorithm, and updates the average of the color data based on the average value of the color data. extracting; and extracting, by the skin and health diagnosis apparatus, a bio-signal based on the average value of the color data, and measuring the user's health indicator using the bio-signal.

Description

A method for measuring a user's skin and health indicators through a video image and a device using the same

The present invention relates to a method for measuring a user's health index through a video image, and more particularly, to a method for measuring a user's health index, wherein the skin and the health diagnosis device are present on the opening of the user's skin. photographing a part, and detecting the coordinates of the skin region of interest or the skin portion from the captured video image; The skin and health diagnosis apparatus calculates the average of the color data in the skin ROI, corrects the detected coordinates of the skin part through a moving average algorithm, and updates the average of the color data based on the average value of the color data. extracting; and extracting, by the skin and health diagnosis apparatus, a biosignal by filtering the average value of the color data, and measuring the user's health indicator using the biosignal.

Cardiovascular disease, the leading cause of death in the world population, accounts for about 21.6% of all deaths in Korea. Cardiovascular disease is increasing rapidly due to an aging population and changes in lifestyle. Cardiovascular diseases include circulatory-related diseases such as ischemic heart disease, stroke, and high blood pressure, and are influenced by various risk factors, including parental genetic factors and environmental factors such as drinking and smoking. It is also closely related to other diseases such as diabetes and obesity.

The traditional tool for estimating arrhythmias, one of the cardiovascular diseases, has been using the method of estimating through the ratio of RR intervals in the ECG waveform. Unlike an electrocardiogram waveform, which is an electrical signal of the heart, it is difficult to observe waveforms other than R waves in a pulse wave measuring blood flow according to a pulse.

Arterial stiffness refers to the degree of stiffness due to decreased elasticity of the arteries. The most important factor that determines the stiffness of arteries is age, and with age, changes in arterial wall tissue occur, resulting in decreased elasticity and increased stiffness.

Arterial stiffness is an important measure for predicting the risk of cardiovascular disease. Patients with increased arterial stiffness are classified as a high-risk group for future cardiovascular disease.

And the pulse wave propagation velocity (PWV) is a useful index for estimating arterial stiffness. The pulse wave propagation velocity (PWV) changes with age. In young people, the pulse wave propagation velocity (PWV) is relatively low because the arteries are elastic, but as the aging progresses, the arteries become less elastic and the pulse wave propagation velocity (PWV) increases. As a method of measuring the pulse wave propagation velocity (PWV), it is classified according to the measurement site, and among them, baPWV (brachial-ankle PWV), which is relatively easy to record a waveform, is used. In the conventional baPWV, the velocity was calculated using the difference value and distance of the pulse wave velocity between the two points in the pulse waves extracted from the ankle and upper arm.

As described above, in the prior art, arrhythmia or arterial stiffness was measured for diagnosing cardiovascular disease, but a device for accurately measuring it is expensive and expensive to measure, so there is a lot of difficulty in periodically treating and measuring it.

Also, conventionally, there is an apparatus for diagnosing a user's skin on the fly through a skin diagnosis apparatus, but there is a limitation in that only the skin condition can be measured.
On the other hand, as a prior art document related to the present invention, Patent Registration No. 10-1738278 (published date: 17.05.22) exists.

Accordingly, the present inventor intends to propose a method for measuring a user's skin and health index through a video image and an apparatus using the same.

An object of the present invention is to solve all of the above problems.

Another object of the present invention is to perform self-diagnosis using a video image and to provide customized content to a corresponding user through the self-diagnosis result.

Another object of the present invention is to take a video image, extract an average value of color data from the captured video image, and diagnose a user's health index through this.

In addition, the present invention proposes a technique for diagnosing health by using an image in addition to the configuration of diagnosing skin using an image.

In order to achieve the object of the present invention as described above and to realize the characteristic effects of the present invention to be described later, the characteristic configuration of the present invention is as follows.

According to one aspect of the present invention, in a method for measuring a user's health index, the skin and health diagnosis apparatus photograph a part of the user's skin, and determine the coordinates of the skin region of interest or the skin part from the captured image image. detecting; The skin and health diagnosis apparatus calculates the average of the color data in the skin ROI, corrects the detected coordinates of the skin part through a moving average algorithm, and updates the average of the color data based on the average value of the color data. extracting; and extracting, by the skin and health diagnosis apparatus, a biosignal by filtering the average value of the color data, and measuring the user's health indicator using the biosignal.

According to another aspect of the present invention, there is provided a skin and health diagnosis apparatus for measuring a user's health index, comprising: a communication module for transmitting and receiving information to and from an external device; an opening into which a part of the user's body can be inserted; a skin measurement module including a camera module and a lighting module located in the opening; photographing a part of the user's skin using the skin measurement module, detecting the coordinates of the skin ROI or the skin part from the photographed video image, calculating an average of color data in the skin ROI, and the detection After correcting the coordinates of a part of the skin with a moving average algorithm, the average value of the updated color data is extracted based on this, and the average value of the color data is filtered to extract a biosignal, and the biosignal is used. To provide a skin and health diagnosis device including a control module for measuring the health index of the user.

According to the present invention, the following effects are obtained.

The present invention has an effect of performing self-diagnosis using a video image and providing customized content to the user through the self-diagnosis result.

In addition, the present invention has an effect of diagnosing a user's health index by photographing a video image, extracting an average value of color data from the photographed video image, and thus diagnosing a user's health index

1 is a view showing the appearance of a skin and health diagnosis apparatus according to an embodiment of the present invention.
2 is a diagram showing a schematic configuration of a skin and health diagnosis apparatus according to an embodiment of the present invention.
3 is a diagram illustrating a process up to measuring a user's health index according to an embodiment of the present invention.
4 is a diagram illustrating coordinates before and after applying a moving average algorithm according to an embodiment of the present invention.
5 is a diagram illustrating in detail a process of measuring a user's health indicator from a biosignal according to an embodiment of the present invention.
6 is a view showing a normal pulse wave and a PVC generation section according to an embodiment of the present invention.
7 is a diagram illustrating a process of calculating vascular elasticity and vascular age according to an embodiment of the present invention.
8 is a diagram illustrating a scatter diagram of an actual distance to a camera according to a detected face width on an image coordinate system according to an embodiment of the present invention.
9 is a diagram illustrating a regression curve estimating an actual distance per pixel according to a distance from a camera based on a scatter diagram according to an embodiment of the present invention.
10 is a diagram illustrating the provision of customized content using related information stored in a database according to an embodiment of the present invention.
11 is a view showing an intake guide according to food and health according to an embodiment of the present invention.
12 is a diagram showing the PWV distribution of 100 data of 20 adult males and females.
13 is a diagram showing the distribution of baPWV in Korean adult men and women according to age.
14 is a diagram showing the baPWV normal range.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS [0010] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS [0010] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS [0023] Reference is made to the accompanying drawings, which show by way of illustration specific embodiments in which the present invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the present invention. It should be understood that the various embodiments of the present invention are different but need not be mutually exclusive. For example, certain shapes, structures, and characteristics described herein with respect to one embodiment may be embodied in other embodiments without departing from the spirit and scope of the invention. In addition, it should be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the present invention. Accordingly, the detailed description set forth below is not intended to be taken in a limiting sense, and the scope of the invention, if properly described, is limited only by the appended claims, along with all scope equivalents to those claimed. Like reference numerals in the drawings refer to the same or similar functions throughout the various aspects.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings in order to enable those of ordinary skill in the art to easily practice the present invention.

1 is a view showing the appearance of a skin and health diagnosis apparatus according to an embodiment of the present invention.

2 is a diagram showing a schematic configuration of a skin and health diagnosis apparatus according to an embodiment of the present invention.

The present invention extracts a biosignal from a skin region of interest in a video image of a user's body taken by the skin and health diagnosis apparatus 100, and uses the extracted biosignal to determine the pulse rate per minute, respiration rate per minute, pulse intensity, Health indicators such as blood pressure, bradycardia/tachycardia/irregular arrhythmia, blood volume, blood viscosity, abnormal pulse, vascular elasticity, and vascular age can be measured. In the present invention, the measured health index is stored in the database 140, and based on this, it can be provided to the user in the form of a health-log or customized content can be provided. For reference, the shape of the skin and health diagnosis apparatus 100 shown in FIG. 1 corresponds to one example, and various shapes may exist.

In the present invention, as an example of various color systems, pulses per minute, respirations per minute, pulse intensity, Blood pressure, bradycardia/tachycardia/irregular arrhythmia, blood volume, blood viscosity, abnormal pulse, vascular elasticity, and vascular age can be extracted or estimated. A detailed process for this will be described later.

Referring to FIG. 2 , the skin and health diagnosis apparatus 100 includes a control module 110 , a skin measurement module 120 , a communication module 130 , a database 140 , an input module 150 , an output module 160 , etc. may include.

First, the control module 110 controls the overall process in the present invention, specifically, receives an image image from the skin measurement module 120, extracts a bio-signal using it, and a health indicator through the extracted bio-signal. can be diagnosed

The skin measurement module 120 is present in the openings 10 and 20, and a lighting module 121 for illuminating a part of the user's body, such as hands and feet, and a camera for photographing parts of the user's body, such as hands and feet. at least one of the modules 122 . Since it may have a shape other than the shape of the skin and health diagnosis apparatus 100 of FIG. 1(a) (eg, FIG. 1(b)), it may be possible to photograph not only the hands and feet but also other body parts such as the face and neck. have. In particular, the camera module 122 can take a moving image as well as a simple image, and as will be described later, can diagnose or measure skin through the captured moving image, and ultimately can diagnose the health of the user. The video image described in the present invention mainly assumes a moving image, and a simple image will not be excluded. Accordingly, in some cases, a moving image image and a simple image may be photographed at the same time.

In addition, the skin and health diagnosis device 100 includes two (two or more) openings 10 and 20 into which a body part can be inserted, as shown in FIG. (120a, 120b) may be included. That is, any one of the skin measurement modules 120a is included in one of the openings 10, and the other skin measurement module 120b is included in the other opening 20, so that two or more skin measurement modules are included at the same time. At ( 120a and 120b ), two or more parts of the user's body may be photographed. For reference, a wall separating the two openings 10 and 20 may be removed to be used as one opening.

On the other hand, as another embodiment, the face diagnosis apparatus 100 having the form as shown in FIG. 1(b) instead of FIG. 1(a) may exist. In this case, the user can take an image of the face through the skin measurement module located on the front part 15 in a sitting state. The user can insert the face into the concave part 25 of the face diagnosis apparatus 100 and photograph the face through the camera module located on the front part 15 . In addition, when a video image is not captured, a sliding cover may cover the concave portion 25 .

The communication module 130 may be implemented using various communication technologies. That is, Wi-Fi (WIFI), WCDMA (Wideband CDMA), HSDPA (High Speed Downlink Packet Access), HSUPA (High Speed Uplink Packet Access), HSPA (High Speed Packet Access), Mobile WiMAX (Mobile WiMAX), WiBro (WiBro) , LTE (Long Term Evolution), Bluetooth (bluetooth), infrared data association (IrDA), NFC (Near Field Communication), Zigbee, wireless LAN technology, etc. may be applied. In addition, when a service is provided by being connected to the Internet, TCP/IP, which is a standard protocol for information transmission on the Internet, may be followed or a Global Positioning System (GPS) technology may be used. In the present invention, communication with the external device 200 may be performed through the communication module 130 . Here, the external device 200 may include various devices such as a user terminal, an external server, and a cloud.

The database 140 may store necessary information, such as a measurement result of a user's health index, user's body information, content information, and an image image, as will be described later, and may provide it to the control module 110 . For reference, unlike FIG. 2 , the database 140 is not included in the skin and health diagnosis apparatus 100 but exists outside, and information may be transmitted and received with the skin and health diagnosis apparatus 100 through the communication module 130 . have.

Both the input module 150 and the output module 160 are controlled by the control module 110, and the input module 150 may include any device capable of input, such as a touch screen 30, a keypad, and a mouse, and output The module 160 may include various devices such as a printer and a display device. Unlike FIG. 2 , the input module 150 and the output module 160 are not included in the skin and health diagnosis apparatus 100 , but may exist outside.

As the external device 200 connected to the skin and health diagnosis apparatus 100 of the present invention, various devices such as a user's terminal, an external server, and a cloud may be included. For reference, the terminal is a digital device including a function for performing communication, and has a memory means such as a desktop computer, a notebook computer, a workstation, a PDA, a web pad, a mobile phone, etc. Any digital device equipped with a . In particular, the user terminal may include a health indicator diagnosis program (application) dealt with in the present invention.

3 is a diagram illustrating a process up to measuring a user's health index according to an embodiment of the present invention.

First, the skin and health diagnosis apparatus 100 may obtain the user's body information (eg height, weight, gender, age, blood pressure, pulse, body temperature, etc.) through the input module 150 ( S210 ). In addition, personal information such as a user's name may be input. The user may input information such as his/her height, weight, gender, age, blood pressure, pulse, and body temperature through the touch screen 30 and the like, and may store the information in the database 140 . Since the database 140 can store each user by identification code or the like, the user's diagnosis result or body information can be continuously updated or initialized in the database 140 .

In addition, the user's body information can be directly input through the input module 150 , and the user's body information recorded in the database 140 can be transferred to the control module 110 of the skin and health diagnosis apparatus 100 . You might be able to receive this. In some cases, the skin and health diagnosis apparatus 100 interworks with an application installed on the terminal 200 worn by the user, so that the user's body information (blood pressure, pulse, body temperature) changed due to the user's movement is displayed in real time. may be provided.

Next, the skin and health diagnosis apparatus 100 uses the camera module 121 included in the skin measurement module 120 to use a part of the user's skin (eg hands, back of hands, nails, feet, feet, toenails, face, etc.) may be photographed, and coordinates of a skin region of interest or a part of the skin may be detected from the acquired video image ( S220 ).

Also, the skin and health diagnosis apparatus 100 may support image capturing by irradiating light to a part of the user's skin using the lighting module 122 . Here, the lighting module 122 may be an LED module, and by sequentially irradiating normal light, polarized light, and UV light, individual image images according to each of the normal light, polarized light, and UV light may be obtained. A skin condition (normal light), a skin condition related to epidermal pigmentation (polarization), dermal pigmentation, a skin type, and a skin condition related to porphyrin (UV light) may be measured by each light of the lighting module 122 . In this case, the skin index may be measured in relation to the photographed skin condition of the user, and the measured skin index may include elasticity, oil-water content, oil-water balance, wrinkle index, skin tone, and the like. According to the above skin index, the skin and health diagnosis apparatus 100 may recommend a skin care method or cosmetics.

For reference, when the skin care method and cosmetics are recommended, the measurement results affected by dry or wet weather can be corrected based on the weather information included in the diagnosis results, and the dry or wet weather, the weather with strong UV rays and how to take care of the skin when the weather affects the skin, such as weather with a lot of fine dust, may be included in the skin diagnosis result.

In addition, the control module 110 may provide the user with skin diagnosis results in which the user's nasolabial folds, acne, freckles, eyelash lengths, and dots included in the captured video image are indicated to the user.

For example, when the oil-water content of the hand is measured to be low, a skin care method that requires the use of a hand cream with high moisturizing power may be recommended, and information about at least one cosmetic that increases the oil-water content may be included in the skin diagnosis result. However, the skin diagnosis result is not limited to the above example, and may include information on skin care methods and recommended cosmetics for other items related to skin conditions such as elasticity of hands or feet, wrinkle index, and skin tone.

The position and size of the photographed skin region of interest may vary and may be flexibly adjusted. The apparatus 100 for diagnosing skin and health may detect or set the skin ROI using various algorithms, and in this case, the algorithm for detecting the skin ROI may use various conventionally known techniques.

The control module 110 of the skin and health diagnosis apparatus 100 may calculate or calculate an average of color data in a skin ROI in an image being photographed or already photographed.

Here, for example, various values may be used for color data. For example, (1) color data may use at least one of an R value, a G value, and a B value based on the RGB color system. That is, color average data for at least one of the R value, the G value, and the B value may be used as the color data. (2) Also, the skin and health diagnosis apparatus 100 may convert the RGB color system into another color system. For example, the skin and health diagnosis apparatus 100 may convert the RGB color system into various color systems such as YUV, HSV, YCbCr, and YCgCo. In this case, the color data may use one of the color difference components that are less affected by the surrounding environment (illuminance, etc.). For example, in the case of YCbCr, at least one of a Cb value and a Cr value may be used. In the case of YCgCo, at least one of a Cg value and a Co value may be used. Furthermore, any one of the two color difference components that is more robust to changes in illuminance may be used. For example, in the case of YCgCo, only the Cg value can be used. In this case, the control module 110 of the skin and health diagnosis apparatus 100 may extract an average value of Cg color data of the skin region as color data. (3) Furthermore, the color data may be a value obtained by applying a weight to at least one color component in various color systems such as RGB, YUV, HSV, YCbCr, YCgCo, and the like and combining the same. When color components are combined, the color data may be a value obtained by adding different weights according to color systems and types of color components. The skin and health diagnosis apparatus 100 may change the face image having the RGB color system to the YCgCo color system. Hereinafter, it is assumed that the skin and health diagnosis apparatus uses the Cg value obtained from YcgCo. The control module 110 may change the video image having the RGB color system to the YCgCo color system using Equation 1 below.

<Equation 1>

As another example, when the image is an infrared image captured by an infrared camera, an average value of gray color data may be extracted and used in the skin region.

However, the control module 110 may extract the biosignal by filtering the average of the color data to the updated average value of the color data, rather than directly using the average of the color data. This is to obtain a stabilized average value of color data, which will be described together with FIG. 4 .

4 is a diagram illustrating coordinates before and after applying a moving average algorithm according to an embodiment of the present invention. Fig. 4(a) shows the state before applying the moving average algorithm, and Fig. 4(b) shows the state after applying the moving average algorithm.

Specifically, the control module 110 may detect the coordinates of a part of the user's skin through the skin measurement module 120 . In this case, the coordinates may be regarded as coordinates of the skin region.

As shown in FIG. 4( a ), the control module 110 of the skin and health diagnosis apparatus 100 has a limitation in extracting a biosignal from a stable position because the coordinates of the skin region detected in every frame change. can Accordingly, the control module 110 can detect stable coordinates by applying the moving average algorithm to the coordinates (x, y, width, height) used for detecting the skin region.

The skin and health diagnosis apparatus 100 can efficiently extract biosignals from blood vessels of the skin by applying a moving average algorithm to the coordinates in order to stabilize the image coordinates of the skin region detected every frame in the input image. In this case, the moving average algorithm may use a moving average filter (MAF).

After all, the control module 110 of the skin and health diagnosis apparatus 100 corrects the detected coordinates of a part of the skin through a moving average algorithm, and then extracts the average value of the color data obtained by updating (safety) the average of the color data based on this. (S230) can be done.

The control module 110 may extract the biosignal by filtering the average value of the color data. The biosignal at this time can be regarded as a pulse wave signal. The skin and health diagnosis apparatus 100 may estimate a cardiovascular health index by using the calculated bio-signals.

Specifically, the control module 110 of the skin and health diagnosis apparatus 100 may detect a biosignal (pulse wave signal) by applying a band pass filter (BPF) to the average value of the color data. Of course, although it is unstable, the biosignal may be detected by applying a band pass filter (BPF) to the average of the color data before applying the moving average algorithm (MAF).

Also, the control module 110 of the skin and health diagnosis apparatus 100 may apply a Fast Fourier Transform (FFT) to the average value of color data. For example, the control module 110 extracts the Cg signal by calculating the average of the Cg values in each frame, and uses the extracted Cg signal to use FFT (Fast Fourier Transform) to convert the largest frequency component in the pulse frequency domain of the pulse wave. cycle can be determined. Pulse wave refers to a wavelength at which blood waves and propagates in the heart, and can be used to measure heart rate (HRV), pulse, respiration, pulse strength, blood pressure, and the like.

The control module 110 may observe the signal in the frequency domain in order to estimate the pulse wave signal from the Cg signal. For example, in a normal case, the control module 110 may measure a pulse wave per minute from about 40 to 200 depending on the degree of stability or excitation, and accordingly, the area observed in the frequency domain may be limited from 0.65 Hz to 3.4 Hz. have.

5 is a diagram illustrating in detail a process of measuring a user's health indicator from a biosignal according to an embodiment of the present invention.

The control module 110 of the skin and health diagnosis apparatus 100 may measure the user's health index by using the extracted bio-signals and body information (S240). Specifically, by diagnosing at least one or more of pulse rate per minute, respiration rate per minute, pulse intensity, blood pressure, bradycardia/tachycardia/irregular arrhythmia, blood volume, blood viscosity, abnormal pulse, vascular elasticity, and vascular age to determine the user's health index can be measured That is, the user's health is diagnosed through the bio-signals extracted from the video image.

For example, the pulse may be estimated (11) as the maximum frequency component in the pulse-related frequency band (0.67 to 4 Hz) of the biosignal. Respiration can be estimated as the maximum frequency component in the respiration-related frequency band (0.13 to 0.35 Hz) of the biosignal (12). The pulse intensity can be estimated (13) by using the amplitude of the pulse wave performed by BPF in the pulse-related frequency band of the biosignal. The blood pressure may be estimated (14) by using a pulse transit time (PTT) of a pulse wave calculated from two skin regions of interest. Bradycardia/tachycardia/arrhythmia can be estimated (15) by applying the estimated pulse rate to the bradycardia standard (60 beats or less) and the tachycardia standard (100 beats or more). Irregular arrhythmias can be estimated (15) using the pulse wave R-R interval ventricular premature beat (PVC) standard and ACF slope change. The blood volume can be estimated (16) by using the slope of the next maximum peak from the maximum peak in the pulse wave frequency region (0 to 4.2 Hz). The blood viscosity can be estimated (17) by using the amplitude at the peak point of the pulse wave, the first and second derivatives, and the spring constant K. The odd pulse can be estimated (18) by using the degree of the odd pulse in which the difference in the number of pulses and the intensity are significantly different in each region of the inhalation and exhalation sections. The vascular elasticity can estimate the pulse wave propagation velocity (PWV) by calculating the time difference and the ROI distance of the pulse waves calculated from two skin regions of interest (19). Vascular age can be estimated by applying the estimated pulse wave propagation velocity (PWV) value and the average PWV of Korean adults (19). A detailed description of this is as follows.

In the case of a pulse, the control module 110 of the skin and health diagnosis apparatus 100 processes the MAF of the maximum frequency component or frequency amplitude value of 0.67 to 4 Hz, which is a pulse-related frequency band, from the extracted biosignal, thereby considering the pulse change. can be used to estimate the pulse rate.

In the case of respiration, it can be estimated based on the frequency characteristics of the respiration-related frequency (0.13 to 0.35 Hz) band in the same way as the pulse.

In the case of pulse intensity, it can be provided in the form of a graph according to the amplitude of the pulse wave processed by BPF (Band Pass Filtering) in the pulse-related frequency band.

Blood pressure can be estimated using pulse waves calculated from two skin regions of interest. In the present invention, the skin and health diagnosis apparatus 100 may detect the first skin ROI and the second skin ROI by using the plurality of skin measurement modules 120a and 120b. At this time, the first skin area of interest is detected by the camera module 121a of the first skin measurement module 120a included in the first opening 10 of the skin and health diagnosis apparatus 100, and the second opening 20 ), the second skin region of interest may be detected by the camera module 121b of the second skin measurement module 120b included in the .

Also, unlike the above, the first skin ROI and the second skin ROI may be detected from the detected video image using one skin measurement module 120 . In this case, only one of the first skin measurement module 120a or the second skin measurement module 120b may be used.

In the case of blood pressure, it may be related to the user's body information (height, weight, etc.) and the pulse wave transmission time. Therefore, by applying multiple regression for blood pressure estimation, a regression model can be obtained with the pulse wave propagation time, height, and weight of the pulse waves extracted from two regions of interest as independent variables.

In the present invention, the improved blood pressure can be estimated using Equation 1 below by additionally applying the correlation between the blood pressure estimated in the regression model and the blood pressure measured through the device in addition to the regression model. Equation 1 shows an example of a regression curve equation for improved blood pressure estimation.

<Equation 2>

In Equation 1, x may represent the blood pressure estimated using the body information and the pulse wave transmission time, y may represent the improved blood pressure by applying the relationship between the blood pressure measured by the device and the estimated blood pressure, and y1 is the systolic, y2 may represent a diastolic blood pressure.

In addition, the control module 110 of the skin and health diagnosis apparatus 100 is a method for estimating bradycardia, tachyarrhythmia, and irregular arrhythmia, which are representative examples of arrhythmias, a method based on a pulse rate and a method using a pulse rate RR interval. can be provided.

Bradycardia and tachycardia can be estimated based on the resting pulse rate, and a pulse rate of less than 60 beats per minute can be classified as bradycardia, and a pulse rate of more than 100 beats per minute can be classified as tachycardia.

In the case of ventricular premature contraction (PVC), one of irregular arrhythmias, it can be calculated by using the standard according to the interval of the R wave, which is the peak of the pulse wave, and the ACF (Auto Correlation Function) slope accordingly.

In the case of ventricular premature beat (PVC), the control module 110 may estimate using the R wave just before the occurrence and the interval of two consecutive R waves thereafter. Equation 2 for estimating ventricular premature contraction (PVC) is as follows.

<Equation 3>

Equation 3 may show an equation for the PVC estimation method using the R-R interval.

N may represent the number of R peaks (N: number of R peaks), and RR _var may represent a continuous change in RR (RR _var : variation of continuous RR). And RR _th1 is 78% of the total RR interval, which can be introduced to estimate the case where the initial peak interval, which is a characteristic of PVC, is short and the subsequent peak interval is long.

Through Equation 3 described above, the control module 110 may estimate the PVC after detecting the peak of the pulse wave.

In addition, the computer device may use the ACF slope of the classified PVC and the normal pulse wave as another method for estimating the PVC.

First, in the case of PVC, additional characteristics other than the R-R interval may appear, such as the R-R interval and attenuation of the generated R wave. Therefore, an ACF capable of detecting a sudden change in a pulse wave can be used.

6 is a view showing a normal pulse wave and a PVC generation section according to an embodiment of the present invention.

As shown in FIG. 6(a) , in the case of a normal pulse wave, a signal in the form of a sine wave is observed, so that the linearity of the slope can be guaranteed when ACF is applied. As shown in FIG. 6(b) , the ACF at the point where the PVC occurs may show the ACF result of an irregular slope. In the present invention using the slope shown in FIG. 6, PVC can be detected using ACF in irregular changes in the pulse wave, such as the R-R interval and attenuation of the R wave.

The skin and health diagnosis apparatus 100 may use Equation 4 below to calculate the change in the above-described slope. Equation 4 can calculate the ACF slope change.

<Equation 4>

N disclosed in Equation 4 may represent that the number of peaks is divided by 2 and then 1 is subtracted (N: (number of peak/2)-1).

The skin and health diagnosis apparatus 100 may perform PVC detection using the slope of the ACF by learning a threshold value for classifying a normal pulse wave and a PVC using the calculated slope change value (G).

In the case of blood volume, it can be estimated using the slope value between peaks in the frequency domain of the biosignal and the user's body information (sex, weight). The control module 110 of the skin and health diagnosis apparatus 100 may set a frequency band (0 to 4.2 Hz) related to blood volume in a frequency domain in the measured biosignal, and calculate a frequency domain peak of the set band.

Thereafter, the control module 110 may estimate the blood volume by applying the calculated inter-peak slope value G to Equation 5 below. Equation 5 may represent a blood volume estimation equation.

<Equation 5>

In the case of blood viscosity, it can be estimated using the peak amplitude of the pulse wave calculated from the skin image, the second derivative of the peak point, and the spring constant K.

The spring constant K represents the ratio of the linear displacement of the elasticity of the vessel and can be estimated as a pulse wave. This can be calculated through Equation (6). Equation 6 is an estimate of the spring constant K.

<Equation 6>

In addition, the blood viscosity can be calculated by applying the spring constant K estimated through Equation 6 to Equation 7 which is the blood viscosity estimation equation below. Equation 7 may be an equation for estimating a blood viscosity estimation equation.

<Equation 7>

In addition, the control module 110 may determine the abnormal pulse when the difference in blood pressure during inhalation and exhalation is 10 mmHg or more, and may occur in the case of pericardial tamponade, pulmonary embolism, pericardial effusion, pneumothorax, and the like. In the present invention, the abnormal pulse can be detected using the respiratory waveform and the pulse wave calculated from the skin image.

The control module 110 divides the inspiratory (lower peak to the upper peak) and the expiratory (upper peak to the lower peak) section in the respiration waveform calculated from the biosignal, and calculates the amplitude difference of the pulse wave in each section to calculate the inspiration and expiratory section It is possible to estimate the abnormality pulse by the ratio of the amplitude.

In the diagnosis of abnormal pulse, it can be provided by expressing the degree into four categories. Table 1 is a table organized by classifying abnormal pulse states.

<Table 1>

As described in Table 1, when the pulse wave amplitude ratio between inspiration and expiration is ratio _r > 0.65, it may be normal. And it can be dangerous if 0.15>= ratio _r.

7 is a diagram illustrating a process of calculating vascular elasticity and vascular age according to an embodiment of the present invention.

8 is a diagram illustrating a scatter diagram of an actual distance to a camera according to a detected face width on an image coordinate system according to an embodiment of the present invention.

9 is a diagram illustrating a regression curve estimating an actual distance per pixel according to a distance from a camera based on a scatter diagram according to an embodiment of the present invention.

The skin and health diagnosis apparatus 100 may photograph a part of the user's body using the camera module 121 and set a first skin ROI and a second skin ROI based thereon ( S710 ).

In particular, in the process of calculating the vascular elasticity and vascular age, the first skin ROI and the second skin ROI may be set from the image image acquired from one camera module 121 . Of course, depending on the setting, the first skin ROI and the second skin ROI may be set from each of the camera modules 121a and 121b.

In addition, in the case of vascular elasticity and vascular age, the skin region of interest may be set based on the arterial section. This is to more accurately detect vascular elasticity. For example, the skin ROI may be set based on a facial artery section, a wrist artery section, and the like.

The control module 110 of the skin and health diagnosis apparatus 100 may extract biosignals from the first skin ROI and the second skin ROI ( S720 ), and may respectively calculate pulse waves. The calculated pulse wave may be delayed and generated in the same blood vessel with a certain time difference.

Accordingly, the control module 110 may calculate a time difference between the two pulse waves in order to estimate a pulse wave velocity (PWV) representing a vascular elasticity. In order to calculate the time difference between the two pulse waves, in the present invention, the time difference may be calculated through the delay Lag having the maximum value in the CCF result S730 of the two pulse waves using a CCF (Cross Correlation Function). Equation 8 is an equation for performing CCF of two pulse waves.

<Equation 8>

The CCF generated as a result of Equation 8 indicates the correlation between the two pulse waves p1 and p2, and the delay time of the pulse wave can be calculated from the sample value from the origin to the position having the largest value.

The PWV estimation may be calculated based on the delay time of the pulse wave and the distance between the two measurement locations (S760). In order to calculate the actual distance of the skin region of interest shown in the skin (ex face skin, hand and foot skin) video image that is the measurement location, in the present invention, the distance estimation method between the camera and the skin and the actual distance estimation technique of the image coordinate system are applied (S740, S750) can be done. The control module 110 estimates the distance to the camera based on the skin region (photographed) in the video image of the captured skin region to derive the actual distance between the first skin region of interest and the second region of interest of the skin, and estimates You can use the distance to the camera.

In order to derive the actual distance between the camera lens and the skin area, the control module 110 may check the correlation between the skin area width on the image coordinate system according to the detection of the skin area and the actual distance between the camera and the skin area.

FIG. 8 shows a scatter diagram of an actual distance from a camera according to the detected face width on an image coordinate system after photographing a face region through the camera module 121 according to an embodiment of the present invention. In the graph shown in FIG. 8 , a horizontal direction may indicate a face width, and a vertical direction may indicate a distance from the camera.

For reference, since it is assumed that the face region is photographed in the case of FIG. 8 , the skin and health diagnosis apparatus 100 having a shape different from that of the skin and health diagnosis apparatus 100 of FIG. 1 will be used.

It can be seen from the graph shown in FIG. 8 that as the face width increases, the actual distance to the camera decreases non-linearly. Accordingly, the control module 110 may estimate the actual distance to the camera from the face width by applying a quadratic regression curve by reflecting the nonlinear characteristic. For example, the regression equation calculated in the 640 X 480 shooting environment is shown in Equation 9.

<Equation 9>

In Equation 9, the distance is the actual distance (cm) between the estimated camera lens and the face, and W may represent the width of the detected face on the image coordinate system. The control module 110 may convert the distance between the two ROIs from an existing pixel unit to an actual distance scale by using the estimated distance from the camera.

Also, the control module 110 may track the 10 cm marker expressed on the image coordinate system according to the distance from the camera, and calculate the ratio of cm per pixel according to the distance between the camera and the object using Equation 10. Equation 10 may represent an actual distance per pixel conversion equation. 9 is a graph showing the regression curve calculated in Equation (10). In the graph shown in FIG. 9 , a horizontal direction may indicate a distance to the camera, and a vertical direction may indicate a distance of 1 pixel.

<Equation 10>

The actual distance in cm can be calculated by multiplying W calculated in Equation 10 by a pixel value between two ROIs detected in a photographed skin region (eg, a face region, etc.).

In addition, the control module 110 of the skin and health diagnosis apparatus 100 calculates the actual distance D between the two regions of interest in cm and the delay time PPT of the two pulse waves calculated through the CCF result in order to estimate the PWV. By substituting into Equation 11, the PWV can be calculated (S760).

<Equation 11>

The control module 110 of the skin and health diagnosis apparatus 100 may estimate the blood vessel age based on the calculated PWV and the diagnosis (S770).

10 is a diagram illustrating the provision of customized content using related information stored in a database according to an embodiment of the present invention.

Referring to FIG. 10 , the database 140 of the skin and health diagnosis apparatus 100 includes user-specific body information, health index measurement result information, content information, and the like. The information may be continuously updated as the process of the present invention is performed. For example, the skin and health diagnosis apparatus 100 may continuously update the database 140 after acquiring the user's body information and health index measurement results.

In addition, the user-specific body information, health indicator measurement result information, content information, etc. stored in the database 140 will be stored in a state corresponding to each other. For example, content recommendation information may be stored to correspond to each health indicator measurement result, and content recommendation information may be stored to correspond to body information. In addition, health index measurement results or customized content information corresponding to each user classified by each identification code may be stored.

For reference, as described above, the database 140 may not be included in the skin and health diagnosis apparatus 100 unlike in FIG. 2 . That is, the database 140 may correspond to an external database such as an external server or cloud. In this case, the skin and health diagnosis apparatus 100 may transmit/receive necessary information to and from an external database through the communication module 130 .

The control module 110 of the skin and health diagnosis apparatus 100 combines or combines body information for each user, health index measurement result information, content information, etc. stored in the database 140 to log each user's health according to the user's health index. (Health-log) can be generated and the generated information can be provided to the user as a diagram. The health log may correspond to recording and managing a user's health index.

In addition, the control module 110 of the skin and health diagnosis apparatus 100 may generate and output a notification message for motivation to the user based on the health log, or may provide customized content. That is, the control module 110 may calculate the degree of state change based on the user's health log and recommend personalized content according to the state.

For example, the recommended personalized content may include personalized music, a pulse-matched and normal pulse induction sound source, a healing frequency applied sound source, a personalized exercise guide, and a personalized intake guide. That is, the personalized content can be applied to prevent and improve representative cardiovascular disease content such as music, diet, and exercise. The present invention is not limited thereto, and additional content other than the described content is possible.

11 is a view showing an intake guide according to food and health according to an embodiment of the present invention.

The database 140 may include personalized intake guide information as content information.

Specifically, the skin and health diagnosis apparatus 100 may acquire a food picture (FIG. 11(a)) through an external device 200 (ex user terminal, external server) or its own camera module 121 . In addition, the name of the food may be input from the user.

The control module 110 can classify based on a food photo or food name (FIG. 11 (b)), and input the classification result into a calorie model for each food to estimate the calories of the food (FIG. 11 (c)) can do. The control module 110 may continuously update such information in the database 140 .

In the case of cardiovascular disease, there may be a variety of recommended foods and foods to avoid. Therefore, the control module 110 of the skin and health diagnosis apparatus 100 provides dietary information that the user is not familiar with, such as recommended daily intake, recommended food, and food to avoid (FIG. 11 (d)) to prevent cardiovascular health. can It will be able to provide customized intake guides not only for cardiovascular health but also for other diseases.

In addition, the control module 110 of the skin and health diagnosis apparatus 100 may present a customized exercise according to the user's body information and health index, and provide the exercise intensity under a setting. In particular, it is possible to provide a personalized exercise guide by controlling the current user's running speed by providing a location-based service linked to a GPS sensor of a user's personal terminal connected to the skin and health diagnosis apparatus 100 .

In addition, the control module 110 of the skin and health diagnosis device 100 provides sound source recommendations that help improve and improve cardiovascular function according to the diagnosis results of health indicators, and a sound source reflecting the real-time pulse state of the user and a sound source for inducing a normal pulse rate. It can be printed out to induce cardiovascular prevention and improvement.

A sound source applied with a healing frequency that induces a positive cardiovascular effect through music therapy by emphasizing the healing frequency in the sound source input by the user separately from the pulse tuning sound source may be included. For example, representative healing frequencies are 528Hz, 432Hz, 936Hz, and the effect of each frequency was tested through various studies.

In addition, in the present invention, a sound source reflecting the healing frequency may be generated through the steps of emphasizing the healing frequency in the frequency domain of the sound source input by the user using the healing frequency and converting the sound source of the emphasized frequency domain into the time domain. .

It can also be processed as a sound source to which a technique emphasizing a healing frequency in a sound source reflecting the user's current or normal pulse wave through the pulse-controlled sound source generation technique and sequential processing described as another embodiment.

The skin and health diagnosis apparatus 100 may store, in the DB, performance details according to the content recommendation for the prevention and improvement of representative cardiovascular diseases such as music, diet, and exercise. The present invention relates to a health-log for preventing and improving cardiovascular disease by integrating the above-described diagnostic method and content recommendation elements, and another embodiment may exist by combining additional elements other than the above elements. have.

On the other hand, the control module 110 of the skin and health diagnosis apparatus 100 prints the measured user's health index through the output module 160 or transmits it to the external device 200 through the communication module 130 . can

As an experiment to measure vascular elasticity using skin images, the results of the PWV measurement experiment in an experimental group consisting of 20 men and women aged 20-30 years are as follows.

12 shows the PWV distribution of 100 data of 20 adult males and females, FIG. 13 shows the Korean adult male and female baPWV distribution according to age, and FIG. 14 shows the normal range of baPWV.

12 to 14, as an experiment of measuring vascular elasticity using skin images according to an embodiment of the present invention, a PWV measurement experiment was performed in an experimental group consisting of 20 men and women aged 20-30 years, and the frequency of the results is shown in Fig. The histogram shown in Fig. 12 is the same.

In the present invention, in order to amplify the value of baPWV, which is a commonly used pulse wave transmission velocity measurement method to measure the elasticity of blood vessels, in the same range as the value, the baPWV value according to the age of Korean adult men and women announced by the Exercise Prescription Center in Yongin is utilized. did.

In the scatter diagram shown in FIG. 13, an amplification constant α that matches the presented age-dependent value with the average PWV value calculated in the present invention was calculated and applied to the calculated PWV.

For diagnosis of PWV value, normal abnormal diagnosis according to age was performed by applying the baPWV normal range threshold value to the PWV value multiplied by the amplification constant α. 14 shows the average line and the upper and lower limits of normal according to age.

As shown in FIG. 14 , PWV diagnosis was performed according to the age of the user by applying the range of the graph. In addition, by inputting the PWV value calculated from the skin image and outputting the value of the age axis, the age of the blood vessel was estimated and provided.

The embodiments according to the present invention described above may be implemented in the form of program instructions that can be executed through various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the computer-readable recording medium may be specially designed and configured for the present invention, or may be known and used by those skilled in the computer software field. Examples of the computer-readable recording medium include hard disks, magnetic media such as floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magneto-optical media such as floppy disks. media), and hardware devices specially configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware device may be configured to operate as one or more software modules to perform processing according to the present invention, and vice versa.

In the above, the present invention has been described with specific matters such as specific components and limited embodiments and drawings, but these are provided to help a more general understanding of the present invention, and the present invention is not limited to the above embodiments. , those of ordinary skill in the art to which the present invention pertains can devise various modifications and variations from these descriptions.

Therefore, the spirit of the present invention should not be limited to the above-described embodiments, and not only the claims described below, but also all modifications equivalently or equivalently to the claims described below belong to the scope of the spirit of the present invention. will do it

100: skin and health diagnosis device
110: control module
120: skin measurement module
121: camera module
122: lighting module
130: communication module
140: database
150: input module
160: output module
200: external device (server)

Claims (10)

A method for measuring a user's health index, the method comprising:
(a) a skin and health diagnosis apparatus photographing a part of the user's skin present on the opening, and detecting the coordinates of the skin region of interest or the skin part from the acquired image image;
(b) the skin and health diagnosis apparatus calculates the average of color data in the skin ROI, corrects the coordinates of the detected skin part through a moving average algorithm, and updates the average of the color data based on this extracting an average value of color data; and
(c) extracting, by the skin and health diagnosis apparatus, a bio-signal based on the average value of the color data, and measuring the user's health index using the bio-signal,
In step (c), at least one or more of pulse rate per minute, respiration rate per minute, pulse strength, blood pressure, bradycardia, tachycardia, irregular arrhythmia, blood volume, blood viscosity, abnormal pulse, vascular elasticity, and vascular age is measured as the health indicator, ,
The blood pressure is estimated through Equation 1 below,
<Formula 1>

(where x is blood pressure estimated using body information and pulse wave transmission time, y1 is systolic blood pressure, y2 is diastolic blood pressure)
Ventricular premature contraction (PVC), one of the irregular arrhythmias, is estimated through Equation 2 below based on the interval between the R wave just before the occurrence of the ventricular premature contraction and the two consecutive R waves thereafter.
<Formula>

(However, N is the number of R peaks (N: number of R peaks), RRvar is continuous RR change (RRvar: variation of continuous RR), RRth1 is 78% of the total RR interval) According to claim 1,
In a state in which body information including the height and weight of the user is obtained from an input module or database,
The skin and health diagnosis device uses the body information and the bio-signals in step (c) to provide pulse rate per minute, respiration rate per minute, pulse intensity, blood pressure, bradycardia/tachycardia/irregular arrhythmia, blood volume, blood viscosity, and abnormal pulse rate. , vascular elasticity, and diagnosing at least one of vascular age, and measuring the health index of the user. 3. The method of claim 2,
i) the pulse rate per minute is calculated using the maximum frequency component in the pulse-related frequency band of the biosignal, ii) the respiration rate is calculated using the maximum frequency component in the respiration-related frequency band of the biosignal, iii ) the pulse intensity is calculated using the amplitude of a pulse wave subjected to BPF (band pass filtering) as a pulse-related frequency band of the biosignal, iv) the bradycardia/tachycardia arrhythmia is calculated based on the calculated pulse rate, The irregular arrhythmia is calculated using the ventricular premature beat (PVC) standard and the ACF slope change of the pulse wave RR interval, v) the blood volume is calculated using the slope between the peaks in the blood volume related frequency band of the biosignal, vi) The blood viscosity is calculated using the amplitude, the differential value, and the spring constant K at the peak point of the pulse wave, vii) The strange pulse is calculated using the difference in amplitude of the pulse wave during inhalation and exhalation. 3. The method of claim 2,
In a state in which the first skin region of interest and the second skin region of interest are included in the skin region of interest,
i) the blood pressure is calculated using a pulse transit time (PTT) of pulse waves calculated from the first skin ROI and the second skin ROI, ii) the first skin ROI and the second skin ROI Assuming that a pulse wave velocity is obtained using the time difference of the pulse waves calculated in , the vascular elasticity and the vascular age are calculated using the pulse wave velocity. According to claim 1,
In the state that content recommendation information corresponding to each health indicator is stored in the database,
The skin and health diagnosis device configures a health log for recording and managing the user's health index, generates and outputs a notification message for motivation of the user based on the health log, and provides customized content A method characterized in that 6. The method of claim 5,
The customized content is
A method comprising at least one of personalized music, a pulse-matched and normal pulse induction sound source, a sound source applied with a healing frequency, a personalized exercise guide, and a personalized intake guide. According to claim 1,
The method of claim 1, wherein the skin and health diagnosis device continuously updates the user's body information and health index in a database. According to claim 1,
When it is assumed that at least one opening into which the user's body part can be inserted exists in the skin and health diagnosis apparatus,
The method of claim 1, wherein the skin and health diagnosis apparatus captures the video image by using a camera module and an illumination module located in the opening. According to claim 1,
The method of claim 1, wherein the skin and health diagnosis apparatus prints the measured health index of the user through an output module or transmits the measured health index to the user's terminal. In the skin and health diagnosis apparatus for measuring a user's health index,
a communication module for transmitting and receiving information to and from an external device;
an opening into which a part of the user's body can be inserted;
a skin measurement module including a camera module and a lighting module located in the opening;
photographing a part of the user's skin using the skin measurement module, detecting the coordinates of the skin ROI or the skin part from the photographed video image, calculating an average of color data in the skin ROI, and the detection After correcting the coordinates of a part of the skin with a moving average algorithm, the average value of the updated color data is extracted based on this, and the average value of the color data is filtered to extract a biosignal, and the biosignal is used. to include a control module for measuring the user's health index,
At least one of pulse rate per minute, respiratory rate per minute, pulse strength, blood pressure, bradycardia, tachycardia, irregular arrhythmia, blood volume, blood viscosity, abnormal pulse, vascular elasticity, and vascular age is measured as the health index,
The blood pressure is estimated through Equation 1 below,
<Formula 1>

(where x is blood pressure estimated using body information and pulse wave transmission time, y1 is systolic blood pressure, y2 is diastolic blood pressure)
Ventricular premature contraction (PVC), which is one of the irregular arrhythmias, is estimated through Equation 2 below based on the R wave just before the occurrence of the ventricular premature contraction and the interval between the two consecutive R waves.
<Formula 2>

(However, N is the number of R peaks (N: number of R peaks), RRvar is continuous RR change (RRvar: variation of continuous RR), RRth1 is 78% of the total RR interval)

Images