KR20200140469A - Method for measuringing skin and health indicators of an user using a video image and apparatus using the same - Google Patents

Abstract

According to the present invention, provided is a method for measuring health indicators of a user, which includes the steps of: photographing a part of a user′s skin, by a skin and health diagnosis apparatus, and detecting a skin region of interest or coordinates of the part of the skin from an acquired video image; calculating an average of color data in the skin region of interest, correcting the coordinates of the detected part of the skin through a moving average algorithm, and extracting the average color data average value in which the average of the color data is updated based thereon, by the skin and health diagnosis apparatus; and extracting, by the skin and health diagnosis apparatus, a bio-signal based on the average value of the color data, and measuring the health indicator of the user by using the bio-signal.

Description

A method of measuring a user's skin and health indicators through video images, and a device using the same {METHOD FOR MEASURINGING SKIN AND HEALTH INDICATORS OF AN USER USING A VIDEO IMAGE AND APPARATUS USING THE SAME}

The present invention relates to a method of measuring a user's health index through an image image, and more particularly, in a method of measuring a user's health index, the skin and the user's skin having a health diagnosis device on the opening Photographing a part and detecting a skin region of interest or coordinates of the part of the skin from the photographed image image; The skin and health diagnosis apparatus calculates an average of color data in the skin region of interest, corrects the coordinates of the detected part of the skin through a moving average algorithm, and updates the average of the color data based on the average color data average value. Extracting; And extracting, by the skin and health diagnosis apparatus, a biosignal by filtering the average value of the color data, and measuring a health indicator of the user using the biosignal.

Cardiovascular disease, the leading cause of death in the world's population, accounts for about 21.6% of all deaths in Korea. Cardiovascular disease is rapidly increasing due to the aging population and lifestyle changes. Cardiovascular diseases include ischemic heart disease, circulatory system related diseases such as stroke and hypertension, and are affected by various risk factors such as genetic factors of parents and environmental factors such as alcohol and smoking. It is also closely related to other diseases such as diabetes and obesity.

Traditional tools for estimating arrhythmia, one of the cardiovascular diseases, have used a method of estimating through the ratio of the RR interval in the ECG waveform. Unlike the electrocardiogram waveform, which is an electrical signal of the heart, it is not easy to observe other waveforms other than the R wave in the pulse wave measuring blood flow according to the pulse.

Arterial stiffness refers to the degree of rigidity due to the decrease in the elasticity of the artery. The most important factor in determining the stiffness of an artery is softness, and with age, the arterial wall tissue changes with age, resulting in a decrease in elasticity and an increase in stiffness.

Arterial stiffness is an important measure to predict the risk of cardiovascular disease. Patients with increased arterial stiffness are classified as a high risk group for future cardiovascular disease.

And the pulse wave transmission velocity (PWV) is a useful index to estimate the stiffness of the artery. The pulse wave transmission speed (PWV) changes with age. In young people, since the arteries are elastic, the pulse wave transmission rate (PWV) is relatively low, but as the aging progresses, the arteries are not elastic and the pulse wave transmission rate (PWV) increases. As a method of measuring the pulse wave transmission speed (PWV), it is classified according to the measurement part, and among them, baPWV (brachial-ankle PWV) is used, which makes recording of waveforms relatively easy. In the conventional baPWV, the velocity was calculated using the difference value and distance of the pulse wave velocity at two points in the pulse wave extracted from the ankle and upper arm.

As described above, conventionally, arrhythmia or arterial stiffness has been measured in order to diagnose cardiovascular disease, but a device for accurately measuring it is expensive and expensive to measure, so there are many difficulties in periodically treating and measuring.

In addition, conventionally, there has been a device for instantly diagnosing a user's skin through a skin diagnosis device, but there is a limitation in that it can only measure the skin condition.

Accordingly, the present inventors propose a method of measuring a user's skin and health indicators through an image image and an apparatus using the same.

An object of the present invention is to solve all of the above-described problems.

Another object of the present invention is to perform self-diagnosis using a video image and provide customized content to a corresponding user through the self-diagnosis result.

In addition, another object of the present invention is to take an image image, extract an average color data value from the captured image image, and diagnose a user's health indicator through this.

In addition, the present invention proposes a technique for diagnosing health by using an image in addition to a configuration for diagnosing skin using an image.

In order to achieve the object of the present invention as described above, and to realize the characteristic effects of the present invention described later, the characteristic configuration of the present invention is as follows.

According to an aspect of the present invention, in a method of measuring a user's health index, a skin and health diagnosis apparatus photographs a part of the user's skin, and calculates a skin region of interest or a coordinate of the part of the skin in the photographed image image. Detecting; The skin and health diagnosis apparatus calculates an average of color data in the skin region of interest, corrects the coordinates of the detected part of the skin through a moving average algorithm, and updates the average of the color data based on the average color data average value. Extracting; And extracting a biosignal by filtering the average value of the color data by the skin and health diagnosis apparatus, and measuring a health indicator of the user using the biosignal.

Further, according to another aspect of the present invention, there is provided a skin and health diagnosis apparatus for measuring a user's health index, comprising: a communication module for transmitting and receiving information with an external device; An opening through which a part of the user's body can be inserted; A skin measurement module including a camera module and a lighting module positioned in the opening; Using the skin measurement module, a part of the user's skin is photographed, a skin region of interest or coordinates of the skin part are detected from the photographed image image, and an average of color data in the skin region of interest is calculated, and the detection After correcting the coordinates of a portion of the skin through a moving average algorithm, the average value of color data is extracted based on the average of the color data, the average value of the color data is filtered to extract a biosignal, and the biosignal is used. Thus, a skin and health diagnosis apparatus including a control module measuring the user's health indicators is provided.

According to the present invention, the following effects are obtained.

The present invention has the effect of performing self-diagnosis using a video image and providing customized content to a corresponding user through the self-diagnosis result.

In addition, the present invention is effective in capturing an image image, extracting an average color data value from the captured image image, and diagnosing a user's health indicators through this.

1 is a view showing the appearance of a skin and health diagnosis apparatus according to an embodiment of the present invention.
2 is a diagram showing a schematic configuration of a skin and health diagnosis apparatus according to an embodiment of the present invention.
3 is a diagram showing a process of measuring a user's health indicator according to an embodiment of the present invention.
4 is a diagram showing coordinates before and after applying a moving average algorithm according to an embodiment of the present invention.
5 is a diagram showing in detail a process of measuring a user's health indicator from a bio-signal according to an embodiment of the present invention.
6 is a view showing a normal pulse wave and PVC generation section according to an embodiment of the present invention.
7 is a diagram illustrating a process of calculating blood vessel elasticity and blood vessel age according to an embodiment of the present invention.
FIG. 8 is a diagram showing a distribution diagram of an actual distance to a camera according to a face width detected in an image coordinate system according to an embodiment of the present invention.
9 is a view showing a regression curve in which an actual distance per pixel is estimated according to a distance to a camera based on a scatter plot according to an embodiment of the present invention.
10 is a diagram illustrating providing customized content using related information stored in a database according to an embodiment of the present invention.
11 is a view showing an intake guide according to food and health according to an embodiment of the present invention.
12 is a diagram showing the PWV distribution of 100 data of 20 adult men and women.
13 is a diagram showing the distribution of baPWV for men and women in Korea according to age.
14 is a diagram showing the normal range of baPWV.

DETAILED DESCRIPTION OF THE INVENTION The detailed description of the present invention to be described later refers to the accompanying drawings, which illustrate specific embodiments in which the present invention may be practiced. These embodiments are described in detail sufficient to enable a person skilled in the art to practice the present invention. It is to be understood that the various embodiments of the present invention are different from each other, but need not be mutually exclusive. For example, specific shapes, structures, and characteristics described herein may be implemented in other embodiments without departing from the spirit and scope of the present invention in relation to one embodiment. In addition, it is to be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the present invention. Accordingly, the detailed description to be described below is not intended to be taken in a limiting sense, and the scope of the present invention, if properly described, is limited only by the appended claims, along with all scopes equivalent to those claimed by the claims. Like reference numerals in the drawings refer to the same or similar functions over several aspects.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings in order to enable those of ordinary skill in the art to easily implement the present invention.

1 is a view showing the appearance of a skin and health diagnosis apparatus according to an embodiment of the present invention.

2 is a diagram showing a schematic configuration of a skin and health diagnosis apparatus according to an embodiment of the present invention.

The present invention extracts a bio-signal from a skin region of interest of an image image of a part of the user's body photographed by the skin and health diagnosis apparatus 100, and uses the extracted bio-signal to obtain a pulse rate per minute, respiratory rate per minute, and pulse intensity. Health indicators such as blood pressure, bradycardia/tachycardia/irregular arrhythmia, blood volume, blood viscosity, odd pulse, vascular elasticity, and blood vessel age can be measured. The present invention stores the measured health indicators in the database 140, and provides them to the user in the form of a health-log based on this, or provides customized content. For reference, the shape of the skin and health diagnosis apparatus 100 shown in FIG. 1 corresponds to one example, and various shapes may exist.

In the present invention, as an example of various color systems, pulse rate per minute, respiratory rate per minute, pulse intensity, from bio signals extracted by applying BPF (Band Pass Filter) to the average value of Cg color data calculated by converting the RGB color system to the YCgCo color system, Blood pressure, bradycardia/tachycardia/irregular arrhythmia, blood volume, blood viscosity, odd pulse, vascular elasticity, and vascular age can be extracted or estimated. A detailed process for this will be described later.

Referring to FIG. 2, the skin and health diagnosis apparatus 100 includes a control module 110, a skin measurement module 120, a communication module 130, a database 140, an input module 150, an output module 160, etc. It may include.

First, the control module 110 controls the overall process in the present invention, specifically, receives an image image from the skin measurement module 120, extracts a biosignal using it, and uses the extracted biosignal to indicate health indicators. Can be diagnosed.

The skin measurement module 120 is present in the openings 10 and 20, and includes an illumination module 121 for illuminating a user's body part such as hands and feet, and a camera for photographing a user's body part such as hands and feet. It may include at least one of the modules 122. Since it may have a shape other than the shape of the skin and health diagnosis apparatus 100 of FIG. 1(a) (ex FIG. 1(b)), it is possible to take pictures of not only hands and feet, but also other body parts such as face and neck. have. In particular, the camera module 122 may capture not only a simple image but also a moving image, and as will be described later, it may diagnose or measure the skin through the captured moving image, and ultimately diagnose the user's health. The video image described in the present invention mainly assumes a moving image, and a simple image will not be excluded. Therefore, in some cases, a moving image and a simple image may be captured at the same time.

In addition, the skin and health diagnosis apparatus 100 includes two openings 10 and 20 through which a body part can be inserted as shown in FIG. 1(a), and a skin measurement module for each (120a, 120b) may be included. That is, any one skin measurement module 120a is included in one of the openings 10, and another skin measurement module 120b is included in the other opening 20, so that two or more skin measurement modules are simultaneously included. In (120a, 120b), two or more locations of the user's body may be photographed. For reference, a wall separating between the two openings 10 and 20 may be removed and used as one opening.

Meanwhile, as another embodiment, a face diagnosis apparatus 100 having a shape as shown in FIG. 1(b) rather than FIG. 1(a) may also exist. In this case, the user may take an image of the face through the skin measurement module located on the front part 15 while sitting. The user can insert the face into the concave portion 25 in the face diagnosis apparatus 100 and take a picture of the face through the camera module located at the front portion 15. In addition, when the video image is not captured, a sliding-type cover may cover the concave portion 25.

The communication module 130 may be implemented with various communication technologies. In other words, Wi-Fi, WCDMA (Wideband CDMA), HSDPA (High Speed Downlink Packet Access), HSUPA (High Speed Uplink Packet Access), HSPA (High Speed Packet Access), Mobile WiMAX, WiBro , Long Term Evolution (LTE), Bluetooth, infrared data association (IrDA), Near Field Communication (NFC), Zigbee, WLAN technology, etc. may be applied. In addition, when a service is provided by being connected to the Internet, TCP/IP, which is a standard protocol for information transmission on the Internet, may be followed, or a Global Positioning System (GPS) technology may be used. In the present invention, communication with the external device 200 may be performed through the communication module 130. Here, the external device 200 may include various devices such as a user terminal, an external server, and a cloud.

As will be described later, the database 140 may store necessary information, such as a measurement result of a user's health indicator, a user's body information, content information, and an image image, and may provide this to the control module 110. For reference, unlike FIG. 2, the database 140 is not included in the skin and health diagnosis apparatus 100 but exists outside, and information may be transmitted and received with the skin and health diagnosis apparatus 100 through the communication module 130. have.

Both the input module 150 and the output module 160 are controlled by the control module 110, and the input module 150 may include any device capable of input such as a touch screen 30, a keypad, and a mouse, and output The module 160 may include various devices such as a printing device and a display device. Unlike FIG. 2, the input module 150 and the output module 160 are not included in the skin and health diagnosis apparatus 100, but may exist outside.

As an external device 200 connected to the skin and health diagnosis apparatus 100 of the present invention, various devices such as a user's terminal, an external server, and a cloud may be included. For reference, the terminal is a digital device including a function capable of performing communication, and has a memory means such as a desktop computer, a notebook computer, a workstation, a PDA, a web pad, a mobile phone, etc. Any digital device equipped with can be adopted as a user terminal according to the present invention. In particular, the user terminal may include a health indicator diagnosis program (application) covered in the present invention.

3 is a diagram showing a process of measuring a user's health indicator according to an embodiment of the present invention.

First, the skin and health diagnosis apparatus 100 may acquire user's body information (ex height, weight, gender, age, blood pressure, pulse, body temperature, etc.) through the input module 150 (S210). In addition, personal information such as the user's name may be input. The user may input information such as his/her height, weight, gender, age, blood pressure, pulse, and body temperature through the touch screen 30 and the like, and may store the information in the database 140. Since the database 140 can be classified and stored for each user by an identification code or the like, the user's diagnosis result or body information can be continuously updated or initialized in the database 140.

In addition, not only can the user's body information be directly input through the input module 150, but also the user's body information recorded in the database 140 is transferred to the control module 110 of the skin and health diagnosis apparatus 100. You might be able to receive this. In some cases, the skin and health diagnosis apparatus 100 is linked with an application installed in the terminal 200 that the user is wearing, so that the user's body information (blood pressure, pulse, body temperature) that is changed due to the user's movement is recorded in real time. It can also be provided.

Next, the skin and health diagnosis apparatus 100 uses the camera module 121 included in the skin measurement module 120 to use a part of the user's skin (eg, hands, back of hands, nails, feet, insteps, toenails, face, etc.) Is captured, and the coordinates of a skin region of interest or a portion of the skin may be detected from the acquired image image (S220).

In addition, the skin and health diagnosis apparatus 100 may support image capturing by irradiating light onto a part of the user's skin using the illumination module 122. Here, the illumination module 122 may be an LED module, and by sequentially irradiating general light, polarized light, and UV light, individual image images according to each of general light, polarized light, and UV light may be obtained. Skin conditions (normal light), epidermal pigmentation-related skin conditions (polarization), dermal pigmentation, skin types, and skin conditions (UV light) related to porphyrin may be measured by each light of the illumination module 122. In this case, a skin index may be measured in relation to the photographed user's skin condition, and the measured skin index may include elasticity, oil and moisture content, oil and moisture balance, wrinkle index, and skin tone. According to the above skin index, the skin and health diagnosis apparatus 100 may recommend a skin care method or cosmetics.

For reference, when the skin care method and cosmetics are recommended, the measurement result affected by dry or humid weather can be corrected based on the weather information included in the diagnosis result, and dry or humid weather, weather with strong ultraviolet rays. And how to do skin care in a weather that affects the skin, such as a weather with a lot of fine dust, a skin care method may be included in the skin diagnosis result.

In addition, the control module 110 may provide the user with a skin diagnosis result in which the user's nasolabial folds, acne, spots, eyelash lengths and points included in the captured image image are displayed.

For example, when the oil and moisture content of the hand is measured to be low, a skin care method that requires the use of a hand cream with high moisturizing power may be recommended, and information on at least one cosmetic product that increases the oil and moisture content may be included in the skin diagnosis result. However, the skin diagnosis result is not limited to the above example, and may include information on skin care methods and recommended cosmetics for other items related to skin conditions, such as elasticity of hands or feet, wrinkle index, and skin tone.

The location and size of the region of interest of the skin to be photographed may be various and may be flexibly adjusted. The skin and health diagnosis apparatus 100 may detect or set a skin region of interest using various algorithms. In this case, the algorithm for detecting the skin region of interest may use various techniques known in the art.

The control module 110 of the skin and health diagnosis apparatus 100 may calculate or calculate an average of color data in a skin region of interest on an image image that is being photographed or already photographed.

Here, for example, various values may be used for color data. For example, (1) color data may use at least one of an R value, a G value, and a B value based on an RGB color system. That is, color average data for at least one of an R value, a G value, and a B value may be used as the color data. (2) In addition, the skin and health diagnosis apparatus 100 may convert an RGB color system into another color system. For example, the skin and health diagnosis apparatus 100 may convert an RGB color system into various color systems such as YUV, HSV, YCbCr, YCgCo, and the like. In this case, the color data may use one of the color difference components that are less affected by the surrounding environment (illumination, etc.). For example, in the case of YCbCr, at least one of a Cb value or a Cr value may be used. In the case of YCgCo, at least one of a Cg value or a Co value may be used. Furthermore, one of the two color difference components, which is more robust to changes in illuminance, can be used. For example, in the case of YCgCo, only the Cg value can be used. In this case, the control module 110 of the skin and health diagnosis apparatus 100 may extract the average value of the Cg color data of the skin region as color data. (3) Furthermore, the color data may be a value obtained by applying a weight to at least one or more color components in various color systems such as RGB, YUV, HSV, YCbCr, YCgCo, and the like. When color components are combined, the color data may be a value obtained by adding different weights according to a color system and a type of color component. The skin and health diagnosis apparatus 100 may change a face image having an RGB color system to a YCgCo color system, and it is assumed that the skin and health diagnosis apparatus obtains and uses a Cg value from YcgCo. The control module 110 may change an image image having an RGB color system into a YCgCo color system using Equation 1 below.

<Equation 1>

As another example, when the image is an infrared image captured by an infrared camera, an average value of gray color data may be extracted from the skin area and used.

However, the control module 110 may extract the biosignal by applying filtering to the updated color data average value instead of using the average of the color data directly. This is to obtain a stabilized average value of color data, which will be described with reference to FIG. 4.

4 is a diagram showing coordinates before and after applying a moving average algorithm according to an embodiment of the present invention. Fig. 4(a) shows the state before applying the moving average algorithm, and Fig. 4(b) shows the state after applying the moving average algorithm.

Specifically, the control module 110 may detect the coordinates of a part of the user's skin through the skin measurement module 120. At this time, the coordinates can be regarded as the coordinates of the skin area.

As shown in Fig. 4(a), the control module 110 of the skin and health diagnosis apparatus 100 has a limitation in extracting a biosignal from a stable position because the coordinates of the skin area detected in every frame change. I can. Accordingly, the control module 110 may detect stable coordinates by applying a moving average algorithm to coordinates (x, y, width, and height) used for detecting the skin area.

The skin and health diagnosis apparatus 100 may efficiently extract a biosignal from a blood vessel of the skin by applying a moving average algorithm to the coordinates to stabilize the image coordinate of the skin region detected every frame from the input image. In this case, the moving average algorithm may use a moving average filter (MAF).

Eventually, the control module 110 of the skin and health diagnosis apparatus 100 corrects the coordinates of a part of the detected skin through a moving average algorithm, and then extracts an average value of color data obtained by updating (safety) the average of color data based on this. (S230) You can.

The control module 110 may filter the average value of the color data to extract a biosignal. The biological signal at this time can be regarded as a pulse wave signal. The skin and health diagnosis apparatus 100 is capable of estimating a cardiovascular health index using the calculated biometric signal.

Specifically, the control module 110 of the skin and health diagnosis apparatus 100 may detect a bio-signal (a pulse wave signal) by applying a band pass filter (BPF) to the average value of the color data. Of course, although it is unstable, it may be possible to detect a biosignal by applying a band pass filter (BPF) to the average of color data before applying the moving average algorithm (MAF).

In addition, the control module 110 of the skin and health diagnosis apparatus 100 may apply a Fast Fourier Transform (FFT) to the average value of color data. For example, the control module 110 extracts the Cg signal by calculating the average of the Cg values in every frame, and uses the FFT (Fast Fourier Transform) to determine the largest frequency component of the pulse wave. It can be judged by the cycle. Pulse wave refers to a wave of blood that forms waves in the heart and propagates, and can be used to measure heart rate (HRV), pulse, respiration, pulse strength, and blood pressure.

The control module 110 may observe the signal in the frequency domain in order to estimate the pulse wave signal from the Cg signal. For example, the control module 110 may measure a pulse wave per minute from about 40 to 200 depending on the level of stability or excitement in a normal case, and accordingly, the area observed in the frequency domain may be limited to 0.65 Hz to 3.4 Hz. have.

5 is a diagram showing in detail a process of measuring a user's health indicator from a biometric signal according to an embodiment of the present invention.

The control module 110 of the skin and health diagnosis apparatus 100 may measure a user's health indicator by using the extracted biometric signals and body information (S240). Specifically, at least one of pulse rate per minute, respiratory rate per minute, pulse intensity, blood pressure, bradycardia/tachycardia/irregular arrhythmia, blood volume, blood viscosity, odd pulse, vascular elasticity, and vascular age are diagnosed to determine the health indicators of the user. Can be measured. In other words, the health of the user is diagnosed through the biosignal extracted from the image image.

For example, the pulse rate can be estimated (11) as the maximum frequency component in the pulse-related frequency band (0.67 ~ 4 Hz) of the biological signal. Respiration can be estimated (12) as the maximum frequency component in the respiratory-related frequency band (0.13~0.35Hz) of the biological signal. The pulse strength can be estimated (13) by using the amplitude of the pulse wave performed by BPF in the pulse-related frequency band of the biological signal. The blood pressure can be estimated (14) by using the pulse transit time (PTT) of the pulse wave calculated in the two regions of interest in the skin. The bradycardia/tachycardia/arrhythmia can be estimated (15) by applying the estimated pulse rate to the bradycardia criterion (60 times or less) and tachycardia criterion (100 times or more). Irregular arrhythmia can be estimated (15) by using the baseline ventricular premature beat (PVC) of the pulse wave R-R interval and the change in ACF slope. The blood volume can be estimated (16) by using the slope of the next maximum peak from the maximum peak in the pulse wave frequency domain (0 to 4.2 Hz). Blood viscosity can be estimated (17) using amplitude, first and second derivatives, and spring constant K at the peak point of the pulse wave. The odd pulse can be estimated (18) using the degree of the odd pulse in which the difference in pulse rate and intensity are significantly different in each of the inhalation and exhalation intervals. The vascular elasticity can be estimated (19) by calculating the time difference between the pulse waves calculated from the two skin regions of interest and the distance of the region of interest. The blood vessel age can be estimated (19) by applying the estimated pulse wave transmission velocity (PWV) value and the average PVV of male and female Korean adults. This is described in detail as follows.

In the case of a pulse, the control module 110 of the skin and health diagnosis apparatus 100 processes the MAF of the maximum frequency component or frequency amplitude value of 0.67 to 4 Hz, which is a pulse-related frequency band, from the extracted bio-signal in the case of pulse. You can estimate the pulse rate using a lamp.

In the case of respiration, it can be estimated based on the frequency characteristics of the respiration-related frequency (0.13~0.35Hz) band in the same way as the pulse.

In the case of pulse strength, it can be provided in a graph form according to the amplitude of a pulse wave processed by Band Pass Filtering (BPF) in a pulse-related frequency band.

In the case of blood pressure, it can be estimated using pulse waves calculated from two skin regions of interest. In the present invention, the skin and health diagnosis apparatus 100 may detect the first skin region of interest and the second skin region of interest using a plurality of skin measurement modules 120a and 120b. At this time, the first skin region of interest is detected by the camera module 121a of the first skin measurement module 120a included in the first opening 10 of the skin and health diagnosis apparatus 100, and the second opening 20 The second skin region of interest may be detected by the camera module 121b of the second skin measurement module 120b included in ).

In addition, unlike the above, the first skin region of interest and the second region of interest may be detected from an image image detected using one skin measurement module 120. In this case, only one of the first skin measurement module 120a or the second skin measurement module 120b may be used.

In the case of blood pressure, the user's body information (height, weight, etc.) and the pulse wave transmission time may be related. Therefore, by applying multiple regression for estimating blood pressure, a regression model can be obtained with the pulse wave transmission time, height, and weight of the pulse waves extracted from the two ROIs as independent variables.

In the present invention, the improved blood pressure may be estimated using Equation 1 below by additionally applying a correlation between the blood pressure estimated in the regression model and the blood pressure measured through the device in addition to the regression model. Equation 1 shows an example of a regression curve equation for improved blood pressure estimation.

<Equation 2>

In Equation 1, x may represent an estimated blood pressure using body information and a pulse wave transmission time, y may represent an improved blood pressure by applying a relationship between the blood pressure measured by the device and the estimated blood pressure, and y1 is a systolic, y2 may represent the diastolic blood pressure.

In addition, the control module 110 of the skin and health diagnosis apparatus 100 is a method of estimating bradycardia, tachycardia, and irregular arrhythmia, which are representative cases of arrhythmia, and a method based on pulse rate and a method using pulse RR intervals. It can be provided.

In the case of bradycardia and tachycardia, it can be estimated based on the pulse rate at rest, and it can be classified as bradycardia when a pulse rate is less than 60 times per minute, and tachycardia when a pulse rate is more than 100 times.

In the case of ventricular premature contraction (PVC), which is one of the irregular arrhythmia, it can be calculated by using a reference according to the interval of the R wave, which is the peak of the pulse wave, and the corresponding auto correlation function (ACF) slope.

In the case of the ventricular premature heartbeat (PVC), the control module 110 may estimate the R wave immediately before the occurrence and the interval between two consecutive R waves after the occurrence. Equation 2 for estimating ventricular premature contraction (PVC) is as follows.

<Equation 3>

In Equation 3, an equation for the PVC estimation method using the R-R interval can be shown.

N may represent the number of R peaks (N: number of R peaks), and RR _var may represent a variation of continuous RR (RR _var : variation of continuous RR). In addition, RR _th1 is 78% of the total RR interval and can be introduced to estimate the case where the initial peak interval, which is a characteristic of PVC, is short and the peak interval is long thereafter.

Through the above-described Equation 3, the control module 110 may estimate the PVC after detecting the peak of the pulse wave.

In addition, the computer device can use the classified PVC and ACF slope of the standing pulse wave as another method to estimate the PVC.

First, in the case of PVC, additional features other than the R-R interval may appear, such as the R-R interval and attenuation of the generated R wave. Therefore, it is possible to use ACF that can detect sudden changes in pulse waves.

6 is a view showing a normal pulse wave and PVC generation section according to an embodiment of the present invention.

As shown in FIG. 6(a), in the case of a steady pulse wave, a signal is observed in the form of a sine wave, so that when ACF is applied, linearity of the slope can be guaranteed. The ACF at the point where the PVC occurred may show the ACF result of an irregular slope, as shown in FIG. 6(b). In the present invention by using the slope shown in FIG. 6, it is possible to detect PVC using ACF in irregular changes in pulse waves such as R-R intervals and attenuation of R waves.

The skin and health diagnosis apparatus 100 may use Equation 4 below to calculate the change in the inclination described above. Equation 4 can calculate the ACF slope change.

<Equation 4>

N disclosed in Equation 4 may represent that the number of peaks is divided by 2 and then 1 is subtracted (N: (number of peak/2)-1).

The skin and health diagnosis apparatus 100 may perform PVC detection using the slope of ACF by learning a threshold value that is divided into a normal pulse wave and PVC using the calculated slope change value G.

In the case of blood volume, the peak-to-peak slope value in the frequency domain of the biological signal and the user's body information (gender, weight) can be used to estimate. The control module 110 of the skin and health diagnosis apparatus 100 may set a frequency band (0 to 4.2 Hz) related to the blood volume in the frequency domain from the measured bio-signals, and calculate a frequency domain peak of the set band.

Thereafter, the control module 110 may estimate the blood volume by applying the calculated slope value G between peaks to Equation 5 below. Equation 5 may represent a blood volume estimation equation.

<Equation 5>

In the case of blood viscosity, it can be estimated using the peak amplitude of the pulse wave calculated from the skin image, the second derivative value of the peak point, and the spring constant K.

The spring constant K represents the ratio of the linear displacement of the blood vessel elasticity and can be estimated as a pulse wave. This can be calculated through Equation 6. Equation 6 is an estimation equation of the spring constant K.

<Equation 6>

In addition, the blood viscosity may be calculated by applying the spring constant K estimated through Equation 6 to Equation 7 which is the blood viscosity estimation equation below. Equation 7 may be an equation capable of estimating a blood viscosity estimation equation.

<Equation 7>

In addition, the control module 110 may determine when the difference between the blood pressure during inhalation and exhalation of the odd pulse is 10 mmHg or more, and may occur in the case of pericardial pressure, pulmonary embolism, pericardial effusion, pneumothorax, and the like. In the present invention, an odd pulse can be detected using the respiratory waveform and pulse wave calculated from the skin image.

The control module 110 distinguishes between inspiration (lower peak to upper peak) and expiration (upper peak to lower peak) section from the breathing waveform calculated from the bio-signal, and calculates the difference in amplitude of the pulse wave in each section between the intake and exhalation apparatus. The odd pulse can be estimated by the ratio of the amplitude.

In the abnormal pulse diagnosis, the degree can be expressed and provided in four categories. Table 1 is a table organized by dividing odd pulse states.

<Table 1>

As described in Table 1, it may be normal when the pulse wave amplitude ratio between the intake and exhalation ports is ratio _r > 0.65. And if 0.15>= ratio _r , it may be a risk.

7 is a diagram illustrating a process of calculating blood vessel elasticity and blood vessel age according to an embodiment of the present invention.

FIG. 8 is a diagram showing a distribution diagram of an actual distance to a camera according to a face width detected in an image coordinate system according to an embodiment of the present invention.

9 is a view showing a regression curve in which an actual distance per pixel is estimated according to a distance to a camera based on a scatter plot according to an embodiment of the present invention.

The skin and health diagnosis apparatus 100 may photograph a part of the user's body using the camera module 121 and may set a first skin region of interest and a second skin region of interest therefrom (S710).

In particular, in the process of calculating the vascular elasticity and the vascular age, the first skin region of interest and the second region of interest may be set in the image image acquired from one camera module 121. Of course, depending on the setting, the first skin region of interest and the second region of interest may be set from each of the camera modules 121a and 121b.

In addition, in the case of vascular elasticity and vascular age, a skin region of interest may be set around an arterial section. This is to more accurately detect the vascular elasticity. For example, a skin region of interest may be set around a facial artery section, a wrist artery section, and the like.

The control module 110 of the skin and health diagnosis apparatus 100 may extract a bio-signal from the first skin region of interest and the second skin region of interest (S720), and may calculate pulse waves, respectively. The calculated pulse wave may be delayed and generated in the same blood vessel with a certain time difference.

Accordingly, the control module 110 may calculate a time difference between two pulse waves in order to estimate PWV (Pulse Wave Velocity) representing vascular elasticity. In order to calculate the time difference between the two pulse waves, in the present invention, the time difference may be calculated through a delay (Lag) having a maximum value in the CCF result (S730) of the two pulse waves using a cross correlation function (CCF). Equation 8 is an equation for performing CCF of two pulse waves.

<Equation 8>

The CCF generated as a result of Equation 8 indicates the correlation between the two pulse waves p1 and p2, and the delay time of the pulse wave may be calculated through a sample value from the origin to the position having the largest value.

PWV estimation may be calculated through the delay time of the pulse wave and the distance between the two measurement locations (S760). In order to calculate the actual distance of the skin region of interest displayed in the image image of the skin (ex. face skin, hands and feet skin) as the measurement location, the present invention applies a distance estimation technique between the camera and the skin and the actual distance estimation method of the image coordinate system (S740, S750) You can. The control module 110 estimates and estimates the distance to the camera based on the skin area (taken) from the image image of the skin area taken to derive the actual distance between the first skin area of interest and the second area of interest. The distance to the old camera can be used.

In order to derive an actual distance between the camera lens and the skin area, the control module 110 may check the correlation between the skin area width on the image coordinate system according to the skin area detection and the actual distance between the camera and the skin area.

8 is a photograph of a face area through the camera module 121 according to an embodiment of the present invention, and shows a distribution diagram of an actual distance to the camera according to the detected face width in an image coordinate system. In the graph shown in FIG. 8, the horizontal direction may represent the face width and the vertical direction may represent the distance to the camera.

For reference, in the case of FIG. 8, since it is assumed that a facial area is photographed, a skin and health diagnosis apparatus 100 having a different shape from that of the skin and health diagnosis apparatus 100 of FIG. 1 will be used.

It can be seen from the graph shown in FIG. 8 that as the face width increases, the actual distance to the camera decreases non-linearly. Accordingly, the control module 110 may estimate the actual distance to the camera from the face width by applying the second-order regression curve by reflecting the nonlinear characteristics. For example, the regression equation calculated in the 640 X 480 photographing environment is shown in Equation 9.

<Equation 9>

In Equation 9, distance is the actual distance (cm) between the estimated camera lens and the face, and W may represent the width of the detected face in the image coordinate system. The control module 110 may convert the distance between the two ROIs into an actual distance measure in units of pixels using the estimated distance to the camera.

In addition, the control module 110 may track the 10 cm marker expressed on the image coordinate system according to the distance from the camera, and calculate the cm ratio per pixel according to the distance between the camera and the object using Equation 10. Equation 10 may represent the actual distance conversion equation per pixel. 9 is a graph showing the regression curve calculated in Equation 10. In the graph illustrated in FIG. 9, a horizontal direction may indicate a distance to a camera, and a vertical direction may indicate a 1 pixel distance.

<Equation 10>

The actual distance may be calculated in cm by multiplying the W calculated in Equation 10 by a pixel value between two regions of interest detected in a photographed skin region (eg, a face region).

In addition, the control module 110 of the skin and health diagnosis apparatus 100 calculates the calculated actual distance D between the two ROIs in cm units and the delay time PPT of the two pulse waves calculated through the CCF result in order to estimate the PWV. Substitute in Equation 11 to calculate PWV (S760).

<Equation 11>

The control module 110 of the skin and health diagnosis apparatus 100 may estimate the blood vessel age based on the calculated PWV and the diagnosed content (S770).

10 is a diagram illustrating providing customized content using related information stored in a database according to an embodiment of the present invention.

Referring to FIG. 10, the database 140 of the skin and health diagnosis apparatus 100 includes body information for each user, health indicator measurement result information, content information, and the like. The above information may be continuously updated as the process of the present invention is performed. For example, after acquiring the user's body information and health indicator measurement results, the skin and health diagnosis apparatus 100 may continuously update the data in the database 140.

In addition, body information for each user, health indicator measurement result information, and content information stored in the database 140 will be stored in a state corresponding to each other. For example, content recommendation information may be stored to correspond to each health index measurement result, or content recommendation information may be stored to correspond to body information. In addition, health indicator measurement results or customized content information corresponding to each user identified by each identification code may be stored.

For reference, as described above, the database 140 may not be included in the skin and health diagnosis apparatus 100 unlike FIG. 2. That is, the database 140 may correspond to an external database such as an external server or cloud. In this case, the skin and health diagnosis apparatus 100 may transmit and receive necessary information with an external database through the communication module 130.

The control module 110 of the skin and health diagnosis apparatus 100 combines or combines user-specific body information, health indicator measurement result information, and content information stored in the database 140 to log a user-specific health log according to the user's health indicators. (Health-log) can be created and the generated information can be schematically provided to users. The health log may correspond to recording and managing a user's health index.

In addition, the control module 110 of the skin and health diagnosis apparatus 100 may generate and output a notification message for motivating a user based on a health log, or may provide customized content. That is, the control module 110 may calculate the degree of state change based on the user's health log and recommend personalized content according to the state.

For example, the recommended personalized content may include personalized music, pulse-matched and normal-pulse guidance sound sources, healing frequency applied sound sources, personalized exercise guides, and personalized intake guides. That is, as the personalized content, representative cardiovascular disease prevention and improvement content such as music, diet, and exercise can be applied. The present invention is not limited thereto, and additional content other than the described content is possible.

11 is a view showing an intake guide according to food and health according to an embodiment of the present invention.

The database 140 may include personalized intake guide information as content information.

Specifically, the skin and health diagnosis apparatus 100 may acquire food pictures (FIG. 11(a)) through an external device 200 (ex user terminal, external server) or its own camera module 121. In addition, food names and the like may be input from the user.

The control module 110 can classify food based on a picture or food name (Fig. 11(b)), and estimate the calories of food by inputting the classification result into a calorie model for each food (Fig. 11(c)) can do. The control module 110 may continuously update such information in the database 140.

In the case of cardiovascular disease, there may be a variety of recommended foods and foods to avoid. Therefore, the control module 110 of the skin and health diagnosis apparatus 100 can prevent cardiovascular health by providing diet information that the user is not familiar with, such as recommended daily intake, recommended food, and food to avoid (Fig. 11(d)). I can. We will be able to provide customized intake guides for cardiovascular health as well as other diseases.

In addition, the control module 110 of the skin and health diagnosis apparatus 100 may present a customized exercise according to the user's body information and health indicators, and provide the exercise intensity under setting. In particular, by providing a location-based service in conjunction with a GPS sensor of a user's personal terminal connected to the skin and health diagnosis apparatus 100, it is possible to provide a personalized exercise guide by controlling the running speed of the current user.

In addition, the control module 110 of the skin and health diagnosis apparatus 100 recommends a sound source that is helpful for improving and improving cardiovascular function, and a sound source reflecting the real-time pulse state of the user and a sound source for inducing a normal pulse rate according to the result of health indicator diagnosis. It can be printed to induce cardiovascular prevention and improvement.

Apart from the pulse tuning sound source, a sound source applied with a healing frequency that induces a positive cardiovascular effect through music therapy by emphasizing the healing frequency in the sound source input by the user may be included. For example, representative healing frequencies are 528Hz, 432Hz, and 936Hz, and the effects of each frequency were tested through various studies.

In addition, in the present invention, a sound source reflecting the healing frequency may be generated through the steps of emphasizing the healing frequency in the frequency domain of the sound source input by the user using the healing frequency and converting the sound source in the emphasized frequency domain into the time domain. .

Through the pulse-tuning sound source generation technique and sequential processing described as another embodiment, the sound source in which the current pulse wave or the normal pulse wave of the user is reflected may be processed as a sound source to which a technique emphasizing the healing frequency is applied.

The skin and health diagnosis apparatus 100 may store performance details according to content recommendations for preventing and improving typical cardiovascular diseases, such as music, diet, and exercise, in the DB. The present invention relates to a health log for preventing and improving cardiovascular disease by integrating the above-described diagnostic method and content recommendation elements, and another embodiment may exist by combining additional elements other than the above elements. have.

Meanwhile, the control module 110 of the skin and health diagnosis apparatus 100 prints the measured user's health indicators through the output module 160 or transmits the measured user's health indicators to the external device 200 through the communication module 130. I can.

As an experiment for measuring vascular elasticity using skin images, the results of the PWV measurement experiment in an experimental group consisting of 20 men and women aged 20-30 years are as follows.

FIG. 12 shows the distribution of PWV of 100 data of 20 adult men and women, FIG. 13 shows the distribution of baPWV for men and women in Korea according to age, and FIG. 14 shows the normal range of baPWV.

12 to 14, as an experiment of measuring vascular elasticity using a skin image according to an embodiment of the present invention, a PWV measurement experiment was performed in an experimental group consisting of 20 men and women aged 20 to 30, and the frequency of the results is shown in FIG. It is the same as the histogram shown in 12.

In the present invention, in order to amplify to the same range as the value of baPWV, which is a commonly used pulse wave transmission rate measurement method to measure the elasticity of blood vessels, the baPWV value according to the age of Korean adults announced by the Exercise Prescription Center in Yongin is used. I did.

In the scatter plot shown in FIG. 13, an amplification constant α that matches the suggested age-dependent value and the PWV average value calculated in the present invention was calculated and applied to the calculated PWV.

For diagnosis of PWV value, normal abnormal diagnosis according to age was performed by applying the baPWV normal range threshold value from the PWV value multiplied by the amplification constant α. 14 shows the average line and the normal upper and lower limits according to age.

As shown in FIG. 14, PWV diagnosis according to the user's age was performed by applying a range of the graph. In addition, the PWV value calculated from the skin image was input and the age axis value was output to estimate the age of blood vessels.

The embodiments according to the present invention described above may be implemented in the form of program instructions that can be executed through various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded in the computer-readable recording medium may be specially designed and configured for the present invention, or may be known and usable to those skilled in the computer software field. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magnetic-optical media such as floptical disks. media), and a hardware device specially configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of the program instructions include not only machine language codes such as those produced by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware device may be configured to operate as one or more software modules to perform processing according to the present invention, and vice versa.

In the above, the present invention has been described by specific matters such as specific elements and limited embodiments and drawings, but this is provided only to help a more general understanding of the present invention, and the present invention is not limited to the above embodiments. , Anyone with ordinary knowledge in the technical field to which the present invention pertains can make various modifications and variations from these descriptions.

Therefore, the spirit of the present invention is limited to the above-described embodiments and should not be defined, and all modifications that are equally or equivalent to the claims as well as the claims to be described later fall within the scope of the spirit of the present invention. I would say.

100: skin and health diagnosis device
110: control module
120: skin measurement module
121: camera module
122: lighting module
130: communication module
140: database
150: input module
160: output module
200: external device (server)

Claims (10)

In the method of measuring the user's health index,
(a) photographing a part of the user's skin present on the opening of the skin and health diagnosis apparatus, and detecting a skin region of interest or coordinates of the part of the skin from the acquired image image;
(b) The skin and health diagnosis apparatus calculates an average of color data in the skin region of interest, corrects the coordinates of the detected part of the skin through a moving average algorithm, and updates the average of the color data based on this. Extracting an average value of color data; And
(c) the skin and health diagnosis apparatus extracting a bio-signal based on the average value of the color data, and measuring a health indicator of the user by using the bio-signal.
How to include. The method of claim 1,
In a state in which body information including height and weight of the user is obtained from an input device or a database,
The skin and health diagnosis apparatus, using the body information and the bio-signal in the step (c), pulse rate per minute, respiratory rate per minute, pulse intensity, blood pressure, bradycardia/tachycardia/irregular arrhythmia, blood volume, blood viscosity, odd pulse , Vascular elasticity, blood vessel age by diagnosing at least one or more of the user's health indicators. The method of claim 2,
i) The pulse rate per minute is calculated using the maximum frequency component in the pulse-related frequency band of the biological signal, ii) The breath rate per minute is calculated using the maximum frequency component in the breath-related frequency band of the biological signal, iii ) The pulse intensity is calculated using the amplitude of a pulse wave subjected to BPF (Band Pass Filtering) as a pulse-related frequency band of the biological signal, and iv) the bradycardia/tachycardia arrhythmia is calculated based on the calculated pulse rate, The irregular arrhythmia is calculated using the baseline ventricular pulse (PVC) of the pulse wave RR interval and the change in ACF slope, v) the blood volume is calculated using the slope between peaks in the blood volume related frequency band of the biological signal, vi) The blood viscosity is calculated using an amplitude, a differential value, and a spring constant K at the peak point of the pulse wave, and vii) the odd pulse is calculated using a difference in amplitude of the pulse wave during inhalation and exhalation. The method of claim 2,
In a state in which the first skin region of interest and the second skin region of interest are included in the skin region of interest,
i) The blood pressure is calculated using the Pulse Transit Time (PTT) of the pulse wave calculated from the first skin region of interest and the second skin region of interest, ii) the first skin region of interest and the second skin region of interest When it is assumed that the pulse wave velocity is obtained using the time difference of the pulse wave calculated in, the vascular elasticity and the vessel age are calculated using the pulse wave transmission velocity. The method of claim 1,
Content recommendation information corresponding to each health indicator is stored in the database,
The skin and health diagnosis device configures a health log for recording and managing the user's health indicators, generates and outputs a notification message for the user's motivation based on the health log, and provides customized content Method characterized in that to. The method of claim 5,
The above customized content
A method comprising at least one of personalized music, pulse-matched and normal-pulse guidance sound sources, healing frequency applied sound sources, personalized exercise guides, and personalized intake guides. The method of claim 1,
Wherein the skin and health diagnosis device continuously updates the user's body information and health indicators in a database. The method of claim 1,
When it is assumed that at least one opening through which the user's body part can be inserted is present in the skin and the health examination apparatus,
The skin and health diagnosis apparatus, characterized in that for capturing the image image using a camera module and a lighting module located in the opening. The method of claim 1,
The skin and health diagnosis apparatus, wherein the measured health indicators of the user are printed through an output module or transmitted to the user's terminal. In the skin and health diagnosis apparatus for measuring a user's health index,
A communication module for transmitting and receiving information to and from an external device;
An opening through which a part of the user's body can be inserted;
A skin measurement module including a camera module and a lighting module positioned in the opening;
Using the skin measurement module, a part of the user's skin is photographed, a skin region of interest or coordinates of the skin part are detected from the photographed image image, and an average of color data in the skin region of interest is calculated, and the detection After correcting the coordinates of a portion of the skin through a moving average algorithm, the average value of color data is extracted based on the average of the color data, the average value of the color data is filtered to extract a biosignal, and the biosignal is used. Control module to measure the health indicators of the user
Skin and health diagnosis device comprising a.

Images