KR101854246B1 - Skin temperature estimation method using image and skin temperature estimation apparatus using image - Google Patents

Abstract

A method for estimating skin temperature using an image includes the steps of: detecting a skin region in an image of a computer device; extracting color data of the skin region of the computer device; Estimating the skin temperature by applying the color data to the regression analysis function, and estimating the skin temperature by applying the skin temperature estimated by applying the color data to the second regression analysis function.

Description

TECHNICAL FIELD [0001] The present invention relates to a skin temperature estimating method using an image, and a skin temperature detecting apparatus using an image. [0002]

The technique described below relates to a technique for measuring skin temperature using an image.

Techniques have been developed to measure object temperature using images in various fields. The temperature is measured using an infrared camera in various fields such as medical, automobile, construction, and the like. An infrared camera is a device that measures temperature using electromagnetic waves emitted by an object having heat in an electromagnetic wave region that can not be seen by human eyes.

Korean Patent Publication No. 10-2015-0088135

The technique described below is intended to measure the skin temperature using only the image including the skin region.

A method for estimating skin temperature using an image includes the steps of: detecting a skin region in an image of a computer device; extracting color data of the skin region of the computer device; And applying the color data to the analysis function to estimate skin temperature.

A skin temperature detecting apparatus using an image includes a camera for acquiring an image including a skin and color data for a skin region included in the image and applying the color data to a regression analysis function for calculating skin temperature from color data And estimating the skin temperature.

Since the technology described below uses only images, temperature can be conveniently measured in a non-contact manner without expensive equipment such as a thermal imaging camera.

1 is an example of a flowchart of a skin temperature estimation method using an image.
FIG. 2 shows an example of a process of preparing a first regression analysis function using a general image and a thermal image.
FIG. 3 is another example of a process of preparing a second regression analysis function using a general image and a thermal image.
4 is an example of a flowchart for an improved skin temperature estimation method using an image.
5 is an example of a skin temperature estimation process using an image.
6 is an example of a skin temperature detecting apparatus using an image.

The following description is intended to illustrate and describe specific embodiments in the drawings, since various changes may be made and the embodiments may have various embodiments. However, it should be understood that the following description does not limit the specific embodiments, but includes all changes, equivalents, and alternatives falling within the spirit and scope of the following description.

The terms first, second, A, B, etc., may be used to describe various components, but the components are not limited by the terms, but may be used to distinguish one component from another . For example, without departing from the scope of the following description, the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component. And / or < / RTI > includes any combination of a plurality of related listed items or any of a plurality of related listed items.

As used herein, the singular " include "should be understood to include a plurality of representations unless the context clearly dictates otherwise, and the terms" comprises & , Parts or combinations thereof, and does not preclude the presence or addition of one or more other features, integers, steps, components, components, or combinations thereof.

Before describing the drawings in detail, it is to be clarified that the division of constituent parts in this specification is merely a division by main functions of each constituent part. That is, two or more constituent parts to be described below may be combined into one constituent part, or one constituent part may be divided into two or more functions according to functions that are more subdivided. In addition, each of the constituent units described below may additionally perform some or all of the functions of other constituent units in addition to the main functions of the constituent units themselves, and that some of the main functions, And may be carried out in a dedicated manner.

Also, in performing a method or an operation method, each of the processes constituting the method may take place differently from the stated order unless clearly specified in the context. That is, each process may occur in the same order as described, may be performed substantially concurrently, or may be performed in the opposite order.

The technique described below is a technique for estimating skin temperature based on an image including a user's skin. The technique described below can estimate skin temperature using only an image in a device such as a smart device equipped with a camera. The techniques described below can be performed in an apparatus such as a PC, a smart phone, a server, and the like. For convenience of explanation, it is assumed that the computer device estimates the skin temperature.

FIG. 1 is an example of a flowchart of a skin temperature estimation method 50 using an image. First, the camera acquires the source image (51). The source image is a skin-like image including a face. The computer device detects the skin area in the source image captured by the camera (52). The algorithm by which the computer device detects the skin area can utilize various techniques known in the art.

The computer device extracts color data for the skin region (53). The computer device can extract color data for the entire extracted skin area. The computer device may also extract color data for a specific skin region from the acquired image.

Various values can be used for the color data. For example, (1) the color data may use at least one of an R value, a G value, and a B value based on the RGB color scheme. The color data may use an average value for at least one of R value, G value, and B value. (2) The computer device can convert the RGB color scheme to a different color scheme. For example, computer politics can convert the RGB color scheme to various color schemes such as YUV, HSV, YCbCr, YCgCo, and the like. In this case, the color data may use one of color difference components that are less influenced by the surrounding environment (illumination, etc.). For example, in the case of YCbCr, at least one of Cb and Cr can be used. In the case of YCgCo, at least one of Cg and Co may be used. Further, one of the two chrominance components, which is more robust to the variation of the illuminance, can be used. For example, in the case of YCgCo, only Cg can be used. In this case, the computer device can extract the average value of the Cg color data of the skin region as color data. (3) Further, the color data may be a value obtained by combining weights applied to at least one color component in various color systems such as RGB, YUV, HSV, YCbCr, YCgCo, and the like. In the case of combining color components, the color data may be a value obtained by adding different weighted values depending on the color system and the type of color component.

The computer device may select a regression analysis function corresponding to the extracted color data (54). When various kinds of color data are used, different regression analysis functions can be used for each color data. In addition, even when the types of skin regions detected by the computer apparatus are different, a different regression analysis function can be used for each skin region. For example, different regression analysis functions can be used depending on positions such as forehead, ball, nose, under eye, and jaw in the face region. Regression analysis is an analytical method that measures fit between two variables for continuous variables. Regression analysis calculates a function (formula) that calculates a specific value using predetermined sample data in advance. The regression function is expressed as an equation representing a regression line or a regression curve.

The computer device applies the color data to the regression analysis function to ultimately estimate the skin temperature (55).

First, the process of preparing the first regression analysis function will be described. FIG. 2 shows an example of a process of preparing a first regression analysis function using a general image and a thermal image. The computer device can generate a regression analysis function for skin temperature estimation by comparing the color data average value DB extracted from the designated color system with the skin temperature DB of the thermal imaging camera.

The computer device acquires the source image captured by the camera (step a). The computer device detects the face region from the source image and detects the skin region to be analyzed in the face region (step b). The skin region may be at least one or all of the various regions in the face region. Then, the computer device stores the color data for the corresponding skin area in the color data DB (storage device) (step c).

FIG. 2 shows an example of extracting color data for the ball region A. FIG. The computer device continuously stores color data (e.g., average value of Cg data) of the ball area in the DB for a predetermined time.

At the same time, a face image of the same subject is photographed using a thermal imaging camera (step d). That is, a face image is photographed with a thermal camera simultaneously with a general camera. The computer device stores the result of imaging with the thermal imaging camera in the thermal image DB (storage device). The computer device extracts skin temperature data for position (B) that is identical (matched) to skin region (A). When the thermal image temperature DB stores the skin temperature data, the computer device extracts the B region matching the A region in the thermal image temperature DB and determines the temperature for the region. Or the thermal image temperature DB may store temperature data for the B region.

Color data for the skin area and temperature data for the skin area are now provided. The computer device calculates skin temperature regression line and / or regression curve expressions of all or a designated area through a regression analysis using color data and thermal image temperature data (step e). The computer device provides a regression analysis function and stores the function in a regression analysis DB. The regression analysis function prepared in this process is called the first regression analysis function.

In FIG. 2, the color data DB, the thermal image temperature DB, and the regression analysis DB are shown as individual components, but they may physically be the same storage device.

Assume that the forehead, the ball, the nose, the under eye, and the jaw are used in the face region and the skin temperature is estimated using the regression line and the regression curve, respectively.

First, the regression line is a straight line representing a set of points on the scatter diagram, and represents the relationship between the two variables.

The computer device can derive the regression linear equation using the thermal image temperature DB and the color data DB for the same area. The regression linear equation is shown in Equation 1 below.

Where y is the skin temperature and x is the selected color data. a and b are constants. The result obtained by applying the actual data may change the values of the constants a and b according to the color data to be used. Table 1 below shows the linear regression and linear regression for each region of the face region to which the actual Cg color data is applied. The following table shows an example of the average value of Cg color data.

The whole is a case in which the sum of the color data of the forehead, the ball, the nose, the under eye, and the jaw area is used. And the remainder is the case of using color data for each area.

The regression curve is a curve obtained by representing a set of points on a correlation chart as a curve rather than a straight line. Regression curves show the relationship between two variables. The computer device can derive a regression curve expression using the thermal image temperature DB and the color data DB. The regression curve equation is shown in Equation 2 below.

Where y is the skin temperature and x is the selected color data. a, b, and c are constants. The result obtained by applying the actual data may be changed in accordance with the color data to be used, the values of the constants a, b, and c. Table 2 shows the results of each region and the entire regression curve of the face region to which the actual Cg color data is applied.

The whole is a case in which the sum of the color data of the forehead, the ball, the nose, the under eye, and the jaw area is used. And the remainder is the case of using color data for each area.

FIG. 3 is another example of a process of preparing a second regression analysis function using a general image and a thermal image. In FIG. 3, the regression analysis DB is a state in which the first regression analysis function provided in FIG. 2 is stored.

The computer device acquires the source image captured by the camera (step a). The computer device detects the face region from the source image and detects the skin region to be analyzed in the face region (step b). At least one of the whole region or various regions in the face region may be used as the skin region. In FIG. 3, it is assumed that the area indicated by A is used. Then, the computer device extracts color data (for example, average value of Cg) for the corresponding skin area (step c). The computer device estimates the skin temperature by applying the color data extracted from the current image to the first regression analysis function of the regression analysis DB (step d).

On the other hand, the computer device acquires the thermal image for the same area (area indicated by B in Fig. 3) from which the color data is extracted at the same time when the color data is extracted (step e).

The computer device may prepare another regression analysis function using the skin temperature estimated using the first regression analysis function and the skin temperature data stored in the thermal image database (step f). The regression line and / or regression curve calculated in this process is called a second regression analysis function. The computer device also stores a second regression analysis function newly provided in the regression analysis DB. The computer device can estimate the skin temperature finally improved using the second regression analysis function.

The computer device can estimate the skin temperature by using the first regression analysis function and the skin temperature estimated by applying the second regression analysis function as described above.

4 is an example of a flowchart for an improved skin temperature estimation method 80 using images. First, the camera acquires the source image (81). The source image is a skin-like image including a face. The computer device detects the skin region from the source image captured by the camera (82).

The computer device extracts color data for the skin region (83). The computer device can extract color data for the entire extracted skin area. The computer device may also extract color data for a specific skin region from the acquired image.

The computer device may select a first regression analysis function corresponding to the extracted color data (84). The computer device may apply the color data to a first regression analysis function to estimate a baseline skin temperature (85). Further, the computer device may apply the estimated skin temperature to the second regression analysis function described above to estimate an improved skin temperature (86).

5 is an example of a skin temperature estimation process using an image. For convenience of explanation, it is assumed that the skin region is converted into the YCgCo color system and the skin temperature is measured based on the Cg value. It is also assumed that the source image is an image including a face, and a face region including a face is detected as a skin region.

The computer device acquires an image including a face (a process). The computer device detects a face area displayed in a square in the source image (step b).

The computer device detects the skin region in the face region and changes the RGB color scheme of the skin region to the YCgCo system. The computer device calculates an average value of the Cg color data for the skin area (step c).

In the description of FIG. 5, it is explained that RGB is changed to YCgCo in the process c. In fact, changing RGB to a different color scheme (YCgCo) may be done before. After the computer device acquires the source image, the source image may be changed to a different color scheme, or the face region may be changed to another color scheme after the face region is detected. The formula for changing RGB to the YCgCo color scheme is shown in Equation 3 below.

The computer device loads a regression analysis function previously prepared in the regression analysis DB and estimates the improved skin temperature by applying an average value (color data) of the Cg color data to the skin region to the regression analysis function (step d). If there is a first regression analysis function and a second regression analysis function in the regression analysis DB in this process, (1) the computer device can estimate the skin temperature using the first regression analysis function. Or (2) the computer device can estimate the skin temperature using the first regression analysis function, and estimate the improved skin temperature by applying the estimated skin temperature to the second regression analysis function.

6 is an example of a skin temperature detecting apparatus using an image. Fig. 6 is an example of a detection device for detecting skin temperature using the skin temperature estimation method described above.

6 (a) is an example of measuring skin temperature using a user terminal 100 such as a smart phone. The user takes a face with a camera built in the user terminal 100. The user terminal 100 detects a face region in the source image and extracts color data of a predetermined skin region set in advance. The user terminal 100 measures skin temperature by applying color data to a regression analysis function previously provided.

6A, the user terminal 100 includes a camera 110, a storage device 120, a computing device 130, and an output device 140. In FIG. The camera 110 acquires the source image. The arithmetic unit 130 extracts the color data of the skin region included in the source image and applies the color data to the regression analysis function for calculating the skin temperature from the color data to estimate the skin temperature. The storage device 120 may temporarily store the source image. The storage device 120 may also store a regression analysis function used for skin temperature estimation. The output device 140 can output the measured skin temperature.

6 (b) is an example of measuring skin temperature using a device such as a PC. The user takes a face with the camera 210 connected to the computer 220. The computer 220 detects the face region from the source image and extracts color data of a predetermined skin region set in advance. The computer 220 measures skin temperature by applying color data to a regression analysis function previously provided.

6C is an example in which the server 320 at a remote location measures the skin temperature by using the image acquired by the user terminal 310. FIG. The user takes a face with a camera built in the user terminal 300. The user terminal 300 transmits the photographed source image to the server 320 through the network. In this case, the user terminal 300 includes a communication module for data transmission. The server 320 detects the face region from the source image and extracts the color data of a predetermined skin region set in advance. The server 320 measures skin temperature by applying color data to a regression analysis function previously provided. The server 320 may transmit the estimated skin temperature to the user terminal 310. [

In some cases, the user terminal 310 may detect the skin region in the source image and deliver it to the server 320 for the distributed operation. In this case, the server 320 extracts the color data for the skin region, and applies the color data to the regression analysis function to measure the skin temperature. As another example, the user terminal 310 can detect a skin region in a source region and extract color data on a skin region. The user terminal 310 may transmit only the color data of the skin region to the server 320. [ In this case, the server 320 measures the skin temperature by applying the color data received from the regression analysis function.

The present embodiment and drawings attached hereto are only a part of the technical idea included in the above-described technology, and it is easy for a person skilled in the art to easily understand the technical idea included in the description of the above- It will be appreciated that variations that may be deduced and specific embodiments are included within the scope of the foregoing description.

100: user terminal
110: camera
120: Storage device
130:
140: Output device
210: camera
220: Computer
310: User terminal
320: Server

Claims (15)

The computer device detecting a skin region in the image;
Extracting color data of the skin region from the computer device; And
And estimating skin temperature by applying the color data to a regression analysis function that defines a relationship between the color data and the skin temperature,
Wherein the regression analysis function is performed using an image set in advance using the skin temperature measured by the thermal imaging camera and the color data for the same position as the skin region with respect to the skin region. The method according to claim 1,
Wherein the computer device converts an RGB color of the skin region into a different color system and extracts an average value of at least one of chrominance components of the different color system as the color data. The method according to claim 1,
Wherein the computer device changes an RGB color scheme of the skin region into a YCgCo color scheme and extracts an average value of Cg color data as the color data. The method according to claim 1,
Wherein the skin region is detected in a face region included in the image. The method according to claim 1,
Wherein the skin region is a plurality of regions having different positions in the image. The method according to claim 1,
Wherein the computer device extracts the color data based on at least one color scheme of RGB, HSV, YUV, YCgCo, and YCbCr for the skin region. The method according to claim 1,
Wherein the computer device extracts at least one color component from each of at least two color schemes of RGB, HSV, YUV, YCgCo, and YCbCr for the skin region, and adds the weighted values to the at least one color component A method for estimating skin temperature using images. delete The method according to claim 1,
Wherein the regression analysis function is a regression line or a regression curve. The method according to claim 1,
The regression analysis function may include a first regression analysis function provided in advance using the skin temperature measured by the thermal imaging camera and the color data for the same position as the skin region with respect to the skin region,
At least one of a second regression analysis function provided beforehand using the skin temperature estimated by applying the color data on the skin region to the first regression analysis function and the skin temperature measured with the thermal imaging camera on the skin region A skin temperature estimation method using an image including the skin temperature. A camera for acquiring images containing skin; And
An arithmetic unit for extracting color data on a skin region included in the image and applying the color data to a regression analysis function for calculating skin temperature from color data to estimate a skin temperature,
Wherein the regression analysis function uses an image set in advance using the skin temperature measured by the thermal imaging camera and the color data of the same position as the skin region with respect to the skin region. 12. The method of claim 11,
Wherein the computing device changes an RGB color of the skin region to a different color system and extracts an average value of at least one of chrominance components of the different color system as the color data. 12. The method of claim 11,
Wherein the computing device uses the image for extracting the color data based on at least one color scheme of RGB, HSV, YUV, YCgCo, and YCbCr for the skin region. 12. The method of claim 11,
The regression analysis function may include a first regression analysis function provided in advance using the skin temperature measured by the thermal imaging camera and the color data for the same position as the skin region with respect to the skin region,
At least one of a second regression analysis function provided beforehand using the skin temperature estimated by applying the color data on the skin region to the first regression analysis function and the skin temperature measured with the thermal imaging camera on the skin region The skin temperature detecting device using the image. 12. The method of claim 11,
The apparatus according to claim 1, further comprising: a storage device that stores a regression analysis function for a plurality of skin areas, wherein the arithmetic unit calculates skin temperature by using a regression analysis function corresponding to the detected skin area, .

Images