KR20230032101A - Deep learning-based In-person Customized Metabolic Rate Prediction System and Prediction Method - Google Patents

Abstract

The present invention relates to a deep learning-based resident-customized metabolic rate prediction system executed by a computer, for predicting a real-time metabolic rate using only real-time activity images, comprising: a metabolic rate acquisition unit (100) which receives a resident's real-time heart rate measured by a wearable device (10) capable of measuring heart rate and calculates the resident's real-time metabolic rate; an image acquisition unit (200) which receives real-time activity images of the resident acquired through a camera (20); a data pre-processing unit (300) which pre-processes the images of the image acquisition unit (200); a deep learning unit (400) which generates a real-time metabolic rate calculation model of the resident through deep learning using the real-time metabolic rate of the metabolic rate acquisition unit (100) and the pre-processed activity images as input values; and a deep learning prediction unit (500) which predicts the real-time metabolic rate of the resident by inputting the resident's real-time activity images acquired from the image acquisition unit (200) into the real-time metabolic rate calculation model.

Description

Deep learning-based In-person Customized Metabolic Rate Prediction System and Prediction Method}

The present invention relates to a metabolic rate prediction system and prediction room. Specifically, it relates to a deep learning-based occupant-customized metabolic rate prediction system and prediction method.

The metabolic rate refers to the amount of heat generated by the occupant's activities. Metabolism is one of the factors used to evaluate occupant's thermal comfort.

In general, it is difficult to predict the value of metabolic rate due to unexpected activities of occupants, so in the construction field, metabolic rate is assumed to be a fixed value when controlling air conditioners. Therefore, there is a problem in that the metabolic rate according to the actual activity of the occupant is not accurately reflected.

In order to improve this problem, Korea Patent Registration No. 10-2233157 suggested a method for calculating occupant activity using deep learning-based occupant pose classification. In this prior art, there is a difference in that it is an attempt to calculate the amount of activity (metabolism) according to the actual activity of the occupant.

However, in the case of Korean Patent Registration No. 10-2233157, a classification model is used to specifically classify an occupant's activity, and a metabolic value determined during the activity is calculated. Therefore, deep learning here is only used to classify the occupant's image as an indoor activity pose, and after classifying the occupant's pose, a specific activity (metabolic amount) is applied as a fixed value to a specific pose. Therefore, there is still a problem in that fixing the real-time motion change of the occupant to a specific value does not predict the natural change of speech according to the actual occupant's behavior change.

(Document 1) Korea Patent Registration No. 10-2233157 (2021.03.23)

The deep learning-based occupant-customized metabolic rate prediction system and prediction method according to the present invention have the following challenges.

First, real-time metabolic rate is calculated using the real-time heart rate of occupants.

Second, real-time activity images of occupants and real-time metabolism are deep-learned to calculate real-time metabolism according to real-time activity images.

Third, through data pre-processing that deletes the background excluding occupants, the analysis is focused on the activity images of occupants.

Fourth, with the real-time activity change metabolic rate calculation model derived through deep learning learning, real-time metabolic rate is predicted by inputting only real-time activity images without heart rate information.

The problems of the present invention are not limited to those mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the description below.

The present invention is a deep learning-based occupant-customized metabolic rate prediction system performed by a computer, comprising: a metabolic acquisition unit that receives a real-time heart rate of an occupant measured by a wearable device capable of measuring heart rate and calculates a real-time metabolic amount of the occupant; an image acquisition unit that receives real-time activity images of occupants acquired through the camera; a data pre-processing unit that pre-processes the image of the image acquiring unit; a deep learning learning unit generating a real-time metabolic rate calculation model of a occupant through deep learning using the real-time metabolic rate of the metabolic rate acquisition unit and the preprocessed activity image as input values; and a deep learning prediction unit for predicting the real-time metabolic rate of the occupant by inputting the activity image of the occupant acquired in real time by the image acquisition unit to the real-time metabolic calculation model. .

In the present invention, the wearable device includes a watch-type device, a band-type device, a glasses-type device, a necklace-type device, a helmet-type device, and a clothing-type device, and the camera may include a Kinect camera.

In the present invention, the metabolic rate obtaining unit may calculate the real-time metabolic rate of the occupant by using the received real-time heart rate information of the occupant and previously input sex, weight, and age information of the occupant.

In the present invention, the data pre-processing unit may delete a portion of the activity image acquired by the image acquiring unit, except for occupants.

In the present invention, the camera uses a Kinect camera, acquires an RGB image and a depth image indicating perspective through the Kinect camera, detects an occupant from an input image based on the obtained depth image, , By masking to the RGB image, parts other than occupants can be removed.

In the present invention, the real-time metabolic calculation model of the deep learning learning unit

ReLU is adopted as the activation function of Equation 1, batch normalization and SGD optimizer are adopted, and the initial target function of the real-time metabolic rate calculation model may have Equation 2.

In the present invention, the real-time metabolic rate calculation model may have a final target function of Equation 4 by adding the normalization function of Equation 3 below.

The present invention is a deep learning-based occupant-customized metabolic rate prediction method performed by a computer, wherein a metabolic rate acquisition unit receives the real-time heart rate of an occupant measured by a wearable device capable of measuring heart rate, and calculates the real-time metabolic rate of the occupant; S100; Step S200 of receiving real-time activity images of occupants obtained through a camera by an image acquisition unit; S300 step of pre-processing the image of the image acquisition unit by the data pre-processing unit; Step S400 in which a real-time metabolic rate calculation model of an occupant is generated by a deep learning learning unit through deep learning using the real-time metabolic rate of the metabolic acquisition unit and the preprocessed activity image as input values; and a step S500 of predicting the real-time metabolic rate of the occupant by the deep learning prediction unit by inputting the activity image of the occupant acquired in real time by the image acquisition unit to the real-time metabolic rate calculation model.

In step S100 according to the present invention, the metabolic rate obtaining unit may calculate the real-time metabolic rate of the occupant by using the received real-time heart rate information of the occupant and previously input sex, weight, and age information of the occupant.

In step S300 according to the present invention, the data pre-processing unit may delete a part except for occupants from the activity image obtained by the image acquisition unit.

In the present invention, the camera uses a Kinect camera, obtains an RGB image and a depth image displaying a sense of perspective through the Kinect camera, and based on the acquired depth image, occupants are captured in the input image. can be detected and masked on the RGB image to remove parts other than occupants.

The present invention may be combined with hardware and implemented as a computer program stored in a computer-readable recording medium in order to execute the deep learning-based occupant-customized metabolic rate prediction method according to claim 8 by a computer.

The deep learning-based occupant-customized metabolic rate prediction system and prediction method according to the present invention have the following effects.

First, there is an effect of calculating the real-time metabolic rate, not the fixed metabolic rate, using the real-time heart rate of the occupant.

Second, there is an effect of calculating the real-time metabolic amount according to the real-time activity image by deep learning the real-time activity image of the occupant and the real-time metabolic amount.

Third, through data pre-processing that deletes the background except for occupants, there is an effect of focusing on the activity images of occupants and analyzing them.

Fourth, a real-time activity change metabolic rate calculation model derived through deep learning learning has the effect of predicting real-time metabolic rate by inputting only real-time activity images without heart rate information.

The effects of the present invention are not limited to those mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

1 shows an embodiment of a deep learning-based occupant-customized metabolic rate prediction system according to the present invention.
2 is a schematic diagram of a deep learning learning process and a deep learning prediction process.
3 shows a flow chart of a deep learning-based occupant-customized metabolic rate prediction method according to the present invention.
4 shows an embodiment of data pre-processing according to the present invention.
5A to 5C are examples of deleting a background except for a occupant in a data pre-processing unit. FIG. 5A shows an input image, FIG. 5B shows an occupant image, and FIG. 5C shows a background removal image.
6A to 6H show an embodiment of various actual activities of occupants.
7 is an example of an actual metabolic value (Ground truth) and an artificial intelligence predicted value according to various activity changes of occupants.
FIG. 8 shows an example in which the image of FIG. 5 is compared before and after excluding the background.
9 shows an embodiment of a residual block of an artificial neural network.
10 shows one embodiment of the METNet model structure.

Hereinafter, with reference to the accompanying drawings, embodiments of the present invention will be described so that those skilled in the art can easily practice it. As can be easily understood by those skilled in the art to which the present invention pertains, the embodiments described below may be modified in various forms without departing from the concept and scope of the present invention. Where possible, identical or similar parts are indicated using the same reference numerals in the drawings.

The terminology used in this specification is only for referring to specific embodiments and is not intended to limit the present invention. As used herein, the singular forms also include the plural forms unless the phrases clearly indicate the opposite.

As used herein, the meaning of "comprising" specifies particular characteristics, regions, integers, steps, operations, elements, and/or components, and other specific characteristics, regions, integers, steps, operations, elements, components, and/or components. It does not exclude the presence or addition of groups.

All terms including technical terms and scientific terms used in this specification have the same meaning as commonly understood by a person of ordinary skill in the art to which the present invention belongs. The terms defined in the dictionary are additionally interpreted as having a meaning consistent with the related technical literature and the currently disclosed content, and are not interpreted in an ideal or very formal meaning unless defined.

Expressions related to directions used in this specification, for example, expressions of front/back/left/right, top/bottom, and longitudinal/lateral directions may be interpreted with reference to directions disclosed in the drawings.

In the case of the present invention, as a customized model that selects the metabolic rate value using the heart rate of the occupant, a value closer to the actual occupant's metabolic rate is presented.

Occupants perform various activities indoors, and learning is performed using artificial intelligence deep learning technology, and based on the learned contents, metabolic rate values are predicted for each action, and finally, customized metabolic rate prediction is possible in real time.

In the present invention, we propose a metabolic detection technology tailored to occupants by utilizing deep learning computer vision technology and a wearable device capable of measuring heart rate.

If the present invention is used, it is possible to derive a metabolic value customized for occupants rather than a generalized tabular average metabolic value, which can help create a better thermal environment.

The Kinect camera usable in the present invention can secure high precision without being involved in changes in the occupant's space (surrounding environment) by incorporating a technology for excluding the background except for the occupants.

2 is a schematic diagram of a deep learning learning process and a deep learning prediction process. 3 shows a flow chart of a deep learning-based occupant-customized metabolic rate prediction method according to the present invention.

The whole process is as follows. The metabolic rate (MET) is calculated in real time through a wearable device, and the corresponding image is simultaneously taken through a camera. It learns (Deep Learning) the metabolic value according to the captured image and the corresponding image. After that, customized metabolic rate can be calculated in real time according to the occupant's activity without a wearable device.

Hereinafter, the present invention will be described with reference to the drawings. For reference, the drawings may be partially exaggerated in order to explain the features of the present invention. In this case, it is preferable to interpret in light of the whole purpose of this specification.

1 shows an embodiment of a deep learning-based occupant-customized metabolic rate prediction system according to the present invention.

As shown in FIG. 1, the deep learning-based occupant-customized metabolic rate prediction system according to the present invention includes a metabolic amount acquisition unit 100, an image acquisition unit 200, a data pre-processing unit 300, a deep learning learning unit 400, and It includes a deep learning prediction unit 500.

The present invention is a deep learning-based occupant customized metabolic rate prediction system performed by a computer. The metabolic rate acquisition unit (100 ); an image acquisition unit 200 that receives real-time activity images of occupants acquired through the camera 20; a data pre-processing unit 300 that pre-processes the image of the image acquiring unit 200; a deep learning learning unit 400 for generating a real-time metabolic rate calculation model of an occupant through deep learning using the real-time metabolic rate of the metabolic acquisition unit 100 and the preprocessed activity image as input values; and a deep learning prediction unit 500 that predicts the real-time metabolic rate of the occupant by inputting the activity image of the occupant acquired in real time by the image acquisition unit 200 to the real-time metabolic calculation model.

The wearable device 10 according to the present invention includes a watch-type device, a band-type device, a glasses-type device, a necklace-type device, a helmet-type device, and a clothing-type device.

The metabolic rate acquisition unit 100 according to the present invention receives the real-time heart rate of an occupant measured by the wearable device 10 capable of measuring the heart rate, and calculates the real-time metabolic rate of the occupant.

The metabolic rate acquisition unit 100 according to the present invention may calculate the real-time metabolic rate of the occupant by using the transmitted real-time heart rate information of the occupant and previously input sex, weight, and age information of the occupant.

The 'metabolic amount' according to the present invention means the total metabolic amount including the basal metabolic rate and the active metabolic rate.

In the present invention, metabolic conversion using heart rate follows the metabolic evaluation method Level 3 standard of ISO 8996, an international standard organization.

The metabolic rate evaluation formula is as follows.

은 대사량 값이다.

는 휴식시 대사량값이고, 단위는 W/

이다.

는 심박수이다.

는 휴식시 열적 중립상태의 심박수이다. RM은 심박수의 증가를 나타낸 수이다. here,

is the metabolic value.

is the resting metabolic rate, and the unit is W/

am.

is the heart rate.

is the thermally neutral heart rate at rest. RM is a number representing an increase in heart rate.

The RM calculation formula is as follows.

Here, MWC refers to the maximum working capacity. At this time, gender, age, and weight are calculated. A is the age P is the weight.

When the metabolic value is calculated in W/m2, according to ANSI/ASHRAE Standard 55, the metabolic value is proportionally converted to 58.2 W/m2 for 1 MET.

The image acquisition unit 200 according to the present invention may receive real-time activity images of occupants acquired through the camera 20 .

The camera 20 according to the present invention includes a Kinect camera. The Kinect sensor is a low-cost depth camera that can provide not only depth information but also RGB images and joint tracking information in real time.

The data pre-processing unit 300 according to the present invention may pre-process the image of the image acquiring unit 200 .

4 shows an embodiment of data pre-processing according to the present invention.

Data preprocessing is to minimize the impact of the previous activity on the MET of the next activity. In the case of the embodiment of FIG. 4 , data for the preceding 1 minute and the following 2 minutes per activity were removed.

5A to 5C are examples of deleting a background except for a occupant in a data pre-processing unit. FIG. 5A shows an input image, FIG. 5B shows an occupant image, and FIG. 5C shows a background removal image.

FIG. 8 shows an example in which the image of FIG. 5 is compared before and after excluding the background.

The data pre-processing unit 300 according to the present invention may delete parts other than occupants from the activity image obtained by the image acquisition unit 200 .

The present invention utilizes a technique for removing a background portion of an image that causes deterioration in action recognition performance by utilizing visual attention. The performance of the action recognition technology can be improved by using the image with the background removed using visual attention for the deep learning model developed in this study.

In the present invention, Microsoft's Kinect camera is used as an embodiment in order to remove a background portion containing unnecessary information for recognizing an occupant's behavior. As shown in FIG. 5, an RGB image and a depth image displaying perspective are obtained through the Kinect camera, and a occupant is detected from the input image based on the acquired depth image, and the part except for the occupant is masked in the RGB image can be removed.

For reference, in this specification and drawings, the term 'background' of an image means a part of the image excluding 'occupant'.

The Kinect camera used in the present invention can secure high precision without being involved in changes in the occupant's space (surrounding environment) by applying a technology that excludes the background except occupants.

As a learning step of the deep learning learning unit 400 according to the present invention, the occupant performs various activities, and the camera acquires this appearance as image data. In addition, real-time metabolic rate (MET) is calculated by calculating the occupant's real-time heart rate data obtained through the wearable device and the occupant's sex, weight, and age. Then, using the image and the amount of metabolism as input values, deep learning technology is used to learn the amount of metabolism in the image.

In the case of the prediction step of the deep learning prediction unit 500 according to the present invention, it is possible to calculate the metabolic rate tailored to the occupant only with real-time images obtained through a camera without a wearable device.

2 is a schematic diagram of a deep learning learning process and a deep learning prediction process.

The MET prediction method of the present invention is shown in FIG. 2. First of all, in the sensing stage, various life activities of occupants are observed using the ㅋKiKinect camera, and at the same time, occupant information is obtained from wearable devices. In AI (artificial intelligence), deep learning is used to learn image information and metabolic information about behavior. Based on AI's accumulated database, without the help of any wearable device, AI enables Metabolic Equivalent Task (MET) prediction with only the Kinect camera, a non-contact sensor.

3 shows a flow chart of a deep learning-based occupant-customized metabolic rate prediction method according to the present invention.

A detailed MET prediction method is shown in FIG. 3 . First, by using the Kinect camera to sense the occupant's behavior, five images per second are obtained. At the same time, heart rate data is collected through a wearable device. The MET value is calculated according to the international metabolic rate standard ISO 8996 Level 3 method using the collected heart rate data and the experimenter's unchanging information such as gender, weight, and age as input data. The calculated MET value and the image file obtained through the Kinect camera are deep-learned in an AI system. Finally, a person's MET can be predicted with only the Kinect camera, a non-contact sensor, without information from a wearable device.

The deep learning learning unit 400 according to the present invention may generate a real-time metabolic rate calculation model of an occupant through deep learning using the real-time metabolic rate of the metabolic rate acquisition unit 100 and the preprocessed activity image as input values.

9 shows an embodiment of a residual block of an artificial neural network. 10 shows one embodiment of the METNet model structure.

To improve the stability of the metabolic calculation model, Rectified Linear Unit (ReLU) was adopted as the activation function, and batch normalization and stochastic gradient descent (SGD) optimizers were adopted.

Since ReLU has non-linearity, it can alleviate the problems of gradient vanishing and gradient explosion. In addition, by using batch normalization, we reduce the covariance shift problem inside artificial neural networks, as demonstrated in batch normalization. Finally, we adopted the SGD optimizer.

This is very effective to train a model stably and quickly. The initial target function of the entire model is L1 Loss (Equation 2) ld , and the SGD optimizer can solve an optimization problem that reduces the target function to a minimum.

Here, f(x) is an activation function and x is an input value.

는 n개의 입력값 x 중에서 i번째 입력값이다.where L1Loss is the initial target function, x is the input value, n is the number of input values (batchsize), y is the label,

is the ith input value among n input values x.

However, the metabolic rate calculation model still has a bias to make predictions that are most similar to the data used for learning. The output model is trained in the direction of minimizing the average absolute error by predicting the median value of the training data by the target function, which may cause a different result than expected. Therefore, the normalization function of Equation 3 is added. This function penalizes models that simultaneously predict similar values during the learning process.

That is, the real-time metabolic rate calculation model according to the present invention may have a final target function of Equation 4 below by adding the normalization function of Equation 3 below.

는 n개의 입력값 x 중에서 i번째 입력값이고,

는 n개의 x평균이다.Here, REG is a regularization function, n is the number of input values (batchsize),

is the ith input value among n input values x,

is the x mean of n.

Therefore, the final target function according to the present invention is shown in Equation 4 below.

Here, O is the final target function, L1Loss is the initial target function, and REG is the regularization function.

In the case of METNet according to the present invention, all CNN layers have ReLU as an activation function. The first CNN layer of each residual block sets the filter movement interval to 2 pixels for downsampling. And apply batch normalization. In the present invention, an SGD optimizer with a learning rate of 5e-5, a momentum of 0.9, and a variable reduction ratio of 5e-4 is used.

Meanwhile, the present invention may be implemented as a method for predicting metabolic rate. The prediction method invention has the same practical technical configuration as the prediction system invention described above, but the category of the invention is different. The overlapping details can be borrowed from the prediction system invention, and hereinafter, the technical configuration will be mainly described.

The present invention is a deep learning-based occupant-customized metabolic rate prediction method performed by a computer, in which the metabolic rate acquisition unit 100 receives the real-time heart rate of the occupant measured by the wearable device 10 capable of measuring the heart rate, and measures the real-time metabolic rate of the occupant S100 step of calculating; Step S200 in which the image acquisition unit 200 receives the real-time activity image of the occupant acquired through the camera 20; A step S300 in which the data pre-processing unit 300 pre-processes the image of the image acquisition unit 200; Step S400 in which the deep learning learning unit 400 generates a real-time metabolic rate calculation model of the occupant through deep learning using the real-time metabolic rate of the metabolic rate acquisition unit 100 and the preprocessed activity image as input values; and a step S500 in which the deep learning prediction unit 500 inputs the activity image of the occupant acquired in real time by the image acquisition unit 200 to the real-time metabolic rate calculation model to predict the real-time metabolic rate of the occupant.

Since steps S100 and S200 are to acquire real-time information, it is preferable to perform them simultaneously.

In step S100, the metabolic rate acquisition unit 100 may calculate the real-time metabolic rate of the occupant by using the transmitted real-time heart rate information of the occupant and previously input sex, weight, and age information of the occupant.

In step 300, the data pre-processing unit 300 may delete a portion of the activity image acquired by the image acquisition unit 200, except for occupants.

The camera uses a Kinect camera, obtains an RGB image and a depth image indicating perspective through the Kinect camera, detects an occupant from the input image based on the obtained depth image, and RGB By masking the image, parts other than occupants can be removed.

The real-time metabolic calculation model of the deep learning learning unit 400 adopts ReLU as an activation function of Equation 1, adopts batch normalization and an SGD optimizer, and the initial target function of the real-time metabolic calculation model is mathematical You can have Equation 2.

The real-time metabolic rate calculation model may have a final target function of Equation 4 below by adding a normalization function of Equation 3 below.

Meanwhile, the present invention may be implemented as a computer program.

Specifically, the present invention may be implemented as a computer program stored in a computer-readable recording medium in order to execute the deep learning-based occupant-customized metabolic rate prediction method according to the present invention by a computer in combination with hardware.

Methods according to embodiments of the present invention may be implemented in the form of programs readable by various computer means and recorded on computer-readable recording media. Here, the recording medium may include program commands, data files, data structures, etc. alone or in combination. Program instructions recorded on the recording medium may be those specially designed and configured for the present invention, or those known and usable to those skilled in computer software. For example, recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CDROMs and DVDs, and magneto-optical media such as floptical disks. optical media), and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of the program command may include a high-level language that can be executed by a computer using an interpreter, as well as a machine language generated by a compiler. These hardware devices may be configured to act as one or more software modules to perform the operations of the present invention, and vice versa.

The embodiments described in this specification and the accompanying drawings merely illustrate some of the technical ideas included in the present invention by way of example. Therefore, since the embodiments disclosed in this specification are intended to explain rather than limit the technical spirit of the present invention, it is obvious that the scope of the technical spirit of the present invention is not limited by these embodiments. All modified examples and specific examples that can be easily inferred by those skilled in the art within the scope of the technical idea included in the specification and drawings of the present invention should be construed as being included in the scope of the present invention.

10: wearable device
20 : camera
100: metabolic rate acquisition unit
200: image acquisition unit
300: data pre-processing unit
400: deep learning learning unit
500: deep learning prediction unit

Claims (14)

As a deep learning-based occupant-customized metabolic rate prediction system performed by a computer,
a metabolic rate acquisition unit receiving a real-time heart rate of an occupant measured by a wearable device capable of measuring heart rate and calculating a real-time metabolic rate of the occupant;
an image acquisition unit that receives real-time activity images of occupants acquired through the camera;
a data pre-processing unit that pre-processes the image of the image acquiring unit;
a deep learning learning unit generating a real-time metabolic rate calculation model of a occupant through deep learning using the real-time metabolic rate of the metabolic rate acquisition unit and the preprocessed activity image as input values; and
A deep learning-based occupant customized metabolic rate prediction system comprising a deep learning prediction unit for predicting the real-time metabolic rate of the occupant by inputting the activity image of the occupant acquired in real time by the image acquisition unit to the real-time metabolic calculation model. The method of claim 1,
The wearable device includes a watch-type device, a band-type device, a glasses-type device, a necklace-type device, a helmet-type device, and a clothing-type device,
The camera is a deep learning-based occupant-customized metabolic rate prediction system, characterized in that it comprises a Kinect camera. The method of claim 1,
The metabolic rate acquisition unit calculates the real-time metabolic rate of the occupant using the transmitted real-time heart rate information of the occupant and previously input sex, weight and age information of the occupant. Deep learning-based occupant customized metabolic rate prediction system. The method according to claim 1, wherein the data pre-processing unit
Deep learning-based occupant-customized metabolic rate prediction system, characterized in that in the activity image acquired by the image acquisition unit, deleting parts other than the occupant. The method of claim 4,
The camera uses a Kinect camera,
Acquires RGB image and depth image that displays perspective through Kinect camera, detects occupants in the input image based on the acquired depth image, and removes parts other than the occupant by masking the RGB image A deep learning-based metabolic rate prediction system customized for occupants. The method of claim 1,
The real-time metabolic rate calculation model of the deep learning learning unit is
Adopting ReLU as the activation function in Equation 1, adopting Batch Normalization and the SGD optimizer,
The initial target function of the real-time metabolic calculation model is a deep learning-based occupant-customized metabolic prediction system, characterized in that it has Equation 2 below.
[Equation 1]

(Where f(x) is the activation function and x is the input value.)

[Equation 2]

(Where L1Loss is the initial target function, x is the input value, n is the number of input values (batchsize), y is the label,

is the ith input value among n input values x.) The method of claim 6,
The real-time metabolic rate calculation model is characterized in that it has a final target function of Equation 4 below by adding the normalization function of Equation 3 below.
[Equation 3]

(Where REG is the regularization function, n is the number of input values (batchsize),

is the ith input value among n input values x,

is the x mean of n.)
[Equation 4]

(Where O is the final target function, L1Loss is the initial target function, and REG is the regularization function.) As a deep learning-based occupant-specific metabolic prediction method performed by a computer,
Step S100 in which the metabolic rate obtaining unit receives the real-time heart rate of the occupant measured by the wearable device capable of measuring the heart rate and calculates the real-time metabolic rate of the occupant;
Step S200 of receiving real-time activity images of occupants obtained through a camera by an image acquisition unit;
S300 step of pre-processing the image of the image acquisition unit by the data pre-processing unit;
Step S400 in which a real-time metabolic rate calculation model of an occupant is generated by a deep learning learning unit through deep learning using the real-time metabolic rate of the metabolic acquisition unit and the preprocessed activity image as input values; and
Deep learning based occupant customized metabolic rate prediction, characterized in that it comprises a step S500 of predicting real-time metabolic rate of the occupant by inputting the activity image of the occupant acquired in real time by the image acquisition unit to the real-time metabolic rate calculation model by the deep learning prediction unit. method. The method of claim 8,
In step S100, the metabolic rate acquisition unit calculates the real-time metabolic rate of the occupant using the received real-time heart rate information of the occupant and previously input sex, weight, and age information of the occupant. . The method of claim 8,
In step S300, the data pre-processing unit deletes a part except for the occupant from the activity image acquired by the image acquisition unit. The method of claim 10,
The camera uses a Kinect camera,
Through the Kinect camera, an RGB image and a depth image that displays perspective are obtained, and based on the obtained depth image, an occupant is detected from the input image, and the part except for the occupant is removed by masking the RGB image. Deep learning-based occupant-customized metabolic rate prediction method, characterized in that. The method of claim 8,
The real-time metabolic rate calculation model of the deep learning learning unit is
Adopting ReLU as the activation function in Equation 1, adopting Batch Normalization and the SGD optimizer,
The deep learning-based occupant-customized metabolic rate prediction method, characterized in that the initial target function of the real-time metabolic rate calculation model is Equation 2 below.
[Equation 1]

(Where f(x) is the activation function and x is the input value.)

[Equation 2]

(Where L1Loss is the initial target function, x is the input value, n is the number of input values (batchsize), y is the label,

is the ith input value among n input values x.) The method of claim 12,
The real-time metabolic rate calculation model is characterized in that it has a final target function of Equation 4 below by adding the normalization function of Equation 3 below.
[Equation 3]

(Where REG is the regularization function, n is the number of input values (batchsize),

is the ith input value among n input values x,

is the x mean of n.)
[Equation 4]

(Where O is the final target function, L1Loss is the initial target function, and REG is the regularization function.) A computer program stored in a computer-readable recording medium in order to execute the deep learning-based occupant-customized metabolic rate prediction method according to claim 8 by a computer, combined with hardware.

Images