KR102615747B1 - Method, device and system for providing food calorie estimation and user customized body change information visualization service using artificial intelligence model - Google Patents

Abstract

According to one embodiment, in a method of estimating food calories using an artificial intelligence model and providing a user-customized body change information visualization service performed by a device, the camera of the first user terminal is used to determine what the first user wants to eat. When photographing the first food is performed, receiving a first image generated by photographing the first food from the first user terminal; generating a first input signal by encoding the first image; Inputting the first input signal to a first artificial intelligence model learned to recognize the type and amount of food through image analysis; When the type of food is recognized as the first food and the amount of food is recognized as the first weight through the first input signal, a first output indicating the first food and the first weight is generated from the first artificial intelligence model. acquiring a signal; Analyzing that the first user eats the first food of the first weight based on the first output signal; When the calories per unit weight of the first food are confirmed to be a first value, calculating a second value by multiplying the first value and the first weight; And when it is confirmed that the first user has eaten the first food, estimating the calorie intake of the first user to the second value, estimating calories in food using an artificial intelligence model and customizing the body of the user. A method for providing a change information visualization service is provided.

Description

Method, device and system for estimating food calories and providing user-customized body change information visualization service using an artificial intelligence model {METHOD, DEVICE AND SYSTEM FOR PROVIDING FOOD CALORIE ESTIMATION AND USER CUSTOMIZED BODY CHANGE INFORMATION VISUALIZATION SERVICE USING ARTIFICIAL INTELLIGENCE MODEL}

The following embodiments relate to technology for estimating the calories of food using an artificial intelligence model and providing a user-customized body change information visualization service.

As modern society becomes increasingly westernized and industrialized, the number of people suffering from overweight or obesity is gradually increasing. Overweight or obesity is a serious problem for individuals that causes various diseases such as adult diseases, but it also causes enormous social costs to society, making it one of the serious social problems of modern times.

Therefore, in order to solve overweight or obesity, dieting is widely practiced to the extent that it is considered essential for modern people. Additionally, recently, as people's desire for beauty has increased, diet has been in the spotlight not only as a solution to overweight or obesity, but also as a means to maintain a beautiful body and health.

For this type of diet, it is essential to be aware of the calories you consume and the calories you burn.

In a conventional calorie manager, calories consumed are registered only when the user manually enters information about the food they consume, and calories burned are registered only when the user manually enters information about the exercise they performed. There are difficulties in managing.

Therefore, there is a need for research and development on technology that can increase user convenience in managing calories by estimating calories consumed and calories consumed.

Korean Patent No. 10-2247695 Korean Patent No. 10-1741502 Korean Patent No. 10-1562364 Korean Patent Publication No. 10-2018-0005016

According to one embodiment, the purpose is to estimate the calories of food using an artificial intelligence model and provide a user-customized body change information visualization service.

The object of the present invention is not limited to the object mentioned above, and other objects not mentioned can be clearly understood from the description below.

According to one embodiment, in a method of estimating food calories using an artificial intelligence model and providing a user-customized body change information visualization service performed by a device, the camera of the first user terminal is used to determine what the first user wants to eat. When photographing the first food is performed, receiving a first image generated by photographing the first food from the first user terminal; generating a first input signal by encoding the first image; Inputting the first input signal to a first artificial intelligence model learned to recognize the type and amount of food through image analysis; When the type of food is recognized as the first food and the amount of food is recognized as the first weight through the first input signal, a first output indicating the first food and the first weight is generated from the first artificial intelligence model. acquiring a signal; Analyzing that the first user eats the first food of the first weight based on the first output signal; When the calories per unit weight of the first food are confirmed to be a first value, calculating a second value by multiplying the first value and the first weight; And when it is confirmed that the first user has eaten the first food, estimating the calorie intake of the first user to the second value, estimating calories in food using an artificial intelligence model and customizing the body of the user. A method for providing a change information visualization service is provided.

The method of providing food calorie estimation and user-customized body change information visualization service using the artificial intelligence model includes biometric information of the first user and the Receiving activity information of a first user; determining whether the first user is exercising based on the biometric information of the first user; If it is determined that the first user is exercising from the first time based on the biometric information of the first user received at the first time, the activity of the first user received at the first time Encoding information to generate a second input signal; Inputting the second input signal to a second artificial intelligence model learned to recognize the type of exercise through analysis of activity information; When the type of exercise is recognized as a first exercise through the second input signal, obtaining a second output signal representing the first exercise from the second artificial intelligence model; Analyzing that the first user is performing the first exercise based on the second output signal; If it is determined that the first user is not exercising at the second time based on the biometric information of the first user received at the second time after the first time, setting a first period until a second time point; When the calories per unit time of the first exercise are confirmed to be a third value, calculating a fourth value by multiplying the third value and the length of the first period; And it may further include estimating calories consumed by the first user as the fourth value as a result of the first user performing the first exercise during the first period.

A method of estimating food calories using an artificial intelligence model and providing a user-customized body change information visualization service includes collecting the calorie intake of the first user during a predetermined reference period and calculating the total calorie intake of the first user. ; calculating the active metabolic rate of the first user by collecting calories consumed by the first user during the reference period; calculating the basal metabolic rate of the first user based on the gender, age, weight, and height of the first user; Generating matching information by matching the total calorie intake of the first user, the active metabolic rate of the first user, and the basal metabolic rate of the first user; Generating a third input signal by encoding the matching information; Inputting the third input signal to a third artificial intelligence model learned to identify body changes through analysis of total calorie intake, active metabolic rate, and basal metabolic rate; When the physical change of the first user is identified through the third input signal, obtaining a third output signal representing the physical change of the first user from the third artificial intelligence model; analyzing changes in the body of the first user during the reference period based on the third output signal; estimating the amount of change in the body of the first user based on a result of analyzing the change in the body of the first user; A first graph is generated using the amount of change in the body of the first user, a second graph is generated using a preset target value of the first user, and the difference between the amount of change in the body of the first user and the target value of the first user is generated. Generating a third graph using; And based on the first graph, the second graph, and the third graph, it may further include generating body change information visualizing the first user's body change and goal achievement status.

According to one embodiment, by estimating calories consumed and calories consumed using an artificial intelligence model, user convenience can be increased by automating calorie management.

Meanwhile, the effects according to the embodiments are not limited to those mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the description below.

1 is a diagram schematically showing the configuration of a system according to an embodiment.
Figure 2 is a flowchart illustrating a process for estimating a user's calorie intake using an artificial intelligence model according to an embodiment.
Figure 3 is a flow chart to explain the process of estimating the user's calories consumed using an artificial intelligence model according to an embodiment.
Figure 4 is a flowchart for explaining the process of generating body change information visualizing the user's body change and goal achievement status using an artificial intelligence model according to an embodiment.
Figure 5 is a flowchart illustrating a process for estimating a user's calorie intake by confirming the actual amount eaten according to an embodiment.
Figure 6 is a flow chart to explain the process of estimating the user's calories consumed by dividing sections according to exercise intensity according to an embodiment.
Figure 7 is an exemplary diagram of the configuration of a device according to an embodiment.

Hereinafter, embodiments will be described in detail with reference to the attached drawings. However, various changes can be made to the embodiments, so the scope of the patent application is not limited or limited by these embodiments. It should be understood that all changes, equivalents, or substitutes for the embodiments are included in the scope of rights.

Specific structural or functional descriptions of the embodiments are disclosed for illustrative purposes only and may be modified and implemented in various forms. Accordingly, the embodiments are not limited to the specific disclosed form, and the scope of the present specification includes changes, equivalents, or substitutes included in the technical spirit.

Terms such as first or second may be used to describe various components, but these terms should be interpreted only for the purpose of distinguishing one component from another component. For example, a first component may be named a second component, and similarly, the second component may also be named a first component.

When a component is referred to as being “connected” to another component, it should be understood that it may be directly connected or connected to the other component, but that other components may exist in between.

The terms used in the examples are for descriptive purposes only and should not be construed as limiting. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, terms such as “comprise” or “have” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are not intended to indicate the presence of one or more other features. It should be understood that this does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by a person of ordinary skill in the technical field to which the embodiments belong. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and unless explicitly defined in the present application, should not be interpreted in an ideal or excessively formal sense. No.

In addition, when describing with reference to the accompanying drawings, identical components will be assigned the same reference numerals regardless of the reference numerals, and overlapping descriptions thereof will be omitted. In describing the embodiments, if it is determined that detailed descriptions of related known technologies may unnecessarily obscure the gist of the embodiments, the detailed descriptions are omitted.

Embodiments may be implemented in various types of products such as personal computers, laptop computers, tablet computers, smart phones, televisions, smart home appliances, intelligent vehicles, kiosks, and wearable devices.

In an embodiment, an artificial intelligence (AI) system is a computer system that implements human-level intelligence, and unlike existing rule-based smart systems, it is a system in which machines learn and make decisions on their own. As artificial intelligence systems are used, recognition rates improve and sellers' tastes can be more accurately understood, and existing rule-based smart systems are gradually being replaced by deep learning-based artificial intelligence systems.

Artificial intelligence technology consists of machine learning and element technologies using machine learning. Machine learning is an algorithmic technology that classifies/learns the characteristics of input data on its own, and elemental technology is a technology that mimics the functions of the human brain such as cognition and judgment by utilizing machine learning algorithms such as deep learning, including linguistic understanding and visual It consists of technical areas such as understanding, reasoning/prediction, knowledge expression, and motion control.

The various fields where artificial intelligence technology is applied are as follows. Linguistic understanding is a technology that recognizes and applies/processes human language/characters and includes natural language processing, machine translation, conversation systems, question and answer, and voice recognition/synthesis. Visual understanding is a technology that recognizes and processes objects like human vision, and includes object recognition, object tracking, image search, person recognition, scene understanding, spatial understanding, and image improvement. Inferential prediction is a technology that judges information to make logical inferences and predictions, and includes knowledge/probability-based reasoning, optimization prediction, preference-based planning, and recommendations. Knowledge expression is a technology that automatically processes human experience information into knowledge data, and includes knowledge construction (data creation/classification) and knowledge management (data utilization). Motion control is a technology that controls the autonomous driving of vehicles and the movement of robots, and includes motion control (navigation, collision, driving), operation control (behavior control), etc.

Generally, in order to apply machine learning algorithms to real life, learning is performed using a trial and error method due to the nature of the basic methodology of machine learning. In particular, deep learning requires hundreds of thousands of iterations. It is impossible to execute this in an actual physical external environment, so instead, the actual physical external environment is virtually implemented on a computer and learning is performed through simulation.

1 is a diagram schematically showing the configuration of a system according to an embodiment.

Referring to FIG. 1, a system according to an embodiment may include a plurality of user terminals 100 and devices 200 that can communicate with each other through a communication network.

First, a communication network can be configured regardless of the communication mode, such as wired or wireless, and can be implemented in various forms to enable communication between servers and between servers and terminals.

Each of the plurality of user terminals 100 may be implemented as a computing device with a communication function, and may be implemented as, for example, a mobile phone, a desktop PC, a laptop PC, a tablet PC, a smartphone, etc., but is not limited thereto. It can also be implemented as various types of communication devices that can be connected to external servers.

The plurality of user terminals 100 are terminals used by users who wish to estimate food calories and receive customized body change information visualization services. The first user terminal 110 is used by the first user, and the second user terminal 110 is used by the second user. It may include a second user terminal 120, etc.

Each of the plurality of user terminals 100 may be configured to perform all or part of the calculation function, storage/reference function, input/output function, and control function of a typical computer. A plurality of user terminals 100 may be configured to communicate with the device 200 in a wired or wireless manner.

Each of the plurality of user terminals 100 is connected to a web page operated by a person or organization providing services using the device 200, or is developed and distributed by a person or organization providing services using the device 200. An application can be installed. Each of the plurality of user terminals 100 may be linked to the device 200 through a web page or application.

Each of the plurality of user terminals 100 may access the device 200 through a web page or application provided by the device 200.

Hereinafter, for convenience of explanation, the operation of the first user terminal 110 will be mainly described, but of course, other user terminals such as the second user terminal 120 can perform the operation of the first user terminal 110 instead. am.

The device 200 may be its own server owned by a person or organization that provides services using the device 200, a cloud server, or a p2p (peer-to-peer) set of distributed nodes. It may be possible. The device 200 may be configured to perform all or part of the calculation function, storage/reference function, input/output function, and control function of a typical computer. The device 200 may be equipped with at least one artificial intelligence model that performs an inference function.

The device 200 may be configured to communicate wired or wirelessly with a plurality of user terminals 100, control the operation of each of the plurality of user terminals 100, and display certain information on the screen of each of the plurality of user terminals 100. You can control whether to display it or not.

The device 200 is implemented as a server that provides food calorie estimation and user-customized body change information visualization services, and can provide a platform for food calorie estimation and body change information provision.

Meanwhile, for convenience of explanation, only the first user terminal 110 and the second user terminal 120 are shown in FIG. 1 among the plurality of user terminals 100, but the number of terminals may vary depending on the embodiment. . As long as the processing capacity of the device 200 allows, the number of terminals is not particularly limited.

According to one embodiment, the device 200 uses artificial intelligence to recognize the type and amount of food through image analysis and, through this, can estimate the user's calorie intake. A detailed description regarding this will be provided later with reference to FIG. 2 .

Additionally, the device 200 utilizes artificial intelligence to recognize the type of exercise through analysis of activity information and, through this, can estimate the user's calories consumed. A detailed description regarding this will be provided later with reference to FIG. 3 .

In addition, the device 200 uses artificial intelligence to identify physical changes through analysis of total calorie intake, active metabolic rate, and basal metabolic rate, and through this, generates physical change information visualizing the user's physical changes and goal achievement status. You can. A detailed description regarding this will be provided later with reference to FIG. 4 .

In the present invention, artificial intelligence (AI) refers to technology that imitates human learning ability, reasoning ability, perception ability, etc. and implements this with a computer, and includes concepts such as machine learning and symbolic logic. may include. Machine Learning (ML) is an algorithmic technology that classifies or learns the characteristics of input data on its own. Artificial intelligence technology is a machine learning algorithm that analyzes input data, learns the results of the analysis, and makes judgments or predictions based on the results of the learning. Additionally, technologies that mimic the functions of the human brain, such as cognition and judgment, using machine learning algorithms can also be understood as the category of artificial intelligence. For example, technical fields such as verbal understanding, visual understanding, reasoning/prediction, knowledge representation, and motion control may be included.

Machine learning can refer to the process of training a neural network model using experience processing data. Machine learning can mean that computer software improves its own data processing capabilities. A neural network model is built by modeling the correlation between data, and the correlation can be expressed by a plurality of parameters. A neural network model extracts and analyzes features from given data to derive correlations between data. Repeating this process to optimize the parameters of the neural network model can be called machine learning. For example, a neural network model can learn the mapping (correlation) between input and output for data given as input-output pairs. Alternatively, a neural network model may learn the relationships by deriving regularities between given data even when only input data is given.

An artificial intelligence learning model or neural network model may be designed to implement the human brain structure on a computer, and may include a plurality of network nodes with weights that simulate neurons of a human neural network. A plurality of network nodes may have a connection relationship with each other by simulating the synaptic activity of neurons in which neurons exchange signals through synapses. In an artificial intelligence learning model, multiple network nodes are located in layers of different depths and can exchange data according to convolutional connection relationships. The artificial intelligence learning model may be, for example, an artificial neural network model (Artificial Neural Network), a convolution neural network (CNN) model, etc. As an example, an artificial intelligence learning model may be machine-learned according to methods such as supervised learning, unsupervised learning, and reinforcement learning. Machine learning algorithms for performing machine learning include Decision Tree, Bayesian Network, Support Vector Machine, Artificial Neural Network, and Ada-boost. , Perceptron, Genetic Programming, Clustering, etc. can be used.

Among them, CNN is a type of multilayer perceptrons designed to use minimal preprocessing. CNN consists of one or several convolution layers and general artificial neural network layers on top of them, and additionally utilizes weights and pooling layers. Thanks to this structure, CNN can fully utilize input data with a two-dimensional structure. Compared to other deep learning structures, CNN shows good performance in both video and audio fields. CNNs can also be trained via standard back propagation. CNNs have the advantage of being easier to train and using fewer parameters than other feedforward artificial neural network techniques.

Convolutional networks are neural networks that contain sets of nodes with bound parameters. The increasing size of available training data and the availability of computational power, combined with algorithmic advances such as piecewise linear unit and dropout training, have led to significant improvements in many computer vision tasks. For extremely large data sets, such as those available for many tasks today, overfitting is not critical, and increasing the size of the network improves test accuracy. Optimal use of computing resources becomes a limiting factor. For this purpose, distributed, scalable implementations of deep neural networks can be used.

Figure 2 is a flowchart illustrating a process for estimating a user's calorie intake using an artificial intelligence model according to an embodiment.

Referring to FIG. 2 , first, in step S201, the device 200 may receive a first image generated by photographing the first food from the first user terminal 110.

Specifically, the first user terminal 110 may generate a first image by photographing the first food that the first user wants to eat using a camera provided in the first user terminal 110, and the device (200) When the first food that the first user wants to eat is photographed using the camera of the first user terminal 110, the first food generated by photographing the first food from the first user terminal 110 Images can be received.

In step S202, the device 200 may generate a first input signal by encoding the first image.

Specifically, the device 200 may generate a first input signal by encoding pixels of the first image into color information. Color information may include, but is not limited to, RGB color information, brightness information, and saturation information. The device 200 can convert color information into a numerical value and encode an image in the form of a data sheet including this value.

In step S203, the device 200 may input the first input signal to the first artificial intelligence model learned in advance. Here, the first artificial intelligence model may be trained to recognize the type and amount of food through image analysis.

The first artificial intelligence model is an algorithm that receives an input signal generated by encoding an image, recognizes the type and amount of food in the image through the input signal, and outputs an output signal indicating the type and amount of the recognized food. It can be.

The first artificial intelligence model according to one embodiment is implemented as a convolutional neural network, and the convolutional neural network consists of a feature extraction neural network and a classification neural network, and the feature extraction neural network sequentially stacks the input signal into a convolutional layer and a pooling layer. do. The convolution layer contains convolution operations, convolution filters, and activation functions. The calculation of the convolution filter is adjusted according to the matrix size of the target input, but a 9X9 matrix is generally used. The activation function generally uses, but is not limited to, the ReLU function, sigmoid function, and tanh function. The pooling layer is a layer that reduces the size of the input matrix and uses a method of extracting representative values by grouping pixels in a specific area. The calculation of the pooling layer generally uses average or maximum values, but is not limited to these. The operation is performed using a square matrix, and a 9X9 matrix is generally used. The convolution layer and the pooling layer are alternately repeated until the input becomes sufficiently small while maintaining the difference.

According to one embodiment, a classification neural network has a hidden layer and an output layer. In the convolutional neural network implemented as the first artificial intelligence model, there are generally three or more hidden layers, and the number of nodes in each hidden layer is set to 100, but in some cases, it can be set to more. The activation function of the hidden layer uses the ReLU function, sigmoid function, and tanh function, but is not limited to these. There are a total of two output layer nodes in the convolutional neural network, and the output values for the type of food and the output value for the amount of food can be output through the two output layer nodes, respectively.

In step S204, when the type of food is recognized as the first food and the amount of food is recognized as the first weight through the first input signal, the device 200 selects the first food and the first weight from the first artificial intelligence model. A first output signal representing the signal can be obtained.

For example, the device 200 inputs a first input signal to the first artificial intelligence model, and when the type of food is recognized as a hamburger and the amount of food is recognized as 500g through the first input signal, the first artificial intelligence model A first output signal representing hamburger and 500g may be obtained from the model.

That is, the first artificial intelligence model may use the first input signal generated by encoding an image as input, and information indicating the type and amount of food as output.

Encoding according to one embodiment may be accomplished by storing color information for each pixel of an image in the form of a numerical data sheet. The color information may include RGB color, brightness information, and saturation information contained in one pixel. , but is not limited to this.

According to one embodiment, the first artificial intelligence model may be composed of a feature extraction neural network and a classification neural network. The feature extraction neural network may perform the task of separating food and background in an image of food, and the classification neural network may perform the task of separating food and background from an image of food. You can perform the task of determining what type of food is in an image and how much food is in the image.

The method by which a feature extraction neural network distinguishes between food and background is based on the data sheet of the input signal encoding the image, where the change in each value of color information is more than 30% in 6 or more of the 8 pixels including one pixel. A group of pixels that are detected as objects can be used as the boundary between food and background, but are not limited to this.

The feature extraction neural network sequentially stacks the input signal into a convolutional layer and a pooling layer. The convolution layer contains convolution operations, convolution filters, and activation functions. The calculation of the convolution filter is adjusted according to the matrix size of the target input, but a 9X9 matrix is generally used. The activation function generally uses, but is not limited to, the ReLU function, sigmoid function, and tanh function. The pooling layer is a layer that reduces the size of the input matrix and uses a method of extracting representative values by grouping pixels in a specific area. The calculation of the pooling layer generally uses average or maximum values, but is not limited to these. The operation is performed using a square matrix, and a 9X9 matrix is generally used. The convolution layer and the pooling layer are alternately repeated until the input becomes sufficiently small while maintaining the difference.

The classification neural network can identify the type of food in the image by checking the shape of the food distinguished from the background through a feature extraction neural network and checking whether it is similar to a predefined representative food for each food. At this time, information stored in the database can be used to compare foods.

The classification neural network prioritizes the task of identifying the type of food, and can even determine the amount of food depending on the shape and size of the identified food.

A classification neural network has a hidden layer and an output layer. Generally, there are more than 5 hidden layers, and each hidden layer has 80 nodes, but in some cases, it can be set to more. The activation function of the hidden layer uses the ReLU function, sigmoid function, and tanh function, but is not limited to these. There can be a total of two output layer nodes in the classification neural network.

The output of the classification neural network is an output value for the type of food and an output value for the amount of food, which can indicate the type of food and the amount of food. For example, if the output value is (1, 300), the type of food is hamburger and the amount of food is 300g, and if the output value is (2, 500), the type of food is pizza and the amount of food is 300g. You can indicate that the amount is 500g.

According to one embodiment, the first artificial intelligence model may learn by receiving the first learning signal generated by the corrected answer input by the user when the user discovers a problem in the output through the first artificial intelligence model. The problem with output through the first artificial intelligence model may be that an output value indicating a different type of food is output, or an output value indicating a different amount of food is output.

The first learning signal according to one embodiment is created based on the error between the correct answer and the output value, and in some cases, SGD using delta, a batch method, or a method following a backpropagation algorithm may be used. The first artificial intelligence model performs learning by modifying the existing weights using the first learning signal, and may use momentum in some cases. A cost function can be used to calculate the error, and the Cross entropy function can be used as the cost function.

The learning device in which learning of the first artificial intelligence model is performed may be the same device as the device 200 that recognizes the type and amount of food through image analysis using the learned first artificial intelligence model, or may be a separate device. . Below, the process by which the first artificial intelligence model is learned is explained.

The first artificial intelligence model includes an input layer through which training samples are input and an output layer through which training outputs are output, and may be learned based on the difference between the training outputs and the first labels. Here, the first labels may be defined based on representative images for each type of food and representative images for each weight of each food. The first artificial intelligence model is connected to a group of multiple nodes, and is defined by weights between connected nodes and an activation function that activates the nodes.

The learning device may learn the first artificial intelligence model using the Gradient Descent (GD) technique or the Stochastic Gradient Descent (SGD) technique. The learning device may use a loss function designed by the outputs and labels of the first artificial intelligence model.

The learning device can calculate the training error using a predefined loss function. The loss function may be predefined with a label, output, and parameter as input variables, where the parameter may be set by weights in the first artificial intelligence model. For example, the loss function may be designed in the form of MSE (Mean Square Error), entropy, etc., and various techniques or methods may be employed in embodiments in which the loss function is designed.

The learning device can find weights that affect the training error using a backpropagation technique. Here, the weights are relationships between nodes in the first artificial intelligence model. The learning device can use the SGD technique using labels and outputs to optimize the weights found through the backpropagation technique. For example, the learning device can update the weights of the loss function defined based on the labels, outputs, and weights using the SGD technique.

According to one embodiment, the learning device may acquire labeled training food images from a database. The learning device can acquire pre-labeled information on food images set by type of food and food images set by weight of each food, and the food images can be labeled according to the type and amount of pre-classified food. .

According to one embodiment, the learning device can acquire 1000 labeled training food images and generate first training food vectors based on the labeled training food images. Various methods can be employed to extract the first training food vectors.

According to one embodiment, the learning device may obtain first training outputs by applying the first training food vectors to the first artificial intelligence model. The learning device may learn the first artificial intelligence model based on the first training outputs and first labels. The learning device may learn the first artificial intelligence model by calculating training errors corresponding to the first training outputs and optimizing the connection relationship of nodes in the first artificial intelligence model to minimize the training errors. Through this, the first artificial intelligence model can recognize the type and amount of food in the image.

In step S205, the device 200 may analyze that the first user eats the first food of the first weight based on the first output signal.

That is, when the first output signal representing the first food and the first weight is obtained, the device 200 confirms the first food and the first weight based on the first output signal, and through this, the first user It can be analyzed as eating the first food of the first weight.

The device 200 may transmit a notification message to the first user terminal 110 notifying that the first user has photographed the first food in order to eat the first food of the first weight. At this time, when a request for modification of at least one of the first food and the first weight is received through confirmation of the notification message, the device 200 generates a first learning signal through the contents of the modification request and sends the first artificial intelligence model to the first artificial intelligence model. By inputting the first learning signal, the first artificial intelligence model can be learned and updated through the first learning signal.

In step S206, the device 200 may check the calories per unit weight of the first food as a first value. At this time, the device 200 may check how many calories are consumed per unit weight of the first food based on the first food information and set the confirmed calories as the first value. Here, the first food information may be obtained through a database or an external server.

In step S207, when the calories per unit weight of the first food are confirmed to be a first value, the device 200 may calculate a second value by multiplying the first value and the first weight.

For example, if the calories per unit weight of the first food are set to 2 calories per 1g and the first weight is confirmed to be 300g, the device 200 provides 600 calories calculated through (2 2 It can be calculated as a number.

In step S208, if it is confirmed that the first user ate the first food, the device 200 may estimate the first user's calorie intake as a second value.

Specifically, when the first user captures the first food through the first user terminal 110 and then eats the captured first food, the first food displayed on the screen of the first user terminal 110 When the meal completion button is selected, the device 200 may receive a meal completion processing request for the first food from the first user terminal 110, and when the meal completion processing request for the first food is received, the first user It can be confirmed that the first food has been eaten, and if it is confirmed that the first user has eaten the first food, the calorie intake of the first user is estimated as a second value, and the second value is added to the calorie intake history of the first user. You can register by doing this.

Figure 3 is a flow chart to explain the process of estimating the user's calories consumed using an artificial intelligence model according to an embodiment.

Referring to FIG. 3, first, in step S301, the device 200 receives the first user's biometric information and the first user's activity information every preset period from the first wearable device worn on the first user's body. You can.

According to one embodiment, a plurality of wearable devices and the device 200 may be directly connected, each of the plurality of wearable devices is connected to a plurality of user terminals 100, and the plurality of user terminals 100 are connected to the device 200. In connection with, a plurality of wearable devices and devices 200 may be indirectly connected through a plurality of user terminals 100. To this end, a plurality of wearable devices may have a corresponding relationship with a plurality of user terminals 100. For example, while the first wearable device is connected to the first user terminal 110, the first user terminal 110 may be connected to the device 200.

Each of the plurality of wearable devices may be implemented in the form of accessories such as glasses, watches, rings, etc., or in the form of clothing such as clothes and underwear, but is not limited thereto, and may be implemented in various forms of devices depending on the embodiment.

Each of the plurality of wearable devices may be equipped with various sensors to generate biometric information of the user. At this time, the sensor may include a heart rate sensor, a blood pressure sensor, an electrocardiogram sensor, a temperature sensor, etc. In addition, various sensors for measuring the user's biological signals may be included.

For example, the first wearable device may measure heart rate, blood pressure, electrocardiogram, body temperature, etc. while worn on the first user's body, and generate biometric information of the first user based on the measurement results. Here, the biometric information of the first user may include at least one of a heart rate measurement, a blood pressure measurement, an electrocardiogram measurement, and a body temperature measurement.

Each of the plurality of wearable devices may be equipped with various sensors to generate user activity information. At this time, the sensor may include an acceleration sensor, a gyro sensor, a tilt sensor, a GPS sensor, etc. In addition, various sensors for measuring the user's activity signals may be included.

For example, the first wearable device may measure speed, rotation, tilt, position, etc. while worn on the first user's body, and generate activity information of the first user based on the measurement results. Here, the first user's activity information may include at least one of a speed measurement value, a rotation measurement value, a tilt measurement value, and a location measurement value.

That is, when the first wearable device is worn on the first user's body and generates the first user's biometric information and the first user's activity information at preset periods, the device 200 generates The first user's biometric information and the first user's activity information may be received at preset intervals.

For example, when the first wearable device generates the first user's biometric information and the first user's activity information once every 10 seconds, the device 200 generates the first user's biometric information and the first user's activity information once every 10 seconds. Activity information of the first user may be received.

In step S302, the device 200 may determine whether the first user is exercising based on the first user's biometric information.

That is, the device 200 receives the first user's biometric information from the first wearable device every preset period, and each time the first user's biometric information is received, based on the received biometric information of the first user. , it can be determined whether the first user is exercising.

Specifically, the device 200 verifies at least one of the heart rate measurement value, blood pressure measurement value, electrocardiogram measurement value, and body temperature measurement value of the first user based on the biometric information of the first user, and uses the confirmed measurement value as a standard. It is possible to determine whether the first user is exercising.

For example, when the device 200 determines that the first user's heart rate measurement value is higher than the reference value, the device 200 determines that the first user is exercising, and determines that the first user's heart rate measurement value is lower than the reference value. If so, it can be determined that the first user is not exercising.

In step S303, the device 200 may check whether the first user is exercising as a result of determining whether the first user is exercising.

If it is confirmed in step S303 that the first user is not exercising, the process may return to step S301 and begin again with the process of receiving the first user's biometric information and the first user's activity information.

If it is confirmed that the first user is exercising in step S303, in step S304, the device 200 determines whether the first user is exercising from the first time based on the biometric information of the first user received at the first time. If it is determined that the user is in a state of doing , the second input signal can be generated by encoding the activity information of the first user received at the first time.

That is, the device 200 determines whether the first user is exercising based on the biometric information of the first user received at the first time. If it is determined that the first user is exercising, the device 200 determines whether the first user is exercising. From point 1, it can be determined that the first user is exercising, and the activity information of the first user received at the first point in time can be encoded to generate a second input signal.

Specifically, in order to use the activity information as input to an artificial intelligence model, the device 200 may perform normal information processing on the activity information and generate a second input signal.

In step S305, the device 200 may input a second input signal to a pre-trained second artificial intelligence model. Here, the second artificial intelligence model may be trained to recognize the type of exercise through analysis of activity information.

The second artificial intelligence model may be an algorithm that receives an input signal generated by encoding activity information, recognizes the type of exercise through the input signal, and outputs an output signal indicating the recognized type of exercise.

In other words, the second artificial intelligence model can consider the activity information, analyze what type of exercise the user is doing, and output an output signal indicating the type of analyzed exercise.

In step S306, if the type of exercise is recognized as the first exercise through the second input signal, the device 200 may obtain a second output signal representing the first exercise from the second artificial intelligence model. At this time, the second output signal may include information representing the first movement.

For example, the device 200 inputs a second input signal to the second artificial intelligence model, and when the type of exercise is recognized as the first exercise through the second input signal, the device 200 performs the first exercise from the second artificial intelligence model. A second output signal indicating the second exercise can be obtained, and if the type of exercise is recognized as the second exercise through the second input signal, a second output signal indicating the second exercise can be obtained from the second artificial intelligence model.

In other words, the second artificial intelligence model can recognize the type of exercise through the input signal and output information indicating the type of recognized exercise. To this end, the second artificial intelligence model may be pre-trained to analyze what type of exercise the user's activity is based on exercise information classified by exercise stored in the database.

The learning device in which learning of the second artificial intelligence model is performed may be the same device as the device 200 that recognizes the type of exercise through analysis of activity information using the learned second artificial intelligence model, or may be a separate device. . Below, the process by which the second artificial intelligence model is learned is explained.

First, the learning device can generate input based on activity information.

Specifically, the learning device may perform a preprocessing process on activity information. The pre-processed activity information can be used as input to the second artificial intelligence model, or the input can be generated through normal processing to remove unnecessary information.

Next, the learning device can apply input to the second artificial intelligence model. The second artificial intelligence model may be an artificial neural network learned according to reinforcement learning. The second artificial intelligence model may be a Q-Network, DQN (Depp Q-Network), or relational network (RL) structure suitable for outputting abstract inference through reinforcement learning.

The second artificial intelligence model learned according to reinforcement learning can be updated and optimized by reflecting the evaluation of various rewards.

For example, the first compensation can increase the compensation value by comparing the user's speed measurement value with the reference speed for each exercise and analyzing the exercise with the smallest difference as the exercise the user is doing, and the second compensation is the user's rotation measurement. The compensation value can be increased by analyzing the exercise with the smallest difference as the exercise the user is doing by comparing the value with the reference rotation for each exercise, and the third compensation can be used to compare the user's tilt measurement value with the reference rotation for each exercise and have the smallest difference. If a small exercise is analyzed as the exercise the user is doing, the compensation value can be increased, and the fourth compensation is compensated by comparing the user's position measurement value with the reference position for each exercise and analyzing the exercise with the smallest difference as the exercise the user is doing. The value may increase. At this time, the speed measurement value, rotation measurement value, tilt measurement value, and position measurement value can be confirmed through activity information, and the reference speed, reference rotation, reference tilt, and reference position for each exercise can be confirmed through exercise information divided by exercise. It can be.

Next, the learning device can obtain an output from the second artificial intelligence model. At this time, the output of the second artificial intelligence model may be information indicating the type of exercise the user is doing. In other words, the second artificial intelligence model can recognize the type of exercise the user is doing through analysis of activity information and output information about the type of exercise recognized.

Next, the learning device can evaluate the output of the second artificial intelligence model and pay a reward.

For example, the learning device compares the user's speed measurement value and the reference speed for each exercise, and analyzes the exercise with the smallest difference as the exercise the user is doing. If the learning device analyzes the exercise being performed by the user, it awards a large amount of the first reward, and the user's rotation measurement value and the reference speed for each exercise are analyzed. If the reference rotation is compared and the exercise with the smallest difference is analyzed as the exercise the user is doing, a large amount of secondary compensation is awarded, and the user's tilt measurement value is compared with the reference inclination for each exercise to determine whether the user is doing the exercise with the smallest difference. If analyzed as exercise, more third rewards can be awarded, and if the user's position measurement value is compared with the reference position for each exercise and the exercise with the smallest difference is analyzed as the exercise the user is doing, more fourth rewards can be awarded.

Next, the learning device may update the second artificial intelligence model based on the evaluation.

Specifically, the learning device sets certain states so that the expected value of the sum of rewards is maximized in an environment where the second artificial intelligence model recognizes the type of exercise the user is doing. The second artificial intelligence model can be updated through the process of optimizing the policy that determines the actions to be taken.

For example, if there is no problem with the result of analyzing the type of exercise the first user is doing as the first exercise based on the activity information of the first user, the learning device determines the type of exercise the first user is doing. A second learning signal is generated indicating that there is no problem with the results analyzed by the first exercise, and the second learning signal is applied to the second artificial intelligence model to detect a second user who performs activities similar to those of the first user. When analyzing the type of exercise, the second artificial intelligence model can be updated through the process of training the second artificial intelligence model to analyze the type of exercise the second user is doing as the first exercise.

Meanwhile, the process of optimizing the policy can be accomplished through estimating the maximum value of the expected sum of rewards or the maximum value of the Q-function, or estimating the minimum value of the loss function of the Q-function. Estimation of the minimum value of the loss function can be done through stochastic gradient descent (SGD), etc. The process of optimizing the policy is not limited to this, and various optimization algorithms used in reinforcement learning can be used.

The learning device can gradually update the second artificial intelligence model by repeating the learning process of the second artificial intelligence model described above. Through this, the learning device can learn a second artificial intelligence model that recognizes and outputs the type of exercise the user is doing.

That is, when the learning device analyzes the type of exercise the user is doing, it can learn the second artificial intelligence model by reflecting reinforcement learning through the first to fourth rewards, etc. and adjusting the analysis standard.

In step S307, the device 200 may analyze that the first user is doing the first exercise based on the second output signal.

That is, when the second output signal representing the first exercise is obtained, the device 200 determines the type of exercise the first user is doing as the first exercise based on the second output signal, and through this, the first exercise It can be analyzed that the user is doing the first exercise.

In step S308, the device 200 may receive biometric information of the first user and activity information of the first user every preset period from the first wearable device.

In step S309, the device 200 may determine whether the first user is exercising based on the first user's biometric information.

In step S310, the device 200 may check whether the first user is exercising as a result of determining whether the first user is exercising.

If it is confirmed that the first user is exercising in step S310, the process returns to step S301 and the process of receiving the first user's biometric information and the first user's activity information may be performed again.

If it is confirmed in step S310 that the first user is not exercising, in step S311, the device 200 determines that the first user is not exercising at the second time, based on the biometric information of the first user received at the second time. If it is determined that the user is not exercising, the first period can be set from the first time point to the second time point. Here, the second time point is a time point after the first time point.

For example, if the first time point is 4:03:10 and the second time point is 4:05:20, the device 200 uses the difference between the first time point and the second time point, 2 minutes 10 seconds, as the first time point. It can be set by period.

In step S312, the device 200 may check the calories per unit time of the first exercise as a third value. At this time, the device 200 may check how many calories are consumed per unit time of the first exercise based on the first exercise information and set the confirmed calories as the third value. Here, the first exercise information may be obtained through a database or an external server.

In step S313, the device 200 may calculate the fourth numerical value by multiplying the third numerical value by the length of the first period.

For example, if the calories per unit time of the first exercise are set to 2 calories per second and the length of the first period is confirmed to be 3 minutes, the device 200 converts 3 minutes into 180 seconds, and then 360 calories calculated through (2

In step S314, the device 200 may estimate the calories consumed by the first user as a fourth value as a result of the first user performing the first exercise during the first period.

Specifically, it is confirmed that the first user is exercising at a first time while the first user is not exercising, and the first user is exercising at a second time while the first user is exercising. If it is determined that the first user is not exercising, the device 200 determines that the first user exercised from the first time to the second time, analyzes that the first user performed the exercise during the first period, and determines that the first user has exercised during the first period. The calorie consumption of the first user may be estimated as a fourth value, and the fourth value may be added and registered in the calorie consumption history of the first user.

Figure 4 is a flowchart for explaining the process of generating body change information visualizing the user's body change and goal achievement status using an artificial intelligence model according to an embodiment.

Referring to FIG. 4, first, in step S401, the device 200 may calculate the total calorie intake of the first user by collecting the calories consumed by the first user during the reference period. Here, the reference period may be set differently depending on the embodiment.

For example, the device 200 may calculate the total calorie intake of the first user by collecting details added to the calorie intake of the first user during the day, based on the calorie intake details of the first user.

In step S402, the device 200 may calculate the active metabolic rate of the first user by collecting the calories consumed by the first user during the reference period.

For example, the device 200 may calculate the active metabolic rate of the first user by collecting details added to the calories consumed by the first user during the day, based on the details of calories consumed by the first user.

In step S403, the device 200 may calculate the basal metabolic rate of the first user based on the first user's gender, age, weight, and height. At this time, the device 200 may check the gender, age, weight, and height of the first user based on the first user information and calculate the basal metabolic rate of the first user. Here, the first user information may be obtained through a database or the first user terminal 110.

When calculating the basal metabolic rate of the first user, the device 200 may calculate the basal metabolic rate of the first user through various known methods, such as the Harris Benedict equation and the Miffin Sategel equation.

In step S404, the device 200 may generate matching information by matching the first user's total calorie intake, the first user's active metabolic rate, and the first user's basal metabolic rate.

In step S405, the device 200 may generate a third input signal by encoding matching information.

Specifically, in order to use the matching information as input to an artificial intelligence model, the device 200 may perform normal information processing on the matching information and generate a third input signal.

In step S406, the device 200 may input a third input signal to a pre-trained third artificial intelligence model. Here, the third artificial intelligence model may be trained to identify physical changes through analysis of total calorie intake, active metabolic rate, and basal metabolic rate.

The third artificial intelligence model receives an input signal generated by encoding matching information matching total calorie intake, active metabolic rate, and basal metabolic rate, then analyzes the user's body changes through the input signal, and outputs the analysis result. It may be an algorithm that outputs a signal.

In other words, the third artificial intelligence model can predict and analyze how the user's body will change by considering total calorie intake, active metabolic rate, and basal metabolic rate, and output an output signal representing the analyzed result.

In step S407, when the change in the first user's body is identified through the third input signal, the device 200 may obtain a third output signal representing the change in the first user's body from the third artificial intelligence model.

For example, the device 200 inputs a third input signal to the third artificial intelligence model, and when the change in the first user's body is analyzed as weight loss through the third input signal, the device 200 determines the weight loss from the third artificial intelligence model. A third output signal indicating an increase in muscle mass can be obtained, and if the physical change of the first user is analyzed as an increase in muscle mass through the third input signal, a third output signal indicating an increase in muscle mass can be obtained from the third artificial intelligence model. . At this time, the third output signal may include information indicating how much the body weight will be reduced when the first user's physical change is analyzed as a weight loss, and when the first user's physical change is analyzed as an increase in muscle mass, the third output signal may include information indicating how much the body weight will be reduced. It may contain information indicating how much muscle mass will be increased.

In other words, the third artificial intelligence model can identify changes in the user's body through input signals and output an output signal representing the identified body changes. To this end, the third artificial intelligence model may be pre-trained to analyze how the user's body has changed based on body change information classified by situation stored in the database.

The learning device in which learning of the third artificial intelligence model is performed may be the same device as the device 200 that determines physical changes by analyzing total calorie intake, active metabolic rate, and basal metabolic rate using the learned third artificial intelligence model, It may be a separate device. Below, the process by which the third artificial intelligence model is learned is explained.

First, the learning device can generate input based on matching information that matches total calorie intake, active metabolic rate, and basal metabolic rate.

Specifically, the learning device may perform a preprocessing process on matching information that matches total calorie intake, active metabolic rate, and basal metabolic rate. The pre-processed matching information can be used as input to the third artificial intelligence model, or the input can be generated through normal processing to remove unnecessary information.

Next, the learning device can apply input to a third artificial intelligence model. The third artificial intelligence model may be an artificial neural network learned according to reinforcement learning. The third artificial intelligence model may be a Q-Network, DQN (Depp Q-Network), or relational network (RL) structure suitable for outputting abstract inference through reinforcement learning.

The third artificial intelligence model learned according to reinforcement learning can be updated and optimized by reflecting the evaluation of various rewards.

For example, for the 5th compensation, if the total calorie intake is greater than the sum of the active metabolic rate and basal metabolic rate, the compensation value can be increased if the analysis shows that the weight has increased more, and for the 6th compensation, the compensation value can be higher if the total calorie intake is greater than the combined active metabolic rate and basal metabolic rate. If the value is less than the added value, the compensation value can be higher if analyzed as a greater weight loss, and the seventh compensation is analyzed as a greater increase in muscle mass if the activity ratio calculated by dividing the active metabolic rate by the basal metabolic rate is higher than the reference rate. The compensation value may be higher, and the eighth compensation may be higher if the activity rate calculated by dividing the active metabolic rate by the basal metabolic rate is analyzed as a lower decrease in muscle mass than the reference rate.

Next, the learning device can obtain the output from the third artificial intelligence model. At this time, the output of the third artificial intelligence model may be information indicating changes in the user's body. In other words, the third artificial intelligence model can identify changes in the user's body by analyzing matching information that matches total calorie intake, active metabolic rate, and basal metabolic rate, and output information about the identified physical changes. At this time, the information about the physical change may include any one of maintaining the status quo, weight loss, weight gain, muscle mass loss, and muscle mass increase. In the case of weight loss, it includes information indicating how much the weight will be reduced, and weight increase. In the case of a decrease in muscle mass, it includes information indicating how much the body weight will increase, in the case of a decrease in muscle mass, it includes information indicating how much the muscle mass will be decreased, and in the case of an increase in muscle mass, it may include information indicating how much the muscle mass will increase. .

Next, the learning device can evaluate the output of the third artificial intelligence model and pay compensation.

For example, if the learning device analyzes that the total calorie intake is greater than the sum of active metabolic rate and basal metabolic rate, the greater the weight gain, it awards the fifth reward more, and if the total calorie intake is greater than the sum of active metabolic rate and basal metabolic rate, the learning device awards the fifth reward more. If it is analyzed that the lower the weight, the greater the weight loss, the 6th reward is awarded more, and if the activity ratio calculated by dividing the active metabolic rate by the basal metabolic rate is analyzed to be higher than the standard rate, the 7th reward is awarded more. , if the activity rate calculated by dividing the active metabolic rate by the basal metabolic rate is analyzed to be lower than the standard rate, the greater the decrease in muscle mass, the 8th compensation can be awarded more.

Next, the learning device may update the third artificial intelligence model based on the evaluation.

Specifically, the learning device determines actions to be taken in specific states so that the expected value of the sum of rewards is maximized in an environment where a third artificial intelligence model analyzes changes in the user's body. The third artificial intelligence model can be updated through the process of optimizing the policy that determines actions.

For example, the learning device has a problem with the results of analyzing the physical changes of the first user based on matching information that matches the first user's total calorie intake, the first user's active metabolic rate, and the first user's basal metabolic rate. If there is none, a third learning signal is generated indicating that there is no problem with the results of analyzing the first user's body changes, and the third learning signal is applied to the third artificial intelligence model to obtain a calorie intake similar to that of the first user; When analyzing the physical changes of a second user who has an active metabolic rate and basal metabolic rate, through the process of learning a third artificial intelligence model to analyze the physical changes of the second user similarly to the physical changes of the first user. , the third artificial intelligence model can be updated.

Meanwhile, the process of optimizing the policy can be accomplished through estimating the maximum expected value of the sum of rewards or the maximum value of the Q-function, or estimating the minimum value of the loss function of the Q-function. Estimation of the minimum value of the loss function can be done through stochastic gradient descent (SGD), etc. The process of optimizing the policy is not limited to this, and various optimization algorithms used in reinforcement learning can be used.

The learning device can gradually update the third artificial intelligence model by repeating the learning process of the third artificial intelligence model described above. Through this, the learning device can learn a third artificial intelligence model that analyzes and outputs changes in the user's body.

That is, when analyzing changes in the user's body, the learning device can learn a third artificial intelligence model by reflecting reinforcement learning through the fifth to eighth rewards, etc. and adjusting the analysis standard.

In step S408, the device 200 may analyze changes in the first user's body during the reference period based on the third output signal.

That is, when the third output signal representing the change in the body of the first user is obtained, the device 200 determines how and how much the change in the body of the first user has changed based on the third output signal, and through this, sets the standard Changes in the first user's body can be analyzed during the period.

For example, based on the third output signal, the device 200 may determine the physical change of the first user during the reference period as one of maintaining status quo, weight loss, weight gain, muscle mass decrease, and muscle mass increase, and weight loss. If , check how much weight was lost during the baseline period. If weight gain, check how much weight was gained during the baseline period. If muscle mass was decreased, check how much muscle mass was lost during the baseline period. If muscle mass was increased, check how much muscle mass was lost during the baseline period. , you can check how much muscle mass has increased during the standard period.

In step S409, the device 200 may estimate the amount of change in the first user's body based on the result of analyzing the change in the first user's body.

For example, if the physical change of the first user is confirmed to be the status quo, the device 200 may estimate the amount of physical change of the first user as 0, and if the physical change of the first user is confirmed to be 1 kg weight loss, the device 200 may estimate the amount of physical change of the first user as 0. The amount of physical change of the first user can be estimated as a weight loss of 1kg, and if the first user's physical change is confirmed to be a 1kg weight increase, the amount of physical change of the first user can be estimated as a 1kg weight increase, and the first user's physical change can be estimated as a 1kg weight increase. If the physical change is confirmed to be a 1kg decrease in muscle mass, the first user's physical change can be estimated as a 1kg decrease in muscle mass, and if the first user's physical change is confirmed to be a 1kg increase in muscle mass, the first user's physical change can be estimated to be a 1kg increase in muscle mass. It can be estimated as

In step S410, the device 200 may generate a first graph using the amount of change in the first user's body.

For example, if the first user's body weight is set to 60 kg and the amount of change in the first user's body is confirmed to be reduced by 1 kg, the device 200 may generate a first graph showing the change from 60 kg to 59 kg.

In step S411, the device 200 may generate a second graph using the first user's target value.

For example, if the first user's target weight is set to 55kg, the device 200 may generate a second graph using 55kg as the baseline.

In step S412, the device 200 may generate a third graph using the difference between the amount of change in the first user's body and the first user's target value.

For example, if the first user's weight changes from 60 kg to 59 kg and the first user's target value is set to a weight of 55 kg, the device 200 changes the difference between the first user's weight and the target value from 5 kg to 4 kg. A third graph can be created.

In step S413, the device 200 may generate body change information visualizing the first user's body change and goal achievement status based on the first graph, second graph, and third graph. At this time, the device 200 may generate body change information that displays the first graph, second graph, and third graph individually, and displays the body by integrating the first graph, second graph, and third graph into one. Change information can also be generated.

When the device 200 generates body change information, it may transmit the generated body change information to the first user terminal 110 .

Figure 5 is a flowchart illustrating a process for estimating a user's calorie intake by confirming the actual amount eaten according to an embodiment.

Referring to FIG. 5 , first, in step S501, the device 200 may receive a second image generated by photographing the remaining first food from the first user terminal 110.

Specifically, the first user terminal 110 uses a camera provided in the first user terminal 110 to photograph the first food remaining after the first user eats the first food to create a second image. Can be generated, and when the device 200 uses the camera of the first user terminal 110 to photograph the first food remaining after the first user eats the first food, the first user terminal 110 A second image generated by photographing the remaining first food may be received from 110 .

In step S502, the device 200 may generate a 1-1 input signal by encoding the second image.

In step S503, the device 200 may input the 1-1 input signal to the first artificial intelligence model learned in advance.

In step S504, when the type of food is recognized as the first food and the amount of food is recognized as the second weight through the 1-1 input signal, the device 200 selects the first food and the second food from the first artificial intelligence model. A 1-1 output signal representing the weight can be obtained.

In step S505, the device 200 may analyze that the first user left the first food of the second weight based on the 1-1 output signal.

In step S506, the device 200 may calculate the third weight by subtracting the second weight from the first weight.

That is, the device 200 subtracts the second weight, which is the weight of the first food left after eating, from the first weight, which is the weight of the first food before eating, and calculates the third weight, which is the weight of the food actually eaten by the first user. It can be calculated.

In step S507, the device 200 may calculate the fifth value by multiplying the first value and the third weight.

In step S508, the device 200 may estimate the first user's calorie intake as a fifth value.

That is, the device 200 may check the weight of food actually eaten by the first user and estimate the first user's calorie intake as the fifth value rather than the second value.

Figure 6 is a flow chart to explain the process of estimating the user's calories consumed by dividing sections according to exercise intensity according to an embodiment.

Referring to FIG. 6, in step S601, when the biometric information of the first user is a heart rate measurement value, the device 200 monitors the heart rate measurement value of the first user for a first period from the first wearable device. You can receive the results. To this end, the first wearable device may continuously measure and monitor the heart rate of the first user during the first period and generate monitoring results for the heart rate measurement value of the first user during the first period.

In step S602, based on the monitoring result, the device 200 may classify a section in the first period in which the measured heart rate value of the first user is lower than the first reference value as the first section. Here, the first reference value may be set to distinguish whether the user is resting or exercising, and may be set differently depending on the embodiment. That is, if the heart rate measurement value is lower than the first reference value, it may be analyzed that the user is resting, and if the heart rate measurement value is higher than the first reference value, it may be analyzed as the user exercising.

In step S603, based on the monitoring results, the device 200 may classify a section in the first period in which the measured heart rate value of the first user is higher than the first reference value but lower than the second reference value as a second section. Here, the second reference value may be set to distinguish whether the user is engaging in low-intensity exercise or high-intensity exercise, and may be set differently depending on the embodiment. That is, if the heart rate measurement value is lower than the second reference value, it may be analyzed that the user is engaging in low-intensity exercise, and if the heart rate measurement value is higher than the second reference value, the user may be analyzed as engaging in high-intensity exercise.

In step S604, based on the monitoring result, the device 200 may classify a section in the first period in which the measured heart rate value of the first user is higher than the second reference value as a third section.

In step S605, the device 200 may set the first section as a rest section of the first exercise.

In step S606, the device 200 may set the second section as the low-intensity exercise section of the first exercise.

In step S607, the device 200 may set the third section as the high-intensity exercise section of the first exercise.

In step S608, the device 200 may calculate the sixth value by multiplying the first weight, the third value, and the length of the first section. Here, the first weight may be set differently depending on the embodiment, and for example, may be set to a number less than 1.

For example, if the first weight is set to 0.9, the third value is set to 2 calories per second, and the length of the first section is confirmed to be 50 seconds, device 200 determines (0.9 50), the 90 calories calculated can be calculated as the sixth number.

In step S609, the device 200 may calculate the seventh numerical value by multiplying the second weight, the third numerical value, and the length of the second section. Here, the second weight is set to a higher value than the first weight and may be set differently depending on the embodiment, for example, may be set to 1.

For example, if the second weight is set to 1, the third value is set to 2 calories per second, and the length of the second section is confirmed to be 50 seconds, device 200 determines (1 50), 100 calories can be calculated as the 7th number.

In step S610, the device 200 may calculate the eighth numerical value by multiplying the third weight, the third numerical value, and the length of the third section. Here, the third weight is set to a higher value than the second weight and may be set differently depending on the embodiment, for example, may be set to a number greater than 1.

For example, if the third weight is set to 1.1, the third value is set to 2 calories per second, and the length of the third section is confirmed to be 50 seconds, the device 200 calculates (1.1 50), 110 calories can be calculated as the 8th number.

In step S611, the device 200 may calculate the ninth numerical value by adding the sixth numerical value, the seventh numerical value, and the eighth numerical value.

In step S612, the device 200 may estimate the calories consumed by the first user during the first period as a ninth value.

That is, the device 200 divides the rest section, low-intensity exercise section, and high-intensity exercise section according to the exercise intensity of the first user, checks the length of each section, and calculates the calories consumed by the first user during the first period into the fourth section. It can be estimated using the 9th figure, not the numerical value.

Figure 7 is an exemplary diagram of the configuration of a device according to an embodiment.

Device 200 according to one embodiment includes a processor 210 and memory 220. The processor 210 may include at least one device described above with reference to FIGS. 1 to 6 or may perform at least one method described above with reference to FIGS. 1 to 6 . A person or organization using the device 200 may provide services related to some or all of the methods described above with reference to FIGS. 1 to 6 .

The memory 220 may store information related to the methods described above or store a program in which methods described later are implemented. Memory 220 may be volatile memory or non-volatile memory.

The processor 210 can execute programs and control the device 200. The code of the program executed by the processor 210 may be stored in the memory 220. The device 200 is connected to an external device (eg, a personal computer or a network) through an input/output device (not shown) and can exchange data through wired or wireless communication.

The device 200 may be used to learn an artificial intelligence model or use a learned artificial intelligence model. Memory 220 may include a learning or learned artificial intelligence model. The processor 210 may learn or execute the artificial intelligence model algorithm stored in the memory 220. The learning device that trains the artificial intelligence model and the device 200 that uses the learned artificial intelligence model may be the same or may be separate.

The embodiments described above may be implemented with hardware components, software components, and/or a combination of hardware components and software components. For example, the devices, methods, and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, and a field programmable gate (FPGA). It may be implemented using one or more general-purpose or special-purpose computers, such as an array, programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. A processing device may execute an operating system (OS) and one or more software applications that run on the operating system. Additionally, a processing device may access, store, manipulate, process, and generate data in response to the execution of software. For ease of understanding, a single processing device may be described as being used; however, those skilled in the art will understand that a processing device includes multiple processing elements and/or multiple types of processing elements. It can be seen that it may include. For example, a processing device may include a plurality of processors or one processor and one controller. Additionally, other processing configurations, such as parallel processors, are possible.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., singly or in combination. Program instructions recorded on the medium may be specially designed and configured for the embodiment or may be known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -Includes optical media (magneto-optical media) and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, etc. Examples of program instructions include machine language code, such as that produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

Software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing unit to operate as desired, or may be processed independently or collectively. You can command the device. Software and/or data may be used on any type of machine, component, physical device, virtual equipment, computer storage medium or device to be interpreted by or to provide instructions or data to a processing device. , or may be permanently or temporarily embodied in a transmitted signal wave. Software may be distributed over networked computer systems and thus stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

Although the embodiments have been described with limited drawings as described above, those skilled in the art can apply various technical modifications and variations based on the above. For example, the described techniques are performed in a different order than the described method, and/or components of the described system, structure, device, circuit, etc. are combined or combined in a different form than the described method, or other components are used. Alternatively, appropriate results may be achieved even if substituted or substituted by an equivalent.

Therefore, other implementations, other embodiments, and equivalents of the claims also fall within the scope of the following claims.

Claims (3)

In a method of estimating food calories using an artificial intelligence model and providing a user-customized body change information visualization service performed by a device,
When the first food that the first user wants to eat is photographed using the camera of the first user terminal, receiving a first image generated by photographing the first food from the first user terminal;
generating a first input signal by encoding the first image;
Inputting the first input signal to a first artificial intelligence model learned to recognize the type and amount of food through image analysis;
When the type of food is recognized as the first food and the amount of food is recognized as the first weight through the first input signal, a first output indicating the first food and the first weight is generated from the first artificial intelligence model. acquiring a signal;
Analyzing that the first user eats the first food of the first weight based on the first output signal;
When the calories per unit weight of the first food are confirmed to be a first value, calculating a second value by multiplying the first value and the first weight; and
When it is confirmed that the first user has eaten the first food, estimating the calorie intake of the first user to the second value,
Receiving biometric information of the first user and activity information of the first user every preset period from a first wearable device worn on the body of the first user;
determining whether the first user is exercising based on the biometric information of the first user;
If it is determined that the first user is exercising from the first time based on the biometric information of the first user received at the first time, the activity of the first user received at the first time Encoding information to generate a second input signal;
Inputting the second input signal to a second artificial intelligence model learned to recognize the type of exercise through analysis of activity information;
When the type of exercise is recognized as a first exercise through the second input signal, obtaining a second output signal representing the first exercise from the second artificial intelligence model;
Analyzing that the first user is performing the first exercise based on the second output signal;
If it is determined that the first user is not exercising at the second time based on the biometric information of the first user received at the second time after the first time, setting a first period until a second time point;
When the calories per unit time of the first exercise are confirmed to be a third value, calculating a fourth value by multiplying the third value and the length of the first period; and
As a result of the first user performing the first exercise during the first period, estimating the calories burned by the first user as the fourth value,
calculating the total calorie intake of the first user by collecting the calorie intake of the first user during a predetermined reference period;
calculating the active metabolic rate of the first user by collecting calories consumed by the first user during the reference period;
calculating the basal metabolic rate of the first user based on the gender, age, weight, and height of the first user;
Generating matching information by matching the total calorie intake of the first user, the active metabolic rate of the first user, and the basal metabolic rate of the first user;
Generating a third input signal by encoding the matching information;
Inputting the third input signal to a third artificial intelligence model learned to identify body changes through analysis of total calorie intake, active metabolic rate, and basal metabolic rate;
When the physical change of the first user is identified through the third input signal, obtaining a third output signal representing the physical change of the first user from the third artificial intelligence model;
analyzing changes in the body of the first user during the reference period based on the third output signal;
estimating the amount of change in the body of the first user based on a result of analyzing the change in the body of the first user;
A first graph is generated using the amount of change in the body of the first user, a second graph is generated using a preset target value of the first user, and the difference between the amount of change in the body of the first user and the target value of the first user is generated. Generating a third graph using; and
Based on the first graph, the second graph, and the third graph, it further includes generating body change information visualizing the body change and goal achievement status of the first user,
The step of estimating the calorie intake of the first user to the second value is,
When the first food remaining after the first user eats the first food is photographed using the camera of the first user terminal, the first food remaining from the first user terminal is photographed. receiving a second image generated by:
Generating a 1-1 input signal by encoding the second image;
Inputting the 1-1 input signal to the first artificial intelligence model;
When the type of food is recognized as the first food and the amount of food is recognized as the second weight through the 1-1 input signal, a device representing the first food and the second weight is derived from the first artificial intelligence model. Obtaining a 1-1 output signal;
Analyzing that the first user left the first food of the second weight based on the 1-1 output signal;
calculating a third weight by subtracting the second weight from the first weight;
calculating a fifth value by multiplying the first value and the third weight; and
A step of estimating the calorie intake of the first user to the fifth value,
The step of estimating the calories consumed by the first user to the fourth value is,
If the biometric information of the first user is a heart rate measurement value, receiving a monitoring result of the heart rate measurement value of the first user during the first period from the first wearable device;
Based on the monitoring result, a section in which the heart rate measurement value of the first user is lower than a preset first reference value in the first period is classified as a first section, and the heart rate measurement value of the first user in the first period is classified as a first section. A section that is higher than the first reference value but lower than a preset second reference value is classified as a second section, and a section in which the heart rate measurement value of the first user is higher than the second reference value in the first period is classified as a third section. steps;
Setting the first section as a rest section of the first exercise, setting the second section as a low-intensity exercise section of the first exercise, and setting the third section as a high-intensity exercise section of the first exercise. step;
calculating a sixth value by multiplying a preset first weight, the third value, and the length of the first section;
estimating calories consumed by the first user during the first section to the sixth value;
calculating a seventh value as a value obtained by multiplying a second weight set to a higher value than the first weight, the third value, and the length of the second section;
estimating calories consumed by the first user during the second section to the seventh value;
calculating an eighth numerical value as a value obtained by multiplying a third weight set to a value higher than the second weight, the third numerical value, and the length of the third section;
estimating calories consumed by the first user during the third section to the eighth value;
calculating a ninth numerical value by adding up the sixth numerical value, the seventh numerical value, and the eighth numerical value; and
Comprising: estimating calories consumed by the first user during the first period to the ninth value,
Method for estimating food calories and providing customized body change information visualization service using an artificial intelligence model. delete delete