KR102615445B1 - Method, apparatus and system for providing nutritional information based on fecal image analysis - Google Patents

Abstract

A method, device, and system for providing necessary nutritional information based on stool image analysis are disclosed. A method of providing necessary nutritional information based on stool image analysis according to an embodiment includes acquiring stool images corresponding to an identifier, extracting lump objects that are not fully digested objects from the stool images, and first training lump objects. And the first labels - the first labels are information on foods corresponding to the training chunk objects - are used to classify the chunk objects into incompletely digested food information using a first neural network learned in advance, and the chunk objects are classified into incompletely digested food information. Necessary nutrient information corresponding to the identifier is generated based on the incompletely digested food information corresponding to the identifier.

Description

Method for providing nutritional information based on stool image analysis, device and system {METHOD, APPARATUS AND SYSTEM FOR PROVIDING NUTRITIONAL INFORMATION BASED ON FECAL IMAGE ANALYSIS}

The embodiments below relate to a method, device, and system for providing nutritional information based on stool image analysis.

As background technology related to the embodiments, Republic of Korea Patent Publication KR 10-2020-0141776 A discloses a method and device for infant health management using stool images. Specifically, the prior literature includes the steps of collecting stool images as well as at least one of vital sign data and hospital record data in a time-series manner over a preset time, and deriving stool status information through analysis of each of the collected stool images. A step of generating stool status information for judgment by normal state, disease state, and health status based on the stool status information and the acquired data and storing it in a user database, and thereafter, acquiring stool images of infants and young children; Thereafter, a step of generating stool status information through analysis of the acquired stool image, and a comparison between stool status information, normal status stored in the user database, and stool status change information for judgment by disease status and health status, A step of determining whether stool status information for judgment that matches within a preset threshold range exists in the user database; and, as a result of the judgment, if there is stool status information for judgment that matches, generating status judgment information for infants and toddlers. Disclosed is a stool image-based infant health management method including the step of transmitting status determination information to a pre-registered user terminal.

Through this, the prior literature uses at least one of the hospital record data, feeding record data, and vital sign data in addition to the infant's stool image to generate stool status information for judgment by condition of the infant and then uses the stool image based on this. By determining the infant's condition, it can help manage the infant's health.

In addition, Republic of Korea Patent Publication KR 10-2020-0119135 A discloses a stool condition analysis method using a neural network module and a system for the same. Specifically, the prior literature provides temperature information and humidity information for a predetermined time period from the temperature sensor, humidity sensor, gyro sensor, gas sensor, and image sensor installed in the mounting port of the automatic excrement disposal device mounted on the patient's body. , a processing unit configured to collect the patient's posture information, gas information, and image information and perform a pre-processing step on the collected information, and a temperature value that is input information output from the processing unit after the pre-processing step is performed, Disclosed is a server that includes a neural network module that receives humidity values, posture values, gas concentration values, gas generation times, and image values and outputs stool condition values of the patient.

Through this, the prior literature can provide a method for analyzing a patient's stool condition using a neural network module and a server for this.

However, prior literature acquires stool images corresponding to identifiers, extracts lump objects that are not fully digested objects from the stool images, classifies lump objects into incompletely digested food information, and divides the lump objects into incompletely digested food information. It does not disclose technical features for generating required nutrient information corresponding to the identifier based on it.

Republic of Korea Patent Publication KR 10-2020-0141776 A Republic of Korea Patent Publication KR 10-2020-0119135 A

Embodiments seek to provide necessary nutritional information based on stool image analysis.

Embodiments seek to provide recommended food information based on required nutritional information.

Embodiments seek to provide feedback on the digestive state after consuming food according to recommended food information.

A method of providing nutritional information based on stool image analysis according to an embodiment of the present invention includes the steps of acquiring stool images corresponding to an identifier; Extracting lump objects, which are objects that have not been completely digested, from the stool images; Incompletely digesting the chunk objects using a first neural network previously learned based on first training chunk objects and first labels - the first labels are information on foods corresponding to the training chunk objects - Classifying food information; and generating required nutrient information corresponding to the identifier based on incompletely digested food information corresponding to the lump objects.

In one embodiment, the step of generating recommended food information corresponding to the identifier may be further included based on the necessary nutrient information.

In one embodiment, acquiring at least one second stool image after consuming food according to the recommended food information; determining the digestion state of the food according to the recommended food information using the second stool image; generating feedback information about food according to the recommended food information based on the digestion state of the food; And it may further include providing the feedback information to the user corresponding to the identifier.

In one embodiment, the step of acquiring the stool images includes acquiring the stool images based on a predetermined period for acquiring stool images corresponding to the identifier and the number of stool images, the predetermined period and The number of stool images may be predefined based on at least one of age, gender, breastfeeding status, and weight corresponding to the identifier.

In one embodiment, the step of extracting the lump objects includes detecting candidate objects having a specific pattern from the feces images; Classifying the candidate objects as food or foreign substances using a second neural network learned in advance; and classifying the candidate object classified as food into a lump object.

In one embodiment, acquiring first training stool images; extracting the first training chunk objects from the first training stool images; Obtaining the first labels, which are information on foods corresponding to the first training chunk objects; applying the first training chunk objects to the first neural network to generate training outputs corresponding to the first training chunk objects; and training the first neural network based on the training outputs and the first labels.

In one embodiment, the classifying the incompletely digested food information may include obtaining sizes and numbers of the lump objects included in the stool images; and applying the obtained distribution information of the sizes and numbers to a pre-trained third neural network to generate incompletely digested food information corresponding to the stool images.

In one embodiment, acquiring training stool images; extracting the first training chunk objects from the training stool images; extracting first training distribution information, which is distribution information of sizes and numbers of the first training chunk objects, from the first training chunk objects; Obtaining second labels that are incompletely digested food information corresponding to the first training distribution information; applying the first training distribution information to the third neural network to generate training outputs corresponding to the first training distribution information; and training the third neural network based on the training outputs and the second labels.

In one embodiment, forming necessary nutrient information corresponding to the identifier may include assigning a weight to each of the lump objects in consideration of the sizes and numbers of the lump objects included in the feces images; calculating a digestibility index by adding up the assigned weights for each necessary nutrient corresponding to each of the lump objects; And it may include calculating the digestibility ranking of the identifier based on the digestibility index.

The computer program according to an embodiment of the present invention may be stored in a computer-readable recording medium in order to execute the above-described method of providing necessary nutritional information based on stool image analysis on a computer.

An apparatus for providing necessary nutritional information based on stool image analysis according to an embodiment of the present invention includes: a memory for acquiring and storing stool images corresponding to identifiers; and extract lump objects, which are objects that have not been fully digested, from the stool images, and create first training lump objects and first labels, wherein the first labels are information on foods corresponding to the training lump objects. Classifies the lump objects into incompletely digested food information using a first neural network previously learned based on, and provides required nutrient information corresponding to the identifier based on the incompletely digested food information corresponding to the lump objects. It may include a processor that generates.

Embodiments may provide necessary nutritional information based on stool image analysis results for multiple users and provide recommended food information based on the necessary nutritional information to help each user consume appropriate food. Additionally, the efficiency of food recommendation can be improved by providing feedback information using the stool image analysis results after the user consumes the recommended food.

1 is a diagram illustrating a system for providing nutritional information based on stool image analysis according to an embodiment.
Figure 2 is a diagram for explaining learning of a neural network according to an embodiment of the present invention.
Figure 3 is an exemplary diagram of a stool image according to an embodiment of the present invention.
Figure 4 is a diagram for explaining a nutritional information providing device according to an embodiment.
Figure 5 is a flowchart of a method for providing necessary nutritional information according to an embodiment.
Figure 6 is an exemplary diagram of the configuration of a device according to an embodiment.

Hereinafter, embodiments will be described in detail with reference to the attached drawings. However, various changes can be made to the embodiments, so the scope of the patent application is not limited or limited by these embodiments. It should be understood that all changes, equivalents, or substitutes for the embodiments are included in the scope of rights.

Specific structural or functional descriptions of the embodiments are disclosed for illustrative purposes only and may be modified and implemented in various forms. Accordingly, the embodiments are not limited to the specific disclosed form, and the scope of the present specification includes changes, equivalents, or substitutes included in the technical spirit.

Terms such as first or second may be used to describe various components, but these terms should be interpreted only for the purpose of distinguishing one component from another component. For example, a first component may be named a second component, and similarly, the second component may also be named a first component.

When a component is referred to as being “connected” to another component, it should be understood that it may be directly connected or connected to the other component, but that other components may exist in between.

The terms used in the examples are for descriptive purposes only and should not be construed as limiting. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, terms such as “comprise” or “have” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are not intended to indicate the presence of one or more other features. It should be understood that this does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by a person of ordinary skill in the technical field to which the embodiments belong. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and unless explicitly defined in the present application, should not be interpreted in an ideal or excessively formal sense. No.

In addition, when describing with reference to the accompanying drawings, identical components will be assigned the same reference numerals regardless of the reference numerals, and overlapping descriptions thereof will be omitted. In describing the embodiments, if it is determined that detailed descriptions of related known technologies may unnecessarily obscure the gist of the embodiments, the detailed descriptions are omitted.

Embodiments may be implemented in various types of products such as personal computers, laptop computers, tablet computers, smart phones, televisions, smart home appliances, intelligent vehicles, kiosks, and wearable devices.

An artificial intelligence (AI) system is a computer system that implements human-level intelligence, and unlike existing rule-based smart systems, it is a system in which machines learn and make decisions on their own. As artificial intelligence systems are used, their recognition rates improve and they can more accurately understand user preferences, and existing rule-based smart systems are gradually being replaced by deep learning-based artificial intelligence systems.

Artificial intelligence technology consists of machine learning and element technologies using machine learning. Machine learning is an algorithmic technology that classifies/learns the characteristics of input data on its own, and elemental technology is a technology that mimics the functions of the human brain such as cognition and judgment by utilizing machine learning algorithms such as deep learning, including linguistic understanding and visual It consists of technical areas such as understanding, reasoning/prediction, knowledge expression, and motion control.

The various fields where artificial intelligence technology is applied are as follows. Linguistic understanding is a technology that recognizes and applies/processes human language/characters and includes natural language processing, machine translation, conversation systems, question and answer, and voice recognition/synthesis. Visual understanding is a technology that recognizes and processes objects like human vision, and includes object recognition, object tracking, image search, person recognition, scene understanding, spatial understanding, and image improvement. Inferential prediction is a technology that judges information to make logical inferences and predictions, and includes knowledge/probability-based reasoning, optimization prediction, preference-based planning, and recommendations. Knowledge expression is a technology that automatically processes human experience information into knowledge data, and includes knowledge construction (data creation/classification) and knowledge management (data utilization). Motion control is a technology that controls the autonomous driving of vehicles and the movement of robots, and includes motion control (navigation, collision, driving), operation control (behavior control), etc.

Generally, in order to apply machine learning algorithms to real life, learning is performed using a trial and error method due to the nature of the basic methodology of machine learning. In particular, deep learning requires hundreds of thousands of iterations. It is impossible to execute this in an actual physical external environment, so instead, the actual physical external environment is virtually implemented on a computer and learning is performed through simulation.

1 is a diagram illustrating a system for providing nutritional information based on stool image analysis according to an embodiment.

As shown in FIG. 1, the stool image analysis-based nutritional information providing system 100 includes a plurality of user terminals 110-1, 110-2, ..., 110-n, a nutritional information providing device 120, and a database. It may include (130) and network (N). According to one embodiment, the database 130 is shown as being configured separately from the nutritional information providing device 120, but the present invention is not limited thereto, and the database 130 may be provided within the nutritional information providing device 120. For example, the nutritional information providing device 120 may include multiple neural networks for performing machine learning algorithms.

The nutritional information providing device 120 receives stool for a preset period from any one user terminal (for example, 110-1) among the plurality of user terminals 110-1, 110-2, ..., 110-n. Images can be acquired. According to one embodiment, the nutritional information providing device 120 determines a predetermined period of time (e.g., 3 days, 1 week, 1 month, etc.) for acquiring stool images corresponding to the identifier and the number of stool images (e.g., The stool images can be acquired based on (10, 15, 100, etc.), and the predetermined period and number of stool images are determined in advance based on at least one of age, gender, breastfeeding status, and weight corresponding to the identifier. can be defined. For example, the nutritional information providing device 120 acquires first training stool images, extracts first training lump objects from the first training stool images, and provides information on foods corresponding to the first training lump objects. obtain first labels, apply the first training chunk objects to the first neural network, generate training outputs corresponding to the first training chunk objects, and based on the training outputs and the first labels, 1 A neural network can be trained.

The nutritional information providing device 120 may extract lump objects, which are objects that have not been fully digested, from stool images acquired from the user terminal 110-1. According to one embodiment, the nutritional information providing device 120 detects candidate objects with a specific pattern from stool images, classifies the candidate objects as food or foreign substances using a pre-trained second neural network, and Candidate objects classified as can be classified as lump objects.

The nutritional information providing device 120 may classify chunk objects into incompletely digested food information using a first neural network previously learned based on the first training chunk objects and first labels. According to one embodiment, the first labels may indicate information on foods corresponding to training chunk objects. For example, the nutritional information providing device 120 acquires the sizes and numbers of lump objects included in stool images, and applies the distribution information of the obtained sizes and numbers to a previously learned neural network to create stool images. Information on incompletely digested foods corresponding to these can be generated. According to another embodiment, the nutritional information providing device 120 extracts first training chunk objects from training stool images, and provides distribution information of the sizes and numbers of the first training chunk objects from the first training chunk objects. Extract first training distribution information, obtain second labels that are incompletely digested food information corresponding to the first training distribution information, and apply the first training distribution information to a neural network to train corresponding to the first training distribution information. Outputs may be generated and a third neural network may be trained based on the training outputs and the second labels.

The nutritional information providing device 120 may generate necessary nutritional information corresponding to the identifier based on incompletely digested food information corresponding to the lump objects. According to one embodiment, the nutritional information providing device 120 assigns a weight to each of the lump objects in consideration of the sizes and numbers of lump objects included in the stool images, and determines the necessary weight corresponding to each of the lump objects. The digestibility index is calculated by adding up the weights assigned to each nutrient, and the digestibility ranking of the identifier can be calculated based on the digestibility index.

The nutritional information providing device 120 may generate recommended food information corresponding to the identifier based on the necessary nutritional information. According to one embodiment, the nutritional information providing device 120 may generate recommended food information using a mapping table stored in the database 130, or may generate recommended food information using a pre-trained neural network. there is. For example, recommended food information may include information on health functional foods, health foods, and alternative foods. Health functional foods refer to ingredients that can help your health processed into a food form that can be easily consumed, health foods refer to foods eaten to maintain and improve health, and substitute foods refer to foods that replace specific foods. It can indicate foods that can be consumed.

The nutritional information providing device 120 acquires at least one second stool image after consuming food according to the recommended food information, and determines the digestion state of the food according to the recommended food information using the second stool image. , Based on the digestion state of the food, feedback information about the food according to the recommended food information can be generated, and the feedback information can be provided to the user corresponding to the identifier.

The network (N) can perform wireless or wired communication between a plurality of user terminals (110-1, 110-2, ..., 110-n), nutritional information providing device 120, database 130, etc. there is. For example, networks include long-term evolution (LTE), LTE Advanced (LTE-A), code division multiple access (CDMA), wideband CDMA (WCDMA), Wireless BroadBand (WiBro), wireless fidelity (WiFi), and Bluetooth ( Wireless communication can be performed using methods such as Bluetooth), NFC (near field communication), GPS (Global Positioning System), or GNSS (global navigation satellite system). For example, the network may be configured to perform wired communications such as universal serial bus (USB), high definition multimedia interface (HDMI), recommended standard 232 (RS-232), or plain old telephone service (POTS). .

The database 130 can store various data. Data stored in the database 130 is acquired, processed, or acquired by a plurality of user terminals 110-1, 110-2, ..., 110-n, and at least one component of the nutritional information providing device 120. , used data, may include software (e.g., a program). Database 130 may include volatile and/or non-volatile memory. As an embodiment, the database 130 includes stool images acquired from a plurality of user terminals 110-1, 110-2, ..., 110-n, lump objects formed in the nutritional information providing device 120, and incomplete Information on digested food and required nutrients can be stored.

In the present invention, the program is software stored in the database 130, and is an operating system for controlling the resources of a plurality of user terminals 110-1, 110-2, ..., 110-n and the nutritional information providing device 120. , applications and/or middleware that provides various functions to the application so that the application can utilize the resources of multiple user terminals (110-1, 110-2, ..., 110-n) and the nutritional information providing device 120, etc. may include.

In the present invention, artificial intelligence (AI) refers to technology that imitates human learning ability, reasoning ability, perception ability, etc. and implements this with a computer, and includes concepts such as machine learning and symbolic logic. may include. Machine Learning (ML) is an algorithmic technology that classifies or learns the characteristics of input data on its own. Artificial intelligence technology is a machine learning algorithm that analyzes input data, learns the results of the analysis, and makes judgments or predictions based on the results of the learning. Additionally, technologies that mimic the functions of the human brain, such as cognition and judgment, using machine learning algorithms can also be understood as the category of artificial intelligence. For example, technical fields such as verbal understanding, visual understanding, reasoning/prediction, knowledge representation, and motion control may be included.

Machine learning can refer to the process of training a neural network model using experience processing data. Machine learning can mean that computer software improves its own data processing capabilities. A neural network model is built by modeling the correlation between data, and the correlation can be expressed by a plurality of parameters. A neural network model extracts and analyzes features from given data to derive correlations between data. Repeating this process to optimize the parameters of the neural network model can be called machine learning. For example, a neural network model can learn the mapping (correlation) between input and output for data given as input-output pairs. Alternatively, a neural network model may learn the relationships by deriving regularities between given data even when only input data is given.

An artificial intelligence learning model or neural network model may be designed to implement the human brain structure on a computer, and may include a plurality of network nodes with weights that simulate neurons of a human neural network. A plurality of network nodes may have a connection relationship with each other by simulating the synaptic activity of neurons in which neurons exchange signals through synapses. In an artificial intelligence learning model, multiple network nodes are located in layers of different depths and can exchange data according to convolutional connection relationships. The artificial intelligence learning model may be, for example, an artificial neural network model (Artificial Neural Network), a convolution neural network (CNN) model, etc. As an example, an artificial intelligence learning model may be machine-learned according to methods such as supervised learning, unsupervised learning, and reinforcement learning. Machine learning algorithms for performing machine learning include Decision Tree, Bayesian Network, Support Vector Machine, Artificial Neural Network, and Ada-boost. , Perceptron, Genetic Programming, Clustering, etc. can be used.

Among them, CNN is a type of multilayer perceptrons designed to use minimal preprocessing. CNN consists of one or several convolution layers and general artificial neural network layers on top of them, and additionally utilizes weights and pooling layers. Thanks to this structure, CNN can fully utilize input data with a two-dimensional structure. Compared to other deep learning structures, CNN shows good performance in both video and audio fields. CNNs can also be trained via standard back propagation. CNNs have the advantage of being easier to train and using fewer parameters than other feedforward artificial neural network techniques.

Convolutional networks are neural networks that contain sets of nodes with bound parameters. The increasing size of available training data and the availability of computational power, combined with algorithmic advances such as piecewise linear unit and dropout training, have led to significant improvements in many computer vision tasks. For extremely large data sets, such as those available for many tasks today, overfitting is not critical, and increasing the size of the network improves test accuracy. Optimal use of computing resources becomes a limiting factor. For this purpose, distributed, scalable implementations of deep neural networks can be used.

Figure 2 is a diagram for explaining learning of a neural network according to an embodiment of the present invention.

As shown in FIG. 2, the learning device can train the neural network 124 to extract lump objects included in feces images. According to one embodiment, the learning device may be a separate entity from the nutritional information providing device 120, but is not limited thereto.

The neural network 124 includes an input layer 122 through which training samples are input and an output layer 126 through which training outputs are output, and may be learned based on the difference between the training outputs and labels. Here, labels may be defined based on lump objects, information on foods corresponding to lump objects, and information on incompletely digested foods. The neural network 124 is connected to a group of a plurality of nodes, and is defined by weights between connected nodes and an activation function that activates the nodes.

The learning device may learn the neural network 124 using a gradient descent (GD) technique or a stochastic gradient descent (SGD) technique. The learning device can use a loss function designed by the outputs and labels of the neural network.

The learning device can calculate the training error using a predefined loss function. The loss function may be predefined with a label, output, and parameter as input variables, where the parameter may be set by weights in the neural network 124. For example, the loss function may be designed in the form of MSE (Mean Square Error), entropy, etc., and various techniques or methods may be employed in embodiments in which the loss function is designed.

The learning device can use the backpropagation technique to find weights that affect the training error. Here, the weights are relationships between nodes in the neural network 124. The learning device can use the SGD technique using labels and outputs to optimize the weights found through the backpropagation technique. For example, the learning device can update the weights of the loss function defined based on the labels, outputs, and weights using the SGD technique.

According to one embodiment, the learning device may acquire first training stool images and extract first training lump objects from the first training stool images. The learning device may acquire pre-labeled information (first labels) for each of the first training chunk objects, and obtain first labels that are information on foods corresponding to the first training chunk objects.

According to one embodiment, the learning device may generate first training feature vectors based on the appearance features, pattern features, and color features of the first training chunk objects. Various methods can be employed to extract features.

According to one embodiment, the learning device may obtain training outputs by applying the first training feature vectors to the neural network 124. The learning device may train the neural network 124 based on the training outputs and the first labels. The learning device may learn the neural network 124 by calculating training errors corresponding to the training outputs and optimizing the connection relationships of nodes within the neural network 124 to minimize the training errors. The nutritional information providing device 120 may extract lump objects from stool images using the trained neural network 124.

According to another embodiment, the learning device may acquire second training stool images and extract second training lump objects from the second training stool images. The learning device may acquire pre-labeled information (second labels) for each of the second training chunk objects, and may acquire second labels that are information on food or foreign substances corresponding to the second training chunk objects. there is.

According to another embodiment, the learning device may generate second training feature vectors based on the appearance features, pattern features, and color features of the second training chunk objects. Various methods can be employed to extract features.

According to another embodiment, the learning device may obtain training outputs by applying the second training feature vectors to the neural network 124. The learning device may train the neural network 124 based on the training outputs and the second labels. The learning device may learn the neural network 124 by calculating training errors corresponding to the training outputs and optimizing the connection relationships of nodes within the neural network 124 to minimize the training errors. The nutritional information providing device 120 may extract lump objects from stool images using the trained neural network 124.

According to another embodiment, the learning device may acquire third training stool images and extract third training lump objects from the third training stool images. The learning device may extract first training distribution information, which is distribution information of the sizes and numbers of the third training chunk objects, from the third training chunk objects. The learning device may acquire pre-labeled information (third labels) for each of the first training distribution information, and may obtain third labels that are incompletely digested food information corresponding to the first training distribution information.

According to another embodiment, the learning device may generate third training feature vectors based on the appearance features, pattern features, and color features of the first training distribution information. Various methods can be employed to extract features.

According to another embodiment, the learning device may obtain training outputs by applying the third training feature vectors to the neural network 124. The learning device may train the neural network 124 based on the training outputs and the third labels. The learning device may learn the neural network 124 by calculating training errors corresponding to the training outputs and optimizing the connection relationships of nodes within the neural network 124 to minimize the training errors. The nutritional information providing device 120 may extract incompletely digested food information from stool images using the trained neural network 124.

According to another embodiment, the learning device may acquire fourth training stool images and extract fourth training lump objects from the fourth training stool images. The learning device may extract fourth training incompletely digested food information from fourth training chunk objects. The learning device may acquire pre-labeled information (fourth labels) for each of the fourth training incompletely digested food information, such as a health functional food and health food corresponding to the fourth training fourth training incompletely digested food information. , fourth labels, which are information on alternative foods, can be obtained. Health functional foods refer to ingredients that can help your health processed into a food form that can be easily consumed, health foods refer to foods eaten to maintain and improve health, and substitute foods refer to foods that replace specific foods. It can indicate foods that can be consumed.

According to another embodiment, the learning device may generate fourth training feature vectors based on the appearance features, pattern features, and color features of the fourth training incompletely digested food information. Various methods can be employed to extract features.

According to another embodiment, the learning device may obtain training outputs by applying the fourth training feature vectors to the neural network 124. The learning device may learn the neural network 124 based on the training outputs and the fourth labels. The learning device may learn the neural network 124 by calculating training errors corresponding to the training outputs and optimizing the connection relationships of nodes within the neural network 124 to minimize the training errors. The nutritional information providing device 120 may extract necessary nutritional information from stool images using the trained neural network 124.

Figure 3 is an exemplary diagram of a stool image according to an embodiment of the present invention.

As shown in FIG. 3, the nutritional information providing device 120 selects an object that has not been completely digested from stool images obtained from a plurality of user terminals 110-1, 110-2, ..., 110-n. You can extract lumped objects (MO). According to one embodiment, the nutritional information providing device 120 detects candidate objects with a specific pattern from acquired stool images, classifies the candidate objects as food or foreign substances using a pre-trained neural network, and Things classified as can be extracted as lump objects (MO).

Depending on the state of teeth and digestive system formation during the infant's growth process, the foods that are digested may vary for each infant. Some infants may not be able to digest broccoli, and some may not be able to digest carrots. Food that the infant cannot digest is excreted as stool, and the nutritional information providing device 120 can extract these indigestible foods from the stool image. According to one embodiment, the nutritional information providing device 120 can label and learn the classification of chunk objects based on a machine learning algorithm such as deep learning, and perform classification according to the learned model. there is.

When acquiring a plurality of stool images, the nutritional information providing device 120 receives the first part stool image from the first user terminal (e.g., 110-1), and sends the second part stool image to which the second user account is logged in. Receives a second part feces image from a user terminal (e.g., 110-2), generates a first feces image by combining the first part feces image and the second part feces image, and generates a first feces image from the first feces image. A first ratio, which is the ratio occupied by the part feces image, is obtained, and a second ratio is obtained, which is the ratio occupied by the second part feces image in the first feces image. If the first ratio is confirmed to be greater than the second ratio, the first ratio is obtained. Register the user account information as the first author information of the first feces image, and register the second user account information as the second author information of the first feces image. If the first ratio is confirmed to be smaller than the second ratio, , registering the information of the second user account as the first author information of the first feces image, registering the information of the first user account as the second author information of the first feces image, and registering the information of the first user account as the second author information of the first feces image. 1 Check whether any of the information in the user account and the information in the second user account match, and determine that the original author information of the first feces image and the information in the first user account and the information in the second user account do not all match. If confirmed, the first feces image is determined to be a secondary work created by modifying the second feces image, and if the first ratio is confirmed to be greater than the second ratio, the information of the first user account is converted to the second feces image. 1 Register as author information, register the information of the second user account as the second author information of the first stool image, and register it as the original author of the first stool image to the third user terminal (for example, 110-n) The information of the logged in third user account is registered as third author information indicating the third author of the first credit image, and if the first ratio is confirmed to be smaller than the second ratio, the information of the second user account is registered as third author information indicating the third author of the first credit image. You can register as the first author information of the image, register the information of the first user account as the second author information of the first side image, and register the information of the third user account as the third author information of the first side image. there is.

Figure 4 is a diagram for explaining a nutritional information providing device according to an embodiment.

As shown in FIG. 4, the nutritional information providing device 120 may include one or more processors 121, one or more memories 123, and/or a transceiver 125. As an example, at least one of these components of the nutritional information providing device 120 may be omitted, or other components may be added to the nutritional information providing device 120. Additionally or alternatively, some components may be integrated and implemented, or may be implemented as a single or plural entity. At least some of the internal and external components of the nutritional information providing device 120 communicate with each other through a bus, general purpose input/output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI). It is connected and can send and receive data and/or signals.

One or more processors 121 may control at least one component of the nutritional information providing device 120 connected to the processor 121 by running software (eg, commands, programs, etc.). Additionally, the processor 121 can perform various operations related to the present invention, such as calculation, processing, data generation, and processing. Additionally, the processor 121 may load data, etc. from one or more memories 123 or store them in one or more memories 123 .

As described above, one or more processors 121, through the transceiver 125, transmits data to one user terminal (e.g., 110-n) among a plurality of user terminals 110-1, 110-2, ..., 110-n. -1) Fecal images can be received in real time or non-real time in the form of digital packets.

One or more processors 121 may extract lump objects, which are objects that have not been completely digested, using feces images received from the user terminal 110-1 through the transceiver 125. According to one embodiment, the one or more processors 121 obtain first training stool images, extract first training lump objects from the first training stool images, and store food items corresponding to the first training lump objects. Obtain first labels that are information, apply the first training chunk objects to the first neural network, generate training outputs corresponding to the first training chunk objects, and based on the training outputs and the first labels, By training the first neural network, lump objects can be extracted from stool images.

One or more processors 121 may classify chunk objects into incompletely digested food information using a first neural network previously learned based on the first training chunk objects and first labels. For example, the first labels may indicate information on foods corresponding to training chunk objects.

One or more processors 121 may generate required nutrient information corresponding to the identifier based on incompletely digested food information corresponding to the lump objects.

One or more memories 123 may store various data. Data stored in the memory 123 is data acquired, processed, or used by at least one component of the nutritional information providing device 120, and may include software (e.g., commands, programs, etc.) . Memory 123 may include volatile and/or non-volatile memory. In the present invention, a command or program is software stored in the memory 123, and an operating system, application, and/or application for controlling the resources of the nutritional information providing device 120 utilizes the resources of the nutritional information providing device 120. It may include middleware that provides various functions to the application.

One or more memories 123 include stool images received from the above-described plurality of user terminals 110-1, 110-2, ..., 110-n through the network N, and chunks formed by one or more processors 121. Objects, incompletely digested food information, required nutrient information, etc. can be stored. Additionally, one or more memories 123 may store instructions that allow one or more processors 121 to perform operations when executed by one or more processors 121 .

As an example, the nutritional information providing device 120 may further include a transceiver 125. The transceiver 125 provides wireless or wired communication between a plurality of user terminals 110-1, 110-2, ..., 110-n, the nutritional information providing device 120, the database 130, and/or other devices. It can be done. For example, the transceiver 126 may support enhanced Mobile Broadband (eMBB), Ultra Reliable Low-Latency Communications (URLLC), Massive Machine Type Communications (MMTC), long-term evolution (LTE), LTE Advance (LTE-A), UMTS (Universal Mobile Telecommunications System), GSM (Global System for Mobile communications), CDMA (code division multiple access), WCDMA (wideband CDMA), WiBro (Wireless Broadband), WiFi (wireless fidelity), Bluetooth, NFC ( Wireless communication can be performed using methods such as near field communication, GPS (Global Positioning System), or GNSS (global navigation satellite system). For example, the transceiver 125 can perform wired communication according to a method such as universal serial bus (USB), high definition multimedia interface (HDMI), recommended standard232 (RS-232), or plain old telephone service (POTS). there is.

As an embodiment, one or more processors 121 control the transceiver 125 to obtain information from a plurality of user terminals 110-1, 110-2, ..., 110-n, and the nutritional information providing device 120. can do. Information obtained from a plurality of user terminals 110-1, 110-2, ..., 110-n and the nutritional information providing device 120 may be stored in one or more memories 123.

As an example, the nutritional information providing device 120 may be of various types. For example, nutritional information providing device 120 may be a portable communication device, a computer device, or a device based on one or a combination of one or more of the foregoing devices. The nutritional information providing device 120 of the present invention is not limited to the devices described above.

Various embodiments of the nutritional information providing device 120 according to the present invention may be combined with each other. Each embodiment can be combined according to the number of cases, and the embodiment of the nutritional information providing device 120 created by combining the embodiments also falls within the scope of the present invention. Additionally, the internal and external components of the nutritional information providing device 120 according to the present invention described above may be added, changed, replaced, or deleted depending on the embodiment. Additionally, the internal/external components of the above-described nutritional information providing device 120 may be implemented as hardware components.

According to one embodiment, the nutritional information providing device 120 may collect stool images of a specific infant (A). For example, the nutritional information providing device 120 may specify a period and collect stool images during the period, or may specify a number and collect stool images corresponding to the corresponding number. The nutritional information providing device 120 can form a stool list (including M stool information) of a specific infant (A) using the collected stool images, and extract lump objects from each of the collected stool images. . The lump objects may represent pieces of food that were not fully digested in a particular infant (A).

The nutritional information providing device 120 may detect a unique pattern area in the collected stool images based on color or shape. The nutritional information providing device 120 determines the food (e.g., carrots, broccoli, etc.) that is continuously incompletely digested from M stools of a specific infant (A), and determines the food (e.g., carrots, broccoli, etc.) that is required for the specific infant (A) from the determination result. Nutrient information can be derived. According to one embodiment, the nutritional information providing device 120 may label food for each lump object and classify the lump objects using a machine learning algorithm such as deep learning based on color, shape, etc.

The nutritional information providing device 120 may recommend (curate) the optimal health functional food or alternative food combination for a specific infant (A) based on the derived nutritional information. The nutritional information providing device 120 may reanalyze the digestion state of the optimal health functional food or alternative food combination and provide the results as feedback to the guardian of the specific infant (A).

The nutritional information providing device 120 may derive nutritional information required for a specific infant (A) based on the size and number of repetitive lump objects based on the distribution of lump objects extracted from the collected stool images.

Figure 5 is a flowchart of a method for providing necessary nutritional information according to an embodiment.

Although the process steps, method steps, algorithms, etc. are described in the flow chart of FIG. 5 in a sequential order, such processes, methods, and algorithms may be configured to operate in any suitable order. In other words, the steps of the processes, methods and algorithms described in various embodiments of the invention do not need to be performed in the order described herein. Additionally, although some steps are described as being performed asynchronously, in other embodiments, some such steps may be performed concurrently. Additionally, the illustration of a process by depiction in the drawings does not mean that the illustrated process excludes other variations and modifications thereto, and that any of the illustrated process or steps thereof may be incorporated into any of the various embodiments of the invention. It does not imply that more than one is required, nor does it imply that the illustrated process is preferred.

As shown in Figure 5, in step S510, stool images corresponding to the identifier are obtained. For example, referring to FIGS. 1 to 4, the nutritional information providing device 120 is one of a plurality of user terminals 110-1, 110-2, ..., 110-n (for example, For example, stool images can be obtained from 110-1) through the network (N).

In step S520, lump objects, which are objects that have not been completely digested, are extracted from feces images. For example, referring to FIGS. 1 to 4 , the nutritional information providing device 120 may extract lump objects, which are objects that have not been fully digested, from the stool images obtained in step S510. According to one embodiment, the nutritional information providing device 120 may extract lump objects from stool images using a pre-trained neural network.

In step S530, chunk objects are classified into incompletely digested food information using a first neural network previously learned based on the first training objects and first labels. For example, referring to FIGS. 1 to 4, the nutritional information providing device 120 extracts information in step S520 using a first neural network previously learned based on the first training objects and first labels. The lumped objects can be classified as incompletely digested food information.

In step S540, necessary nutrient information corresponding to the identifier is formed based on incompletely digested food information corresponding to the lump objects. For example, referring to FIGS. 1 to 4, the nutritional information providing device 120 provides necessary nutritional information corresponding to a specific identifier based on incompletely digested food information corresponding to the lump objects classified in step S530. can be created.

Figure 6 is an exemplary diagram of the configuration of a device according to an embodiment.

Device 401 according to one embodiment includes a processor 402 and memory 403. The device 401 according to one embodiment may be the server or terminal described above. The processor may include at least one device described above with reference to FIGS. 1 to 5 or may perform at least one method described with reference to FIGS. 1 to 5 . The memory 403 may store information related to the above-described method or store a program in which the above-described method is implemented. Memory 403 may be volatile memory or non-volatile memory.

The processor 402 can execute programs and control the device 401. The code of the program executed by the processor 402 may be stored in the memory 403. The device 401 is connected to an external device (eg, a personal computer or a network) through an input/output device (not shown) and can exchange data.

The embodiments described above may be implemented with hardware components, software components, and/or a combination of hardware components and software components. For example, the devices, methods, and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, and a field programmable gate (FPGA). It may be implemented using one or more general-purpose or special-purpose computers, such as an array, programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. A processing device may execute an operating system (OS) and one or more software applications that run on the operating system. Additionally, a processing device may access, store, manipulate, process, and generate data in response to the execution of software. For ease of understanding, a single processing device may be described as being used; however, those skilled in the art will understand that a processing device includes multiple processing elements and/or multiple types of processing elements. It can be seen that it may include. For example, a processing device may include a plurality of processors or one processor and one controller. Additionally, other processing configurations, such as parallel processors, are possible.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., singly or in combination. Program instructions recorded on the medium may be specially designed and configured for the embodiment or may be known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -Includes optical media (magneto-optical media) and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, etc. Examples of program instructions include machine language code, such as that produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

Software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing unit to operate as desired, or may be processed independently or collectively. You can command the device. Software and/or data may be used on any type of machine, component, physical device, virtual equipment, computer storage medium or device to be interpreted by or to provide instructions or data to a processing device. , or may be permanently or temporarily embodied in a transmitted signal wave. Software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

Although the embodiments have been described with limited drawings as described above, those skilled in the art can apply various technical modifications and variations based on the above. For example, the described techniques are performed in a different order than the described method, and/or components of the described system, structure, device, circuit, etc. are combined or combined in a different form than the described method, or other components are used. Alternatively, appropriate results may be achieved even if substituted or substituted by an equivalent.

Therefore, other implementations, other embodiments, and equivalents of the claims also fall within the scope of the following claims.

Claims (1)

In a method performed by a processor,
Obtaining stool images corresponding to the identifier;
extracting lump objects from the stool images;
Using a first neural network pre-trained based on first training chunk objects and first labels - the first labels being information corresponding to the training chunk objects - the chunk objects are converted to predefined information. Classifying into groups; and
Based on the classification result, generating stool image analysis information corresponding to the identifier.
Including,
The step of extracting the lump objects is
Detecting candidate objects with a specific pattern from the feces images;
Classifying the candidate objects as food or foreign substances using a second neural network learned in advance; and
Classifying the candidate object classified as food into a lump object
Including,
Stool image analysis method.