KR102546355B1 - Method, apparatus and system for providing nutritional information based on fecal image analysis - Google Patents

Abstract

Disclosed are a method, apparatus, and system for providing necessary nutritional information based on stool image analysis. A method for providing necessary nutrition information based on stool image analysis according to an embodiment includes acquiring stool images corresponding to identifiers, extracting chunk objects, which are objects that are not fully digested, from the stool images, and first training chunk objects. and classifying chunk objects into incompletely digested food information using a first neural network pre-learned based on the first labels, wherein the first labels are information on foods corresponding to the training chunk objects, and Required nutrient information corresponding to the identifier is generated based on the incompletely digested food information corresponding to the .

Description

Stool image analysis-based nutrition information providing method, 　apparatus and system

The following embodiments relate to a method, device, and system for providing nutrition information based on stool image analysis.

As background art related to the embodiments, 　Korean Patent Laid-Open Publication 　KR 10-2020-0141776 A discloses a method and apparatus for managing health of infants and young children using stool images. Specifically, in the preceding literature, time-sequentially collecting at least one or more of vital sign data and hospital record data along with stool images for a predetermined time period, and deriving stool condition information through analysis of each collected stool image The step of generating and storing stool condition information for determination by normal state, disease state, and health state in a user database based on stool state information and acquired data, and then, obtaining stool images of infants, Thereafter, the step of generating stool condition information through analysis of the acquired stool image, and the stool state information and stool state information and Determining whether feces condition information matched within a predetermined threshold range exists in the user database, and, as a result of the determination, if there is matched stool condition information, generating infant condition determination information; Disclosed is a stool image-based health management method for infants and young children, which includes transmitting state determination information to pre-registered user-side terminals.

Through this, the preceding literature uses at least one or more of hospital record data, feeding record data, and vital sign data along with stool images of infants to generate stool condition information for determining each condition of infants, and then use the stool image based on this. By determining the condition of infants and young children, it is possible to help in health management of infants and young children.

In addition, Korean Patent Laid-Open Publication No. KR 10-2020-0119135 A discloses a stool condition analysis method using a neural network module and a system therefor. Specifically, the prior literature provides temperature information and humidity information for a predetermined time period, respectively, from a temperature sensor, a humidity sensor, a gyro sensor, a gas sensor, and an image sensor installed at a mounting hole of an automatic excreta disposal device mounted on a patient's body. , a processing unit configured to collect the patient's posture information, gas information, and image information and perform a pre-processing step on the collected information, and a temperature value that is input information output from the processing unit after the pre-processing step is performed; Disclosed is a server including a neural network module that receives a humidity value, a posture value, a gas concentration value, a gas generation frequency, and an image value and outputs a stool condition value of the patient.

Through this, the prior literature can provide a method for analyzing a patient's stool condition using a neural network module and a server therefor.

However, prior literatures acquire stool images corresponding to identifiers, extract chunk objects, which are objects that are not completely digested, from stool images, classify chunk objects into incompletely digested food information, and classify chunk objects into incompletely digested food information. It does not disclose technical features for generating the necessary nutrient information corresponding to the identifier based on.

Republic of Korea Patent Publication KR 10-2020-0141776 A Republic of Korea Patent Publication KR 10-2020-0119135 A

Embodiments are intended to provide necessary nutritional information based on stool image analysis.

Embodiments attempt to provide recommended food information based on necessary nutritional information.

Embodiments are intended to provide feedback on a digestive state after eating food according to recommended food information.

A method for providing nutrition information based on stool image analysis according to an embodiment of the present invention includes acquiring stool images corresponding to identifiers; extracting lump objects, which are objects that are not fully digested, from the stool images; Incomplete digestion of the chunk objects by using a first neural network trained in advance based on first training chunk objects and first labels, wherein the first labels are information on foods corresponding to the training chunk objects. classifying into food information; and generating necessary nutrient information corresponding to the identifier based on incompletely digested food information corresponding to the chunk objects.

As an embodiment, the method may further include generating recommended food information corresponding to the identifier based on the necessary nutrient information.

As an example, acquiring at least one second stool image after eating food according to the recommended food information; determining a digestion state of food according to the recommended food information by using the second stool image; generating feedback information about food according to the recommended food information based on a digestion state of the food; and providing the feedback information to a user corresponding to the identifier.

As an embodiment, the obtaining of the feces images includes obtaining the feces images based on a predetermined period for obtaining feces images corresponding to the identifier and the number of feces images, and the predetermined period and The number of stool images may be predefined based on at least one of age, gender, breastfeeding status, and weight corresponding to the identifier.

As an embodiment, the extracting of the chunk objects may include detecting candidate objects having a specific pattern from the stool images; classifying the candidate objects as food or foreign substances using a pre-learned second neural network; and classifying the candidate objects classified as food into lump objects.

In one embodiment, obtaining first training stool images; extracting the first training chunk objects from the first training stool images; acquiring the first labels that are information of foods corresponding to the first training chunk objects; applying the first training chunk objects to the first neural network to generate training outputs corresponding to the first training chunk objects; and training the first neural network based on the training outputs and the first labels.

As an embodiment, the classifying of the incompletely digested food information may include acquiring sizes and numbers of the mass objects included in the stool images; and generating incompletely digested food information corresponding to the feces images by applying the obtained distribution information of sizes and numbers to a pretrained third neural network.

As one embodiment, obtaining training stool images; extracting the first training chunk objects from the training stool images; extracting first training distribution information, which is distribution information of sizes and numbers of the first training chunk objects, from the first training chunk objects; obtaining second labels that are incompletely digested food information corresponding to the first training distribution information; generating training outputs corresponding to the first training distribution information by applying the first training distribution information to the third neural network; and training the third neural network based on the training outputs and the second labels.

As an embodiment, the forming of the necessary nutrient information corresponding to the identifier may include assigning a weight to each of the chunk objects in consideration of sizes and numbers of the chunk objects included in the stool images; Calculating a digestibility index by summing up the assigned weights for each necessary nutrient corresponding to each of the chunk objects; and calculating a ranking of digestibility of the identifier based on the digestibility index.

The computer program according to an embodiment of the present invention may be stored in a computer readable recording medium to execute the method for providing necessary nutritional information based on feces image analysis on a computer.

An apparatus for providing required nutrition information based on stool image analysis according to an embodiment of the present invention includes: a memory for obtaining and storing stool images corresponding to identifiers; and extracting chunk objects, which are objects that are not fully digested, from the feces images, and first training chunk objects and first labels, wherein the first labels are information on foods corresponding to the training chunk objects. The chunk objects are classified into incompletely digested food information using a first neural network pre-learned based on , and necessary nutrient information corresponding to the identifier based on the incompletely digested food information corresponding to the chunk objects. It may include a processor that generates.

Embodiments provide necessary nutritional information based on stool image analysis results of multiple users, and provide recommended food information based on the necessary nutritional information, so that each user can consume appropriate foods. In addition, it is possible to improve the efficiency of food recommendation by providing feedback information using a stool image analysis result after the user consumes the recommended food.

1 is a diagram for explaining a stool image analysis-based nutrition information providing system according to an embodiment.
2 is a diagram for explaining learning of a neural network according to an embodiment of the present invention.
3 is an exemplary view of a stool image according to an embodiment of the present invention.
4 is a diagram for explaining a nutrition information providing device according to an embodiment.
5 is a flowchart of a method for providing necessary nutritional information according to an embodiment.
6 is an exemplary diagram of a configuration of a device according to an embodiment.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. However, since various changes can be made to the embodiments, the scope of the patent application is not limited or limited by these embodiments. It should be understood that all changes, equivalents or substitutes to the embodiments are included within the scope of rights.

Specific structural or functional descriptions of the embodiments are disclosed for illustrative purposes only, and may be modified and implemented in various forms. Therefore, the embodiments are not limited to the specific disclosed form, and the scope of the present specification includes changes, equivalents, or substitutes included in the technical spirit.

Although terms such as first or second may be used to describe various components, such terms should only be construed for the purpose of distinguishing one component from another. For example, a first element may be termed a second element, and similarly, a second element may be termed a first element.

It should be understood that when an element is referred to as being “connected” to another element, it may be directly connected or connected to the other element, but other elements may exist in the middle.

Terms used in the examples are used only for descriptive purposes and should not be construed as limiting. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, terms such as "include" or "have" are intended to designate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, but one or more other features It should be understood that the presence or addition of numbers, steps, operations, components, parts, or combinations thereof is not precluded.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the art to which the embodiment belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and unless explicitly defined in the present application, they should not be interpreted in an ideal or excessively formal meaning. don't

In addition, in the description with reference to the accompanying drawings, the same reference numerals are given to the same components regardless of reference numerals, and overlapping descriptions thereof will be omitted. In describing the embodiment, if it is determined that a detailed description of a related known technology may unnecessarily obscure the gist of the embodiment, the detailed description will be omitted.

The embodiments may be implemented in various types of products such as personal computers, laptop computers, tablet computers, smart phones, televisions, smart home appliances, intelligent vehicles, kiosks, and wearable devices.

An artificial intelligence (AI) system is a computer system that implements human-level intelligence, and unlike existing rule-based smart systems, machines learn and judge on their own. The more AI systems are used, the higher the recognition rate and the more accurate understanding of user preferences. Conventional rule-based smart systems are gradually being replaced by deep learning-based AI systems.

Artificial intelligence technology consists of machine learning and element technologies using machine learning. Machine learning is an algorithm technology that classifies/learns the characteristics of input data by itself, and element technology is a technology that uses machine learning algorithms such as deep learning to mimic functions such as recognition and judgment of the human brain. It consists of technical fields such as understanding, inference/prediction, knowledge expression, and motion control.

The various fields where artificial intelligence technology is applied are as follows. Linguistic understanding is a technology for recognizing and applying/processing human language/characters, including natural language processing, machine translation, dialogue systems, question and answering, voice recognition/synthesis, and the like. Visual understanding is a technology for recognizing and processing objects like human vision, and includes object recognition, object tracking, image search, person recognition, scene understanding, space understanding, image improvement, and the like. Inference prediction is a technique of reasoning and predicting logically by judging information, and includes knowledge/probability-based reasoning, optimization prediction, preference-based planning, and recommendation. Knowledge expression is a technology that automatically processes human experience information into knowledge data, and includes knowledge construction (data creation/classification) and knowledge management (data utilization). Motion control is a technology for controlling the autonomous driving of a vehicle and the movement of a robot, and includes motion control (navigation, collision, driving), manipulation control (action control), and the like.

In general, in order to apply machine learning algorithms to real life, learning is performed in a trial and error method due to the nature of the basic methodology of machine learning. In particular, deep learning requires hundreds of thousands of iterations. It is impossible to execute this in an actual physical external environment, so instead, the actual physical external environment is virtually implemented on a computer and learning is performed through simulation.

1 is a diagram for explaining a stool image analysis-based nutrition information providing system according to an embodiment.

As shown in FIG. 1, the stool image analysis-based nutritional information providing system 100 includes a plurality of user terminals 110-1, 110-2, ..., 110-n, a nutritional information providing device 120, and a database. (130) and network (N). According to one embodiment, the database 130 is illustrated as being configured separately from the nutrition information providing device 120, but is not limited thereto, and the database 130 may be included in the nutrition information providing device 120. For example, the nutritional information providing device 120 may include a plurality of neural networks for performing a machine learning algorithm.

The nutrition information providing device 120 provides feces for a preset period of time from any one user terminal (eg, 110-1) among a plurality of user terminals 110-1, 110-2, ..., 110-n. images can be obtained. According to an embodiment, the nutrition information providing device 120 may use a predetermined period (eg, 3 days, 1 week, 1 month, etc.) and the number of stool images (eg, 3 days, 1 week, 1 month, etc.) to obtain stool images corresponding to the identifier. 10, 15, 100, etc.), the stool images may be obtained, and the predetermined period and the number of stool images may be determined in advance based on at least one of age, gender, breastfeeding status, and weight corresponding to the identifier. can be defined For example, the nutrition information providing device 120 acquires first training stool images, extracts first training chunk objects from the first training stool images, and provides information on foods corresponding to the first training chunk objects. Obtain first labels of , apply the first training chunk objects to the first neural network, generate training outputs corresponding to the first training chunk objects, and based on the training outputs and the first labels, 1 You can train a neural network.

The nutrition information providing device 120 may extract lump objects, which are objects that are not completely digested, from feces images obtained from the user terminal 110-1. According to an embodiment, the nutrition information providing device 120 detects candidate objects having a specific pattern from stool images, classifies the candidate objects as food or foreign substances using a pre-learned second neural network, and Candidate objects classified as can be classified as chunk objects.

The nutrition information providing apparatus 120 may classify chunk objects into incompletely digested food information by using a first neural network previously learned based on the first training chunk objects and the first labels. According to an embodiment, the first labels may indicate information on foods corresponding to training chunk objects. For example, the nutrition information providing apparatus 120 obtains the sizes and numbers of mass objects included in stool images, and applies distribution information of the obtained sizes and numbers to a pretrained neural network to obtain stool images. Incompletely digested food information corresponding to them may be generated. According to another embodiment, the nutritional information providing apparatus 120 extracts first training chunk objects from training stool images, and distribution information of sizes and numbers of first training chunk objects from the first training chunk objects. Extract first training distribution information, obtain second labels that are incompletely digested food information corresponding to the first training distribution information, apply the first training distribution information to a neural network, and train training corresponding to the first training distribution information Outputs may be generated, and a third neural network may be trained based on the training outputs and the second labels.

The nutrition information providing device 120 may generate necessary nutrient information corresponding to an identifier based on incompletely digested food information corresponding to chunk objects. According to an embodiment, the nutrition information providing apparatus 120 assigns a weight to each of the lump objects in consideration of the sizes and numbers of the lump objects included in the feces images, and determines the necessity corresponding to each of the lump objects. The digestibility index is calculated by adding the weights assigned to each nutrient, and the digestibility ranking of the identifier may be calculated based on the digestibility index.

The nutrition information providing device 120 may generate recommended food information corresponding to the identifier based on the necessary nutrient information. According to an embodiment, the nutrition information providing device 120 may generate recommended food information using a mapping table stored in the database 130 or may generate recommended food information using a pretrained neural network. there is. For example, recommended food information may include information on health functional foods, health foods, and alternative foods. Health functional food refers to a food that can be easily consumed by processing ingredients that can help health, health food refers to food eaten to maintain and promote health, and substitute food refers to a food that replaces a specific food It can indicate foods that can be eaten.

The nutrition information providing device 120 acquires at least one second stool image after eating the food according to the recommended food information, determines the digestion state of the food according to the recommended food information using the second stool image, and , Based on the digestive state of the food, it is possible to generate feedback information about the food according to the recommended food information, and provide the feedback information to the user corresponding to the identifier.

The network N may enable wireless or wired communication between the plurality of user terminals 110-1, 110-2, ..., 110-n, the nutritional information providing device 120, the database 130, and the like. there is. For example, the network may include long-term evolution (LTE), LTE Advanced (LTE-A), code division multiple access (CDMA), wideband CDMA (WCDMA), wireless broadband (WiBro), wireless fidelity (WiFi), Bluetooth ( Bluetooth), near field communication (NFC), global positioning system (GPS), or global navigation satellite system (GNSS). For example, the network may perform wired communication according to a method such as universal serial bus (USB), high definition multimedia interface (HDMI), recommended standard 232 (RS-232), or plain old telephone service (POTS). .

The database 130 may store various data. Data stored in the database 130 is acquired, processed, or obtained by at least one component of the plurality of user terminals 110-1, 110-2, ..., 110-n and the nutritional information providing device 120. , as used data, may include software (eg: program). Database 130 may include volatile and/or non-volatile memory. As an embodiment, the database 130 includes stool images obtained from a plurality of user terminals 110-1, 110-2, ..., 110-n, lump objects formed by the nutrition information providing device 120, and incomplete Digested food information, necessary nutrient information, etc. can be stored.

In the present invention, the program is software stored in the database 130, and an operating system for controlling resources of the plurality of user terminals 110-1, 110-2, ..., 110-n and the nutritional information providing device 120. , Middleware that provides various functions to applications so that applications and/or applications can utilize resources of a plurality of user terminals (110-1, 110-2, ..., 110-n) and nutritional information providing device 120, etc. can include

In the present invention, artificial intelligence (AI) means a technology that imitates human learning ability, reasoning ability, perception ability, etc., and implements it with a computer, and concepts such as machine learning and symbolic logic can include Machine learning (ML) is an algorithm technology that classifies or learns the characteristics of input data by itself. Artificial intelligence technology is a machine learning algorithm that analyzes input data, learns the result of the analysis, and can make judgments or predictions based on the result of the learning. In addition, technologies that mimic the functions of the human brain, such as recognition and judgment, using machine learning algorithms, can also be understood as artificial intelligence. For example, technical fields such as linguistic understanding, visual understanding, inference/prediction, knowledge expression, and motion control may be included.

Machine learning may refer to processing that trains a neural network model using experience of processing data. Through machine learning, computer software could mean improving its own data processing capabilities. A neural network model is constructed by modeling a correlation between data, and the correlation may be expressed by a plurality of parameters. The neural network model derives a correlation between data by extracting and analyzing features from given data, and optimizing the parameters of the neural network model by repeating this process can be referred to as machine learning. For example, a neural network model may learn a mapping (correlation) between an input and an output with respect to data given as an input/output pair. Alternatively, even when only input data is given, the neural network model may learn the relationship by deriving a regularity between given data.

An artificial intelligence learning model or a neural network model may be designed to implement a human brain structure on a computer, and may include a plurality of network nodes having weights while simulating neurons of a human neural network. A plurality of network nodes may have a connection relationship between them by simulating synaptic activities of neurons that transmit and receive signals through synapses. In the artificial intelligence learning model, a plurality of network nodes can send and receive data according to a convolutional connection relationship while being located in layers of different depths. The artificial intelligence learning model may be, for example, an artificial neural network model, a convolutional neural network model (CNN), and the like. As an embodiment, the artificial intelligence learning model may be machine-learned according to methods such as supervised learning, unsupervised learning, and reinforcement learning. Machine learning algorithms for performing machine learning include Decision Tree, Bayesian Network, Support Vector Machine, Artificial Neural Network, Ada-boost , Perceptron, Genetic Programming, Clustering, etc. may be used.

Of these, CNNs are a type of multilayer perceptrons designed to use minimal preprocessing. A CNN consists of one or several convolution layers and general artificial neural network layers on top, and additionally utilizes weights and pooling layers. Thanks to this structure, CNN can fully utilize the input data with a two-dimensional structure. Compared to other deep learning structures, CNN shows good performance in both video and audio fields. CNNs can also be trained via standard back-propagation. CNNs are easier to train than other feedforward artificial neural network techniques and have the advantage of using fewer parameters.

Convolutional networks are neural networks that contain sets of nodes with bound parameters. The increasing size of available training data and the availability of computational power, combined with algorithmic advances such as piecewise linear unit and dropout training, have greatly improved many computer vision tasks. With huge data sets, such as those available for many tasks today, overfitting is not critical, and increasing the size of the network improves test accuracy. Optimal use of computing resources becomes a limiting factor. To this end, a distributed, scalable implementation of deep neural networks can be used.

2 is a diagram for explaining learning of a neural network according to an embodiment of the present invention.

As shown in FIG. 2 , the learning device may train the neural network 124 to extract mass objects included in stool images. According to one embodiment, the learning device may be a separate subject different from the nutrition information providing device 120, but is not limited thereto.

The neural network 124 includes an input layer 122 into which training samples are input and an output layer 126 which outputs training outputs, and may be learned based on differences between the training outputs and labels. Here, the labels may be defined based on chunk objects, information on foods corresponding to the chunk objects, and incompletely digested food information. The neural network 124 is connected to a group of a plurality of nodes, and is defined by weights between connected nodes and an activation function that activates the nodes.

The learning device may train the neural network 124 using a gradient descent (GD) technique or a stochastic gradient descent (SGD) technique. The learning device may use a loss function designed by outputs and labels of the neural network.

The learning device may calculate a training error using a predefined loss function. The loss function may be predefined with labels, outputs and parameters as input variables, where the parameters may be set by weights in the neural network 124. For example, the loss function may be designed in a mean square error (MSE) form, an entropy form, and the like, and various techniques or methods may be employed in an embodiment in which the loss function is designed.

The learning device may find weights affecting the training error using a backpropagation technique. Here, weights are relationships between nodes in neural network 124 . The learning device may use the SGD technique using labels and outputs to optimize the weights found through the backpropagation technique. For example, the learning device may update weights of a loss function defined based on labels, outputs, and weights using the SGD technique.

According to an embodiment, the learning device may obtain first training stool images, and extract first training chunk objects from the first training stool images. The learning device may obtain pre-labeled information (first labels) for each of the first training chunk objects, which is information on foods corresponding to the first training chunk objects.

According to an embodiment, the learning device may generate first training feature vectors based on appearance features, pattern features, and color features of the first training chunk objects. Various methods may be employed to extract features.

According to an embodiment, the learning device may obtain training outputs by applying the first training feature vectors to the neural network 124 . The learning device may train the neural network 124 based on the training outputs and the first labels. The learning device may train the neural network 124 by calculating training errors corresponding to training outputs and optimizing a connection relationship of nodes in the neural network 124 to minimize the training errors. The nutritional information providing device 120 may extract lump objects from stool images using the neural network 124 that has been trained.

According to another embodiment, the learning device may acquire second training stool images, and extract second training chunk objects from the second training stool images. The learning device may acquire pre-labeled information (second labels) for each of the second training chunk objects, which is information on food or foreign substances corresponding to the second training chunk objects. there is.

According to another embodiment, the learning device may generate second training feature vectors based on appearance features, pattern features, and color features of the second training chunk objects. Various methods may be employed to extract features.

According to another embodiment, the learning device may obtain training outputs by applying the second training feature vectors to the neural network 124 . The learning device may train the neural network 124 based on the training outputs and the second labels. The learning device may train the neural network 124 by calculating training errors corresponding to training outputs and optimizing a connection relationship of nodes in the neural network 124 to minimize the training errors. The nutritional information providing device 120 may extract lump objects from stool images using the neural network 124 that has been trained.

According to another embodiment, the learning device may acquire third training stool images, and extract third training chunk objects from the third training stool images. The learning device may extract first training distribution information, which is distribution information of sizes and numbers of third training chunk objects, from the third training chunk objects. The learning device may obtain pre-labeled information (third labels) for each of the first training distribution information, which is incompletely digested food information corresponding to the first training distribution information.

According to another embodiment, the learning device may generate third training feature vectors based on the appearance features, pattern features, and color features of the first training distribution information. Various methods may be employed to extract features.

According to another embodiment, the learning device may obtain training outputs by applying the third training feature vectors to the neural network 124 . The learning device may train the neural network 124 based on the training outputs and the third labels. The learning device may train the neural network 124 by calculating training errors corresponding to training outputs and optimizing a connection relationship of nodes in the neural network 124 to minimize the training errors. The nutritional information providing device 120 may extract incompletely digested food information from stool images using the trained neural network 124 .

According to another embodiment, the learning device may acquire fourth training stool images, and extract fourth training chunk objects from the fourth training stool images. The learning device may extract fourth training incompletely digested food information from the fourth training chunk objects. The learning device may acquire pre-labeled information (fourth labels) for each of the fourth training incompletely digested food information, including a health functional food and health food corresponding to the fourth training incompletely digested food information. , it is possible to obtain fourth labels that are information on alternative foods. Health functional food refers to a food that can be easily consumed by processing ingredients that can help health, health food refers to food eaten to maintain and promote health, and substitute food refers to a food that replaces a specific food It can indicate foods that can be eaten.

According to another embodiment, the learning device may generate fourth training feature vectors based on appearance features, pattern features, and color features of the fourth training fourth training incompletely digested food information. Various methods may be employed to extract features.

According to another embodiment, the learning device may obtain training outputs by applying the fourth training feature vectors to the neural network 124 . The learning device may train the neural network 124 based on the training outputs and the fourth labels. The learning device may train the neural network 124 by calculating training errors corresponding to training outputs and optimizing a connection relationship of nodes in the neural network 124 to minimize the training errors. The nutritional information providing device 120 may extract necessary nutritional information from stool images using the trained neural network 124 .

3 is an exemplary view of a stool image according to an embodiment of the present invention.

As shown in FIG. 3, the nutrition information providing device 120 is an object that has not been completely digested from stool images obtained from a plurality of user terminals 110-1, 110-2, ..., 110-n. The imported chunk objects (MO) may be extracted. According to an embodiment, the nutrition information providing device 120 detects candidate objects having a specific pattern from acquired stool images, classifies the candidate objects as food or foreign substances using a pre-learned neural network, and Those classified as may be extracted as lump objects (MO).

Depending on the state in which teeth and digestive organs are formed during the growth process of infants, food to be digested may vary from infant to infant. Some infants cannot digest broccoli, and some infants cannot digest carrots. Foods that infants cannot digest are excreted as feces, and the nutrition information providing device 120 may extract such undigestible foods from the stool image. According to an embodiment, the nutrition information providing device 120 may label and learn the classification of lump objects based on a machine learning algorithm such as deep learning, and perform classification according to the learned model. there is.

When a plurality of stool images are acquired, the nutrition information providing device 120 receives a first part stool image from the first user terminal (eg, 110-1), and provides a second user account logged in. The second part feces image is received from the user terminal (eg, 110-2), the first part feces image and the second part feces image are combined to generate a first feces image, and a first feces image is obtained from the first feces image. A first ratio, which is the ratio occupied by the part stool image, is obtained, and a second ratio, which is the ratio occupied by the second part stool image in the first stool image, is obtained, and when it is confirmed that the first ratio is greater than the second ratio, the first ratio is When the information of the user account is registered as the first author information of the first credit image, the information of the second user account is registered as the second author information of the first credit image, and the first ratio is smaller than the second ratio , Register the information of the second user account as the first author information of the first stool image, register the information of the first user account as the second author information of the first stool image, and register the information of the original author of the first stool image and the first credit image. It is checked whether any one of the information of the first user account and the information of the second user account matches, and the information of the original author of the first credit image, the information of the first user account, and the information of the second user account are all inconsistent. If confirmed, the first feces image is determined as a secondary work created by modification of the second feces image, and if the first rate is greater than the second rate, the information of the first user account is determined as the second work created by the modification of the second feces image. 1 Register the author information, register the information of the second user account as the second author information of the first credit image, and register the original author of the first credit image to the third user terminal (eg, 110-n) The information of the logged-in third user account is registered as third author information representing the third author of the first credit image, and when it is confirmed that the first ratio is smaller than the second ratio, the information of the second user account is registered as the first credit ratio. Register information of the first author of the image, register information of the first user account as information of the second author of the first credit image, and register information of the third user account as information of the third author of the first credit image. there is.

4 is a diagram for explaining a nutrition information providing device according to an embodiment.

As shown in FIG. 4 , the nutrition information providing device 120 may include one or more processors 121 , one or more memories 123 , and/or a transceiver 125 . As an example, at least one of these components of the nutritional information providing device 120 may be omitted or another component may be added to the nutritional information providing device 120 . Additionally or alternatively, some of the components may be integrated and implemented, or implemented as a singular or plural entity. At least some of the internal and external components of the nutritional information providing device 120 communicate with each other through a bus, general purpose input/output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI). Being connected, data and/or signals can be sent and received.

One or more processors 121 may control at least one component of the nutritional information providing device 120 connected to the processor 121 by driving software (eg, instructions, programs, etc.). In addition, the processor 121 may perform operations such as various calculations, processing, data generation, and processing related to the present invention. Also, the processor 121 may load data or the like from one or more memories 123 or store data in one or more memories 123 .

As described above, the one or more processors 121, via the transceiver 125, a plurality of user terminals (110-1, 110-2, ..., 110-n) any one of the user terminal (eg, 110 Feces images from -1) can be received in real time or non-real time in the form of digital packets.

The one or more processors 121 may extract lump objects, which are objects that are not completely digested, using feces images received from the user terminal 110 - 1 through the transceiver 125 . According to one embodiment, one or more processors 121 obtains first training stool images, extracts first training chunk objects from the first training stool images, and sets food items corresponding to the first training chunk objects. Obtain first labels that are information, apply the first training chunk objects to the first neural network, generate training outputs corresponding to the first training chunk objects, and based on the training outputs and the first labels, A first neural network may be trained to extract lump objects from stool images.

The one or more processors 121 may classify the chunk objects into incompletely digested food information by using a first neural network pretrained based on the first training chunk objects and the first labels. For example, the first labels may indicate information on foods corresponding to training chunk objects.

One or more processors 121 may generate necessary nutrient information corresponding to the identifier based on incompletely digested food information corresponding to the chunk objects.

One or more memories 123 may store various data. Data stored in the memory 123 is data acquired, processed, or used by at least one component of the nutritional information providing device 120, and may include software (eg, commands, programs, etc.) . Memory 123 may include volatile and/or non-volatile memory. In the present invention, commands or programs are software stored in the memory 123, and the operating system, application and/or application for controlling the resources of the nutritional information providing device 120 utilize the resources of the nutritional information providing device 120. It can include middleware that provides various functions to applications so that

One or more memories 123 store stool images received from the above-described plurality of user terminals 110-1, 110-2, ..., 110-n through the network N, and a mass formed by one or more processors 121. Objects, incompletely digested food information, necessary nutrient information, and the like can be stored. In addition, the one or more memories 123 may store instructions that cause the one or more processors 121 to perform calculations when executed by the one or more processors 121 .

As an embodiment, the nutritional information providing device 120 may further include a transceiver 125 . The transceiver 125 enables wireless or wired communication between a plurality of user terminals 110-1, 110-2, ..., 110-n, the nutritional information providing device 120, the database 130, and/or other devices. can be done For example, the transceiver 126 may transmit eMBB (enhanced mobile broadband), URLLC (Ultra Reliable Low-Latency Communications), MMTC (Massive Machine Type Communications), LTE (long-term evolution), LTE-A (LTE Advance), UMTS (Universal Mobile Telecommunications System), GSM (Global System for Mobile communications), CDMA (code division multiple access), WCDMA (wideband CDMA), WiBro (Wireless Broadband), WiFi (wireless fidelity), Bluetooth (Bluetooth), NFC ( Wireless communication may be performed according to a method such as near field communication (GPS), global positioning system (GPS), or global navigation satellite system (GNSS). For example, the transceiver 125 may perform wired communication according to a method such as universal serial bus (USB), high definition multimedia interface (HDMI), recommended standard 232 (RS-232), or plain old telephone service (POTS). there is.

As an embodiment, one or more processors 121 control the transceiver 125 to obtain information from a plurality of user terminals 110-1, 110-2, ..., 110-n and the nutritional information providing device 120. can do. Information obtained from the plurality of user terminals 110-1, 110-2, ..., 110-n and the nutritional information providing device 120 may be stored in one or more memories 123.

As an example, the nutrition information providing device 120 may be various types of devices. For example, the nutritional information providing device 120 may be a portable communication device, a computer device, or a device according to a combination of one or more of the foregoing devices. The nutritional information providing device 120 of the present invention is not limited to the above devices.

Various embodiments of the nutrition information providing device 120 according to the present invention may be combined with each other. Each embodiment may be combined according to the number of cases, and the embodiments of the nutritional information providing device 120 made in combination also fall within the scope of the present invention. In addition, internal/external components of the nutrition information providing device 120 according to the present invention described above may be added, changed, replaced, or deleted according to embodiments. In addition, internal/external components of the nutritional information providing device 120 described above may be implemented as hardware components.

According to an embodiment, the nutritional information providing device 120 may collect stool images of a specific infant A. For example, the nutritional information providing device 120 may designate a period to collect feces images for a corresponding period, or may designate a number to collect feces images corresponding to the corresponding number. The nutritional information providing device 120 may form a stool list (including M pieces of stool information) of a specific infant A by using the collected stool images, and may extract lump objects from each of the collected stool images. . The lump objects may represent pieces of food that are not completely digested by the particular infant (A).

The nutritional information providing device 120 may detect a specific pattern region based on the color or shape of the collected stool images. The nutrition information providing device 120 continuously determines food that is incompletely digested (eg, carrots, broccoli, etc.) Nutrient information can be derived. According to an embodiment, the nutrition information providing device 120 may perform labeling of food for each chunk object and classify the chunk objects based on color, shape, etc. using a machine learning algorithm such as deep learning.

The nutrition information providing device 120 may recommend (curate) an optimal health functional food or alternative food combination for a specific infant (A) from the derived nutritional information. The nutrition information providing device 120 may reanalyze the digestion state of the optimal health functional food or alternative food combination and provide the result as feedback to the guardian of the specific infant A.

The nutrition information providing device 120 may derive nutrient information required for a specific infant A based on the size and number of repeated lump objects based on the distribution of lump objects extracted from the collected stool images.

5 is a flowchart of a method for providing necessary nutritional information according to an embodiment.

Although process steps, method steps, algorithms, etc. are described in a sequential order in the flowchart of FIG. 5, such processes, methods and algorithms may be configured to operate in any suitable order. In other words, the steps of the processes, methods and algorithms described in the various embodiments of the invention need not be performed in the order described herein. Also, although some steps are described as being performed asynchronously, in other embodiments some of these steps may be performed concurrently. Further, illustration of a process by depiction in the drawings does not mean that the illustrated process is exclusive of other changes and modifications thereto, and that any of the illustrated process or steps thereof may be one of various embodiments of the present invention. It is not meant to be essential to one or more, and it does not imply that the illustrated process is preferred.

As shown in FIG. 5, in step S510, stool images corresponding to the identifier are obtained. For example, referring to FIGS. 1 to 4 , the nutrition information providing device 120 is configured by one of a plurality of user terminals 110-1, 110-2, ..., 110-n (eg, For example, stool images may be obtained from 110-1) through the network N.

In step S520, lump objects, i.e. objects that are not completely digested, are extracted from the stool images. For example, referring to FIGS. 1 to 4 , the nutrition information providing device 120 may extract lump objects, which are objects that are not fully digested, from the stool images acquired in step S510. According to an embodiment, the nutrition information providing apparatus 120 may extract lump objects from stool images using a pretrained neural network.

In operation S530, the chunk objects are classified into incompletely digested food information using a first neural network previously learned based on the first training objects and the first labels. For example, referring to FIGS. 1 to 4 , the nutritional information providing apparatus 120 uses a first neural network trained in advance based on first training objects and first labels to perform extraction in step S520. The lumped objects may be classified as incompletely digested food information.

In step S540, necessary nutrient information corresponding to the identifier is formed based on the incompletely digested food information corresponding to the chunk objects. For example, referring to FIGS. 1 to 4 , the nutritional information providing device 120 provides necessary nutrient information corresponding to a specific identifier based on incompletely digested food information corresponding to the lump objects classified in step S530. can create

6 is an exemplary diagram of a configuration of a device according to an embodiment.

Device 401 according to an embodiment includes a processor 402 and a memory 403 . The device 401 according to an embodiment may be the above-described server or terminal. The processor may include at least one device described above with reference to FIGS. 1 to 5 or may perform at least one method described above with reference to FIGS. 1 to 5 . The memory 403 may store information related to the above method or store a program in which the above method is implemented. Memory 403 may be volatile memory or non-volatile memory.

The processor 402 may execute a program and control the device 401 . Codes of programs executed by the processor 402 may be stored in the memory 403 . The device 401 may be connected to an external device (eg, a personal computer or network) through an input/output device (not shown) and exchange data.

The embodiments described above may be implemented as hardware components, software components, and/or a combination of hardware components and software components. For example, the devices, methods and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate (FPGA). array), programmable logic units (PLUs), microprocessors, or any other device capable of executing and responding to instructions. A processing device may run an operating system (OS) and one or more software applications running on the operating system. A processing device may also access, store, manipulate, process, and generate data in response to execution of software. For convenience of understanding, there are cases in which one processing device is used, but those skilled in the art will understand that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that it can include. For example, a processing device may include a plurality of processors or a processor and a controller. Other processing configurations are also possible, such as parallel processors.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. Program commands recorded on the medium may be specially designed and configured for the embodiment or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. - includes hardware devices specially configured to store and execute program instructions, such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter, as well as machine language codes such as those produced by a compiler. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

Software may include a computer program, code, instructions, or a combination of one or more of the foregoing, which configures a processing device to operate as desired or processes independently or collectively. The device can be commanded. Software and/or data may be any tangible machine, component, physical device, virtual equipment, computer storage medium or device, intended to be interpreted by or provide instructions or data to a processing device. , or may be permanently or temporarily embodied in a transmitted signal wave. Software may be distributed on networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer readable media.

As described above, although the embodiments have been described with limited drawings, those skilled in the art can apply various technical modifications and variations based on the above. For example, the described techniques may be performed in an order different from the method described, and/or components of the described system, structure, device, circuit, etc. may be combined or combined in a different form than the method described, or other components may be used. Or even if it is replaced or substituted by equivalents, appropriate results can be achieved.

Therefore, other implementations, other embodiments, and equivalents of the claims are within the scope of the following claims.

Claims (1)

In a method performed by a processor,
obtaining stool images corresponding to the identifier;
extracting lump objects, which are objects that are not fully digested, from the stool images;
Incomplete digestion of the chunk objects by using a first neural network trained in advance based on first training chunk objects and first labels, wherein the first labels are information on foods corresponding to the training chunk objects. classifying into food information; and
Generating recommended food information corresponding to the identifier based on incompletely digested food information corresponding to the chunk objects,
A method for providing necessary nutritional information based on stool image analysis.

Images