KR102683228B1 - A method, device and program for measuring food - Google Patents

Abstract

The present invention relates to a method of measuring food. The present invention calculates the amount of food based on information on the size, depth and capacity of a plurality of spaces formed in a dish containing food and images taken from a photographing means.

Description

{A method, device and program for measuring food}

The present invention relates to a method of measuring food, and specifically to a method of measuring the amount of food using volume.

Recently, as the number of people trying to maintain a healthy diet, such as well-being and diet, has increased, the demand for food measurement technology is increasing.

Using this, even in places where food is served to multiple people, such as schools, businesses, military, government offices, hospitals, etc., the amount of food served to people is measured and the amount of food served and the amount of leftover food are measured, thereby measuring demand and supply. There are many advantages, such as efficient distribution management in anticipation of food distribution and the ability to manage the calories of those receiving food.

However, since most of the currently disclosed technologies are limited to simple image search of images captured through cameras, their accuracy is significantly reduced, and as the accuracy in image search is low, larger errors occur in the next steps such as calorie calculation. There is a problem with doing so.

Accordingly, the present inventor came up with an invention that can accurately analyze and calculate food rather than simply searching for food through images.

Republic of Korea Patent Publication No. 10-2005-0083210 (2005.08.26)

The present invention to solve the problems described above can provide a food measurement method that calculates the volume of food using height information for each pixel of an image captured through a stereo camera or depth measuring device.

Additionally, the present invention can provide a food measurement method that corrects the volume of food calculated using information on the serving dish.

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned can be clearly understood by those skilled in the art from the description below.

A food measuring method according to an embodiment of the present invention for solving the above-described problem includes the steps of an analysis unit analyzing an image of a serving dish (a dish served to a user), and recognizing one or more foods contained in the dish; A calculation unit calculating the volume of each food using dishware information and the image of the food in the dishware image; and calculating the weight of each food using the food information and the calculated volume of each food, wherein the server determines the size, depth, and It includes a database storing tableware information including capacity and food information including weight per volume for each food.

In addition, the food measurement server according to an embodiment of the present invention to solve the above-mentioned problem, dishes information including an image of empty dishes, the size, depth and capacity of a plurality of spaces formed in the dishes, and the weight per volume of each food. A database storing food information, including: An analysis unit that analyzes images of serving dishes (tableware served by the user) and recognizes one or more foods contained in the dishes; and a calculation unit that calculates the volume of each food using the dish information and the food image in the dish image, and calculates the weight of each food using the food information and the calculated volume of each food.

In addition, the food measuring device according to an embodiment of the present invention for solving the above-described problem includes an image of empty dishes, dishware information including the size, depth, and capacity of a plurality of spaces formed in the dishes, and weight per volume for each food. A database storing food information, including: An analysis unit that analyzes images of serving dishes (tableware served by the user) and recognizes one or more foods contained in the dishes; and a calculation unit that calculates the volume of each food using the dish information and the food image in the dish image, and calculates the weight of each food using the food information and the calculated volume of each food.

In addition to this, another method for implementing the present invention, another system, and a computer-readable recording medium recording a computer program for executing the method may be further provided.

According to the present invention as described above, there is an effect of calculating the volume of food accurately by calculating the volume of food using height information for each pixel of an image captured through a stereo camera or depth measuring device.

In addition, the present invention has the effect of calculating the volume of food more accurately by correcting the volume of food calculated through information about the size, depth, and capacity of the space for serving dishes.

The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

1 is a flowchart of a food measuring method according to an embodiment of the present invention.
Figure 2A is a perspective view of tableware according to an embodiment of the present invention.
Figure 2B is a perspective view of a tableware with an identification code attached to the back or side.
Figure 3 is a perspective view of a tableware containing food according to an embodiment of the present invention.
Figure 4 is a perspective view of the tableware in Figure 3 with two foods stacked in one space.
Figure 5 is a perspective view of tableware without spatial division.
Figure 6 is a flowchart of a method for providing an advanced local model to increase accuracy in determining the type of food.
Figure 7 is a flow chart of a food measuring method according to another embodiment of the present invention.
Figure 8 is a perspective view of a table for explaining a background target.
Figure 9 is a flowchart of a method of providing restaurant operation information and management information of diners according to an embodiment of the present invention.
Figure 10 is a block diagram of a food measurement server according to an embodiment of the present invention.
Figure 11 is a detailed block diagram of the management unit of Figure 10.
Figure 12 is a block diagram of a food measuring device according to an embodiment of the present invention.
Figures 13 to 16 are perspective views of a food measuring device according to an embodiment of the present invention.
Figure 17 is a screen of a food measuring device according to an embodiment of the present invention.
Figure 18 is a block diagram of a food measurement server according to an embodiment of the present invention.
Figure 19 is a block diagram of a user terminal according to an embodiment of the present invention.

The advantages and features of the present invention and methods for achieving them will become clear by referring to the embodiments described in detail below along with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below and may be implemented in various different forms. The present embodiments are merely provided to ensure that the disclosure of the present invention is complete and to provide a general understanding of the technical field to which the present invention pertains. It is provided to fully inform the skilled person of the scope of the present invention, and the present invention is only defined by the scope of the claims.

The terminology used herein is for describing embodiments and is not intended to limit the invention. As used herein, singular forms also include plural forms, unless specifically stated otherwise in the context. As used in the specification, “comprises” and/or “comprising” does not exclude the presence or addition of one or more other elements in addition to the mentioned elements. Like reference numerals refer to like elements throughout the specification, and “and/or” includes each and every combination of one or more of the referenced elements. Although “first”, “second”, etc. are used to describe various components, these components are of course not limited by these terms. These terms are merely used to distinguish one component from another. Therefore, it goes without saying that the first component mentioned below may also be a second component within the technical spirit of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used in this specification may be used with meanings commonly understood by those skilled in the art to which the present invention pertains. Additionally, terms defined in commonly used dictionaries are not to be interpreted ideally or excessively unless clearly specifically defined.

Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings.

Prior to explanation, the meaning of terms used in this specification will be briefly explained. However, since the explanation of terms is intended to aid understanding of the present specification, it should be noted that if it is not explicitly described as limiting the present invention, it is not used in the sense of limiting the technical idea of the present invention.

With reference to Figures 1 to 5, a food measuring method according to an embodiment of the present invention will be described. The food measurement method according to an embodiment of the present invention is performed by a computer, where the computer refers to the food measurement server 100 or the food measurement device 200. That is, filming is performed by the food measurement device 200, but other steps may be performed by the food measurement server 100 or the food measurement device 200.

Figure 1 is a flow chart of a food measuring method according to an embodiment of the present invention, Figure 2A is a perspective view of a tableware according to an embodiment of the present invention, and Figure 2B is a perspective view of a tableware with an identification code attached to the back or side. , Figure 3 is a perspective view of a tableware containing food according to an embodiment of the present invention, Figure 4 is a perspective view of a tableware with two foods stacked in one space in Figure 3, and Figure 5 is a tableware with no space division. It is a perspective view, and Figure 6 is a flowchart of a method of providing an advanced local model to increase accuracy in determining the type of food.

The food measuring method according to an embodiment of the present invention can be performed at the distribution or dispensing port of a restaurant, where the restaurant is a self-service restaurant such as a cafeteria or buffet, and the user eats while moving the utensils 500 to receive food. It may be a place that provides meals to groups such as schools, companies, or the military. Here, the restaurant may have a distribution port for receiving food or utensils 500 and a dispensing port for discarding leftover food and returning the utensils 500 after a meal, but is not limited to this.

However, the food measuring method according to an embodiment of the present invention is not limited to being performed in a restaurant, especially a self-service restaurant, and may be performed in other types of restaurants or at home, and may be performed at each location.

In addition, for convenience of explanation, the description will be made using the tableware 500 in the shape of a plate with a space 510 defined therein, but the shape of the tableware 500 is not limited thereto.

Referring to Figure 1, the receiver 110 receives an image of a tableware containing food photographed through the photographing unit 250 of the food measuring device 200 (step S20).

The receiving unit 110 normalizes the image of the received tableware containing food, and normalization adjusts or transforms the image of the received tableware containing food according to predetermined standards (e.g., size, brightness, rotation, tilt, etc.). means that

There are no restrictions on the configuration of the photographing unit 250 as long as it can acquire image data and depth data. For example, the photographing unit 250 may include at least one of an RGB camera, a 2D camera, a 3D camera, a Time of Flight (ToF) camera, a light field camera, a stereo camera, an event camera, and an infrared camera. , there are no restrictions on the configuration as long as image data and depth data can be measured.

Meanwhile, in addition to taking pictures of food or dishes containing food, the photographing unit 250 may take pictures of biometric information of the person eating the meal to identify the person eating the meal. Here, the biometric information of the eating subject may be, but is not limited to, the eating subject's face, iris, or fingerprint. One photographing unit 250 can simultaneously or separately photograph food or dishes containing food and the biometric information of the eating subject. However, the photographing unit 250a for photographing food or dishes containing food and the biometric information of the eating subject are The photographing unit 250b for photographing may be separated.

In some embodiments, when photographing dishes 500 moving along the conveyor belt 270, the photographing unit 250 of the food measuring device 200 may acquire depth data in addition to image data, even if it includes a single camera. You can. Using a single photographing unit 250, images are taken when the dishes 500 are in the first position on the conveyor belt 270 and when the dishes 500 are moved on the conveyor belt 270 and are in the second position. This is because the same effect as photographing stationary dishes 500 using the photographing unit 250 including a plurality of cameras can be obtained.

Because a camera is a device that recognizes light, it is greatly affected by the environment, such as light reflection. Therefore, it is necessary to minimize the impact of the environment by taking pictures using the photography unit 250 that includes two or more cameras. However, even when taking pictures using the photographing unit 250 that includes two or more cameras, correction for this may be affected by the environment.

Therefore, in order to optimize according to the characteristics and environment of the restaurant, the computer learns the situation in the field through images taken by the learning module, and recognizes the environmental state based on filter processing, fixed objects, and reference objects (background targets described later). You can do it.

On the other hand, when photographing the dishes 500 using the food measuring device 200 according to an embodiment of the present invention, the dishes 500 are photographed from the upper side by the photographing unit 250 and the food measuring device ( 200 may detect identification information 530 for the tableware 500 or identification information for the person eating the meal. That is, the person eating can be identified using the food measuring device 200 simultaneously or separately from the tableware image receiving step (S20) (S21).

Specifically, referring to Figure 2B, identification information for the tableware 500 may be included in the tableware 500, and may be physically included in the tableware 500 in the form of, for example, a QR code (530, Quick Response Code) or a barcode. In addition, identification information in the form of various codes, such as combinations of numbers or letters, shapes, and symbols, may be included in the tableware 500. In addition, identification information about the tableware 500 may be recorded in various semiconductor devices such as RFID chips and included in the tableware 500. For this purpose, the food measuring device 200 includes an identification information detection unit 251, and when the photographing unit 250 photographs the tableware 500 from the upper side, the identification information detection unit 251 also detects the tableware 500. It can be placed in the food measuring device 200 to detect identification information.

For example, as shown in Figure 2B, when the QR code 530 is located on the back or side of the tableware 500, the identification information detection unit 251 is located on the lower side facing the photographing unit 250 or detects the QR code ( 530), but the position of the identification information detection unit 251 is not limited to this, and when the photographing unit 250 photographs the upper surface of the tableware 500, the identification information of the tableware 500 is simultaneously captured. If it can be detected, it can be placed in various locations.

In some embodiments, when the identification information of the tableware 500 is located on the upper side of the tableware 500, the food measuring device 200 may not include a separate identification information detection unit 251, and the photographing unit 250 ) can also be used to confirm the identification information of the tableware 500 by analyzing the measured tableware image.

In addition, identification information about the person eating can be detected by the authentication unit 160, and for example, methods such as facial recognition, iris recognition, or fingerprint recognition that confirm biometric information by taking images of the person eating can be applied. You can.

In addition, the authentication unit 160 recognizes tags containing identification information in the form of various codes, such as tags containing an RFID chip held by the person eating, QR codes, bar codes, combinations of numbers or letters, shapes, and symbols. By doing so, identification information about the person eating can be detected.

When using the food measurement method according to an embodiment of the present invention, various information can be calculated using the measured results. The food measurement device 200 collects the identification information 530 for the tableware 500 and the person eating the meal. There may be differences in the information calculated depending on whether any of the identification information can be recognized.

First, both the food measuring device 200 installed in the dispensing port and the food measuring device 200 installed in the dispensing port cannot recognize the identification information 530 for the tableware 500 and can only recognize the identification information for the person eating. If so, information about each meal subject and information about the group including the meal subject can be calculated using the food measurement method according to this embodiment.

On the contrary, both the food measuring device 200 installed in the dispensing port and the food measuring device 200 installed in the dispensing port do not recognize the identification information about the person eating, but only the identification information 530 about the tableware 500 can be recognized. If so, information about each meal subject cannot be calculated using the food measurement method according to this embodiment, but only information about the group including the meal subject can be calculated.

In addition, both the identification information 530 for the tableware 500 and the identification information for the person eating can be recognized in the food measuring device 200 installed in the dispensing port, and the food measuring device 200 installed in the dispensing port can recognize If the identification information 530 for the tableware 500 is not recognized and only the identification information for the person eating can be recognized, the identification information 530 for the tableware 500 is collected from the food measuring device 200 installed in the dispensing port. Even if the food is not recognized, the identification information 530 for the tableware 500 held by a specific meal subject can be specified using the information at the distribution port, so each meal can be measured using the food measurement method according to this embodiment. Information about the subject and information about the group that includes the subject of the meal can be calculated.

Meanwhile, the food identification device 30 may not include a separate authentication unit 160 and may receive identification information from the person eating through an input device (not shown) of the food measurement device 200.

Accordingly, in some embodiments, in the step (S520) of the receiving unit 110 receiving the tableware image photographed through the photographing unit 250, the receiving unit 110 receives identification information about the tableware 500 and the person eating the meal in addition to the tableware image. At least one of the identification information for can be received together.

Next, referring to FIG. 1, the correction unit 150 may perform image correction of dishes containing food (step S25).

More specifically, the correction unit 150 may use image data of dishes containing food to remove noise components, recognize the tilt of the dish image, and correct it to a horizontal image. This corrected image can be used in the volume calculation step (S50) described later.

When taking pictures of dishes in a restaurant, the pictures may be taken without shaking while the dishes are placed in a specific position. However, in some cases, the photography unit 250 takes pictures of the dishes 500 moving on the conveyor belt 270, and the person eating There are cases where the tableware 500 is photographed through the food measuring device 200 while holding the dishware 500, and in some embodiments, the user walking while holding the dishware 500 is captured in the photographing unit 250. It may also be filmed by

Therefore, since the tableware image may be photographed unclearly and the tableware image may be photographed at a tilted angle, the correction unit 150 recognizes the shake or tilt of the tableware image and determines whether the tableware image has the least shake or tilt. Select the tableware image and correct the tableware image to a horizontal image to accurately calculate the volume of food.

First, the correction unit 150 may analyze a plurality of standardized tableware images to select the tableware image with the least shaking among the standardized tableware images. For example, the correction unit 150 may analyze the image for each pixel. You can check the degree of blurring and select the clearest tableware image with the least blur.

Subsequently, the correction unit 150 may calculate the distance from the photographing unit 250 to the plurality of exposed areas in the tableware image, targeting the exposed area of the tableware 500 without being obscured by food, and may calculate the distance from the plurality of exposed areas to the plurality of exposures. The tilt of the tableware 50 can be calculated from the tableware image using the calculated distance to the exposed area. Subsequently, the correction unit 150 may correct the photographed tableware image to be horizontal using the calculated tilt.

Specifically, the correction unit 150 may extract the shape of the dishware 500 using depth information from the depth dish image. Since there is a difference in depth information between the tableware 500 and the background other than the tableware 500 in the tableware image, the correction unit 150 can extract the tableware 500 from the tableware image using this.

The correction unit 150 checks flat areas among the exposed areas of the tableware 500 that are not covered by food, determines at least three points among the flat areas, and checks the depth information of the three or more points. do.

Here, checking the depth information of three or more points is to create a virtual plane using three or more points and check the slope of the plane, and the correction unit 150 determines the depth of three or more points. The tilt of the tableware 500 can be calculated using the information, and the photographed dish image can be corrected using the calculated tilt.

In some embodiments, after determining at least three points in a flat area, instead of checking the depth information of the three or more points, the average depth information about the surrounding points of each point is calculated for the three or more points. By using depth information, the accuracy of the calculated slope can be increased by using relatively more information to calculate the slope.

In some embodiments, in the process of calculating the slope of the tableware 500 using the depth information of three or more points, a gradient value according to the line connecting each point can be derived, and through this, the tableware 500 ) can be calculated and the photographed tableware image can be corrected using the calculated slope.

In some embodiments, the correction unit 150 determines the outline of the tableware 50 in the tableware image, and determines the outline of the tableware 50 in the photographed tableware image and the tableware in the non-tilted tableware image for use as a reference ( By comparing the outline of the tableware 50, the inclination of the tableware 50 in the photographed tableware image can be calculated. Next, the correction unit 150 may correct the photographed dish image using the calculated tilt.

In addition, since noise may occur in the image data of two tableware containing food received through the receiver 110 depending on the situation in the field (shaking, light reflection, etc.), noise can be removed by comparing the two tableware images. there is.

Next, referring to Figure 1, the extraction unit 120 extracts the food image data accommodated in each space 510 of the tableware 500 from the image data of the tableware containing food (step S530).

More specifically, since the tableware image includes images of the tableware 500 and food, this means extracting food image data from the tableware image. For example, the extraction unit 120 may extract food image data using a model learned using an artificial intelligence algorithm to distinguish between food and non-food. Here, when the tableware 500 includes a plurality of spaces 510, the extractor 120 may extract food image data for each space 510.

Meanwhile, referring to FIG. 5, it is shown that a plurality of foods are accommodated in one dishware 500 that is not spaced. Since the space within the dishware 500 is not divided, the extraction unit 120 is used to store a plurality of foods. It can be recognized as one whole food image 550 included. In this case, the extraction unit 120 must additionally perform an extraction process to separate the entire food image data into individual food image data.

Next, referring to Figure 1, the determination unit 130 determines the type of each food through the extracted food image data (step S40).

The determination unit 130 may determine the type of food using the extracted food image data. For example, the determination unit 130 may include a model learned based on images to recognize food types, but the method for determining food types is not limited to this.

In order to increase the recognition rate of food types in the food measurement method according to this embodiment, the model learned using an artificial intelligence algorithm can be optimized for individual subjects using the food measurement method in the following manner. An individual subject may be, for example, but is not limited to, an individual restaurant or individual dining entity.

First, referring to FIG. 6, the receiver 110 receives optimization information to optimize a model learned from an individual object such as a restaurant or person eating (step S31).

The learned model that is targeted for optimization is the master model, and the master model is trained to distinguish food types using image data or processed results taken from the food measuring device 200 or the user terminal 300 as a data set. This is an advanced model.

The master model is built using all captured image data or processed results used in food measurement methods as a data set, so in order to determine the type of specific food image data, the master model uses dozens, hundreds, or thousands of food types. The class determines what type of food the specific food image data is.

Optimizing the master model is to reduce the number of food type classes that must be considered to identify the food type of specific food image data. To this end, optimization information for individual targets, such as restaurants or diners, can be obtained.

That is, the receiving unit 110 can receive optimization information for individual targets. Optimization information may be information that has a high probability of being consumed by each individual object or consumed by a meal subject using the individual object, such as menu information sold or provided by an individual restaurant or types of food frequently consumed by an individual meal subject.

Next, referring to FIG. 6, the determination unit 130 calculates a local model by optimizing a class in the master model based on the optimization information (step S32).

The determination unit 130 selects a food type class that is highly related to the individual object based on the optimization information, and reduces the food type class considered in the master model to a food type class that is highly related to the individual object. By optimizing the classes of the master model, a local model for that individual object is produced.

Next, referring to FIG. 6, the determination unit 130 advances the local model by learning using image data related to individual objects (step S33).

The determination unit 130 may advance the local model by learning the calculated local model using image data related to the individual object or results processed in relation to the individual object.

Here, the image data related to an individual object is image data taken within an individual restaurant as an individual object or the result of processing image data taken within an individual restaurant, or image data taken by an individual dining subject as an individual object or an individual meal. It may be the result of processing image data captured by the subject.

Meanwhile, in the process of advancing the local model by learning the calculated local model using image data related to individual objects or results processed related to individual objects, the judging unit 130 uses artificial intelligence. In the process of learning using an intelligent algorithm, learning is carried out by changing various variables such as learning parameters and learning cycles, and the advancement process can be carried out by using the learning decoration with the highest accuracy among the learning methods.

Next, referring to FIG. 6, the determination unit 130 may provide an advanced local model, and using this, the determination unit 130 determines the type of each food through the extracted food image data, thereby improving accuracy. It can be increased (step S34).

In some embodiments, a preceding step may further include the receiving unit 110 receiving food list information provided by the restaurant (step S10).

In this case, in the step of determining the type of food (step S40), the determination unit 130 may determine the type of each food by matching the extracted food image data and food list information.

In this way, the determination unit 130 may independently determine the type of food by analyzing the food image data, or may determine the type of food by matching it with stored food list information. A method of determining the type of food is not limited to this.

Next, referring to FIG. 1, the calculation unit 140 calculates the volume of each food using height information (i.e., 3D distance data) for each pixel of the extracted food image data (step S50).

As mentioned above, since the tableware image was captured through the photographing unit 250 capable of acquiring depth data, the food image data includes height information (3D distance data) for each pixel as depth data. Using this, you can calculate the volume of each food.

Here, the computer includes a database 170 in which tableware information including the size, depth, and capacity of the plurality of spaces 510 formed in the tableware 500 and the tableware image are stored.

In addition, the volume calculation step (step S50) is a step in which the correction unit 150 corrects the volume of each food calculated using tableware information including the size, depth, and capacity of the space 510 in which each food is accommodated. may be included.

Since the depth of each space 510 is different depending on the type of tableware 500, the correction unit 150 can more accurately determine the degree of food accumulation by using the tableware information. Since data on the tableware 500 used in a restaurant is stored in the database 170 of the computer according to this embodiment, when the food measurement method according to this embodiment is used, the volume of each food extracted using the tableware information can be calculated more accurately.

For example, when the determination unit 130 determines that the type of food contained in a specific space 510 within the tableware 500 is liquid, the calculation unit 140 determines the location where the liquid and the space 510 meet. The volume of liquid can be calculated using dish information including the size, depth, and capacity of the space 510.

When liquid food such as 'soup' is accommodated in a specific space 510, a portion inevitably occurs where the 'soup' and the space 510 come into contact at a certain height. The calculation unit 140 uses this to recognize the location where liquid food and the corresponding space 510 meet in the dishware image, and uses the dishware information (size, depth, capacity) of the corresponding space 510 to determine the volume of the food. is calculated.

For example, the part where the 'station' and the space 510 come into contact are shown in the 'A' area shown in Figure 3, and the calculation unit 140 stores the liquid through the same part as the A area in the image data. It is possible to check the location where the spaces 510 meet. Additionally, the calculation unit 140 can calculate the volume of 'soup' by calculating the volume up to the height of area 'A' using dishware information (eg, capacity).

In some embodiments, additional correction may be performed on one dish 500 that is not spaced as shown in FIG. 5 . If it is determined from the food image data in the volume calculation step that a plurality of foods are accommodated in an undifferentiated space within one dishware 500, the calculation unit 140 provides image data for each of the plurality of foods to the extraction unit 120. Request and calculate the volume of each food.

Then, the calculation unit 140 calculates the volume of the entire food using the extracted entire food image data, and compares the total volume of each food calculated in the volume calculation step with the volume calculated through the entire food image data. , the volume of each food can be corrected.

In some embodiments, the volume calculation step (step S50) may further include calculating the volume of the stacked food (step S51).

More specifically, a plurality of foods may be stacked in a specific space 510 within the tableware 500 . Therefore, when the calculation unit 140 recognizes that different foods are stacked in a specific space 510 within the tableware 500, the image data of the food located on the upper side, the calculated volume information, and the corresponding space 510 The size, depth, and volume information are used to calculate the volume of food located below.

Figure 4 is a diagram illustrating a fried egg layered on rice in the space 510 where the rice is accommodated in the dishware. The calculation unit 140 calculates the size and volume of the fried egg using dishware information and image data of the fried egg.

In addition, the calculation unit 140 can predict the height of the rice accommodated in the space 510 using the size and volume information of the fried egg, and the volume of the rice accommodated in the space 510 using the tableware information. can be calculated.

Specifically, the area where the food located on the lower side (e.g., rice) is covered by the food located on the upper side (e.g., a fried egg) is calculated, and the food located on the upper side (e.g., eggs) among the foods located on the lower side (e.g., rice) is calculated. By calculating the height of the area covered by the food located on the upper side (e.g., a fried egg) by using the height of the area that is not covered by the food located on the lower side (e.g., rice), the food located on the upper side (e.g., rice) is calculated. , the volume of the area obscured by the fried egg) can be calculated.

For this purpose, the height of the area where the food located on the lower side (e.g., rice) is covered by the food located on the upper side (e.g., a fried egg) is the height of the area where the food located on the lower side (e.g., rice) is covered by the food located on the upper side (e.g., a fried egg). ), but is not limited to this.

In some embodiments, the computer's database 170 may pre-store data on foods stacked on top of other foods (eg, fried eggs, fish, seaweed, etc.). For example, data on food mainly stacked on the top of the food may include volume data or weight data, and through this, the calculation unit 140 may quickly calculate the volume of the food stacked on the top.

Meanwhile, the volume calculation step (step S50) may further include a step (step S52) in which the correction unit 150 corrects a noise component photographed with the tableware in the tableware image. That is, the correction unit 150 detects a noise component that is not food but has a volume within the image data of the tableware containing food, and the volume of the noise component detected from the volume of the food calculated in the step of calculating the volume of each food. A step of modifying the correction to exclude may be further included.

Here, the noise component is not food but has a volume, and the volume of the noise component may result in the volume of the food being calculated to be larger than it actually is. Therefore, in order to accurately calculate the volume of food, the correction unit 150 recognizes the noise component in the captured image and performs correction to exclude the volume of the noise component from the volume of the food calculated by the calculation unit 140. You can.

For example, hands or utensils (e.g., spoons, forks, knives, etc.) may be volumetric noise components that are not food, and other examples may include, but are not limited to, lids of dairy products. For example, noise information about the object corresponding to the noise component may be stored in advance in the computer's database 170, and the correction unit 150 recognizes the noise component in the captured image and calculates it by the calculation unit 140. Correction can be performed to exclude the volume of the corresponding noise component from the calculated volume of food.

For example, in the step of correcting the noise component, the correction unit 150 recognizes the image of the user's hand in the tableware image and excludes the volume value of the hand from the volume value of the food calculated by the calculation unit 140. Correction can be performed. Specifically, the tableware 500 may be photographed by the food measuring device 200 while the user is holding the tableware 500, but there is an error in calculating the volume of the food due to the image data of the hand holding the tableware 500. Since this may occur, the correction unit 150 performs correction to exclude the volume value for body parts such as the hand from the volume value of the food calculated through the image data of the tableware containing the food, thereby performing the food volume calculation step (S50) ) can increase accuracy.

In one embodiment, the correction unit 150 is trained in advance on image data for empty dishes 500 and cutlery, recognizes when a cutlery is included in the tableware image, and performs correction to exclude the volume value of the cutlery. You can.

More specifically, the correction unit 150 detects the presence of the user's hand or dishes in image data of dishes containing food. Additionally, if the user's hand or tableware is included in the tableware image, the volume value of the user's hand or tableware is excluded from the volume value of the food calculated through the image data of the tableware containing food.

Due to the configuration and operations of the correction unit 150 described above, the food measuring method according to the embodiment of the present invention has the effect of increasing accuracy when subsequent steps are performed.

Next, referring to FIG. 1, the calculation unit 140 calculates meal information including the weight of each food using the volume information and food information of each food calculated in step S50 (step S60).

The computer database 170 stores food information including weight per volume of each food, calorie information per weight of each food, and nutritional information per weight of each food.

Foods have different weights per volume and different calories depending on the type. Therefore, in an embodiment of the present invention, food information including the weight per volume of each food is stored in the database 170, and the calculation unit 140 uses this to calculate the weight of the food from the volume information of the food.

Using this, after the weight calculation step (step S60), the calculation unit 140 can calculate the calories of the food contained in the user's serving dish using the calculated weight information of each food, and in addition to the nutrition consumed by the user. Ingredient information can be calculated, and additional specific information, such as allergy precautions, can be calculated based on the food or the nutritional ingredients contained in the food.

In addition, the calculation unit 140 may calculate price information for the food using the type of food and the volume or weight of each food.

The calculation unit 140 may provide information included in the above-described meal information to the display unit 210 of the food measuring device 200 or the display unit 310 of the user terminal 300. The UI of the screen provided on the display units 210 and 310 will be described later.

For example, the calculation unit 140 may make the information displayed on the display unit 210/310 of the feeding port different from the information displayed on the display unit 210/310 of the discharging port. For example, the display unit 210/310 of the food distribution unit may display information on food types, calorie information, nutritional information, allergy information, and price information, and the display unit 210/310 of the food distribution unit may display information on food types, calorie information, nutritional information, allergy information, and price information. In addition to the above information, information about the actual amount consumed may be additionally displayed. However, the meal information calculated by the calculation unit 140 is not limited to this, and various information can be displayed without limitation using photographed information or information stored in the database 170.

Accordingly, the meal subject uses cash, card, bank transfer, QR code tagging, RFID tagging, facial recognition, etc. according to the price information provided to the payment unit 240 or the user terminal 300 of the food measuring device 200. You can proceed with payment through .

In one embodiment, a restaurant may be equipped with a weight measuring device (not shown) that can measure the weight of the tableware 500, but is not limited thereto, and the receiver 110 may be equipped with a weight measuring device (not shown) provided in the restaurant. ), and the correction unit 150 matches the total weight of each food and empty dishes calculated in step S560 with the received weight information. A step of correcting the weight of each food may be further included.

The above-mentioned correction step can be selectively adopted and used, and since the weight may be measured differently from the actual due to the user's hand or various factors during the weight measurement process, the weight information is measured differently to the extent that it exceeds the threshold. If so, the calibration step may not be performed.

Hereinafter, with reference to FIGS. 7 and 8 , a food measuring method according to another embodiment using the user terminal 300 as the food measuring device 200 will be described. However, the description will focus on the differences from the food measuring method according to an embodiment of the present invention, and description of the same content will be omitted. Figure 7 is a flowchart of a food measurement method according to another embodiment of the present invention, and Figure 8 is a perspective view of a table for explaining a background target.

The food measurement method according to another embodiment of the present invention is performed by a computer, where the computer refers to the food measurement server 100 or the user terminal 300. That is, shooting is performed by the user terminal 300, but other steps may be performed by the food measurement server 100 or the user terminal 300.

Here, the user terminal 300 is a movable portable terminal, and a service application related to a food measurement method is installed. The user terminal 300 may acquire image data and depth data. To this end, the user terminal 300 may include at least one of an RGB camera, a 2D camera, a 3D camera, a Time of Flight (ToF) camera, a light field camera, a stereo camera, an event camera, and an infrared camera. there is.

For example, the user terminal 300 may be a computer, an Ultra Mobile PC (UMPC), a workstation, a net-book, a Personal Digital Assistant (PDA), a portable computer, a web tablet, or a wireless phone. (wireless phone), mobile phone, smart phone, e-book, PMP (portable multimedia player), portable game console, navigation device, black box Alternatively, it may be one of electronic devices such as a digital camera, and may include any user device capable of installing and executing a food measurement application related to the food measurement server 100 or the food measurement device 200.

First, referring to FIG. 7, the receiving unit 110 receives a tableware image photographed through the photographing unit 350 of the user terminal 300 (step S20).

Here, food may be photographed according to a photographing request from a service application installed on the user terminal 300, and a guide on how to photograph food, angle, etc. may be provided to the user of the service application in the form of video or sound.

In one embodiment, the service application may request that food be photographed more than once from different angles. When not using a 3D camera or ToF camera, if food is photographed more than once from different angles, depth data can be obtained by comparing image data of two or more dishes containing food. In addition, even if the purpose is not to acquire depth data, if images are taken more than twice, the accuracy increases in the subsequent calculation and correction stages.

Next, referring to FIG. 7, the correction unit 150 may perform tableware image correction (step S25).

In one embodiment, the tableware image receiving step (step S10) is performed when food is photographed through the photographing unit 350 of the user terminal 300, and the user measured through the gyro sensor (not shown) of the user terminal 300 Tilt information of the terminal 300 may be received. The correction unit 150 may correct the tilt of image data of dishes containing photographed food using tilt information of the user terminal 300.

According to an embodiment of the present invention, in the process of the user holding the user terminal 300 and photographing food through the photographing unit 350, tilting at a predetermined angle may occur, so the food is captured by performing the above-described steps. The image data of the dishes contained therein can be corrected to a horizontal state.

Next, referring to FIG. 7, the extraction unit 120 extracts the food image data accommodated in the space 510 of the tableware 500 from the image data of the tableware containing food (step S30).

Since the tableware image captured through the user terminal 300 does not include only food, only food image data is extracted from the image in this step to accurately calculate the volume.

The extraction unit 120 accurately extracts food from the background using a function to recognize and extract food from the background. If two or more foods are included, the image data of the two or more foods are extracted separately, and this process A model learned using an artificial intelligence algorithm can be used.

Next, referring to FIG. 7, the determination unit 130 determines the type of each food through the extracted food image data (step S40).

The determination unit 130 determines the food type of each food image data extracted by the extraction unit 120 using a model learned using an artificial intelligence algorithm. In some embodiments, when a plurality of decision results with similar accuracy are output, the determination unit 130 outputs a plurality of decision results to the user terminal 300 and inputs the correct result from the user through a service application of the user terminal 300. You can choose.

Next, referring to FIG. 7, the calculation unit 140 calculates the volume of each food using height information (i.e., 3D distance data) for each pixel of the extracted food image data (step S50).

The volume calculation step (step S50) may include a volume correction step (S53). In one embodiment, a background target may be used for volume correction in the volume correction step (step S53).

In the volume correction step (step S53), the determination unit 130 recognizes the background target included in the received tableware image, and the correction unit 150 uses the size information and location information of the recognized background target in step S50. The calculated volume of each food can be corrected.

Referring to Figure 8, the correction unit 150 may perform volume correction of each food by comparing the background target and the height information of each food, and may perform volume correction of each food by comparing the background target and the size information of each food. You can also perform .

Here, the background target is something that is photographed together with the food or photographed separately through the photographing unit 350, and the tableware 500, the table 540, the spoon, the user's finger, etc. containing the food can be used. However, it is not limited to this, and the background target can be applied to any object of normal size. For example, the background target can be applied to any object of a fixed size, such as coins, banknotes, smartphones (by product type), etc.

In some embodiments, the background target may be separately photographed through the photographing unit 350 of the user terminal 300 and size information may be previously stored in the database 170. Accordingly, the correction unit 150 can predict the volume of food with high accuracy from the food image data extracted based on the size or height of the background target.

As a first example, the service application may allow the user to photograph the user's hand from a certain distance through the photographing unit 350. Alternatively, the service application may allow the user to actually measure and input the actual size of the user's hand/finger.

Then, the receiver 110 analyzes the image of the user's hand and stores information about the actual size of the user's hand in the database 170.

Afterwards, when the determination unit 130 recognizes that the user's hand is included in the received tableware image, it sets the user's hand as the background target, and the correction unit 150 sets the user's hand as the background target based on the location information and size information of the background target. Thus, the volume of each food calculated by the calculation unit 140 can be corrected.

As a second example, the service application may allow the user to photograph dishes 500, tables 540, spoons, etc. used at home from a certain distance through the photographing unit 350. At this time, the table 540 may refer to a dining table or dining table where the user places the tableware 500 and eats.

*Specifically, the computer analyzes the image of the tableware 500 used by the user at home captured through the photographing unit 350 of the user terminal 300, and determines the size, depth, and capacity of the tableware 500. Tableware 500 information including information may be stored in the database 170.

More specifically, since height information for each pixel is stored in the image captured through the photographing unit 350 of the user terminal 300 according to an embodiment of the present invention, the calculating unit 140 stores empty dishes 500 When receiving image data for the tableware, the size, depth and capacity information for the tableware can be calculated using the height information for each pixel of the image data of the tableware, which means storing this in the database 170 as time information.

And, as a result of analyzing the food image, if there is food contained in the tableware 500 among the food, the calculation unit 140 calculates each food using the size, depth, and capacity information of the tableware 500 containing the food. Perform volume correction.

If the size, depth, and capacity information of the dishes 500 are known as above, a correction step can be performed to obtain more accurate results.

In some embodiments, the service application may request the user to measure the actual size of the dishes 500, table 540, spoon, etc. and input it into the user terminal 300 (e.g., the width and height of the table 540). or circumference length information)

As a third example, we explain the case of using a background target that is not related to food. For example, a user can capture a 500 won coin along with a food image and input that the 500 won coin was photographed as a background target through a service application.

Accordingly, the determination unit 130 recognizes the 500 won coin (background target) included in the received image of the tableware containing food, and the correction unit 150 recognizes the 500 won coin (background target) based on the size information of the 500 won coin (background target). The calculated volume of each food is corrected.

In this way, objects that are generally used by most people and have a fixed size can be applied as background targets, so the user can easily photograph surrounding objects together with food and have the correction unit correct the volume of each food.

In addition, location information (restaurants, cafeterias, etc.), weather information (ultraviolet rays, reduced food selection), user status information (on diet, exercise, fighting illness, medication, etc.), personal preference information, and surrounding device information are provided in various ways. Accuracy can be increased by correcting the volume.

As a fourth example, the computer stores image data taken through the user terminal 300 of a plurality of users, information calculated through the image data, and information about the restaurant matching the place where the image was taken, thereby creating big data. It can be built.

For example, when user A orders menu C at restaurant B and photographs menu C through the user terminal 300, the type of food included in menu C is determined, the volume of each food is calculated, and the computer This information is stored in the database 170 and used as big data.

Then, food image data is received through the user terminal 300 of a specific user, and the location information where the food image was taken and the determined type of food are matched with data in the database 170 in the computer, resulting in a previous calculation of the matching details. can be detected, and using this, the calculated volume information of each food can be corrected.

For example, when user D photographs food at a location matching restaurant B and transmits food image data, the location information is matched to big data to determine that user D is located at restaurant B, and the transfer is matched with the menu ordered. Correction of the calculated volume information can be performed by querying the result data.

In this regard, the opposite case is also applicable. More specifically, data about food is stored first, and when the user knows what food they ordered and ate when paying at a restaurant, the stored data can be retrieved and volume and nutritional information can be recorded.

In summary, when taking pictures by the user terminal 300, the service application may request the user to take pictures of the background target with an image or sound before taking pictures of the food or to take pictures together with the food. Then, the determination unit 130 recognizes the received background target, and the correction unit 150 can correct the volume of each food calculated by the calculation unit 140 based on the location information and size information of the background target. there is.

Next, referring to FIG. 7, the calculation unit 140 calculates meal information including the weight of each food using the volume information and food information of each food calculated in step S50 (step S60).

Using the food measurement method according to the above-described embodiment of the present invention, a computer can measure the amount of food (meaning the amount of food before a meal) and the amount of leftover food for each person eating at a restaurant or at home and use these to calculate the amount of food. there is.

In addition, the computer can generate restaurant operation information and management information of the diners using the portion size, remaining portions, and meal amount of the diners eating at the restaurant or at home, and as management information of the diners, for example, the nutritional status of the user. , eating habits, picky eating, etc.

Hereinafter, with reference to FIG. 9, a method for providing restaurant operation information and dining entity management information according to an embodiment of the present invention will be described. Figure 9 is a flowchart of a method for providing restaurant operation information and management information of diners according to an embodiment of the present invention.

First, referring to FIG. 9, the receiver 110 receives appetizer data (appetizer image data) photographed through the dispensing port photographing unit 250 provided in the dispensing port or the photographing unit 350 of the user terminal 300. Obtained, the calculation unit 140 uses this to calculate the amount of food for the person eating (step S110). The detailed description is the same as that described with reference to FIGS. 1 to 8 , and the identification step (step S120) for the person eating the meal may be performed together. Here, the pre-meal tableware refers to the tableware 500 in which food is placed in the tableware 500 of the person eating before the meal.

* Next, referring to FIG. 9, post-prandial meal data (post-prandial meal image data) photographed through the discharge port photographing unit 250 provided in the discharge port or the photographing unit 350 of the user terminal 300 is obtained, and calculated The unit 140 uses this to calculate the amount of leftover food for the meal subject (step S120). The detailed description is the same as that described with reference to FIGS. 1 to 8 , and the identification step (step S120) for the person eating the meal may be performed together. Here, the post-meal tableware refers to the tableware 500 of the person eating after the meal.

Next, referring to FIG. 9, the calculation unit 140 calculates the meal amount of the person eating the meal using the calculated amount of food served and the amount of leftover food (step S130).

Through steps S110 to S120, the computer determines how much rice and side dishes the person eating consumed, by calculating the amount of each food menu provided by the restaurant, the amount of leftovers left after eating, and the amount consumed (amount of meal). It is possible to secure information about how much is left.

Additionally, the computer according to the embodiment of the present invention can perform the following operations to improve the accuracy of calculating the amount of food.

In one embodiment, the calculation unit 140 verifies whether the sum of the meal amount and leftover amount of the meal subject calculated through the above steps matches the ration amount, and if there is a discrepancy according to the verification result, the calculation unit 140 At least one of the meal amount, leftover amount, and ration amount calculated through the above steps can be corrected so that the sum of the meal amount or leftover amount matches the ration amount, and the result is recorded in the database 170.

In addition, the calculation unit 140 verifies whether the sum of the total amount of meals and the total amount of leftovers calculated through the above steps of the restaurant diners matches the total amount of food served, and if there is a discrepancy according to the verification result, the calculation unit 140 (140) can correct at least one of the total amount of meals, the total amount of leftovers, and the total amount of rations calculated through the above steps so that the sum of the total amount of meals and the total amount of leftovers matches the total amount of rations, The results are recorded in the database 170.

In addition, the calculation unit 140 verifies whether the total amount of meals and the total amount of leftover food of the people eating at the restaurant calculated through the above steps matches the total amount of food served, and records it in the database 170. More specifically, the calculation unit 140 verifies whether the difference between the total amount of food served by the diners eating at the restaurant and the total amount of leftover food matches the total amount of food.

In addition, the calculation unit 140 performs verification by comparing the total amount of food prepared in the restaurant with the total amount of leftover food generated in the restaurant, the total amount of leftover food generated in the restaurant, and the total amount of meals of the dining subjects of the restaurant. If a difference occurs according to the results, the calculation unit 140 ensures that the total amount of food prepared in the restaurant matches the total amount of leftover food generated in the restaurant, the total amount of leftover food generated in the restaurant, and the total amount of meals of the people eating in the restaurant. At least one of the total amount of leftover food, the total amount of leftover food, and the total amount of meals can be corrected, for example, the amount of meals can be corrected, and the result can be recorded in the database 170.

Here, the amount of leftover food is the amount of food left after being distributed to the person eating, and the amount of food remaining may be the amount of food remaining without being distributed to the person eating, and the amount of food remaining is calculated through the above steps or through a separate method. can be calculated.

In addition, the database 170 may store the total weight of the food prepared for each meal in the restaurant, the total volume (weight) of the remaining food, and the total weight measurement results of the remaining food, and the calculation unit 140 may store the total weight of the food prepared for each meal in the restaurant. Verification is performed by comparing the total volume (weight) with the total volume (weight) of the remaining food, the total volume (weight) of the remaining food, and the sum of the meal amounts of the restaurant diners calculated through the above steps, and the verification results are If a difference occurs, the calculation unit 140 performs the above steps so that the total amount of food prepared in the restaurant matches the total amount of leftover food generated in the restaurant, the total amount of leftover food generated in the restaurant, and the total amount of meals of the dining subjects of the restaurant. The total amount of meals calculated by the eating subjects can be corrected, and the results can be recorded in the database 170.

In this way, the calculation unit 140 performs verification using various types of information and records the accuracy resulting from calculation/correction, thereby making it possible to identify problems and improve them.

For example, the database 170 may store the results of measuring the total volume (weight) of the amount of food prepared for each meal in a restaurant and the total volume (weight) of the amount of leftover food, and the calculation unit 140 may store the total volume (weight) of the amount of food prepared for each meal in a restaurant. Verification may be performed by comparing the difference between the volume (weight) and the total volume (weight) of the remaining portion and the total amount of meals of the restaurant diners calculated through the above steps, and recorded in the database 170. (For example, if the total volume of Menu A prepared in a restaurant is 200 and all are served without any leftovers, the total volume of Menu A served to diners should be close to 200)

Meanwhile, in order to accurately calculate the amount of food, photos must be taken before and after a meal, but there may be cases where the user accidentally does not take photos after a meal.

Therefore, in a situation where a preset time has elapsed after receiving image data of dishes containing food before a meal, the calculation unit 140 determines that if the image data of dishes containing food after a meal is not captured and received, the image data of dishes containing food after a meal has not been captured. It is determined that this is the case, and the user's expected cutting amount is calculated.

For example, the calculation unit 140 may calculate the user's expected meal amount by considering that the user has left no leftovers, and calculate the user's average meal amount and status information (dieting, increasing weight, exercising, etc.) stored in the database 170, The user's expected meal amount may be calculated based on the degree of hunger, the user's preference for menus included in the pre-meal food image, taste information of the menus, etc. and stored in the database 170.

Next, referring to FIG. 9, the computer management unit 180 generates operation information of the restaurant based on the amount of food served, the amount of leftover food, and the amount of meals of the diners registered in the restaurant (step S140).

Next, referring to FIG. 9, the management unit 180 of the computer generates management information for each meal subject based on the amount of food served, the amount of leftover food, and the meal amount of the meal subjects registered in the restaurant (step S150).

Step S140 refers to generating operational information that helps restaurant operations.

More specifically, the ration control module 181 of the management unit 180 uses information on the total amount of rations prepared for distribution in the restaurant, the total amount of rations for diners whose meals have been completed, and the number of remaining diners. The target ration per person of the remaining meal subjects is calculated and provided to the restaurant management restaurant manager device (not shown).

At this time, the restaurant manager refers to the person in charge of designing the menu within the restaurant and directly distributing meals to the diners, and a restaurant manager device such as a display device 30 is installed in front of the restaurant manager of the restaurant along with the dishes to be served. It may be.

Accordingly, when the rationing amount control module 181 provides information on the target rationing amount per person for the remaining diners through the restaurant manager display, the restaurant manager can use this to adjust the rationing amount for the remaining diners.

In an embodiment of the present invention, a restaurant manager may refer to a person in charge of directly distributing meals to diners in a restaurant as described above, or may refer to a manager who manages the restaurant, or a nutritionist at the restaurant.

Through this configuration, if there is a large amount of food preparation remaining compared to the number of remaining diners, the effect of increasing the target ration amount per person for the remaining diners and reducing food waste can be achieved.

In addition, if the amount of food preparation is insufficient compared to the number of remaining diners, the target ration amount per person for the remaining diners is reduced, which has the effect of preventing the occurrence of people (diners) who cannot eat due to lack of food. You can.

In one embodiment, in the operation information generation step (step S140), the meal group management module 183 checks the affiliation of each meal member, stores the amount of food served, the amount of leftover food, and the amount of meals for each member, and uses this to determine preference for each member. A step of generating information for each affiliation including at least one of food, non-preferred food, amount of leftover food, and nutritional status, and extracting correlation information for each affiliation by matching the information for each affiliation with weather, day of the week, and schedule information for each affiliation. may include.

At this time, in the case of a company, the affiliation of the food service members may mean a company, group, team, etc., in the case of a school, it may mean a department, class, etc., and in the case of the military, it may mean a corps, division, regiment, battalion, company, platoon, etc. It can mean a squad.

In addition, schedule information by affiliation may mean quarters, ongoing projects, and projects in the case of a company; semesters, exam periods, etc. in the case of schools; and various training periods, etc. in the case of the military.

Therefore, according to an embodiment of the present invention, it is possible to identify food tastes by affiliation using information about preferred and non-preferred foods depending on the affiliation of the dining subjects, and to prepare an accurate guide for future food menus. .

In addition, managers can check the amount of leftover food and nutritional status by department.

In addition, by extracting correlation information by matching the weather, day of the week, and schedule information for each affiliation with the information for each affiliation, the manager can check changes in the appetite or food taste of the eating subjects depending on the weather and day of the week. In the case of the military, various It will be possible to understand how soldiers' meal intake changes according to their training schedule.

Using this information, administrators can appropriately adjust food menus and ration amounts according to military affiliation and training schedule, thereby providing appropriate food to soldiers completing the training schedule.

In one embodiment, in the operation information generation step (step S140), if the amount of food served is similar for each restaurant, but the difference in the amount of leftover food is more than a certain amount, the management determines that the restaurant with less leftover food has the superior grade of ingredients and cooking method. After making the decision, the restaurant's food ingredient supply information and food cooking method can be provided to the restaurant manager device of another restaurant.

For example, if two or more restaurants have similar food menus and portion sizes, but there is a big difference in the amount of leftovers, the restaurant with less leftovers has higher quality ingredients and excellent cooking methods, and the restaurant with more leftovers This may mean that the food ingredients are of low quality and the cooking method is poor.

Therefore, the management has the effect of improving the quality of the restaurant by providing information on supply and demand of food ingredients and cooking methods of restaurants with small leftovers to restaurant manager devices of other restaurants.

In one embodiment, in the management information generation step (step S150), the meal subject management module 185 uses the body information, portion size, leftover amount, and meal amount of each meal subject to select preferred foods, non-preferred foods, and It includes generating target-specific information including at least one of the average amount of leftover food, nutritional status, and average meal amount.

In addition, the eating subject management module 185 calculates the physical strength management information and obesity risk of each eating subject using the above-mentioned information for each subject, and the meal amount of the eating subject not set to a diet is reduced below a certain amount for a certain period of time, or If the amount of leftover food increases beyond a certain amount over a certain period of time, it includes the step of setting the subject of the meal as an object of attention.

For example, most soldiers have a regular life and schedule, so they need to receive a steady supply of nutrition, and unless they have personal problems, they will maintain a certain amount of food in most cases. However, if a personal problem occurs and motivation is low and the condition is not good, this may appear in the amount of food eaten, so this can be used to detect individual problems early.

Therefore, when the eating subject's meal amount that is not set to a diet state decreases below a certain amount for a certain period of time, the eating subject management module 185 determines that a problem has occurred in the eating subject and the eating subject's appetite has decreased, and the eating subject's appetite is reduced. Set as an object of caution.

In addition, the eating subject management module 185 determines that when the amount of leftover food of a eating subject that is not set to a diet increases to a certain amount or more for a certain period of time, the amount of food served by the eating subject is similar to the previous level, but the eating subject's appetite decreases and the amount of food consumed is reduced. It is determined that the amount of leftover food is increasing due to the decrease, and the subject of the meal is set as an object of caution.

In this way, the meal subject management module 185 manages each meal subject and sets the meal subject suspected of having a problem as a target of caution, thereby creating a risk of danger before an accident occurs to personnel such as soldiers of interest in places such as the military. This has the effect of detecting signs early and allowing managers to prepare countermeasures such as consultation.

In addition, since the meal subject management module 185 generates information for each subject including information such as the nutritional status and average meal amount of each meal subject, the manager uses this information to determine the characteristics and status of each affiliation as well as individual You can check the characteristics and status.

In one embodiment, the operation information generation step (step S140) calculates the average preferred food, non-preferred food, and meal amount of the dining subjects, and the food menu recommendation module 187 uses this to create one or more recommended menus of the restaurant and the meal preparation amount. It includes the step of deriving and providing it to the restaurant manager device.

In a restaurant where food is served at least once or twice a day, it is a very difficult task for the restaurant manager to determine the food menu taking into account the tastes and nutritional status of the diners every day.

Even considering the nutritional status, there is a problem that a lot of leftover food (food waste) is generated because it does not match the tastes (preferred foods) of the diners, and it becomes difficult to match the nutritional status if it is tailored to the tastes of the diners.

Therefore, the meal menu recommendation module 187 calculates the average preferred food, non-preferred food, and meal amount for each food menu of the diners, and uses this to derive and provide the meal preparation amount along with the recommended menu, allowing restaurant managers and managers to manage the restaurant. It has the effect of helping with management.

Additionally, in order to implement this algorithm, the relevance of each food is stored in the computer database 170. For example, if foods A, B, and C are highly related, and diners at the restaurant prefer food B, it can be determined that they are likely to also prefer foods A and C.

At this time, the food menu recommendation module 187 can derive a recommended menu using preference information and taste evaluation scores of the food menu, which will be explained below.

In one embodiment, the food menu evaluation module 189 of the management unit 180 generates operation information and management information based on the amount of food served, the amount of leftovers, and the amount of meals, and the amount of change in the amount of food served and the amount of change in the amount of leftovers for each of the distributed foods. It is characterized by evaluating the preference of each food menu and the taste evaluation score of the distributed food itself.

The computer's database 170 stores standard values for the amount of food served in a restaurant. When receiving food, if the food appears to be tasty through its appearance or smell, the person eating will increase the amount served, or if the opposite is the case, reduce the amount served, and if the food is tasteless while eating, a lot of the food will be left behind.

The above-described configuration evaluates scores using these characteristics, and more specifically, it is judged based on the following criteria.

First, the food menu evaluation module 189 increases the preference of the food menu when the amount of food served is greater than the standard value and the amount of leftover food is less than the standard value, and increases the taste evaluation score of the food itself.

In this case, during the process of receiving food, the diners (eating subjects) saw the food with their eyes and increased the amount of food served because it was their preferred food. While eating, they judged it to be delicious and consumed all of the food. It means judging that there is little left.

Therefore, the food menu evaluation module 189 evaluates the preference score for the corresponding food menu as high, and also evaluates the taste evaluation score of the served food itself as high, thereby evaluating that the selection of ingredients or the cooking of the food was done properly.

Second, the food menu evaluation module 189 increases the preference of the food menu and decreases the taste evaluation score of the food itself when the amount of food served is greater than the standard value and the amount of leftover food is more than the standard value.

In this case, during the process of being served, the food was visually seen by the diners and the amount of food was increased because it was the food they preferred. However, as they were eating, they judged that the taste was different from what they expected or that it was not tasty, so they ate the food. This means that it is judged that the remaining balance is more than the standard.

Accordingly, the food menu evaluation module 189 increases the preference score for the corresponding food menu, but evaluates the taste evaluation score of the served food itself as low, thereby evaluating that the selection of ingredients or the cooking of the food was not done properly.

Third, the food menu evaluation module 189 lowers the preference of the food menu and increases the taste evaluation score of the food itself when the amount of food served is less than the standard value and the amount of leftover food is less than the standard value.

In this case, during the process of receiving the food, the food was visually seen by the diners and the food was not preferred, so the amount of food was reduced. However, as the meal progressed, they decided that it was surprisingly tasty, so they either consumed all of the food or left only a small amount. It means judging that it is.

Therefore, the food menu evaluation module 189 evaluates the preference score for the corresponding food menu as low, but evaluates the taste evaluation score of the served food itself as high, so that the food is not preferred by the diners, but the selection of ingredients or the food is not preferred. The food was cooked properly and the diners were evaluated as being satisfied.

Fourth, the food menu evaluation module 189 lowers the preference of the food menu and the taste evaluation score of the food itself when the amount of food served is less than the standard value and the amount of leftover food is more than the standard value.

In this case, during the process of being served, the food was seen by the diners with the naked eye, and because it was a food they did not prefer, they reduced the amount of food served, and while eating, they decided that it was not tasty, so they left more of the food so that the leftovers were less than the standard. It means judging that there are many things.

Therefore, the food menu evaluation module 189 evaluates the preference score for the corresponding food menu as low, and also evaluates the taste evaluation score of the served food itself as low, so the food is not the food preferred by the diners and the selection of ingredients It is assessed that the food was not cooked properly.

Due to this configuration of the food menu evaluation module 189, the restaurant manager prepares food menus and ingredients for diners through the diners' preferences (preference scores) for each food menu and the food taste evaluation score. Not only can you do this, but you can also check whether there are any problems in the food cooking process.

In one embodiment, the meal subject management module 185 of the management unit 180 generates nutritional information consumed by each meal subject based on the meal amount of the meal subjects, and determines nutritional intake status based on the nutritional information generated at regular time intervals. Derive and provide information.

At this time, the meal subject management module 185 may provide the derived information to the meal subject's terminal, that is, the user terminal 300, the meal subject's guardian's terminal, the meal subject's manager's terminal, and the restaurant manager's terminal.

In one embodiment, the management unit 180 may perform a program to improve the nutritional status and reduce the amount of leftover food by using the meal amount and eating habits information of the eating subjects.

The management unit 180 determines the nutritional information and eating habits information of each meal subject based on the meal amount and remaining portion of the meal subject. In addition, rewards or penalties are provided according to changes in the nutritional status and amount of leftovers of each meal subject derived using this.

More specifically, the management unit 180 determines whether each meal subject has balanced nutritional intake and correct eating habits based on the amount of leftover food and meal size for each meal, and compares the nutritional status with the previous record at each meal. If it is determined that the nutritional status has improved, rewards such as rewards and points are provided to the person responsible for the meal, and if it is determined that the nutritional status has not improved or has worsened, a penalty such as a punishment is provided.

In addition, the management unit 180 stores and monitors the amount of leftover food for each person eating each meal, and if it is determined that the amount of leftover food for a specific person is decreasing, it provides a reward to the person eating the meal, and determines that the amount of leftover food is not decreasing or is increasing. This means providing a penalty like a punishment to the person responsible for the meal.

In this way, since the nutritional information and eating habits of each person are identified and rewards or penalties are provided according to behavioral changes, the effect is to naturally encourage the person eating to eat a balanced diet without being picky and to reduce the generation of food waste. It comes into play.

For example, when the person eating needs to be managed by a guardian, such as a student or a patient, the eating person management module 185 of the management unit 180 may use images of the eating subject's pre-meal tableware and post-meal tableware obtained through the photography unit 250. The image can be provided to the terminal of the guardian of the person eating the meal.

In addition, the guardian of the person eating can check whether the person eating is eating properly by checking the photo through the application.

In the case of students or patients, continuous management is necessary because they must consume sufficient and balanced nutrition for good health, and it is necessary to provide guardians with information about the student's nutritional status.

Therefore, the meal management module 185 periodically receives and manages the physical information or health information of each student or patient, and derives information on the basic metabolic rate and necessary nutrients for each student by considering the physical information and school schedule, etc. .

In addition, the meal subject management module 185 determines the nutritional status and eating habits of the student or patient based on the student's or patient's meal menu, meal amount data, and basal metabolic rate.

The eating subject management module 185 provides the information secured and determined in this way to the user terminal 300 and the eating subject's guardian terminal, and at the same time determines the nutritional status and eating habits of each eating subject, which is good for the eating subject to consume at home. Recommended menus can be derived and nutritional supplements can be recommended depending on the situation.

For example, if it is determined that student A may be deficient in vitamins A and D as a result of the evaluation of his eating habits and nutritional status, a recommended menu that allows sufficient intake of vitamins A and D is derived, and the degree of deficiency of vitamins A and D is determined. This means recommending nutritional supplements if the condition is judged to be serious.

Here, even if the person eating is not a student or patient, based on the analyzed user's nutritional status, eating habits, picky eating, etc., the computer determines what nutrients the user needs if the user's nutritional status is unbalanced, and if it matches the user's eating habits, A menu for the user's nutritional status can be derived and information provided to the user terminal 300, and advertising effects can be achieved by providing information on various nutritional supplements.

In addition, services that can be provided when using the user terminal 300 will be described. Since the computer can determine the amount consumed by the user for each type of food, it can analyze the user's eating habits, and can use information about the nutrients contained in each food to determine the user's nutritional intake.

In addition, the user's body information can be input through a service application installed on the user terminal 300, and the user's basal metabolic rate can be calculated using this body information.

In addition, using the user's nutritional status and eating habits information derived as above, one or more food menus matching the user can be derived and provided as a service application, and the information can also be provided in conjunction with other food ordering applications.

At this time, the service application can directly input preferred and non-preferred foods from the user, and can input information about the user's allergies and diseases to exclude foods that the user should not eat from the recommended menu.

In addition, the determination unit 130 uses this information to send a warning (notification) to the service application if food that the user should not eat is included in the food image captured through the photographing unit 55 of the portable device 50. You can prevent users from accidentally consuming the food.

In one embodiment, the service application may receive various measurement data in conjunction with wearable devices used and worn by the user. If this configuration is included, the computer can secure the user's basal metabolic rate and health information in more detail, and can also be used as various healthcare information.

The computer collects data on the amount of pre-meal, amount of food, and amount of leftover food for each meal of the user through the user terminal 300 and stores it in the database 170 to form information about the user's nutritional status, eating habits, etc. , the user's healthcare information can be formed by collecting data on the user's physical information and the user's activity index, activity information, exercise information, etc. collected through various health and exercise applications.

In addition, by collecting the user's medical information in conjunction with health management applications and medical information applications, information on customized diet and health supplements is derived from the user's nutritional status, eating habits, healthcare information, and medical information. You can provide it, or you can sell related products directly through a service application.

For example, when signing up for the service of a service application provided by an external server (not shown), a comprehensive checkup may be provided, customized health supplements may be provided to the user, and a customized menu may be delivered once a week. Various types of services can be provided, such as giving.

Hereinafter, a device and server for implementing a food measurement method according to an embodiment of the present invention will be described.

With reference to Figure 10, the food measurement server 100 according to an embodiment of the present invention will be described. Figure 10 is a block diagram of a food measurement server according to an embodiment of the present invention, and Figure 11 is a detailed block diagram of the management unit of Figure 10.

Referring to Figure 5, the food measurement server 100 according to an embodiment of the present invention includes a reception unit 110, an extraction unit 120, a determination unit 130, a calculation unit 140, a correction unit 150, and an authentication unit. It includes a unit 160, a database 170, and a communication unit 190.

However, in some embodiments, the server 10 may include fewer or more components than those shown in FIG. 5 .

The receiving unit 110 receives tableware images photographed through one or more photographing units 250.

The extraction unit 120 extracts food image data accommodated in each space 510 of the tableware 500 from image data of dishes containing food.

The determination unit 130 determines the type of each food through the extracted food image data.

The calculation unit 140 calculates the volume of each food using height information for each pixel of the extracted food image data.

The database 170 stores empty dishware images, dishware information including the size, depth, and capacity of the plurality of spaces 510 formed in the dishware 500, and food information including the weight per volume of each food.

The management unit 180 includes a ration amount control module 181, a ration group management module 183, a ration target management module 185, a ration menu recommendation module 187, and a ration menu evaluation module 189, and each module The function is the same as described above.

The communication unit 190 performs communication to receive images of dishes taken from the photography unit 250 installed in the restaurant, and transmits various information calculated and generated by the server 10 to the terminal of the person in charge or user. can play a role.

The authentication unit 160 authenticates user information received from devices such as the food measuring device 200 and the kiosk device 30, identifies the user, and loads various information about the user.

In addition, information on the weight and calories of the food in the serving dish calculated through the calculation unit 140 of the server 10 is provided to the terminal of the authenticated user through the authentication unit 160. At this time, information can be provided through a service application provided by the server 10 itself.

Additionally, a computer (server, device) according to an embodiment of the present invention can perform the following operations to improve the accuracy of food calculation.

In one embodiment, the calculation unit 140 verifies whether the sum of the meal amount and leftover amount of the meal subject calculated through the above steps matches the ration amount, and if there is a discrepancy according to the verification result, the calculation unit 140 At least one of the meal amount, leftover amount, and ration amount calculated through the above steps can be corrected so that the sum of the meal amount or leftover amount matches the ration amount, and the result is recorded in the database 170.

In addition, the calculation unit 140 verifies whether the sum of the total amount of meals and the total amount of leftovers calculated through the above steps of the restaurant diners matches the total amount of food served, and if there is a discrepancy according to the verification result, the calculation unit 140 (140) can correct at least one of the total amount of meals, the total amount of leftovers, and the total amount of rations calculated through the above steps so that the sum of the total amount of meals and the total amount of leftovers matches the total amount of rations, The results are recorded in the database 170.

More specifically, the calculation unit 140 verifies whether the difference between the total amount of food served by the diners eating at the restaurant and the total amount of leftover food matches the total amount of food.

In addition, the calculation unit 140 performs verification by comparing the total amount of food prepared in the restaurant with the total amount of leftover food generated in the restaurant, the total amount of leftover food generated in the restaurant, and the total amount of meals of the dining subjects of the restaurant. If a difference occurs according to the results, the calculation unit 140 ensures that the total amount of food prepared in the restaurant matches the total amount of leftover food generated in the restaurant, the total amount of leftover food generated in the restaurant, and the total amount of meals of the people eating in the restaurant. At least one of the total amount of leftover food, the total amount of leftover food, and the total amount of meals can be corrected, for example, the amount of meals can be corrected, and the result can be recorded in the database 170.

Here, the amount of leftover food is the amount of food left after being distributed to the person eating, and the amount of food remaining may be the amount of food remaining without being distributed to the person eating, and the amount of food remaining is calculated through the above steps or through a separate method. can be calculated.

More specifically, the database 170 may store the measurement results of the total volume (weight) of food prepared for each meal in a restaurant, the total volume (weight) of remaining food, and the total volume (weight) of remaining food, The calculation unit 140 compares the total volume (weight) of the prepared food with the total volume (weight) of the remaining portions, the total volume (weight) of the remaining portions, and the sum of the meal amounts of the restaurant eaters calculated through the above steps. Verification is performed, and if a difference occurs according to the verification result, the calculation unit 140 determines that the total amount of food prepared in the restaurant is the total amount of leftover food generated in the restaurant, the total amount of leftover food generated in the restaurant, and the meal amount of the people eating at the restaurant. The total amount of meals of the eating subjects calculated through the above steps can be corrected to match the sum of the sum of , and the result can be recorded in the database 170. (For example, if the total volume of Menu A prepared in a restaurant is 200 and all are served without any leftovers, the total volume of Menu A served to diners should be close to 200)

In this way, the calculation unit 140 performs verification using various types of information and records the accuracy resulting from calculation/correction, thereby making it possible to identify problems and improve them.

In one embodiment, the artificial intelligence algorithm of the calculation unit 140 according to an embodiment of the present invention may perform volume recognition accuracy comparison and supplementation using image learning results as a way to verify volume recognition and increase reliability. For example, accuracy can be increased by inputting images of different amounts of bread, making judgments about whether the volume is small or large, and relearning when errors or noise occur.

The food measurement server 100 according to the embodiment of the present invention described above is different from the food measurement method described through FIGS. 1 to 9 only in the category of the invention, and since the content is the same, overlapping descriptions and examples will be omitted.

Hereinafter, Figure 12 is a block diagram of a food measuring device according to an embodiment of the present invention, Figures 13 to 16 are a perspective view of a food measuring device according to an embodiment of the present invention, and Figure 17 is a block diagram of a food measuring device according to an embodiment of the present invention. This is the screen of the food measuring device.

The food measuring device 200 may be provided at a place where food is distributed, and includes a receiving unit 110, an extraction unit 120, a judgment unit 130, a calculation unit 140, a correction unit 150, and an authentication unit 160. ), database 170, communication unit 190, display unit 210, payment unit 240, and photography unit 250. However, in some embodiments, the food measuring device 200 may include fewer or more components than the components shown in FIG. 12 .

Referring to Figures 13 and 14, the food measuring device 200 is provided with a measuring area 220 or a plate 230 on which the tableware 500 is placed. Referring to FIGS. 13 and 14 , the photographing unit 250 photographs the tableware 500 placed on the measurement area 220 or the plate 230.

In addition, the measurement area 220 or the plate 230 has a weight measuring device (not shown) formed inside it to measure the weight of the food dish placed on the upper surface.

The receiving unit 110 receives tableware images photographed through one or more photographing units 250. Here, the photographing unit 250 is positioned to photograph the upper surface of the tableware 500 from the upper side of the tableware 500, and when the identification information detection unit 251 is present, the identification information detection unit 251 is the photographing unit. Simultaneously with the photographing of the tableware 500 by 250, the identification information of the tableware 500 may be positioned so that it can be sensed from the top, bottom, or side.

The extraction unit 120 extracts food image data accommodated in each space 510 of the tableware 500 from image data of dishes containing food.

The determination unit 130 determines the type of each food through the extracted food image data.

The calculation unit 140 calculates the volume of each food using height information for each pixel of the extracted food image data.

The database 170 stores empty dishware images, dishware information including the size, depth, and capacity of the plurality of spaces 510 formed in the dishware 500, and food information including the weight per volume of each food.

The communication unit 190 performs communication to receive images of dishes taken from the photography unit 250 installed in the restaurant, and transmits various information calculated and generated by the server 10 to the terminal of the person in charge or user. can play a role.

The authentication unit 160 authenticates user information received from devices such as the food measuring device 200 and a kiosk device to identify the user and loads various information about the user.

Information on the weight and calories of the food in the serving dish calculated through the calculation unit 140 of the server 10 is provided to the terminal of the authenticated user through the authentication unit 160. At this time, information can be provided through a service application provided by the server 10 itself.

The food measuring device 200 includes a display 240, and may receive user authentication information that can authenticate the user, and may display a captured image through the display.

In addition, when the food measuring device 200 of the type shown in FIG. 13 is provided, when the user places the dishes 500 in the measurement area 220, the photographing unit 250 provided in the measurement area 220 photographs the dishes and You can obtain tableware images and post-meal tableware images. In addition, the food measuring device 200 itself is provided with a payment unit 240 so that the user can measure food and make payments at the same time.

In some embodiments, referring to FIG. 14 (C), the food measuring device 200 may not include the plate 230, and the food measurement device 200 may be photographed by the photographing unit 250 while the eating subject is holding the tableware 500. Detection may be performed by the identification information detection unit 251.

In some embodiments, referring to Figure 15, the food measuring device 200 may be installed in the discharge port where the conveyor belt 270 is installed. While the dishes 500 are moved by the conveyor belt 270, the photographing unit 250 may photograph the dishes 500 from the upper side of the dishes 500.

In some embodiments, the food measuring device 200 may include an imaging unit 250, an identification information detection unit 251, and an authentication unit 160, and the imaging unit 250 is located in a relatively highest area, The authentication unit 160 may be located between the photographing unit 250 and the identification information detection unit 251, and the identification information detection unit 251 may be located in a relatively lowest area. Since the food measuring device 200 has this structure, just by the eating subject stopping in front of the food measuring device 200, the photographing unit 250, the identification information detecting unit 251, and the authentication unit 160 can eat. Necessary information can be obtained from the main body and the utensil 500, and through this, the inconvenience of the main meal can be reduced.

The purpose of positioning the photographing unit 250 at the highest area is to photograph the dishes 500 from above. Assuming that the authentication unit 160 recognizes the face of the eating subject, when the eating subject picks up the tableware 500, the tableware 500 is located below the dining subject's face, so the authentication unit 160 provides identification information. It may be located relatively above the sensing unit 251. Here, the identification information 530 of the tableware 500 is located on the side of the tableware 500 and can be recognized by the identification information detection unit 251 when the person eating is standing holding the tableware 50.

Meanwhile, the identification information detection unit 251 and the authentication unit 160 do not always operate, and can operate only when the tableware 500 is photographed and recognized by the photographing unit 250, through which the photographing unit 250 Recognition of the tableware 500 by 250 serves as a kind of enabler and can effectively operate the identification information detection unit 251 and the authentication unit 160.

Referring to Figure 16, the food measuring device 200 can be placed on a table or hung on a wall. A display unit 210 is located in the front of the food measuring device 200, and a photographing unit 250a for photographing food or dishes containing food is located in front of the food measuring device 200 and a photographing portion 250a for photographing biometric information of the eating subject. An imaging unit 250b is located for.

The food measuring device 200 may be tilted with the upper part inclined toward the front, and the photographing unit 250a for photographing food or dishes containing food is located in the tilted area of the food measuring device 200 and is positioned below. You can take a picture of the tableware (500). Although the photographing unit 250b for photographing biometric information of the person eating is shown not to be in the tilted area, it may be located in the tilted area.

And when the person eating photographs the tableware 500 containing food using the food measuring device 200 or the user terminal 300, the screen of FIG. 17 may be displayed on the display unit 210/310.

For example, an image of a tableware containing food captured by the photographing unit 250/350 may be displayed, and information about the type of food and the corresponding calories may be displayed. And when a beverage, etc. is placed in a cup, the height of the beverage in the cup may be displayed. In addition, information about the exercise and exercise time required to consume calories from food can also be displayed. In addition, nutritional information and price information of the food to be consumed can also be displayed. In addition, FIGS. 18 and 19 differ only in that the user terminal 300 performs the role of the food measuring device 200 compared to FIGS. 10 and 12, respectively, and detailed descriptions are omitted.

The detailed contents of the food measuring device 200 according to the embodiment of the present invention described above are different from the food measuring method described through FIGS. 1 to 9 only in the category of the invention, and since the content is the same, redundant description, Examples are omitted.

The food measuring method according to an embodiment of the present invention described above may be implemented as a program (or application) and stored in a medium to be executed in combination with a server, which is hardware.

The above-mentioned program is C, C++, JAVA, machine language, etc. that the processor (CPU) of the computer can read through the device interface of the computer in order for the computer to read the program and execute the methods implemented as a program. It may include code coded in a computer language. These codes may include functional codes related to functions that define the necessary functions for executing the methods, and include control codes related to execution procedures necessary for the computer's processor to execute the functions according to predetermined procedures. can do. In addition, these codes may further include memory reference-related codes that indicate at which location (address address) in the computer's internal or external memory additional information or media required for the computer's processor to execute the above functions should be referenced. there is. In addition, if the computer's processor needs to communicate with any other remote computer or server in order to execute the above functions, the code uses the computer's communication module to determine how to communicate with any other remote computer or server. It may further include communication-related codes regarding whether communication should be performed and what information or media should be transmitted and received during communication.

The storage medium refers to a medium that stores data semi-permanently and can be read by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specifically, examples of the storage medium include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc., but are not limited thereto. That is, the program may be stored in various recording media on various servers that the computer can access or on various recording media on the user's computer. Additionally, the medium may be distributed to computer systems connected to a network, and computer-readable codes may be stored in a distributed manner.

The steps of the method or algorithm described in connection with embodiments of the present invention may be implemented directly in hardware, implemented as a software module executed by hardware, or a combination thereof. The software module may be RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), Flash Memory, hard disk, removable disk, CD-ROM, or It may reside on any type of computer-readable recording medium well known in the art to which the present invention pertains.

Above, embodiments of the present invention have been described with reference to the attached drawings, but those skilled in the art will understand that the present invention can be implemented in other specific forms without changing its technical idea or essential features. You will be able to understand it. Therefore, the embodiments described above should be understood in all respects as illustrative and not restrictive.

Claims (14)

An artificial intelligence-based leftover analysis device,
a photographing unit that acquires a first image by photographing dishes after a meal;
a determination unit that analyzes the first image based on artificial intelligence and determines the type of leftover food based on the analysis result;
a calculation unit that calculates the amount of leftover food for each type of leftover food;
Evaluate the preferences of the diners for each food menu based on the type of leftovers and the amount of leftovers for the diners at a specific meal location, and prepare recommended menus and meals for the diners based on the results of evaluating the preferences. Meal menu recommendation unit that derives quantity; and
Includes a storage unit where the relevance of each food is stored,
Based on the results of evaluating the preference, the first food menu preferred by the dining subjects is determined, and the degree of relevance for each food is used to determine a food menu that the dining subjects are likely to prefer based on the first food menu. Featuring a choice of 2 food menus,
Artificial intelligence-based leftover analysis device.
According to paragraph 1,
The device is,
Characterized in evaluating the preference of each food menu based on the amount of change in the calculated amount of leftover food,
Artificial intelligence-based leftover analysis device.
According to paragraph 1,
The device is,
Characterized in extracting a food image from the first image by distinguishing between food and non-food in the first image using a pre-trained artificial intelligence model,
Artificial intelligence-based leftover analysis device.
According to paragraph 1,
The photographing unit acquires image data and depth data,
The device is,
Calculate the distance from the first image to a plurality of areas exposed without being obscured by food,
Calculate the slope of the dish image in the first image based on the calculated distance,
Characterized in correcting the tableware image in the first image to be horizontal based on the calculated tilt,
Artificial intelligence-based leftover analysis device.
According to paragraph 4,
The device is,
Confirming a flat area among the exposed areas that are not covered by food in the tableware image in the first image,
Characterized by determining three or more points among the confirmed flat areas and checking depth data for the three or more determined points,
Artificial intelligence-based leftover analysis device.
According to paragraph 1,
The device is,
When a second image of the tableware after the meal is received through the terminal of the person eating, tilt information measured through the gyro sensor of the terminal is received, and based on the received tilt information, the second image is received. 2 Characterized in correcting the tilt of the tableware image in the image,
Artificial intelligence-based leftover analysis device.
According to paragraph 1,
The device is,
It uses a model learned using an artificial intelligence algorithm,
Receive menu information provided by the specific dining location as optimization information,
In order to increase the accuracy of judgment about the type of food, the model is trained based on the optimization information to generate and provide a reduced local model in which the number of food type classes is optimized for the specific dining location.
Artificial intelligence-based leftover analysis device.
According to paragraph 1,
The device is,
If the tableware includes a plurality of spaces, extract a food image for each space,
When the space in the tableware is not separated, the entire food image in the tableware is extracted, and each food image is separately extracted from the entire food image,
Artificial intelligence-based leftover analysis device.
A method performed by a leftover analysis device,
Receiving a first image of dishes after a meal;
Analyzing the first image based on artificial intelligence and determining the type of leftover food based on the analysis result;
Calculating the amount of leftover food for each type of leftover food;
Evaluating preferences for each food menu of the diners at a specific dining location based on the type and amount of leftover food for the diners;
Deriving recommended menus and meal preparation amounts for the dining subjects based on the results of evaluating the preferences;
Based on the results of evaluating the preference, the first food menu preferred by the eating subjects is determined, and the possibility that the eating subjects prefer based on the first food menu using the relevance of each food stored in the device comprising selecting this elevated second food menu,
Artificial intelligence-based leftover analysis method.
A program combined with a hardware computer and stored in a medium to execute the method of claim 9.





delete delete delete delete