KR20200126576A - A method, server, device and program for measuring amount of food - Google Patents

Abstract

The present invention relates to a method for measuring the amount of food. The present invention calculates the amount of food based on information on the size, depth, and capacity of a plurality of spaces formed on a plate containing food, and an image photographed from a photographing means. The method for measuring the amount of food comprises the following steps of: extracting, by an extraction unit, the food image data accommodated in each space of a food plate from food plate image data taken by one or more photographing units; determining, by a determination unit, the type of each food based on the extracted food image data; and calculating, by a calculation unit, the volume of each food by using height information for each pixel of the extracted food image data.

Description

{A method, server, device and program for measuring amount of food}

The present invention relates to a method of measuring the amount of food.

In recent years, as more people want to maintain a healthy diet such as well-being and diet, the demand for a technology for measuring food quantity is increasing.

By using this, even in places where food is served to multiple people, such as schools, companies, military, government offices, hospitals, etc., the amount of food served and fed to people is measured, and the amount of food and residual quantity is measured by measuring the amount of food served and the amount of supply. There are many advantages, such as being able to perform efficient food management in anticipation of and manage the calories of the distributors.

However, since most of the currently released technologies are merely searching for images taken through a camera, their accuracy is remarkably degraded, and since the accuracy in image search is poor, a larger error occurs in the next steps such as calorie calculation. There is a problem.

Accordingly, the applicant of the present invention came up with an invention that can accurately analyze and calculate food, not just searching for food by image.

Republic of Korea Patent Publication No. 10-2005-0083210 (2005.08.26)

The present invention for solving the above-described problem can provide a food amount measurement method of calculating the volume of food by using height information for each pixel of an image photographed through a stereo camera or a depth camera.

In addition, the present invention can provide a food amount measuring method for correcting the volume of food calculated by using the information on the serving plate.

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems that are not mentioned will be clearly understood by those skilled in the art from the following description.

The method of measuring the amount of food according to an embodiment of the present invention for solving the above-described problem includes the steps of: analyzing an image of a serving plate (the user's serving plate) by an analysis unit to recognize one or more foods accommodated in the plate; Calculating, by a calculation unit, the volume of each food using the plate information and the image of the food in the plate image; And calculating the weight of each food by using the food information and the calculated volume of each food by the calculation unit, wherein the server includes a basic plate (pre-dining plate) image and a plurality of spaces formed on the plate. It includes a database in which food information including plate information including size, depth and capacity of each food, and food information including weight per volume for each food is stored.

In addition, the food quantity measurement server according to an embodiment of the present invention for solving the above-described problems includes a basic plate (pre-dining plate) image and plate information including size, depth and capacity for a plurality of spaces formed on the plate, A database storing food information including weight per volume for each food; An analysis unit that analyzes the image of the serving plate (the plate received by the user) and recognizes one or more foods accommodated in the plate; And a calculation unit that calculates the volume of each food using the food plate information and the image of the food in the food plate image, and calculates the weight of each food using the food information and the calculated volume of each food; .

In addition, the food quantity measurement apparatus according to an embodiment of the present invention for solving the above-described problem includes a basic plate (pre-food plate) image and plate information including size, depth, and capacity for a plurality of spaces formed on the plate, A database storing food information including weight per volume for each food; An analysis unit that analyzes the image of the serving plate (the plate received by the user) and recognizes one or more foods accommodated in the plate; And a calculation unit that calculates the volume of each food using the food plate information and the image of the food in the food plate image, and calculates the weight of each food using the food information and the calculated volume of each food; .

In addition to this, another method for implementing the present invention, another system, and a computer-readable recording medium for recording a computer program for executing the method may be further provided.

According to the present invention as described above, it is possible to calculate the volume of food by calculating the volume of food by using height information for each pixel of an image photographed through a stereo camera or a depth camera.

In addition, the present invention has the effect of more accurately calculating the volume of food by correcting the volume of food calculated through information on the size, depth, and capacity of the plate spaces.

The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 is a flowchart of a method for measuring food amount according to an embodiment of the present invention.
2 is an exemplary view of a basic plate image according to an embodiment of the present invention.
3 is an exemplary diagram of an image of a food plate containing food according to an embodiment of the present invention.
4 is a view illustrating that two foods are stacked in one space in FIG. 3.
5 is a block diagram of a food quantity measurement server according to an embodiment of the present invention.
6 is a block diagram of a food amount measuring apparatus according to an embodiment of the present invention.
7 and 8 are exemplary diagrams of a food quantity measuring apparatus according to an embodiment of the present invention.

Advantages and features of the present invention, and a method of achieving them will become apparent with reference to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in a variety of different forms, only the present embodiments are intended to complete the disclosure of the present invention, It is provided to fully inform the technician of the scope of the present invention, and the present invention is only defined by the scope of the claims.

The terms used in the present specification are for describing exemplary embodiments and are not intended to limit the present invention. In this specification, the singular form also includes the plural form unless specifically stated in the phrase. As used in the specification, “comprises” and/or “comprising” do not exclude the presence or addition of one or more other elements other than the mentioned elements. Throughout the specification, the same reference numerals refer to the same elements, and “and/or” includes each and all combinations of one or more of the mentioned elements. Although "first", "second", and the like are used to describe various elements, it goes without saying that these elements are not limited by these terms. These terms are only used to distinguish one component from another component. Therefore, it goes without saying that the first component mentioned below may be the second component within the technical idea of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used in the present specification may be used as meanings that can be commonly understood by those of ordinary skill in the art to which the present invention belongs. In addition, terms defined in a commonly used dictionary are not interpreted ideally or excessively unless explicitly defined specifically.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Prior to the description, the meaning of terms used in the present specification will be briefly described. However, it should be noted that the description of terms is not intended to limit the technical idea of the present invention unless explicitly described as limiting the present invention, since it is intended to help the understanding of the present specification.

Food Service: A place that provides meals to groups, such as schools, companies, and the military.

Distributor: This refers to the location where food is served at the deli.

Exit: Refers to the location where people (to be served) who have finished eating at the bakery return the plate 500.

1 is a flow chart of a method for measuring the amount of food according to an embodiment of the present invention, FIG. 2 is an exemplary diagram of a basic food plate image according to an embodiment of the present invention, and FIG. 3 is a food plate containing food according to an embodiment of the present invention. An exemplary diagram of an image, and FIG. 4 is a diagram illustrating that two foods are stacked in one space 510 in FIG. 3.

A method of measuring food amount according to an embodiment of the present invention will be described with reference to FIGS. 1 to 4.

The food amount measurement method according to the embodiment of the present invention is performed by a computer, and the computer refers to the food amount measuring device 30 or the food amount measuring server 10.

The receiving unit 110 receives the food plate image photographed through the photographing unit 50. (S520 step)

At this time, the serving plate means the plate 500 in a state in which the user has received food on the plate 500, and the basic plate means the basic plate 500 before receiving the food.

The photographing unit 50 includes at least one of a stereo camera and a depth camera, and preferably, two distribution plates photographed through the first photographing unit 53 and the second photographing unit 55 installed at a certain distance apart. Receive the image.

When a stereo camera is included in the photographing unit 50, since one photographing unit 50 includes two photographing means, the calculation unit 140 uses the difference value photographed through the stereo camera to determine the image data. The depth can be calculated.

The depth camera can be applied to any device such as a 3D scanner, a 3D sensor, a Time of Flight (ToF) camera, an infrared (IR) camera, etc. that can photograph a target and acquire 3D distance information (data).

In addition, it is preferable that the first photographing unit 53 and the second photographing unit 55 are spaced a predetermined distance at a position where the meal plate 500 can be photographed from the upper side so that the meal plate 500 can be photographed at different angles. Do.

Since a camera is a device that recognizes light, it is highly affected by environment such as light reflection. Therefore, by taking a picture using two or more photographing units 50 as above, the effect of the environment is minimized, and even if a picture is taken using two or more photographing units 50, the environment may be affected. I need this.

Accordingly, in order to optimize according to the characteristics and environment of the distribution center, the computer learns the situation of the field through the images taken by the learning module, and recognizes the environmental condition based on filter processing, fixed objects, and reference objects.

After step S520, the correction unit 150 may perform image correction.

In more detail, the correction unit 150 may remove a noise component using the image data of the two serving plates, recognize the inclination of the plate image, and correct the image in a horizontal state.

As described above, when taking a picture of the food plate at the bakery, it is possible to stably photograph the food plate by placing it in a specific position, but depending on the circumstances of the bakery, it is also possible to take a picture of the user walking with the plate 500.

Therefore, since the angle of the plate may be inclined, the correction unit 150 recognizes the inclination (inclined information) of the plate image and corrects the plate image to a horizontal image to accurately calculate the volume of the food. To be able to do it.

In addition, since noise may occur in the image data of the two serving plates received through the receiving unit 110 according to the situation (shake, light reflection, etc.) in the field, the noise can be removed by comparing the two plate images. .

Due to the configuration and operations of the above-described correction unit 150, the food quantity measurement method according to the embodiment of the present invention captures an image of a user walking with a food plate without a separate food quantity measurement device in the bakery. Thus, even in the case of obtaining the image of the plate image and the image of the plate, the angle of the plate is corrected to a horizontal state, thereby exerting the effect of increasing the accuracy when the subsequent steps are performed.

Step S520 Next, the extraction unit 120 extracts food image data accommodated in each space 510 of the plate 500 from the plate image data. (S530 step)

More specifically, since the image of the food plate 500 and the food are included in the food plate image, it means that food image data is extracted from the plate 500.

Step S530 Next, the determination unit 130 determines the type of each food through the extracted food image data. (S540 step)

The determination unit 130 may determine the type of food using data photographed through an RGB camera included in the photographing unit 50 or RGB data photographed through a stereo camera.

In this case, as a preceding step, a step of receiving, by the receiving unit 110, information on a list of foods being served at the bakery (step S510) may be further included.

Then, in the determination step (S540), the determination unit 130 may determine the type of each food by matching the extracted food image data and food list information.

In this way, the determination unit 130 may determine the type of food by analyzing the food image data, or may determine the type of food by matching with the stored food list information.

Step S540 Next, the calculation unit 140 calculates the volume of each food by using height information (three-dimensional distance data) for each pixel of the extracted food image data. (S550 step)

As mentioned above, since the food plate image was photographed through the photographing unit 50 including a stereo camera or a depth camera, height information for each pixel (three-dimensional distance data) is included in the food image data, and using this You can calculate the volume of each food.

In one embodiment, a single plate image is generated by correcting two food plate images taken through the first and second photographing units 53 and the second photographing unit 55, and volume calculation is performed using the collected image data. You can also do it.

In one embodiment, the computer includes a database 170 in which food plate information including the size, depth and capacity of the image and the plurality of spaces 510 formed on the plate 500 is stored. Include.

In addition, the volume calculation step (S550) may further include a step of correcting the volume of each food calculated using the size, depth, and capacity information of the space 510 in which each food is accommodated.

In more detail, since data on the food plate 500 used in the food service is stored in the database 170 of the computer, the size (volume) of each extracted food can be more accurately calculated using this.

In addition, since the depth of each space 510 is different according to the type of the plate 500, if the correction unit 150 uses the plate information, it is possible to more accurately determine the degree of food accumulation.

In an embodiment, the volume calculation step (S550) may further include calculating the volume of the stacked food (S555).

In more detail, a plurality of foods may be stacked in a specific space 510 in the plate 500. Therefore, when it is recognized that different foods are stacked on a specific space 510 in the food plate 500, the calculation unit 140, the image data of the food located on the upper side, the calculated volume information, and the corresponding space 510 The volume of food located on the lower side is calculated using the size, depth, and volume information.

3 (a) is a diagram illustrating that fried eggs are stacked on the rice in a space 510 in which rice is accommodated in a distribution plate, and (b) is a diagram illustrating that fish are stacked.

Referring to FIG. 3, for example, the calculation unit 140 calculates the size and volume of the fried egg through plate information and image data of the fried egg.

Then, the calculation unit 140 can predict the height of the rice accommodated in the liver 510 through the size and volume information of the fried egg, and the volume of rice accommodated in the liver 510 using the plate information. Can be calculated.

In addition, in the same manner as in FIG. 3(b), the calculation unit 140 calculates the size and volume of the fish through the plate information and the image of the fish, and uses this to calculate the volume of rice accommodated in the liver 510. .

In one embodiment, the database 170 of the computer may previously store data on foods stacked on top of other foods. (For example, fried eggs, fish, seaweed, etc.)

In addition, when it is determined that the type of food accommodated in the specific space 510 in the plate 500 is a liquid, the calculation unit 140 may provide a location where the liquid and the corresponding space 510 abut and the size of the space 510, The volume of the liquid can be calculated using the depth and volume information.

When food in a liquid form such as'guk' is accommodated in a specific space 510, there is inevitably a portion where the'guk' and the corresponding space 510 abut at a certain height. Using this, the calculation unit 140 recognizes the location where the food in the form of a liquid and the corresponding space 510 contact in the image of the food plate, and uses the space information (size, depth, capacity) of the space 510 to To calculate the volume.

For example, a portion where the'station' and the space 510 abuts is displayed in the'A' area shown in FIG. 3, and the calculation unit 140 contains the liquid and the liquid through the same area as the area A in the image data. The position where the space 510 abuts can be confirmed.

Meanwhile, the volume calculation step (S550) may further include a step of correcting, by the correction unit 150, a noise component photographed together with the food plate in the food plate image. Specifically, the correction unit 150 detects a noise component that is not food and has a volume in the plate image data, and calculates the volume of the noise component detected from the volume of food calculated in the step of calculating the volume of each food. It may further comprise the step of correcting the correction to be excluded.

Here, the noise component is not food and has a volume, and the volume of the food may be calculated larger than the actual volume due to the volume of the noise component. Therefore, the correction unit 150 recognizes the noise component in the captured image in order to accurately calculate the volume of the food, and performs correction to exclude the volume of the noise component from the volume of the food calculated by the calculation unit 140. I can.

For example, a hand or a spoon may be a noise component having a volume while not being food, and the lid of a dairy product may correspond to this, but is not limited thereto.

For example, in the step of correcting the noise component, the correction unit 150 recognizes the image of the user's hand from the food plate image and excludes the volume value of the hand from the volume value of the food calculated by the calculation unit 140 It is possible to perform the correction to make.

For example, as described above, an image may be obtained by photographing a user walking while holding the plate 500 according to the circumstances of the food plate, but an error may occur in calculating the volume of the food due to the image data of the hand holding the plate. In addition, since the correction unit performs a correction that excludes the volume value of the body part such as the hand from the volume value of the food calculated through the plate image data, the accuracy can be increased in the food volume calculation step.

In one embodiment, the correction unit 150 learns in advance about the image data for the empty plate 500 and the spoon, recognizes the spoonful in the image of the plate, and performs correction excluding the volume value of the spoon. can do.

In more detail, the correction unit 150 detects the presence of a user's hand or spoon in the image data of the plate. In addition, when the user's hand or spoon is included in the plate image, the volume value of the user's hand or spoon is excluded from the volume value of the food calculated through the plate image data.

For example, the correction unit 150 may calculate the volume of the food as the volume of the spoon is included in the volume of the food when the spoon is included in the image of the plate image and the image of the after-meal taken through the photographing unit. It is possible to perform correction to exclude the volume from the volume of food calculated through the plate image data.

Due to the configuration and operation of the correction unit 150 described above, the food amount measurement method according to the exemplary embodiment of the present invention has an effect of increasing the accuracy when subsequent steps are performed.

Next to step S550, the calculation unit 140 calculates the weight of each food by using the volume information and food information of each food calculated in step S550. (S560 step)

The database 170 of the computer stores food information including weight per volume for each food.

Different types of foods have different weight per volume and different calories (calories) from each other. Accordingly, in an embodiment of the present invention, food information including food items per volume per food is stored in the database 170, and the calculation unit 140 calculates the weight of the food from the food volume information using this.

In addition, the database 170 may store calorie information per weight for each food.

Using this, the calculation unit 140 calculates the calories of the food accommodated in the user's plate by using the calculated weight information of each food after the weight calculation step (step S560), and calculates the calculation result by the person in charge or the user. It can be provided to the terminal.

In an embodiment, a weight measuring device (not shown) capable of measuring the weight of the plate may be provided in the bakery.

Through this, the receiving unit 110 receives the weight information of the meal plate 500 measured from the weighing device (not shown) provided in the bakery, and the correction unit 150 It may further include the step of correcting the weight of each food by matching the total weight of the weight and the weight of the basic plate and the received weight information.

The above-described correction step can be selectively adopted and used, and since the weight may be measured differently from the actual weight by the user's hand or various factors in the process of measuring the weight, the weight information is measured differently to the extent that it exceeds the threshold. If so, the correction step may not be performed.

By using the food amount measurement method according to an embodiment of the present invention, the computer measures the amount of ration and the amount remaining of each ration target eaten at the deli, and the amount of meal may be calculated using this.

In addition, the computer may generate the operation information of the deli and management information of the rations by using the amount of ration, the amount remaining, and the amount of meals of the rations to be eaten at the deli. (Computer consists of server, device)

In more detail, the computer acquires the meal plate data (meal plate image data) photographed through the meal tray photographing unit 50 provided in the food tray, and uses this to calculate the amount of food to be served.

Then, the computer acquires the after-meal plate data (post-meal image data) photographed through the retirement zone photographing unit 50 provided in the retirement zone, and calculates the remaining amount of the serving target using this.

Next, the computer calculates the amount of meals to be served by using the calculated amount of ration and the amount remaining.

In addition, the computer generates the operation information of the deli based on the amount of ration, remaining amount, and amount of meals registered in the deli, and each serving based on the amount of ration, remaining amount, and amount of meals registered in the deli. It creates management information of targets.

At this time, the meal plate means the plate 500 in which the food is served on the plate 500, and has the same meaning as the plate plate mentioned above.

In this way, by using the method of measuring the amount of food, the device 30, and the server 10 according to the embodiment of the present invention, the amount of ration, the amount of meal, and the amount of remaining meals are calculated to provide information on the ration It becomes possible to process.

5 is a block diagram of a food quantity measurement server 10 according to an embodiment of the present invention.

5, the food amount measurement server 10 according to an embodiment of the present invention includes a receiving unit 110, an extraction unit 120, a determination unit 130, a calculation unit 140, a correction unit 150, and authentication. It includes a module 160, a database 170, and a communication unit 190.

However, in some embodiments, the server 10 may include fewer or more components than the components illustrated in FIG. 5.

The receiving unit 110 receives the image of the food plate photographed through one or more photographing units 50.

The extraction unit 120 extracts food image data accommodated in each space 510 of the food plate 500 from the food plate image data.

The determination unit 130 determines the type of each food through the extracted food image data.

The calculation unit 140 calculates the volume of each food by using height information for each pixel of the extracted food image data.

The database 170 stores the basic food plate image, food plate information including the size, depth, and capacity of the plurality of spaces 510 formed on the plate 500, and food information including the weight per volume of each food.

The communication unit 190 performs communication so that the image of the food plate taken from the photographing unit 50 installed in the food service can be received, and various information generated by the server 10 to the person in charge or the user's terminal. Can play a role in transmitting.

The authentication module 160 identifies a user by authenticating user information received from a device 30 such as the food quantity measurement device 30 and the key-ost device 30, and loads various types of information about the user.

In addition, information on the weight, calories, etc. of foods in the food tray calculated through the calculation unit 140 of the server 10 is provided to the terminal of the authenticated user through the authentication module 160. In this case, information may be provided through a service application provided by the server 10 itself.

In addition, the computer (server, device) according to an embodiment of the present invention may perform the following operations to improve the accuracy of calculating the amount of food.

In one embodiment, the calculation unit 140 verifies whether the sum of the amount of food and the remaining amount of the target food calculated through the above steps matches the amount of ration, and when there is a discrepancy according to the verification result, the calculation unit 140 At least one of the amount of meal, the amount of remaining, and the amount of ration calculated through the above steps may be corrected so that the sum of the amount of food or the amount of leftover equals the amount of ration, and the result is recorded in the database 170.

In addition, the calculation unit 140 verifies whether the sum of the total amount of meals and the total amount of leftovers calculated through the above steps matches the total amount of rations, and calculates when there is a discrepancy according to the verification result. The unit 140 may correct at least one of the total amount of meals calculated through the above steps, the total amount of remaining parts, and the total amount of rations so that the sum of the total amount of meals and the total amount of leftovers matches the total amount of rations, and , And records the result in the database 170.

In more detail, the calculation unit 140 verifies whether the difference value between the total amount of rations and the total amount of leftovers of the serving targets eaten at the deli is identical to the total amount of meals.

In addition, the calculation unit 140 compares the total amount of food prepared in the bakeries with the sum of the total amount of residual food generated in the bakeries, the total amount of remaining food generated in the bakeries, and the total amount of meals of the serving targets of the deli And, if a difference occurs according to the verification result, the calculation unit 140 determines that the total amount of food prepared in the bakery is the total amount of residual food generated in the bakery, the total amount of residual food generated in the bakery, and the At least one of the total amount of remaining meals, the total amount of remaining food, and the total amount of meals may be corrected to match the sum of the total amount of meals, and for example, the amount of meals may be corrected, and the result can be recorded in the database 170 have.

Here, the remaining amount is the amount of food left over by being served by the target to be served, and the amount of remaining food may be the amount of food left without being served to the target, and the amount of remaining food is calculated through the above steps or through a separate method. Can be calculated.

In more detail, the database 170 may store the measurement result of the total volume (weight) of the amount of food prepared for each serving at the bakery (weight), the total volume of the remaining food (weight), and the total volume (weight) of the remaining amount of food. , The calculation unit 140 is a sum of the total volume (weight) of the prepared food amount, the total volume (weight) of the remaining amount, the total volume (weight) of the remaining food, and the total amount of meals of the food service targets calculated through the above steps. Compared with that, if there is a difference according to the verification result, the calculation unit 140 determines that the total amount of food prepared in the bakery is the total amount of residual food generated in the bakery, the total amount of residual food generated in the bakery, and the corresponding It is possible to correct the total amount of meals of the food service targets calculated through the above steps so as to match the sum of the total amount of meals of the food service targets at the restaurant, and the result may be recorded in the database 170. (For example, if the total volume of menu A prepared at the pantry is 200 and all are served with no remaining amount, the total volume of menu A served to the serving targets should approximate 200)

In this way, the calculation unit 140 performs verification using various types of information and records the accuracy due to calculation/correction, so that problems can be identified and improved.

In one embodiment, the artificial intelligence algorithm of the calculation unit 140 according to an embodiment of the present invention may perform volume recognition accuracy comparison and supplementation using image learning results as a method of verifying volume recognition and increasing reliability. For example, it is possible to increase the accuracy by inputting images of different amounts of bread, making a judgment about whether the volume is small or large, and re-learning when an error or noise occurs.

The food quantity measurement food quantity measurement server 10 according to the embodiment of the present invention described above differs only from the food quantity measurement method described with reference to FIGS. 1 to 4 and the category of the invention, and since the contents are the same, duplicate descriptions and examples will be omitted. .

6 is a block diagram of a food amount measuring device 30 according to an embodiment of the present invention, and FIGS. 7 and 8 are exemplary views of a food amount measuring device 30 according to an embodiment of the present invention.

6, the food amount measuring device 30 according to an embodiment of the present invention is a device 30 provided at a place where a food is served, and a receiving unit 110, an extraction unit 120, a determination unit 130, It includes a calculation unit 140, a correction unit 150, an authentication module 160, a database 170, a photographing unit 50, and a communication unit 190.

However, in some embodiments, the food amount measuring device 30 may include a smaller number of components or more components than the components illustrated in FIG. 6.

7 and 8, the food amount measuring device 30 is provided with a plate 220 on which a food plate 500 is placed.

Referring to FIG. 8, the photographing unit 50 photographs the food plate 500 placed on the plate 220, and may have the same configuration as the photographing unit 50 described with reference to FIGS. 1 to 4.

In addition, the plate 220 may have a weight measuring device (not shown) formed therein to measure the weight of the distribution plate placed on the upper surface.

The receiving unit 110 receives the image of the food plate photographed through one or more photographing units 50.

The extraction unit 120 extracts food image data accommodated in each space 510 of the food plate 500 from the food plate image data.

The determination unit 130 determines the type of each food through the extracted food image data.

The calculation unit 140 calculates the volume of each food by using height information for each pixel of the extracted food image data.

The database 170 stores the basic food plate image, food plate information including the size, depth, and capacity of the plurality of spaces 510 formed on the plate 500, and food information including the weight per volume of each food.

The communication unit 190 performs communication so that the image of the food plate taken from the photographing unit 50 installed in the food service can be received, and various information generated by the server 10 to the person in charge or the user's terminal. Can play a role in transmitting.

The authentication module 160 identifies a user by authenticating user information received from a device such as the food quantity measurement device 30 and a keyoist device, and loads various types of information about the user.

Information on the weight, calories, etc. of foods in the food tray calculated through the calculation unit 140 of the server 10 is provided to the terminal of the authenticated user through the authentication module 160. In this case, information may be provided through a service application provided by the server 10 itself.

The food amount measurement device 30 may receive user authentication information for authenticating a user, including the display 240, or may display a photographed image through the display.

In addition, when the food amount measuring device 30 in the form of FIG. 7 is provided, when the user places the food plate 500 on the measurement unit 260, the photographing unit 50 provided in the measurement unit 260 photographs the food plate. An image of a food plate and an image of a meal plate can be obtained. In addition, the payment module 250 is provided in the food quantity measurement device 30 itself, so that the user may measure the food quantity and make payment at the same time.

The detailed contents of the components of the food quantity measurement apparatus 30 according to the embodiment of the present invention described above are only different from the food quantity measurement method described with reference to FIGS. Examples are omitted.

The food amount measurement method according to an embodiment of the present invention described above may be implemented as a program (or application) to be executed in combination with a server, which is hardware, and stored in a medium.

The above-described program is C, C++, JAVA, machine language, etc. that can be read by the computer's processor (CPU) through the device interface of the computer in order for the computer to read the program and execute the methods implemented as a program. It may include a code (Code) coded in the computer language of. Such code may include a functional code related to a function defining necessary functions for executing the methods, and a control code related to an execution procedure necessary for the processor of the computer to execute the functions according to a predetermined procedure. can do. In addition, such code may further include additional information required for the processor of the computer to execute the functions or code related to a memory reference to which location (address address) of the internal or external memory of the computer should be referenced. have. In addition, when the processor of the computer needs to communicate with any other computer or server in the remote in order to execute the functions, the code uses the communication module of the computer to determine how It may further include a communication-related code for whether to communicate, what kind of information or media should be transmitted and received during communication.

The stored medium is not a medium that stores data for a short moment, such as a register, cache, memory, etc., but a medium that stores data semi-permanently and can be read by a device. Specifically, examples of the storage medium include, but are not limited to, ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like. That is, the program may be stored in various recording media on various servers to which the computer can access, or on various recording media on the user's computer. Further, the medium may be distributed over a computer system connected through a network, and computer-readable codes may be stored in a distributed manner.

The steps of a method or algorithm described in connection with an embodiment of the present invention may be implemented directly in hardware, implemented as a software module executed by hardware, or a combination thereof. Software modules include Random Access Memory (RAM), Read Only Memory (ROM), Erasable Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), Flash Memory, hard disk, removable disk, CD-ROM, or It may reside on any type of computer-readable recording medium well known in the art to which the present invention pertains.

In the above, embodiments of the present invention have been described with reference to the accompanying drawings, but those skilled in the art to which the present invention pertains can be implemented in other specific forms without changing the technical spirit or essential features. You can understand. Therefore, the embodiments described above are illustrative in all respects, and should be understood as non-limiting.

10: food quantity measurement server 30: food quantity measurement device
50: photographing unit 53: first photographing unit
55: second photographing unit 110: receiving unit
120: extraction unit 130: determination unit
140: calculation unit 150: correction unit
160: authentication module 170: database
190: communication unit 220: plate
240: display unit 250: payment module
260: measuring unit 500: plate
510: space

Claims (14)

In the method of measuring the amount of food performed by a computer,
Extracting food image data accommodated in each space of the plate from the image data of the plate (the plate received by the user) photographed by the extraction unit through one or more photographing units;
Determining, by a determination unit, the type of each food based on the extracted food image data; And
Comprising, by a calculation unit, calculating the volume of each food using height information for each pixel of the extracted food image data,
The photographing unit is to include at least one of a stereo camera and a depth camera. The method of claim 1,
The extraction step,
Receiving the image data of the two food trays photographed through the first photographing unit and the second photographing unit installed at a predetermined distance from the receiving unit;
Comprising; the step of removing a noise component from the image data of the two serving plate, recognizing the inclination of the plate image to correct the image in a horizontal state; including, method. The method of claim 1,
The correction unit detects a noise component that is not food in the plate image data and has a volume, and corrects correction to exclude the volume of the noise component detected from the volume of the food calculated in the step of calculating the volume of each food. The method further comprising a; The method of claim 1,
The computer,
Includes a database in which food plate information including the size, depth, and capacity of a basic plate (pre-food plate) image and a plurality of spaces formed on the plate is stored,
The volume calculation step,
Compensating the volume of each food calculated using the size, depth, and capacity information of the space in which each food is accommodated; further comprising a method. The method of claim 4,
The database,
Food information including weight per volume for each food is stored,
After the volume calculation step,
The method further comprising: calculating, by a calculation unit, the weight of each food using the calculated volume information of each food and the food information. The method of claim 1,
Receiving information on a list of foods being served at the bakery; further comprising,
The determining step,
The method, characterized in that the determination unit determines the type of each food by matching the extracted food image data with the food list information. The method of claim 1,
The calculation step,
When it is recognized that different foods are stacked in a specific space in the plate, the calculation unit uses image data of the food located on the upper side, calculated volume information, and the size, depth, and capacity information of the space to be located on the lower side. The method further comprising; calculating the volume of. The method of claim 1,
The volume calculation step,
When it is determined that the food accommodated in the specific space in the plate is liquid, calculating a volume of the liquid through information on a location where the liquid and the space contact and the size, depth, and capacity of the space; and further comprising. The method of claim 5,
The calculation unit calculates the amount of food before meals from the pre-meal plate data photographed through the photographing unit provided in the bakery, calculates the amount of remaining meals from the after meal plate data photographed through the photographing unit, and calculates the amount of remaining meals from the pre-meal meal amount and the amount of food after meals. The method further comprising; calculating the amount of ration, remaining amount, and amount of meal. The method of claim 9,
The calculation unit performs verification by comparing the total amount of food prepared in the bakery with the sum of the total amount of leftovers generated at the bakery, the total amount of residual foods generated at the bakery, and the total amount of meals of the serving targets at the serving location. If there is a difference, the calculation unit will determine the amount of meals so that the sum of the amount of food prepared in the bakeries is the sum of the total amount of remaining food in the bakeries, the total amount of remaining food in the bakeries, and Correcting the sum of the; further comprising, the method. The method of claim 10,
The calculation unit verifies whether or not the difference value between the total amount of rations and the total amount of leftovers of the rations at the rations matches the total amount of meals, and if a difference occurs according to the verification result, the calculation unit Correcting the total amount of meals so that the difference value between the total amount of food and the total amount of remaining food matches the total amount of food. An extraction unit for extracting food image data accommodated in each space of the plate from image data of the plate (the plate received by the user) photographed through one or more photographing units;
A determination unit determining the type of each food based on the extracted food image data;
Includes; a calculation unit for calculating the volume of each food by using height information for each pixel of the extracted food image data,
The photographing unit will include at least one of a stereo camera and a depth camera. An extraction unit for extracting food image data accommodated in each space of the plate from image data of the plate (the plate received by the user) photographed through one or more photographing units;
A determination unit determining the type of each food based on the extracted food image data;
Includes; a calculation unit for calculating the volume of each food by using height information for each pixel of the extracted food image data,
The photographing unit, the apparatus comprising at least one of a stereo camera and a depth camera. A program, combined with a computer that is hardware, stored in a medium to execute the method of any one of claims 1 to 11.

Images