KR102329480B1 - Management system of cafeteria and operation method thereof - Google Patents

Abstract

The present invention discloses a food management system. More particularly, the present invention relates to a food service management system and an operating method thereof, which provide food consumed by a user as a food service target and customized information related thereto, and manage leftovers generated in the food service.
According to an embodiment of the present invention, by using a scanner device to identify a user who is a food subject and a plate before and after a meal to obtain the amount of food and the amount of leftover food, based on this, by providing information on the leftovers of each user and useful information related thereto Users can easily determine their eating habits and nutritional status, and there is an effect that can induce users to try to reduce leftovers on their own.

Description

Meal management system and its operation method

The present invention relates to a food service management system, and to a food service management system and an operating method thereof, which provide food consumed by a user to be served and customized information related thereto, and manage leftovers generated in the food service.

Currently, it is common for students, office workers, or meal recipients who receive meals at school, corporate and other group catering centers to have more than one meal in the same restaurant. In particular, students use the school cafeteria for lunch most of the week except for weekends, and it is necessary to provide an environment that allows students in the growing stage to consume various nutrients evenly.

However, students who are not yet accustomed to solving their own meals, such as the lower grades of elementary school, often do not eat well, and if they have a picky eater, they often do not eat side dishes evenly. There is no way to determine whether or not there is, and this is true even for seniors.

In particular, in the case of elementary school students, it can be said that the parents' demands for supervision of their children's unbalanced diet are considerable, and in relation to the students' eating habits, they are contemplating ways to improve the eating speed, amount of food, and leftover habits.

In addition, office workers who frequently use corporate cafeterias want to check and manage their eating habits, and even in hospitals that operate food service systems, nurses try to accurately understand the food intake of inpatients, but currently, accurate data-based dietary information is not available. There are currently no services that can be provided.

Registered Patent Publication No. 10-0975960 (Announcement Date: 2010.08.18.)

The present invention has been devised to solve the above problems, and the present invention is to provide useful information, such as related intake information, according to the amount of leftover food to the food subject in a food service facility that provides group meals in schools, companies, and hospitals, etc. There is a task.

In order to solve the above problems, the food service management system according to an embodiment of the present invention includes: a pre-meal scanner for scanning a recognition code to identify a plate, and obtaining a pre-meal image for a pre-meal plate provided to a user who is a food target; a post-meal scanner that scans an identification code to identify a plate, and scans the post-meal plate provided to the user to obtain a post-meal image of the user's post-meal plate through the identified plate; and a management server for generating intake information for the user based on the before and after images obtained by controlling the pre-meal scanner and the post-meal scanner, wherein the management server is connected to the pre-meal scanner and the post-meal scanner to obtain scanning A device control unit for receiving data, and an image processing unit for generating the intake information in response to the scanning data received by the device control unit, wherein the image processing unit recognizes a plate in the image obtained by the pre-meal scanner and , recognizes the type of food in the plate in the image acquired by the pre-meal scanner.

In addition, it may further include an identification device for identifying the user to be fed.

In addition, the image processing unit recognizes the type of food in the plate by comparing the image acquired by the pre-meal scanner with the learning image accumulated in the knowledge database for a plurality of food-related learning images for machine learning. The type of food is recognized by using the image recognition result for each area.

In addition, the image processing unit recognizing the type of food in the plate is obtained by comparing the image acquired by the pre-meal scanner with the learning image accumulated in the knowledge database for a plurality of food-related learning images for machine learning. The type of food is recognized by matching the recognition result with a predetermined menu.

In addition, the management server further includes a food service management unit including a meal generation module for generating a meal table, the predetermined menu is derived with reference to the menu table.

In addition, after recognizing the type of food in the plate, before the image processing unit calculates the volume of food in the plate, the image processing unit uses at least one of temperature, color, shape, and height to form an area along the boundary of food is set, and the material is recognized according to the type of food for each set area.

In addition, the recognition of the plate by the image processing unit is, in the image acquired by the pre-meal scanner, partitioning the regions in which rice, soup, side dishes or dishes are placed in the plate through the interface formed on the plate, and the direction of the plate It is to determine the area in which food is placed by recognizing

In addition, the calculation of the volume of the food in the plate by the image processing unit is to extract the height within the food boundary using the depth value included in the image obtained by the pre-meal scanner, and calculate the volume using the boundary and the height will be.

In addition, the image processing unit, recognizes the plate in the image acquired by the post-prandial scanner, sets an area along the boundary surface of the food on the plate in the image acquired by the post-meal scanner, and obtains by the post-prandial scanner It calculates the volume of food for each area on the plate using the image, classifies the food, and matches it with the food before a meal.

In addition, when the image processing unit recognizes the food plate, the image processing unit sets the area along the boundary surface of the food on the plate in the image acquired by the post-meal scanner, and the image processing unit classifies the food and matching it with the food before the meal , to generate leftover information by recognizing the ingredients in the menu using the learning image of the knowledge database for machine learning, and calculating the remaining amount for each menu and each material based on the calculated volume amount.

According to an embodiment of the present invention, by using a scanner device to identify a user who is a food subject and a plate before and after a meal to obtain the amount of food and the amount of leftover food, and based on this, by providing each user's intake information and useful information related thereto Users can easily determine their eating habits and nutritional status, and there is an effect that can induce users to try to reduce leftovers on their own.

In addition, according to an embodiment of the present invention, not only the catering subjects, but also the food service center managers, nutritionists, etc. utilize the leftover information obtained through the system to grasp the nutritional intake information of each meal subject as well as food tendencies and eating behaviors. There is an effect that can improve the quality of the meal and induce the reduction of leftovers by reflecting the result in the meal service.

1 is a diagram schematically illustrating the configuration of a food service station to which a food service management system according to an embodiment of the present invention is applied.
Figure 2 is a view showing the structure of the management server of the food management system according to an embodiment of the present invention.
3a and 3b are diagrams showing the structures of the user management unit and the food service management unit constituting the management server of the food service management system according to an embodiment of the present invention.
4 is a view showing the configuration of the image processing unit among the components of the management server of the food service management system according to an embodiment of the present invention.
5 is a view showing an operating method of the food management system according to an embodiment of the present invention.
6A and 6B are diagrams illustrating an image analysis method for generating leftover information of a food service management system according to an embodiment of the present invention.
7 is a view showing the structure of the scanner used in the food management system according to an embodiment of the present invention.

Prior to the description, when a part in the entire specification "includes" or "includes" a certain component, it does not exclude other components, but may further include other components unless otherwise stated. means there is In addition, terms such as "...unit" and "...module" described in the specification mean a unit that processes at least one function or operation, which is hardware, software, or hardware and a combination of software.

In addition, the term "embodiment" herein means serving as an illustration, example, or illustration, but the subject matter of the invention is not limited by such examples. In addition, other similar terms such as "comprising", "comprising", and "having" are used, but when used in the claims, an open transition word that does not exclude any additional or other elements. is used in a similar manner to the term "comprising" as inclusive.

The various techniques described herein may be implemented in conjunction with hardware or software, or a combination of both, where appropriate. As used herein, terms such as “Unit,” “Module,” and “System” likewise refer to computer-related entities, i.e. hardware, a combination of hardware and software, software or It can be treated as equivalent to software at the time of execution. For example, a program module may be configured as equivalent to one component or a combination of two or more components. In addition, in the present invention, both the application program and hardware executed in the server or terminal may be configured in module units, and may be recorded in one physical memory, or may be recorded while being distributed between two or more memories and recording media.

Hereinafter, a food service management system and an operating method thereof according to a preferred embodiment of the present invention will be described with reference to the drawings.

1 is a diagram schematically illustrating the configuration of a food service station to which a food service management system according to an embodiment of the present invention is applied.

Referring to Figure 1, the food management system according to an embodiment of the present invention is a first scanner 100 that identifies the user by scanning the ID card of the user who is the object of the meal, identifies the plate by scanning the recognition code, and provides the A second scanner 110 that acquires a pre-meal image of the pre-meal plate, a third scanner 120 that acquires a post-meal image of the user's post-meal plate through the identified plate by scanning the post-meal plate, and the third scanner 120 It may include a management server 200 for generating intake information based on the before and after meals obtained by controlling the first to third scanners.

The food service management system according to an embodiment of the present invention may be installed in a predetermined food service station 1 that provides food to a large number of cafeterias in a school, a plurality of scanning devices and a management server 200 connected thereto and having a display means ) can be made.

The first scanner 100 serves to identify the subject of the meal, and by scanning the ID card possessed by the user and transmitting the result to the management server 200, the management server 200 refers to the stored personal information Allows the user to determine that he or she is a food subject. Here, the ID card may be a student ID, nameplate, and employee ID, and a method of identifying a user through a facial recognition technique for a user's face photographed through a camera rather than using an ID card according to the designer's intention may also be applied. .

Here, in connection with the user's school or attendance system, the user's appearance is photographed every day at the time of going to work and stored in the database, which refers to the database at the time of identification for future catering, the appearance of the system using vision recognition technology, That is, when a user is identified by applying a vision recognition technology to a face, shoes, clothes, or hair shape, the accuracy can be improved.

Alternatively, a method of recognizing a corresponding user by performing communication with a portable terminal possessed by the user, rather than the scanning method, may be applied.

The second scanner 110 may scan the pre-meal plate 6 containing food through the food distribution port and provide the image to the management server 200 . At this time, as the recognition code formed on one side of the plate 6 before a meal is also scanned, the management server 200 that has received the information distinguishes each plate and determines the amount of food contained in each plate, that is, the amount of food for each user. be able to judge The recognition code may be implemented as a barcode or a QR code.

The third scanner 120 may scan the plate 7 after a meal containing the leftovers before processing the leftovers through the drain hole and provide the image to the management server 200 . Here, as the recognition code of the plate 7 is scanned after a meal, the management server 200 receiving the information can determine the amount of leftovers for each user and each user of the plate 7, and then the management server ( 200) by calculating the leftover information based on the amount of food and the amount of leftovers, it is possible to generate intake information.

The management server 200 may be installed on one side of the food service station 1, preferably near the exit of the food service station 1, and may display intake information generated for each user through the mounted display unit 210. .

This management server 200 can be connected to each of the scanners (100 ~ 130) through a network, the ID card scanning results transmitted from the scanners (100 ~ 130), plate recognition results and images of the plate before and after meals, etc. Through this, the user and his/her intake information will be created and provided.

In particular, in the food service management system according to an embodiment of the present invention, intake information for each user can be quantified by applying an image processing technique to an image obtained through a scanner and provided in the form of figures, texts and graphs. A detailed description of the components of the management server 200 will be described later.

According to the above-described structure, the food service management system according to an embodiment of the present invention is installed in a food service station, checks the user, and acquires the amount of food and leftover food using the scanned image of the plate before and after a meal, and intake based on this By generating information and providing information related to various nutrition related thereto, users can easily check their eating habits.

Hereinafter, a management server included in the food service management system according to an embodiment of the present invention will be described with reference to the drawings.

Figure 2 is a view showing the structure of the management server of the food management system according to an embodiment of the present invention. In the following description, each component constituting the management server of the present invention may be implemented in the form of a program executable by a computer, and may be recorded on a recording medium that can be read and written by the computer and mounted on the management server.

Referring to FIG. 2 , the management server 200 of the present invention includes a display unit 210 that displays leftover information for each user, registers a user as a food subject, and includes any one of the registered users' eating habits and nutritional status. The user management unit 220 that provides nutritional information to, registers the food service, and receives the obtained scanning data in connection with the food service management unit 230 and the first to third scanners that provide operational information related to the operation of the registered food service center. device control unit 240, the image processing unit 250 for generating leftover information and intake information in response to the scanning data received by the device control unit 240, and a database 260 for storing the user's personal information and the operation information of the cafeteria ) may be included.

The display unit 210 may display intake information generated for each user by the management server 200 on the screen. The user who has completed the meal can check the intake information about himself displayed on the display unit 210 of the management server 200 arranged at the exit before leaving the food service after returning the plate.

Here, the display unit 210 may be implemented as a device separate from other components, and may be implemented in a form that is connected to the main body of the management server 200 located in a separate place through a network to display residual information. Alternatively, the display unit 210 may be omitted from the management server 200, and users may be implemented in a form in which the above-described remaining information is provided through their mobile terminal or e-mail.

The user management unit 220 may provide a function of managing the personal information of the meal target. A user who wants to be provided with the service by the food service management system of the present invention must be registered in the management server 200 in advance, and compare the recognition result of the ID card transmitted from the scanner with the personal information stored in the database as a food service target. Registration can be determined.

In addition, the user management unit 220 may manage and provide nutritional information related to eating habits and nutritional status for registered users, and a description of the configuration of the user management unit 220 for implementing these functions will be described later.

The food service management unit 230 may provide a function of managing various types of information about the food service station in which the food service management system of the present invention is installed. Such a food service management function may include creating, updating, deleting, and confirming the maximum number of meals available at the food service center, information related to the affiliated dietitian, and daily, weekly, and monthly diet information, etc. It may be provided to a user with operator authority.

The device control unit 240 may provide a control function of a scanner connected to the management server 200 . The management server 200 is connected to one or more scanners to receive information about the user and the plate therefrom, and the device control unit 240 includes a predetermined communication module and controller, and is connected to each scanner through a network to control it. to receive the necessary information.

The image processing unit 250 may analyze the image of the plate before and after a meal received from the scanner through an image processing technique to calculate the amount of food and leftovers for each user, and generate intake information based thereon.

The database 260 may store personal information, food service information, and the like for performing functions provided by each component of the management server 200 . In addition, various setting values for driving the management server 200 may be stored in the database 260 .

In addition, the management server 200 according to an embodiment of the present invention may further include an advertisement management unit 270 that further provides an additional function for generating revenue in addition to the main function of providing residual information to the user.

As an example, the advertisement management unit 270 reflects personal preference based on personal information stored in the database 260, food in consideration of individual preferences, health supplements in consideration of intake nutritional components, medical devices through eating habits analysis, and personal preference. Advertisements related to restaurants and menus may be displayed through the display unit 210 or provided to the user's mobile terminal. The function provided by the advertisement management unit 270 becomes a revenue model of the food service management system operator.

In addition, as a profit model derived according to an embodiment of the present invention, it is possible to derive a 'premium service' such as a diet management program and intake information, nutritional information, disease probability, and specialized health common sense and recipes for side dishes.

In addition, as a function of public interest, the advertisement management unit 270 may provide the user with a message for promoting or raising awareness about reducing leftovers before a meal. As an example, a message that the effect of environmental protection is effective when the amount of leftovers is reduced may be displayed through the display unit to users who have visited the cafeteria.

According to the above-described structure, the food service management system according to an embodiment of the present invention identifies the user through a plurality of connected scanners, generates leftover information and displays it on the screen, so that the food service users have their own eating habits, as an example, preference You can easily check the ingredients, taste, average meal amount, and speed.

Hereinafter, with reference to the drawings, among the constituent units included in the food service management system according to an embodiment of the present invention, the configuration of the user management unit, the food service management unit, and the image processing unit for implementing the main functions of the present invention will be described in more detail.

3a and 3b are diagrams showing the structures of a user management unit and a food service management unit constituting the management server of the food service management system according to an embodiment of the present invention, and FIG. 4 is a management server of the food service management system according to an embodiment of the present invention. It is a diagram showing the configuration of an image processing part among the configuration parts of .

First, referring to FIG. 3A , the user management unit 220 of the management server of the present invention may be configured with modules corresponding to one or more functions, and these modules are limited to catering subjects and previously issued ID cards to users in the system. The personal information processing module 221 for checking whether to register in the system or managing the personal information of registered users using It may include an image state estimation module 223 for calculating a star image state.

The personal information processing module 221 implements a function of managing the personal information of users registered in the meal management system of the present invention, and the system operator creates a user account by inputting the name, contact information and address of the meal recipient. The personal information processing module can generate and store personal information by receiving the user's eating habits, preferred food, allergy status, current nutritional status, etc. for each account.

The eating habits determination module 222 may determine the eating habits for each user in response to the user's feeding details and store the information. When using the user's cafeteria, the eating habit determination module 222 determines the user's eating habit by using information about the type of food provided at each meal, meal time, and the amount of leftovers according to each meal history. These dietary judgment results may be reflected in personal information.

The nutritional status estimation module 223 may estimate the current nutritional status of each user in response to the user's feeding details and store it as nutritional information. The nutritional status estimation module 223 may calculate the food and amount of food consumed by the user according to the history of the user using the restaurant within a certain period and the leftover information generated during use, and based on this, the user's current nutritional status is analyzed and You can create nutritional information. Such nutritional information may be reflected in personal information along with dietary information.

In addition, referring to FIG. 3b , the food service management unit 230 of the management server of the present invention may include one or more modules, and a food service management module 231 for managing information related to a food service place, which is a place installed in the system of the present invention. ), and a food ingredient inspection module 232 that performs a procedure related to inspection of ingredients delivered to the food service center, and a menu creation module 233 that generates and provides weekly meal schedule information.

The food service management module 231 provides an information management function for a food service restaurant in which the food service management system of the present invention is installed, and can create, update, and delete information related to the maximum number of meals available at the food service center and affiliated nutritionist.

The food material inspection module 232 may provide an inspection procedure function for the food material delivered to the food service restaurant. Food materials for cooking can be delivered to the food service center on a daily basis, and nutritionists belonging to the food service center can input and confirm the process of receiving, inspecting, and storing food in the management server later. Accordingly, the function provided by the food ingredient inspection module 232 may be limitedly provided only to an account with nutritionist authority.

The meal creation module 233 may provide a function that allows nutritionists registered in the system to create a menu for food distributed at the food service center. Nutritionists can create a meal plan at the beginning of each month in consideration of the nutritional status of the caterers, input the meal through the UI provided by the meal creation module 233, and create meal information, and this meal information is managed on a weekly basis. It will be posted on the server 200 and a web site connected thereto.

On the other hand, the food service management system according to an embodiment of the present invention not only identifies the food subject by scanning the user's ID card through one or more scanners installed in the food service station, but also scans the plate before and after a meal to obtain the amount of food and leftovers, Based on this, residual information may be generated.

* In particular, in an embodiment of the present invention, more accurate intake information can be provided by applying an image processing technique to an image acquired through a scanner, and the management server may include an image processing unit for applying the image processing technique. can

4, the image processing unit 250 included in the management server of the present invention is provided with the pre-meal image from the pre-processing module 251 for pre-processing the post-meal image taken after the meal, and the stored image storage 253. The comparison module 252 for comparing with the postprandial image, the leftover estimation module 254 for generating residual information according to the comparison result by the comparison module 252, and the intake in the form of figures, texts or graphs in response to the residual information It may include an intake information generation module 255 for generating and outputting information.

The pre-processing module 251 may perform pre-processing on the image by converting the image taken from the scanner before and after a meal into a predetermined size for comparison and removing noise. The pre-meal image may be stored for each plate identified in the image storage, and when the post-meal image is input thereafter, it is provided to the comparison module 252 after pre-processing.

The comparison module 252 may digitize and compare the pre-prandial image and the post-prandial image. The image is composed of a plurality of pixels having grayscale values, and the comparison module 252 may perform comparison by calculating the difference between the grayscale values of each pixel.

The image storage 253 may store the pre-meal image for each meal plate, and may provide a pre-meal image corresponding to the post-meal image at the request of the comparison module 252 .

The images stored in the image storage 253 may be automatically deleted after a comparison procedure is performed or a predetermined time elapses.

The leftover estimation module 254 may determine the amount of food and the amount of leftover food in response to the pre-meal image and the post-meal image, and calculate the remaining information through the difference calculation. In the image input from the scanner, the part corresponding to gray or white and connected to each side of the image can be determined as the part corresponding to the plate, and the comparison module is based on the increased area of the gray or white region in the postprandial image compared to the preprandial image. Residual information can be calculated.

In addition, the leftover estimation module 254 estimates the amount of reduction in the volume of rice, the height of the soup, and the volume of side dishes based on the image, or a histogram of the brightness distribution of the entire image, and the degree of variation of food according to the difference between the two histograms. can be calculated.

The intake information generation module 255 may generate intake information, which is data in the form of figures, texts, and graphs, according to the leftover information calculated by the leftovers estimation module 254 . Such intake information may be accumulated in personal information, displayed on the screen through the display unit of the management server, or transmitted to the user's terminal through a network.

In addition, the image processing unit 250 according to an embodiment of the present invention may further include a correction means for increasing the accuracy when estimating the degree of variation.

The correction module 256 may correct the residual information by receiving a sensed value from one or more sensors installed in each scanner, and reflecting the sensed value to the degree of variation in the residual estimating module 254 at a certain rate.

That is, the correction module 256 may be equipped with a weight sensor such as a temperature sensor and a load cell in the scanner, and correct the remaining information by generating a correction value using the detected temperature and weight of the plate.

As an example, when the temperature difference between the plate before and after a meal is not large, it can be determined that in particular, food such as soup hardly occurs, and when the difference in weight between the plate before and after a meal is not large, it is determined that the amount of food is very small can do.

In recognizing the types of soup, rice, and side dishes included in a meal, in the case of kimchi, it is generally not more than 30°. can be corrected for

The correction module 256 may generate a correction value as the number of various cases is set for the sensor value provided from the sensors.

According to the above-described structure, the management server according to an embodiment of the present invention provides a management function for catering subjects and food service stations, and at the same time, provides food service subjects with information on the amount of leftover food and food intake according to the amount of food consumed, and nutritional component intake accordingly. information can be provided, etc.

Hereinafter, an operating method of a food service management system according to an embodiment of the present invention will be described with reference to the drawings.

5 is a view showing an operating method of the food management system according to an embodiment of the present invention. In the following description, the subject of each stage becomes a plurality of scanners and management servers constituting the food service management system of the present invention described above, and a component constituting them, even if there is no separate description.

Referring to FIG. 5 , the operating method of the food service management system according to an embodiment of the present invention is an operating method of the food service management system including first to third scanners and a management server, and scans the ID card of the user who is the object of the meal. Step of identifying (S100), scanning the recognition code of the pre-meal plate to identify the plate, and obtaining a pre-meal image for the user's plate (S110), scanning the plate after a meal and scanning the plate after the meal to determine the user's It may include a step (S120) of obtaining a post-meal image for the plate, and a step (S130) of calculating overall information based on the obtained pre-meal image and post-meal image, and generating intake information corresponding to the leftover information.

In the step of identifying the user by scanning the ID card of the user who is the meal target (S100), the first scanner installed in the food service station scans the ID card presented by the user and transmits the information to the management server, and the management server sends the ID card of the corresponding ID card. This is the step of identifying that the user is a food subject.

Next, the step of identifying the plate by scanning the recognition code of the plate before a meal, and obtaining a pre-meal image for the user's plate (S110) is a normal user, that is, when a user identified as a food subject selects a plate and receives a meal, The second scanner scans the recognition code provided on the plate, and at the same time scans the food on the plate to generate a pre-meal plate image, and transmits the information to the management server; Accordingly, it may consist of a step of identifying the corresponding plate and acquiring a pre-meal image. Here, the pre-meal image may be image-processed by the image processing unit of the management server and used to calculate the amount of food.

The step of scanning the plate after a meal and acquiring the post-meal image of the user's plate through the identified plate (S120) is a step in which the image processing unit of the management server acquires the post-meal image, which is an image of the plate after a meal, for each plate. This postprandial image can be used to calculate the amount of leftovers.

* Thereafter, the step (S130) of calculating the remaining information based on the obtained pre-meal and post-meal images, and generating intake information corresponding to the remaining information (S130) is the difference between the amount of food and the amount of leftovers calculated in the steps S110 and S120. It is a step of generating leftover information and intake information corresponding to the actual meal amount through

In particular, in step S130, as the image processing unit calculates the leftover information according to the difference between the amount of food and the amount of leftovers, this leftover information can be viewed as representing the actual amount of meals of the corresponding user. In addition, the present invention provides useful intake information to the user by generating intake information about the user's food intake based on the leftover information and outputting it in the form of figures, texts, graphs, and the like.

In addition, although not shown, after the step S130, the step of displaying the intake information according to the meal and the step of outputting a coupon in response to the intake information may be further included, and according to this step, the user's It is possible to determine the exact amount of intake, and by supplying intake information about the intake to the user, it is possible to provide calorie and nutritional information and provide coffee coupon compensation as various benefits for reducing leftovers.

That is, according to an embodiment of the present invention, it is possible to provide information that is helpful in improving the quality of food service and reducing leftovers by notifying users of accurate and specific preferred food and intake amount for each food to the food service provider.

According to the above-described steps, the operating method of the food service management system according to an embodiment of the present invention can provide intake information according to the actual amount of food to each meal target according to the step of acquiring the image of the plate, and the user can provide such intake information You will be able to check and manage your own eating habits by directly checking the

Hereinafter, with reference to the drawings, a method for generating the leftover information before and after the provision of meals in the food service management system according to an embodiment of the present invention will be described in detail. In the following steps, the executing entity becomes the image processing unit and its comparison module, even if there is no separate description.

6A and 6B are diagrams illustrating an image analysis method for generating leftover information of a food service management system according to an embodiment of the present invention.

First, referring to FIG. 6A , as a step of recognizing a plate ( S200 ), the comparison module of the image processing unit recognizes the plate after the dispensing photographed by the scanner is completed. In detail, zones in which rice, soup, side dishes, and dishes are placed on the plate are partitioned through the boundary surface formed on the plate, and the direction of the plate is recognized to determine the zone in which food is placed.

Next, the step of classifying the food on the plate (S210) is a step in which the comparison module recognizes the type of food for each partitioned area in the plate, and the image acquired by the image sensor, that is, the scanner is related to a plurality of food for machine learning. By comparing the learning image with the learning image accumulated in the knowledge database for the learning image, the food is determined by matching the image recognition result for each area with the menu of the day.

At this time, if the matching success rate is more than 80%, the result is immediately stored in the image storage, and if it is less than 80%, it is classified as undefined food and stored in the image storage.

The step of setting the area along the boundary surface of the food ( S220 ) is a step of accurately setting the boundary surface of the food for each zone. In this case, temperature, color and shape may be considered. As an example, among the foods on the plate, a food boundary that accurately divides between salad and kimchi is set.

Next, the step of recognizing the material according to the menu for each area ( S230 ) is a step of identifying the material constituting the food in the area formed by the food boundary as the food boundary is set while the menu for each area is recognized.

As an example, with respect to the plate partitioned from the first zone to the fifth zone, assuming that the salad is contained in the first zone, cabbage, broccoli, cucumber, carrot, etc., which are ingredients constituting the salad of the first zone, are identified respectively. . In this step S230, machine learning and image comparison procedures may be performed. In this case, even if it is assumed that the same amount of salad is contained in the first zone, the amount of ingredients recognized for each plate may be different.

Next, the step of calculating the volume of food ( S240 ) is a step of recognizing the volume of each food based on the boundary surface for each food using the information obtained through the 3D sensor included in the scanner. The above-described 3D sensor can measure a depth value of a subject, and through this, extract a height within a food boundary surface, and calculate a volume by using the boundary surface and height to calculate the amount of food.

According to the above-described steps, the image processing unit may store the acquired pre-meal image, and then generate the leftover information using the post-meal image obtained by re-photographing the plate on which the meal is completed.

Referring to FIG. 6B , as a step of recognizing the plate ( S300 ), the comparison module of the image processing unit recognizes the plate that the scanner photographed and the plate is completed, and determines the area along the plate boundary.

The step of setting the area along the boundary surface of the food on the plate ( S310 ) is a step of accurately setting the boundary surface of the food for each zone.

Next, the step of calculating the volume of food for each region ( S320 ) is a step of recognizing the volume of each food according to information obtained using the 3D sensor included in the scanner based on the boundary surface for each food. In this step S320, if the comparison module determines that the volume of food is 5% or less compared to the pre-meal image, the remaining amount of menu in the corresponding area is regarded as 0% and calculates the volume of food for other areas, If it is determined that the amount of is more than 5%, the following step S330 is performed.

In the step (S330) of classifying food and matching it with pre-meal food, the ingredients in the menu are recognized using the learning image of the knowledge database for machine learning, and each menu is based on the volume calculated in the steps S310 and S320. and generating residual information by calculating the residual amount for each material.

On the other hand, in order to generate intake information in the above-described food service system operating method, the amount of food and leftovers are obtained by scanning the plate before and after a meal to determine the amount of food. The structure of the scanner used in the food service management system is explained.

7 is a view showing the structure of the scanner used in the food management system according to an embodiment of the present invention.

Referring to FIG. 7 , the scanner 120 according to an embodiment of the present invention includes a body 121 , a support 122 vertically connected to one side of the body 121 , and an upper portion of the support 122 , and the plate The holder 123 to be seated, one side of the body 121 is installed to face the upper portion of the holder 123 , and may include a stereo camera 124 for photographing food on the plate 7 .

The body 121 forms the exterior of the scanner 120, and may be installed in a fixed form near the food outlet of the food service station. A support part 122 may be connected to one side of the body 121 at a predetermined height.

The support part 122 protrudes by a predetermined length from the side, and may be formed in a rectangular shape when viewed from above. The support 122 supports the cradle 123 disposed on the upper portion from the lower portion.

The holder 123 may be seated before and after a meal. The holder 123 may be coupled to be fixed on the support 122 so that the plate faces the lens of the stereo camera 124 to be described later.

In particular, the lower portion of the cradle 123 may be supported and coupled by one or more load cells 125 installed on the support 122 , and these load cells 125 are the cradle 123 and a plate mounted thereon. It can detect the weight of (7) and transmit it to the management server.

In addition, one or more temperature sensors (not shown) for sensing the temperature of the plate seated on the cradle 123 may be mounted on one side of the cradle 123, and the temperature sensed by these temperature sensors is transmitted to the management server and the above-described weight In addition, by measuring the weight and temperature of the food currently on the plate, it can be used as a correction value to calculate the amount of food, the amount of leftovers, or the degree of variation by the management server.

The stereo camera 124 is installed on the body 121, and as it is disposed at a position spaced apart from the support 122 by a predetermined distance, the entire area of the plate 7 on the cradle 123 is installed at a position where it can be scanned. can be

The stereo camera 124 is installed so that the lens faces the cradle 123, each of the plates before and after a meal is photographed to transmit the pre-meal plate image and the post-meal plate image to the management server.

Although many matters are specifically described in the above description, these should be construed as examples of preferred embodiments rather than limiting the scope of the invention. Accordingly, the invention should not be defined by the described embodiments, but should be defined by the claims and equivalents to the claims.

1: cafeteria 3: ID card
6, 7: Plates before and after meals 100 to 120: First to third scanners
200: management server 210: display unit
220: user management unit 230: food service management unit
240: device manager 250: image processing unit
260: database 270: advertisement management unit

Claims (10)

a pre-meal scanner that identifies the plate by scanning the recognition code, and acquires a pre-meal image of the plate before the meal provided to the user who is the target of the meal;
a post-meal scanner that scans an identification code to identify the plate, and scans the plate after a meal provided to the user to obtain a post-meal image of the plate; and
A management server for generating intake information for the user based on the pre-meal or post-meal image obtained by controlling the pre-meal scanner and the post-meal scanner;
The management server,
A database in which images are recorded and stored, including the faces of the food recipients;
Based on the image stored in the database, a user management unit that determines whether the user, photographed from the camera through facial recognition, is a subject to be fed;
a device control unit connected to the pre-meal scanner and the post-meal scanner to receive the acquired scanning data; and
Including an image processing unit for generating the intake information in response to the scanning data received by the device control unit,
The image processing unit recognizes the plate in the pre-meal image obtained by the pre-meal scanner, divides the regions included in the recognized plate through the interface formed on the plate, and based on the direction of the plate, the region Determining an area containing actual food except for an area containing an object other than food, recognizing the type of food contained in the determined area, and using the depth value included in the pre-meal image obtained by the pre-meal scanner Extracting the height of the boundary surface of the food contained in the determined area, calculating the volume of the food contained in the determined area using the boundary surface and the height, and using the learning image of the knowledge database for machine learning of the food Recognizing the ingredients in the menu, and generating leftover information by calculating the amount of leftovers for each menu and each ingredient of the food based on the calculated volume,
The image processing unit analyzes and recognizes the pre-meal image by using the learning image of the knowledge database for machine learning, and uses the recognition result of the pre-meal image and a predetermined menu to determine the type of food contained in the determined area. However, the predetermined menu is characterized in that derived by referring to the diet information provided in the management server, the food management system. According to claim 1,
The management server,
Food management system, characterized in that it further comprises an advertisement management unit that provides a message for raising awareness or publicity about the reduction of leftovers before a meal. delete delete delete delete delete delete delete delete

Images