JP2023135829A - Information processing apparatus, information processing method, and information processing program - Google Patents

Abstract

To provide an information processing apparatus, an information processing method, and an information processing program that can improve a customer satisfaction.SOLUTION: An information processing apparatus includes: a storage unit that stores correspondence information in which ingredients containing meat are associated with a cooking procedure for meat-containing dishes in which the ingredients are used; a first acquisition unit that acquires image information relating to an image of a cooking area where the dish containing the meat is cooked; a specification unit that specifies the ingredient based on the image information acquired by the first acquisition unit; a second acquisition unit that acquires the cooking procedure according to the ingredient specified by the specification unit based on the correspondence information stored in the storage unit; and an output control unit that outputs cooking instruction information relating to a cooking instruction based on the cooking procedure acquired by the second acquisition unit.SELECTED DRAWING: Figure 1

Description

The present disclosure relates to an information processing device, an information processing method, and an information processing program.

2. Description of the Related Art Conventionally, there have been devices (support devices) that support the work procedures of employees at stores (store operations). The support device uses a camera to capture images of the actions of employees and the like in response to the appearance of customers visiting the store. The support device accumulates the captured image information, performs employee evaluation, employee education, etc., and supports store operations (see Patent Document 1).

JP2020-166558A

The support device described in Patent Document 1 supports store operations for employees.
In stores such as restaurants, cooking methods for dishes served at the restaurant may be prescribed. The cooking method for food is generally for the cooking staff to check the cooking procedure and then cook according to the cooking procedure. Nowadays, there is a need to improve customer satisfaction, and it is necessary to check that the cooking staff is following the cooking instructions. Note that the support device described in Patent Document 1 does not improve customer satisfaction related to cooking procedures.

The present disclosure provides an information processing device, an information processing method, and an information processing program that can improve customer satisfaction.

An information processing device according to one aspect includes a storage unit that stores correspondence information that associates meat-containing foodstuffs with cooking procedures for meat-containing dishes using the ingredients, and an image of a cooking area in which meat-containing dishes are cooked. a first acquisition unit that acquires image information about the food; a identification unit that identifies the food item based on the image information acquired by the first acquisition unit; and a food item identified by the identification unit based on the correspondence information stored in the storage unit. and an output control section that outputs cooking instruction information regarding the cooking instruction based on the cooking procedure acquired by the second acquisition section.

According to one aspect, ingredients are identified based on image information regarding an image of a cooking area in which a dish containing meat is cooked, a cooking procedure corresponding to the identified ingredient is acquired based on correspondence information, and a cooking procedure is applied to the cooking procedure. Since cooking instruction information regarding cooking instructions is output based on the cooking instructions, customer satisfaction can be improved.

FIG. 1 is a diagram for explaining an information processing device according to an embodiment. FIG. 1 is a block diagram for explaining an information processing device according to an embodiment. 1 is a flowchart for explaining an information processing method according to an embodiment.

An embodiment will be described below.

[Overview of information processing device 1]
First, an overview of the information processing device 1 according to one embodiment will be described.
FIG. 1 is a diagram for explaining an information processing device 1 according to an embodiment.

The information processing device 1 may be configured as, for example, an instruction device that instructs the cooking procedure of the dish 103, etc. Further, the information processing device 1 may be configured as a warning device or an output device that outputs a warning or the like when the cooking procedure differs from a predetermined cooking procedure, for example.
The information processing device 1 may be a device related to the cooking procedure of dishes using various ingredients including meat, or a device etc. related to the cooking procedure of dishes using various ingredients not including meat. Good too. Dishes that use meat-containing ingredients may be, for example, dishes that are cooked by boiling meat-containing ingredients, or dishes that are cooked using other methods. That is, the information processing device 1 may be a device or the like that is involved in the cooking procedure of a dish 103 using various ingredients 102.
The information processing device 1 may be, for example, a computer such as a server, desktop, laptop, or tablet.
Below, an example of a cooking procedure for a dish using ingredients including meat will be described.

The storage unit 23 (see FIG. 2) of the information processing device 1 may store correspondence information that associates meat-containing ingredients with cooking procedures for meat-containing dishes using the ingredients.
The imaging unit 21 of the information processing device 1 images the cooking area 101 when cooking the dish 103, and generates image information. The cooking area 101 may be, for example, a kitchen of various facilities such as a restaurant or the like. In a facility such as a restaurant, the cooking area 101 is a kitchen when the kitchen where the food 103 is prepared is different from the area (fountain area) where drinks such as fountains such as parfaits, ice cream, and cream soda are prepared. The concept may also include a fountain area and the like.

The information processing device 1 identifies the food material 102 by recognizing the object recorded in the image information generated by the imaging unit 21. The information processing device 1 refers to the correspondence information and obtains a cooking procedure for a dish 103 using the specified ingredients 102. The information processing device 1 outputs cooking instruction information regarding cooking instructions based on the cooking procedure.

[Details of information processing device 1]
Next, details of the information processing device 1 according to one embodiment will be described.
FIG. 2 is a block diagram for explaining the information processing device 1 according to one embodiment.

The information processing device 1 includes, for example, an imaging section 21, a communication section 22, a storage section 23, a display section 24, a control section 11, and the like. The communication section 22, the storage section 23, and the display section 24 may be an embodiment of an output section. The control unit 11 includes, for example, a first acquisition unit 12, a specification unit 13, a second acquisition unit 14, a first estimation unit 15, a second estimation unit 16, a timer 17, an output control unit 18, and the like. The control unit 11 may be configured by, for example, an arithmetic processing unit of the information processing device 1. The control unit 11 (e.g., arithmetic processing unit, etc.) controls each unit (e.g., the first acquisition unit 12, the identification unit 13, the 2 acquisition unit 14, first estimation unit 15, second estimation unit 16, time measurement unit 17, output control unit 18, etc.) may be realized.

The imaging unit 21 may be a camera placed in the cooking area. For example, one or more imaging units 21 may be arranged. The imaging unit 21 images the cooking area and generates image information in which the cooking area is recorded. The imaging unit 21 may capture a moving image or a still image, for example. For example, when capturing still images, the imaging unit 21 may capture images at predetermined timings. The predetermined timing may be various timings including, for example, every predetermined time or every time there is some change in the imaging range by the imaging unit 21.
When there is a change, there may be various changes including, for example, when the object moves or when the color changes. When the object moves, for example, the food may move in order to put the food into a pot. When the color changes, for example, the color of the food may change because the food in the pot is being boiled. As a specific example, the imaging unit 21 continuously captures images and performs image recognition on the images by the control unit 11, and if there is any change as exemplified above as a result of the recognition, the current stage The image information may be generated by capturing an image of the cooking area. The control unit 11 may perform image recognition, for example, by using a trained model that has learned the object, the color of the object, and the like.
The cooking area may include, for example, a kitchen, a fountain area, etc., as described above.

The communication unit 22 is capable of transmitting and receiving various information to and from, for example, a device (external device) (not shown) outside the information processing device 1 . The external device may be, for example, a server, a user terminal used by a user of the information processing device 1, or the like. The user terminal may be, for example, a desktop, laptop, tablet, smartphone, etc.

The storage unit 23 may store various information and programs, for example. An example of the storage unit 23 may be a memory, a solid state drive, a hard disk drive, or the like. Note that the storage unit 23 may be, for example, a storage area on a cloud, a server, or the like.

The storage unit 23 stores correspondence information that associates ingredients with cooking procedures for dishes using the ingredients. The correspondence information may be, for example, information that associates a meat-containing food ingredient with a cooking procedure for a meat-containing dish using that food ingredient. The cooking procedure defines, for example, the timing, cooking content, cooking time, etc. of cooking ingredients, and may define the cooking content of one or more ingredients depending on the passage of time. For example, the cooking procedure may be referred to as a cooking process, a cooking manual, or the like. The cooking procedure may be, for example, various contents such as images, sounds, and texts in which the cooking procedure is recorded, or may be various information that instructs the cooking procedure, and controls the cooking equipment. It may be various types of information.
For example, in the case of business use at restaurants, etc., the amount of food and ingredients to be prepared varies depending on demand forecasts, order details, etc. On the other hand, for home use, for example, the amount of food and food to be cooked is determined by the ingredients you want to cook, the dishes you want to eat, and the family structure.
The "food containing meat" in this embodiment may be, for example, food used for business purposes. Therefore, the amount of dishes and ingredients to be prepared for the "ingredients containing meat" in this embodiment may vary depending on demand forecasts, order details, etc. in the case of business use, for example.

The display unit 24 is, for example, a display capable of displaying various characters, symbols, images, and the like.

The first acquisition unit 12 acquires image information regarding an image of a cooking area where a dish containing meat is cooked. The first acquisition unit 12 may acquire image information from the imaging unit 21, for example. Alternatively, for example, if the image information generated by the imaging unit 21 is stored in a server (not shown), the first acquisition unit 12 may acquire the image information from the server (not shown). Further, the first acquisition unit 12 may acquire image information via, for example, a communication cable and a communication line (communication network), and when the image information is recorded in an external memory (not shown) or the like, When the external memory is inserted into an interface (not shown) provided in the information processing device 1, image information may be acquired from the external memory.

The identification unit 13 identifies food ingredients based on the image information acquired by the first acquisition unit 12. The identification unit 13 may identify the food by performing image recognition based on image information, for example. As an example, the identifying unit 13 may identify the type of food recorded in the image information using a first learned model that has learned the food and the type of the food. As a more specific example, the identification unit 13 uses images of various patterns of meat (for example, various patterns such as shapes and colors) as an example of foodstuffs, and a first image that has learned meat (type of foodstuff). Using the trained model, if meat is recorded in the image information, meat as a food ingredient may be identified. The first learned model may be a model that has learned about ingredients used in dishes cooked in the cooking area where the imaging unit 21 is arranged, in addition to learning about meat as in the above-mentioned example.

The first trained model may be generated by, for example, performing learning by a learning unit (learning device) (not shown) of the information processing device 1. Alternatively, the first trained model may be generated by, for example, performing learning outside the information processing device 1 (for example, a learning device (not shown)). Alternatively, the first learned model may be stored in, for example, a server (not shown). Alternatively, the first learned model may be stored in the storage unit 23, for example. The information processing device 1 receives information from an external learning device or server via a communication unit 22 and an external memory (not shown), or from an internal learning unit or storage unit 23, by a reception unit (configuring the control unit 11). (not shown) may accept the first trained model. The specifying unit 13 acquires, for example, the first trained model received by the receiving unit.

The second acquisition unit 14 acquires a cooking procedure according to the food specified by the identification unit 13 based on the correspondence information stored in the storage unit 23. That is, the second acquisition unit 14 refers to the correspondence information and acquires the cooking procedure according to the specified food ingredient. As an example, when the identified ingredient is meat, the second acquisition unit 14 refers to the correspondence information and acquires the cooking procedure for a dish using meat.

In this case, the second acquisition unit 14 may acquire, for example, a cooking procedure corresponding to the food specified over time. As an example, when meat is first identified by the identification unit 13 based on image information, the second acquisition unit 14 may acquire the cooking procedure of the dish in which the meat is cooked first.

Further, the second acquisition unit 14 may acquire, for example, cooking procedures corresponding to a plurality of ingredients specified according to the passage of time. As an example, when the identifying unit 13 first identifies meat and then identifies onions based on image information, the second acquiring unit 14 cooks the meat first and then the onions. You may also obtain cooking instructions for the dish. Note that the number of food items to be specified is not limited to two as in the above example, and may be three or more food items.

The first estimation unit 15 determines whether the food is cooked according to the cooking procedure based on the ingredients recorded in the image information acquired by the first acquisition unit 12 and appearing in chronological order and the cooking procedure acquired by the second acquisition unit 14. It may be possible to estimate whether As an example, the first estimating unit 15 uses a second learned model that has learned the ingredients that appear in chronological order, the cooking content of the ingredients, and the cooking procedure using the ingredients, and the identifying unit 13 It may be estimated whether the food items specified in time series are cooked according to the cooking procedure. Here, the cooking contents may include various contents related to cooking, such as how to cut ingredients, cooking contents such as boiling, baking, boiling, and steaming, and cooking times for each cooking contents. . The cooking procedure may be, for example, cooking contents in chronological order.

Similarly to the first trained model used by the identification unit 13 described above, the second trained model may be generated by learning in various learning units (learning devices) (not shown), etc. good. Similarly, for example, when the reception unit (not shown) of the information processing device 1 receives the second trained model, the first estimation unit 15 may acquire the second trained model from the reception unit.

That is, the first estimating unit 15 uses, for example, the second trained model to determine whether the ingredients recorded in the image information (ingredients that appear in chronological order) are cooked according to the cooking content, cooking time, etc. Estimate whether there are any.
Further, for example, when estimating that cooking will be performed according to the cooking procedure based on the image information, the first estimation unit 15 may further estimate that all the cooking procedures have been completed. That is, the first estimation unit 15 may estimate that all the cooking procedures have been completed, for example, using the second learned model and image information.

The first estimating unit 15 includes, for example, the ingredients recorded in the image information based on being imaged in real time by the imaging unit 21 (the ingredients identified in real time by the identifying unit 13) and the ingredients acquired by the second acquiring unit 14. Based on the cooking procedure using the food, it may be estimated whether the food is cooked according to the cooking procedure.
Alternatively, the first estimating unit 15 may select, for example, the ingredients recorded in the image information based on images captured in the past by the imaging unit 21 (the ingredients identified by the identifying unit 13) and the ingredients acquired by the second acquiring unit 14. Based on the cooking procedure using the food, it may be estimated whether the food is cooked according to the cooking procedure.

The second estimation unit 16 may estimate the remaining amount of food stored in the utensil used for cooking based on the image information acquired by the first acquisition unit 12. As an example, when food is stored in a large pot (cooking utensil) after cooking, and each time a food order is placed, the food is transferred from the large pot to a container that is served to a customer, and then the food is served to that customer. There is. Note that the above-mentioned food stored in the cooking utensil may include, for example, food transferred from the cooking utensil to the storage container. The second estimation unit 16 may estimate the remaining amount of food in the cooking utensil, for example, based on image information obtained by capturing an image of the cooking utensil. The remaining amount of food may be set in multiple levels, such as large, normal, and small, or may be set in various other amounts. As an example of this case, the second estimation unit 16 uses the third learned model that has learned the cooking utensils and the amount of food in the cooking utensils to estimate the amount of food in the cooking utensils recorded in the image information. The remaining amount may be estimated.

Similarly to the first trained model used by the identification unit 13 described above, the third trained model may be generated by learning in various learning units (learning devices) (not shown), etc. good. Similarly, the second estimation unit 16 may acquire the third trained model from the reception unit, for example, when the reception unit (not shown) of the information processing device 1 receives the third trained model.

The second estimation unit 16 may estimate the time when the food will run out based on the change over time in the amount of food remaining. That is, the second estimating unit 16 may obtain temporal changes in the remaining amount of food, for example, based on temporally different image information. In this case, the second estimating unit 16 may estimate the time when the food will run out based on the change over time in the amount of food left by using various known estimation methods, for example.
For example, if the change over time in the remaining amount of food can be approximated by a linear equation, quadratic equation, etc., the second estimation unit 16 may use the approximate equation to estimate the time when the food will run out. Often, other estimation methods may be used to estimate the time until food will run out.

Further, the second estimating unit 16 may obtain the order status of food by using, for example, a POS (Point of Sale) system that accepts food orders at restaurants and the like. The second estimation unit 16 may estimate that when the number of orders for a dish is relatively large, the remaining amount of that dish decreases relatively quickly. Similarly, when the number of orders for a dish is relatively small, the second estimation unit 16 may estimate that the remaining amount of the dish decreases relatively slowly. That is, the second estimation unit 16 may estimate the time when the food will run out based on the number of food orders. In this case, the second estimation unit 16 may estimate the time when the food will run out, for example, based on at least one of the current number of food orders and the past number of food orders. The number of past food orders is selected from the group of, for example, the number of food orders in the past during the same time period as the current time for which estimation is performed, and the number of past food orders on the same day of the week as the current time for which estimation is performed. There may be at least one.

In addition, the second estimating unit 16 estimates the time (first time) when the remaining amount of food will run out based on changes over time in the remaining amount of food, and calculates the number of current food orders using the POS system and the number of past food orders. The second time may be estimated based on at least one of the number of orders, and the second time may be corrected to be later or earlier than the first time, and the second time may be set as the time when food runs out.

Further, the second estimating unit 16 may obtain the amount of decrease in the amount of food, for example, based on a change over time in the amount of food remaining. Similarly, the second estimation unit 16 may obtain the order amount using, for example, a POS system. In this case, the second estimation unit 16 may obtain the decrease amount or order amount per unit time based on the decrease amount of dishes or the order amount of dishes, for example, based on the decrease amount or order amount in the latest predetermined time. The amount of orders (or the amount of decrease or amount of orders per unit time within the most recent predetermined time) may be acquired. The latest predetermined time may be various times, such as 1 hour, 2 hours, and 3 hours, for example. In this case, the predetermined time may be, for example, a time shorter than the cooking disposal deadline.
The second estimating unit 16 may estimate the cooking amount when cooking the food again, for example, based on the above-mentioned exemplary reduction amount or order amount. As an example, when the reduction amount or the order amount is relatively large, the second estimation unit 16 may relatively increase the cooking amount when cooking the dish again. Further, as an example, when the reduction amount or the order amount is relatively small, the second estimation unit 16 may relatively reduce the cooking amount when cooking the food again.
In this case, the information processing device 1 can, for example, estimate the cooking amount according to the current reduction amount or order amount by considering the most recent reduction amount or order amount of food, and It is possible to reduce the risk that food that has been recooked based on the cooking amount will be discarded without being consumed (risk of food loss).

Further, the second estimating unit 16 calculates the amount of food to be prepared when cooking the food again based on the remaining amount of the food stored in the utensil used for cooking and the above-mentioned example reduction amount or order amount. The amount may be estimated.

Further, the second estimating unit 16 may estimate the order amount of dishes to be sold from now on, for example, based on the order amount of past dishes using the POS system. The past food order amount using the POS system may be, for example, the past order amount on at least one of the same day of the week and the same time zone as the current date and time. On the same day of the week and at the same time, the amount of food ordered tends to be roughly the same.
The second estimating unit 16 may estimate the cooking amount when cooking the food again, for example, based on the above-mentioned example past order amount. In this case, the second estimating unit 16 determines the cooking amount when cooking the food again based on the remaining amount of the food stored in the appliance used for cooking at the current time and the past order amount. may be estimated. That is, the second estimating unit 16 may estimate the cooking amount when cooking the food again by subtracting the remaining amount of the food from the cooking amount of the food estimated based on the past order amount. .

Further, a timer unit 17, which will be described later, starts timekeeping from the timing when cooking is finished, and counts the time until the disposal deadline of the cooked dish. The dishes remaining when the disposal deadline is reached will be discarded.
As described above, the second estimation unit 16 estimates the remaining amount of food based on the image information. At this time, the second estimation unit 16 is based on, for example, changes over time in the remaining amount of food, the amount of recent or past food orders using the POS system, and the time measured by the timer 17, which will be described later. The remaining amount of food that will remain when the disposal deadline is reached may also be estimated.
The second estimation unit 16 calculates, for example, the remaining amount of food stored in the utensils used for cooking at the current time, the amount of food remaining when the disposal deadline is reached, and the change in food over time. The amount of cooking when cooking the food again may be estimated based on the amount of reduction or the amount of past or recent orders using the POS system. That is, the second estimating unit 16 subtracts the remaining amount of the dish from the estimated cooking amount of the dish based on, for example, the amount of decrease in the amount of food over time or the amount of past or recent orders using the POS system. The cooking amount when cooking the food again may be estimated by obtaining the amount A, and adding the remaining amount of the food when the disposal deadline reaches the amount A to the amount B.

The clock section 17 may start clocking when the first estimation section 15 estimates the end of cooking based on the image information. That is, the timer section 17 starts measuring time at the timing when cooking is completed. For example, when the first estimating unit 15 estimates whether the cooking is completed according to the cooking procedure, the timing section 17 determines the timing when the cooking is completed by further estimating whether all the cooking procedures have been completed. The timing at which all cooking procedures are estimated to be completed may be used. The clock section 17 may be, for example, a count-up timer or a count-down timer.

For example, the output control unit 18 controls the cooking procedure acquired by the second acquisition unit 14 , the estimation result by the first estimation unit 15 , the estimation result by the second estimation unit 16 , and the group of time measured by the time measurement unit 17 . The output unit may be controlled to output at least one of (first group). The output section may be the communication section 22, the storage section 23, the display section 24, etc.
That is, the output control unit 18 may, for example, control the communication unit 22 to transmit information regarding at least one of the first groups described above to an external device (not shown). The external device may be, for example, a server, a user terminal, or the like.
For example, the output control unit 18 may control the storage unit 23 to store information regarding at least one of the first groups described above.
For example, the output control unit 18 may control the display unit 24 to display at least one content of the first group described above.
Here, for example, when outputting the cooking procedure acquired by the second acquisition unit 14, the output control unit 18 may control to output the procedure content for each cooking procedure. For example, when the identification unit 13 identifies ingredients based on image information, the output control unit 18 determines whether the current cooking is done based on each cooking content of the cooking procedure and the ingredients used in each cooking content. It may be possible to specify which cooking content in the procedure and control to output the current cooking content or the next cooking content (or the next cooking content). In this case, the output control unit 18 may estimate the cooking procedure (current cooking content), for example, using the estimation result by the first estimation unit 15. As an example, the output control unit 18 estimates which stage of the cooking procedure the cooking content of the dish being cooked is at, and outputs the next cooking content etc. in advance (for example, displaying it on the display unit, etc.) be able to.

The output control unit 18 may control the output unit to output cooking instruction information regarding the cooking instructions based on the cooking procedure acquired by the second acquisition unit 14. The cooking instruction information may be, for example, information recorded in correspondence information stored in the storage unit 23. In other words, the cooking instruction information may be various contents such as images, sounds, and sentences in which cooking procedures are recorded, or may be various information that instructs cooking procedures, and may include information on cooking equipment. It may be various information etc. which control. For example, the output control unit 18 may refer to the correspondence information stored in the storage unit 23 and control the output unit to output cooking instruction information according to the cooking procedure acquired by the second acquisition unit 14. Note that the above-mentioned control of the output control unit 18 includes, for example, outputting cooking instruction information corresponding to the ingredients specified by the specifying unit 13 based on the correspondence information stored in the storage unit 23. Controlling the output may also be included.

For example, the output control unit 18 may control the communication unit 22 to output the cooking instruction information to an external device (not shown). The external device in this case may be, for example, a server, a user terminal, a cooking appliance, or the like.
In this case, for example, upon acquiring the cooking instruction information, the cooking appliance may perform cooking based on the cooking procedure. That is, the information processing device 1 can control a cooking device to automatically cook food, for example.

For example, the output control unit 18 may control the storage unit 23 to store the cooking instruction information.
In this case, the output control unit 18 associates the image information acquired by the first acquisition unit 12, the food ingredients specified by the identification unit 13, the cooking procedure acquired by the second acquisition unit 14, and the cooking instruction information. It may be stored in the storage unit 23 as log information. Thereby, by storing the log information, it becomes possible to use it later for confirmation of cooking procedures, training of cooking staff, etc.

For example, the output control unit 18 may control the display unit 24 to display content based on the cooking instruction information.
In this case, the output control section 18 may control, for example, a speaker (not shown) disposed on the display section 24 or independently disposed to output a sound based on the cooking instruction information. Thereby, the cooking staff can cook while checking the cooking procedure displayed on the display section 24. As an example, even cooking staff with little cooking experience can reliably cook according to the procedure.

The output control unit 18 may control the output unit to output a warning when the first estimation unit 15 estimates that the food is not cooked according to the cooking procedure. The warning may be, for example, various warning contents such as sound, voice, image, text, etc., indicating that the food is not cooked according to the cooking procedure. That is, the warning may be, for example, various warning contents indicating the cooking procedure of the dish (for example, cooking contents, cooking time, etc.). In addition, for example, if ingredients that are not used in a dish (cooking procedure for that dish) (for example, ingredients that are not used in the current or next cooking stage) are recorded in the image information, a warning may be issued according to the cooking procedure. It may also include various warning contents indicating that the food is not cooked properly. Further, the warning may be various warning contents indicating that the cooking is not done according to the cooking procedure, for example, if any cooking content in the cooking procedure is skipped. That is, as an example, when an employee of a restaurant or the like puts in the wrong ingredients or skips cooking contents (skip work), control may be performed to output a warning.
That is, the output control unit 18 may control the communication unit 22 to transmit information regarding the warning to an external device (not shown), for example. The external device may be, for example, a server, a user terminal, or the like. The user terminals here include, for example, terminals used by cooking staff who actually cook in the kitchen of a store, etc., terminals used by various users such as the on-site manager of the store, etc. It may also be a terminal, etc. that is placed in the management headquarters that manages the management headquarters and used by users of the management headquarters.
For example, the output control unit 18 may control the storage unit 23 to store information regarding the warning.
For example, the output control unit 18 may control the display unit 24 to display the content of the warning. In this case, the output control unit 18 may control, for example, to output at least one of a warning sound and a warning voice from a speaker (not shown) disposed on the display unit 24 or independently disposed. .
In this case, the output control unit 18 may, for example, estimate which stage of the cooking procedure the cooking content of the dish being cooked is in, and output the cooking content, etc. at the current stage, as in the case described above, or , the next cooking content, etc. may be output in advance.

The output control unit 18 may control the output unit to output a cooking instruction for the dish when the remaining amount of the dish estimated by the second estimation unit 16 is less than or equal to a threshold value. The cooking instruction may be an instruction to start cooking. The threshold value may be, for example, the lower limit value regarding the amount of food stored, where it is thought that if the number of stored dishes decreases and the number of dishes continues to decrease, it will not be possible to provide food to customers in the relatively near future. good. The threshold value may be set as appropriate depending on the dish, for example.
The output control section 18 may output the cooking instruction by controlling at least one selected from the group of the communication section 22, the storage section 23, and the display section 24, as in the case described above.

The output control unit 18 may control the output unit to output the cooking instruction before the time estimated by the second estimation unit 16 has elapsed. In this case, the output control unit 18 determines the estimated time when the food will run out, based on the cooking time recorded in the correspondence information stored in the storage unit 23 and the time estimated by the second estimation unit 16, for example. The output unit may be controlled to output the cooking instruction at a timing earlier than the cooking time based on the cooking time. That is, the output control unit 18 takes into consideration the cooking time of the dish and outputs a cooking instruction for the dish so that the cooking of the dish is completed before the time estimated by the second estimation unit 16 elapses. It's okay. Therefore, the correspondence information may include a correspondence between a food ingredient, a cooking procedure for a dish using the food ingredient, and a cooking time for the dish (time from the start of cooking to the end of all the procedures).
The output control section 18 may output the cooking instruction by controlling at least one selected from the group of the communication section 22, the storage section 23, and the display section 24, as in the case described above.
Furthermore, for example, when the cooking amount is estimated by the second estimating section 16, the output control section 18 may control the output section so that the cooking amount is included in the cooking instruction and output. .

The output control unit 18 may control the output unit to output the cooking instruction at a timing before the time measured by the clock unit 17 passes a predetermined time set as a food disposal deadline. . For a cooked dish, a deadline for discarding the dish (disposal deadline), such as an expiration date or a best-before date, may be set. For example, various times (predetermined times) are set as the disposal deadline depending on the dish. The output control unit 18 outputs an instruction to cook the food before the time measured by the clock unit 17 passes the disposal deadline (predetermined time) of the stored food. In this case, the output control unit 18 takes into consideration the cooking time of the food, as in the case described above, and the output control unit 18 determines that the cooking time of the stored food is completed before the disposal deadline (predetermined time) elapses. An instruction for cooking the dish may be output so that the cooking process can be completed.

[Information processing method]
Next, an information processing method according to an embodiment will be described.
FIG. 3 is a flowchart for explaining an information processing method according to an embodiment.

In step ST101, the first acquisition unit 12 acquires image information regarding an image of a cooking area where a dish containing meat is cooked.

In step ST102, the identification unit 13 identifies the food material based on the image information acquired in step ST101. Here, as an example, the specifying unit 13 may use the first trained model or the like to specify the type of food material recorded in the image information. The first learned model may be, for example, a model generated by learning ingredients and types of ingredients.

In step ST103, the second acquisition unit 14 acquires a cooking procedure according to the food specified in step ST102, based on the correspondence information stored in the storage unit 23. In this case, the second acquisition unit 14 may acquire, for example, a cooking procedure corresponding to the food specified over time. Further, the second acquisition unit 14 may acquire, for example, cooking procedures corresponding to a plurality of ingredients specified according to the passage of time.

In step ST104, the first estimating unit 15 calculates, based on the ingredients recorded in the image information acquired in step ST101 and appearing in chronological order (the ingredients identified in step ST102) and the cooking procedure acquired in step ST103, Estimate whether the food is cooked according to the cooking instructions. Here, as an example, the first estimating unit 15 uses the second learned model etc. to determine whether the ingredients recorded in the image information (ingredients that appear in chronological order) are cooked according to the cooking content, cooking time, etc. Estimate whether it is. The second learned model may be, for example, a model generated by learning food items that appear in chronological order, cooking contents of the food items, and cooking procedures using the food items.
Further, for example, when estimating that cooking will be performed according to the cooking procedure based on the image information, the first estimation unit 15 may further estimate that all the cooking procedures have been completed.

Step ST105 may be a process performed after a series of processes are completed, that is, after the cooking is completed. That is, in step ST105, the second estimation unit 16 may estimate the remaining amount of the dish (already completed dish) stored in the appliance used for cooking based on the image information in step ST101. The second estimating unit 16 may estimate the time when the food will run out, for example, based on changes over time in the amount of food remaining.
Here, the clock section 17 may start clocking when the end of cooking is estimated based on the image information in step ST104.

Step ST106 is an output process performed both before and after the completion of the dish.

Before the cooking is completed, the following output processing may be performed in step ST106, for example.
In step ST106, the output control unit 18 may control the output unit to output cooking instruction information regarding the cooking instruction based on the cooking procedure acquired in step ST103. The output control unit 18 may control the output unit to output a warning when it is estimated in step ST104 that the food is not cooked according to the cooking procedure. The output section may be, for example, the communication section 22, the storage section 23, the display section 24, etc.

Further, after the cooking is already completed, the following output processing may be performed in step ST106, for example.
In step ST106, the output control unit 18 may control the output unit to output a cooking instruction for the dish, for example, when the remaining amount of the dish estimated in step ST105 is below a threshold value.
For example, the output control unit 18 may control the output unit to output the cooking instruction before the time estimated in step ST105 (the estimated time when the food will run out) has elapsed. For example, the output control unit 18 adjusts the cooking time based on the estimated time when the food will run out, based on the cooking time recorded in the correspondence information stored in the storage unit 23 and the time estimated in step ST105. The output unit may be controlled to output the cooking instruction at a timing earlier than the above.
For example, the output control unit 18 causes the output unit to output the cooking instruction at a timing before the time measured by the clock unit 17 in step ST105 passes a predetermined time set as the food disposal deadline. May be controlled.
The output unit may be, for example, the communication unit 22, the storage unit 23, the display unit 24, etc., as in the case described above.

Each part of the information processing device 1 described above may be realized as a function of a computer processing device or the like. That is, the first acquisition unit 12, the identification unit 13, the second acquisition unit 14, the first estimation unit 15, the second estimation unit 16, the timer 17, and the output control unit 18 (control unit 11) of the information processing device 1, The functions may be realized as a first acquisition function, a specific function, a second acquisition function, a first estimation function, a second estimation function, a timekeeping function, and an output control function (control function) by an arithmetic processing device of a computer or the like.
The information processing program can cause a computer to realize each of the functions described above. The information processing program may be recorded on a computer-readable non-transitory recording medium, such as a memory, a solid state drive, a hard disk drive, or an optical disk. A recording medium may also be referred to as a non-transitory computer-readable medium, for example.
Further, as described above, each part of the information processing device 1 may be realized by a calculation processing device of a computer or the like. The arithmetic processing device and the like are constituted by, for example, an integrated circuit or the like. Therefore, each part of the information processing device 1 may be realized as a circuit that constitutes an arithmetic processing device or the like. That is, the first acquisition unit 12, the identification unit 13, the second acquisition unit 14, the first estimation unit 15, the second estimation unit 16, the timer 17, and the output control unit 18 (control unit 11) of the information processing device 1, It may be realized as a first acquisition circuit, a specific circuit, a second acquisition circuit, a first estimation circuit, a second estimation circuit, a clock circuit, and an output control circuit (control circuit) that constitute an arithmetic processing device of a computer.
The imaging unit 21, communication unit 22, storage unit 23, and display unit 24 (output unit) of the information processing device 1 have, for example, an imaging function including functions such as an arithmetic processing unit, a communication function, and a storage function. It may also be realized as a display function (output function). In addition, the imaging unit 21, communication unit 22, storage unit 23, and display unit 24 (output unit) of the information processing device 1 are configured by, for example, an integrated circuit or the like. It may be realized as a circuit and a display circuit (output circuit). Furthermore, the imaging unit 21, the communication unit 22, the storage unit 23, and the display unit 24 (output unit) of the information processing device 1 are configured by a plurality of devices, so that the imaging unit, the communication device, and the storage unit are configured by a plurality of devices. It may be configured as a device and a display device (output device).

The information processing device 1 can combine one or any plurality of the plurality of units described above.
Although the wording of "information" is used in this disclosure, the wording of "information" can be rephrased as "data" and the wording of "data" can be rephrased as "information."

[Aspects and effects of this embodiment]
Next, one aspect of this embodiment and the effects of each aspect will be described. Note that this embodiment is not limited to each aspect described below, and may be realized by appropriately combining each of the above-mentioned parts. Further, the effects described below are just examples, and the effects achieved by each aspect are not limited to those described below. Moreover, each aspect may exhibit at least one effect described below, for example.

(Aspect 1)
An information processing device according to one aspect includes a storage unit that stores correspondence information that associates meat-containing foodstuffs with cooking procedures for meat-containing dishes using the ingredients, and an image of a cooking area in which meat-containing dishes are cooked. a first acquisition unit that acquires image information about the food; a identification unit that identifies the food item based on the image information acquired by the first acquisition unit; and a food item identified by the identification unit based on the correspondence information stored in the storage unit. and an output control section that outputs cooking instruction information regarding the cooking instruction based on the cooking procedure acquired by the second acquisition section.
That is, the information processing device can notify the outside of the cooking procedure and the like. This allows the cooking staff who cook the food and the manager to check whether the food is being cooked according to the cooking procedure. For example, the cooking staff can cook while checking the cooking procedure. For example, the cooking staff can cook according to the cooking procedure. Further, the cooking appliance can control each part of the cooking device to perform cooking according to the cooking procedure, for example, based on the cooking instruction information.
In addition, although information processing equipment often requires experience in cooking, even cooking staff with relatively little cooking experience can cook by providing cooking instructions. You can cook according to the instructions.
Therefore, the information processing device can provide food of stable quality to customers, and can improve customer satisfaction.
Further, the information processing device can automatically acquire cooking procedures based on the ingredients recorded in the image information. Therefore, when an employee of a restaurant or the like cooks food using ingredients according to a normal procedure, the information processing device can automatically identify a dish and give cooking instructions. In this case, the employee can save time and effort such as selecting a cooking procedure when cooking, and can also prevent the employee from selecting the wrong cooking procedure. That is, the information processing device is different from the case where the user himself/herself acquires cooking procedures (recipes) using various media such as cookbooks, as in the case of cooking at home.

(Aspect 2)
In one embodiment, the information processing device cooks the food according to the cooking procedure based on the ingredients recorded in the image information acquired by the first acquisition unit and appearing in chronological order and the cooking procedure acquired by the second acquisition unit. It is good also as including the 1st estimating part which estimates whether there is.
That is, the information processing device can estimate whether one or more ingredients (ingredients recorded in image information) that appear in chronological order appear in the order according to the cooking procedure and are cooked according to the cooking procedure. I can do it. Thereby, the cooking staff and the manager can check whether the cooking is being performed according to the cooking procedure based on the estimation results.
Further, the information processing device can use the estimation result of whether the cooking is being performed according to the cooking procedure for the training of the cooking staff who cook the food.

(Aspect 3)
In one embodiment of the information processing device, the output control unit may output a warning when the first estimation unit estimates that the food is not cooked according to the cooking procedure.
Thereby, the cooking staff and the manager can confirm, based on the warning, that the cooking is not being performed according to the cooking procedure.

(Aspect 4)
The information processing device in one embodiment may include a second estimating unit that estimates the remaining amount of food stored in an appliance used for cooking based on the image information acquired by the first acquiring unit.
Thereby, the information processing device can notify the cooking staff and the administrator of the remaining amount of food.

(Aspect 5)
In one embodiment of the information processing device, the output control unit may output an instruction to cook the dish when the remaining amount of the dish estimated by the second estimation unit is equal to or less than a threshold value.
Thereby, the information processing device can issue a notification prompting the user to cook the food before the food runs out.

(Aspect 6)
In one aspect of the information processing device, the second estimating unit estimates the time when the remaining amount of food will run out based on changes over time in the amount of remaining food, and the output control unit estimates the amount of time that the second estimating unit estimates. It is also possible to output cooking instructions beforehand.
Thereby, the information processing device can issue a notification prompting the user to cook the food before the food runs out.

(Aspect 7)
In one embodiment of the information processing device, the output control unit is configured to control the estimated time when the food will run out based on the cooking time recorded in the corresponding information stored in the storage unit and the time estimated by the second estimation unit. The instruction for cooking the food may be output at a timing earlier than the cooking time based on the cooking time.
Thereby, the information processing device can issue a notification prompting the user to cook the food before the food runs out.

(Aspect 8)
The information processing device of one aspect includes a timer that starts measuring when the first estimator estimates the end of cooking based on the image information, and the output controller is configured to adjust the time measured by the timer to the end of cooking. The instruction to cook the food may be output at a timing before a predetermined time set as the disposal deadline has elapsed.
Thereby, the information processing device can issue a notification prompting the user to cook the dish before discarding the dish.

(Aspect 9)
In one aspect of the information processing method, a computer including a storage unit that stores correspondence information that associates meat-containing ingredients with cooking procedures for meat-containing dishes using the ingredients, prepares meat-containing dishes. a first acquisition step of acquiring image information regarding the image of the area; a specifying step of specifying the food ingredients based on the image information obtained in the first obtaining step; and a specifying step based on the corresponding information stored in the storage unit. A second acquisition step of acquiring a cooking procedure according to the identified food ingredient; and an output control step of outputting cooking instruction information regarding the cooking instruction based on the cooking procedure acquired in the second acquisition step.
Thereby, the information processing method can achieve the same effects as the information processing device of one embodiment described above.

(Aspect 10)
One aspect of the information processing program includes a storage function for storing, in a computer, correspondence information that associates meat-containing ingredients with cooking procedures for meat-containing dishes using the ingredients, and cooking for cooking meat-containing dishes. A first acquisition function that acquires image information regarding images of the area; a specific function that identifies ingredients based on the image information acquired by the first acquisition function; and a specific function that identifies ingredients based on the corresponding information stored in the memory function. A second acquisition function that acquires a cooking procedure according to the specified food material and an output control function that outputs cooking instruction information regarding cooking instructions based on the cooking procedure acquired by the second acquisition function are realized.
Thereby, the information processing program can achieve the same effects as the information processing apparatus of one embodiment described above.

(Aspect 11)
An information processing device according to one embodiment includes a storage unit that stores correspondence information that associates food ingredients with cooking procedures for dishes using the food ingredients, and a first acquisition unit that acquires image information regarding an image of a cooking area where food is cooked. a specifying section that specifies the food based on the image information acquired by the first obtaining section; and a first obtaining a cooking procedure corresponding to the food specified by the specifying section based on the correspondence information stored in the storage section. and an output control section that outputs cooking instruction information regarding the cooking instruction based on the cooking procedure acquired by the second acquisition section.
Thereby, even with the information processing device of this aspect, the same effects as the information processing device of one aspect described above can be achieved. This aspect can take a similar configuration (or a different configuration) even for an information processing method and an information processing program.

(Aspect 12)
An information processing device according to one aspect includes a storage unit that stores correspondence information that associates food ingredients containing meat with cooking procedures for a stew dish that includes meat using the ingredients, and a cooking area that cooks a stew dish that includes meat. a first acquisition unit that acquires image information regarding the image of the food; a identification unit that identifies the food based on the image information acquired by the first acquisition unit; and an identification unit that identifies the food based on the correspondence information stored in the storage unit. The second acquisition unit acquires a cooking procedure corresponding to the food that has been prepared, and the output control unit outputs cooking instruction information regarding the cooking instruction based on the cooking procedure acquired by the second acquisition unit.
Thereby, even with the information processing device of this aspect, the same effects as the information processing device of one aspect described above can be achieved. This aspect can take a similar configuration (or a different configuration) even for an information processing method and an information processing program.

1 Information processing device 11 Control unit 12 First acquisition unit 13 Specification unit 14 Second acquisition unit 15 First estimation unit 16 Second estimation unit 17 Time measurement unit 18 Output control unit 21 Imaging unit 22 Communication unit 23 Storage unit 24 Display unit 101 Cooking area 102 Ingredients 103 Cooking

Claims (10)

a storage unit that stores correspondence information that associates meat-containing ingredients with cooking procedures for meat-containing dishes using the ingredients;
a first acquisition unit that acquires image information regarding an image of a cooking area where a dish containing meat is cooked;
an identification unit that identifies food ingredients based on the image information acquired by the first acquisition unit;
a second acquisition unit that acquires a cooking procedure according to the food specified by the identification unit based on the correspondence information stored in the storage unit;
an output control unit that outputs cooking instruction information regarding the cooking instructions based on the cooking procedure acquired by the second acquisition unit;
An information processing device comprising: A first method for estimating whether or not the food is cooked according to the cooking procedure based on the ingredients recorded in the image information acquired by the first acquisition section and appearing in chronological order and the cooking procedure acquired by the second acquisition section. The information processing device according to claim 1, further comprising an estimation section. The information processing device according to claim 2, wherein the output control unit outputs a warning when the first estimating unit estimates that the cooking is not done according to the cooking procedure. The information processing apparatus according to claim 2 or 3, further comprising a second estimating section that estimates the remaining amount of food stored in an appliance used for cooking based on the image information acquired by the first acquiring section. The information processing device according to claim 4, wherein the output control unit outputs an instruction to cook the dish when the remaining amount of the dish estimated by the second estimating unit is equal to or less than a threshold value. The second estimation unit estimates the time when the food will run out based on changes over time in the amount of food remaining;
The information processing device according to claim 4, wherein the output control unit outputs the cooking instruction before the time estimated by the second estimation unit has elapsed. The output control unit controls the cooking based on the estimated time when the food will run out, based on the cooking time recorded in the correspondence information stored in the storage unit and the time estimated by the second estimation unit. 7. The information processing apparatus according to claim 6, wherein the information processing apparatus outputs the cooking instruction at a timing earlier than the specified time. comprising a timekeeping unit that starts timing when the first estimating unit estimates the end of cooking based on image information;
8. The information processing apparatus according to claim 7, wherein the output control section outputs the cooking instruction at a timing before the time measured by the clock section passes a predetermined time set as a food disposal deadline. A computer includes a storage unit that stores correspondence information that associates meat-containing ingredients with cooking procedures for meat-containing dishes using the ingredients,
a first acquisition step of acquiring image information regarding an image of a cooking area for cooking a dish containing meat;
a specifying step of specifying the food material based on the image information obtained in the first obtaining step;
a second acquisition step of acquiring a cooking procedure according to the ingredients specified in the identification step, based on the correspondence information stored in the storage unit;
an output control step of outputting cooking instruction information regarding cooking instructions based on the cooking procedure acquired in the second acquisition step;
An information processing method that performs. to the computer,
a memory function that stores correspondence information that associates meat-containing ingredients with cooking procedures for meat-containing dishes using the ingredients;
a first acquisition function that acquires image information regarding an image of a cooking area where a dish containing meat is cooked;
a specific function that identifies food ingredients based on the image information acquired by the first acquisition function;
a second acquisition function that acquires a cooking procedure according to the food specified by the specific function based on the correspondence information stored in the storage function;
an output control function that outputs cooking instruction information regarding cooking instructions based on the cooking procedure acquired by the second acquisition function;
An information processing program that realizes.

Images