JP2016139356A - Cooking support device, cooking support method, and cooking support program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To support cooking.SOLUTION: A cooking support device 100 includes: a reception part 131 which receives a specified dish; a determination part 132 which determines an ingredient used for the dish from an image imaged by an imaging part 114; and a notification part 133 which notifies a user of information regarding cooking of the specified dish received by the reception part 131, in accordance with the ingredient used for the dish determined by determination part 132. For example, notification part 133 displays information regarding cooking of the specified dish superimposed on a display screen displaying a dynamic image being imaged by the imaging part 114.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a cooking support device, a cooking support method, and a cooking support program.

  2. Description of the Related Art Conventionally, a technique for capturing food images and assisting food processing has been provided. For example, as shown in Non-Patent Document 1, an application for displaying a guideline (auxiliary line) for equally dividing a cake or the like is provided.

Cake cutter, Google Play, [online], [Search January 14, 2015], Internet <URL: https://play.google.com/store/apps/details?id=net.newcakecutter&hl=en>

  However, the above-described method can assist the processing of ready-made products such as cakes, but it is difficult to support various cooking operations (hereinafter referred to as “cooking”).

  Then, an object of this invention is to provide the cooking assistance apparatus, the cooking assistance method, and cooking assistance program which can support cooking of cooking.

  In order to solve the above-described problems and achieve the object, the cooking support apparatus includes a reception unit that receives designation of cooking, a determination unit that determines an object used for cooking from an image captured by the imaging unit, and a determination A notification unit configured to notify information related to cooking of the designated dish, which is a dish whose designation is accepted by the accepting unit, based on an object used for the dish determined by the unit.

  According to the present invention, cooking of food can be supported.

FIG. 1 is a diagram illustrating a configuration example of a cooking support apparatus according to an embodiment. FIG. 2 is a diagram illustrating an example of a table stored in the material information storage unit according to the embodiment. FIG. 3 is a diagram illustrating an example of a table stored in the procedure information storage unit according to the embodiment. FIG. 4 is a diagram showing an outline of a procedure for preparing a material according to the embodiment. FIG. 5 is a diagram showing an outline of a procedure for processing the material according to the embodiment. FIG. 6 is a diagram showing an outline of a procedure for making a dish according to the embodiment. FIG. 7 is a flowchart showing a processing procedure for preparing a material according to the embodiment. FIG. 8 is a flowchart showing a processing procedure for processing the material according to the embodiment. FIG. 9 is a flowchart showing a processing procedure for making a dish according to the embodiment. FIG. 10 is a diagram illustrating a computer that executes a cooking support program.

  Hereinafter, embodiments of a cooking support device, a cooking support method, and a cooking support program according to the present application will be described in detail with reference to the drawings. In addition, the cooking assistance apparatus, cooking assistance method, and cooking assistance program which concern on this application are not limited by this embodiment.

[Embodiment]
In the following embodiments, the configuration and processing flow of the cooking assistance device according to the embodiment will be described in order, and then the effects of the embodiment will be described finally.

[Configuration of cooking support device]
First, the structure of the cooking assistance apparatus 100 is demonstrated using FIG. FIG. 1 is a configuration diagram illustrating an outline of a cooking support apparatus according to the present embodiment. The cooking support apparatus 100 described below displays a guide on a display screen that displays a moving image being captured based on a moving image during cooking of a dish, or outputs a guide by voice. As illustrated in FIG. 1, the cooking support apparatus 100 includes an input unit 111, an output unit 112, a communication unit 113, an imaging unit 114, a storage unit 120, and a control unit 130. Hereinafter, the processing of each of these units will be described.

  The input unit 111 is used to input a designation of food. For example, the input unit 111 receives various operations from the user via the display screen 112a such as a tablet terminal by the function of the touch panel. Further, the input unit 111 may accept various operations from the user via a microphone such as a tablet terminal by a voice recognition function.

  The output unit 112 includes a display screen 112a such as a tablet terminal realized by, for example, a liquid crystal display or an organic EL (Electro-Luminescence) display, and displays various types of information. For example, the output unit 112 outputs a moving image being picked up by the image pickup unit 114, information on cooking of the food notified by the notification unit 133 described later, and the like. Specifically, the output unit 112 superimposes and displays information related to cooking on a display screen 112 a that displays a moving image being captured by the imaging unit 114. The output unit 112 may include a speaker or the like, and may output information related to cooking support notified by the notification unit 133. Further, the output unit 112 may display a list of dish names in order to assist the user in specifying dishes.

  The communication unit 113 is realized by a communication circuit or the like, for example. The communication unit 113 transmits / receives information to / from an external information processing apparatus such as a cooking database. For example, the communication unit 113 may receive a list of dish names from an external information processing apparatus or may receive information such as a recipe of a designated dish.

  The imaging unit 114 has a camera function and a video function realized by an imaging element such as a CCD (Charge Coupled Device), and captures a still image or a moving image. For example, the imaging unit 114 captures a moving image of ingredients and cooking utensils (hereinafter, simply referred to as “cooking”) used for cooking when cooking. Note that the imaging unit 114 is not limited to the case where it is built in the cooking assistance device 100, and the imaging unit 114 is provided outside the cooking assistance device 100, and the cooking assistance device 100 is imaged by the imaging unit 114. The transmitted moving image or the like may be received by the communication unit 113.

  The storage unit 120 stores data and programs necessary for various types of processing by the control unit 130. In particular, the storage unit 120 includes a material information storage unit 121 and a procedure information storage unit 122 that are closely related to the present invention. The storage unit 120 is a semiconductor memory device such as a RAM (Random Access Memory), a ROM (Read Only Memory), or a flash memory, or a storage device such as a hard disk or an optical disk.

  The material information storage unit 121 stores information about materials used for cooking. Specifically, the material information storage unit 121 stores information on cooking, materials used for cooking, processing methods of materials, and the like. The materials mentioned here include various materials used for cooking such as vegetables and meat. In the example illustrated in FIG. 2, the material information storage unit 121 stores that the material A and the material B are used for the dish A. In addition, the material information storage unit 121 stores not only the name of the material such as the dish A but also information regarding the image and the amount of the material in the material item.

  In the example shown in FIG. 2, in the case of the dish A, the material information storage unit 121 has the final shape used for the dish A in three stages of the cutting methods A1, A2, and A3, and the material B is the cutting method. It memorizes that it becomes the final shape used for cooking A by two stages of B1 and B2. Here, the material information storage unit 121 stores information on the shape of the vegetable such as an image of the vegetable at the stage of cutting, auxiliary lines for cutting at the stage, and the like in the item of the processing method. For example, in the cutting method A1 of the processing method, an image indicating the first-stage shape of the material A and auxiliary line information indicating the cutting method are stored. That is, in the cutting method A1 of the processing method, the auxiliary line information of the optimal cutting method A1 for the first stage shape of the material A is stored. For example, in the cutting method A2 of the processing method, an image indicating the shape of the second stage of the material A and auxiliary line information indicating the cutting method are stored. That is, in the cutting method A2 of the processing method, the auxiliary line information of the optimal cutting method A2 with respect to the second-stage shape of the material A that has been cut and changed in the first stage is stored.

  In FIG. 2, the cutting method is shown as an example of the processing method, but the processing method may include various processing methods such as tearing and crushing. Moreover, although the example which memorize | stores the information regarding the material used for several dishes, such as dish A, B, C, was shown, you may memorize | store only the information regarding the designated dish. For example, when the dish A is designated, the material information storage unit 121 may store information on the material used for the dish A received from the external information processing apparatus.

  The procedure information storage unit 122 stores information related to a procedure for making a dish. Specifically, the procedure information storage unit 122 stores information on cooking, cooking procedures, and the like. In the example illustrated in FIG. 3, the procedure information storage unit 122 stores that the procedure for preparing the dish A includes the procedure A1, the procedure A2, and the procedure A3. Note that the procedure information storage unit 122 stores, in the work procedure items, instruments and materials used in the procedure, texts indicating the work contents of the procedure, information on the auxiliary image, and the like. The auxiliary image may include, for example, an image of a material used in the procedure.

  The control unit 130 has an internal memory for storing a program that defines various processing procedures and the necessary data, and performs various processes using these programs. Particularly, as closely related to the present invention, A reception unit 131, a determination unit 132, and a notification unit 133 are included.

  The accepting unit 131 accepts designation of a dish. For example, the reception unit 131 receives a dish input via the input unit 111 as a designated dish.

  The determination unit 132 determines an object used for cooking from the image captured by the imaging unit 114. Specifically, the determination part 132 determines the material used for cooking from an image. Moreover, the determination part 132 determines the material used for a designated dish from an image. Moreover, the determination part 132 determines the cooking utensil used for a designated dish from an image. The determination unit 132 may appropriately use various techniques of image processing such as pattern recognition as a technique for determining an object used for cooking from the image captured by the imaging unit 114. For example, the determination unit 132 performs pattern matching using the material image stored in the material information storage unit 121 and the image captured by the imaging unit 114, thereby specifying the image in the image captured by the imaging unit 114. You may determine whether the material used for cooking is included.

  The notification unit 133 notifies information related to cooking of the designated dish, which is a dish whose designation is received by the reception unit 131, based on the thing used for the dish determined by the determination unit 132. Specifically, the notification unit 133 causes the display screen 112a that displays the moving image being captured by the imaging unit 114 to be superimposed and displayed on the display screen 112a. In addition, the notification unit 133 relates to information that is not included in the material determined by the determination unit 132 among the materials required for the specified dish, or that is not necessary for the specified dish among the materials determined by the determination unit 132. Notify information.

  Further, the notification unit 133 notifies information related to the processing method for the material determined by the determination unit 132 based on the designated dish. For example, the notification unit 133 notifies information on how to cut the material based on the designated dish as a processing method for the material. Further, the notification unit 133 notifies information related to the procedure for making the designated dish using the cooking utensil determined by the determination unit 132 based on the specified dish. For example, the notification unit 133 performs the above-described notification via the output unit 112. Specifically, the notification unit 133 displays the above-described notification on the display screen 112a or the like of the output unit 112 or outputs it as a sound through the speaker or the like of the output unit 112.

  Here, the example which notifies the information regarding cooking of designation | designated cooking based on the thing used for the cooking determined by the determination part 132 is demonstrated using the example of FIGS. Specifically, in the following, an example in which information related to cooking of a designated dish is displayed on a display screen 112a that displays a moving image being captured by the imaging unit 114 will be described. As shown in FIGS. 4 to 6, a portable terminal as a whole can be adopted as the cooking assistance device 100. For example, a dedicated terminal that performs cooking support, a smartphone or a tablet terminal in which a cooking support application is installed, or the like can be employed for the cooking support device 100. For example, when the cooking assistance apparatus 100 is a smartphone in which a cooking assistance application is installed, the control unit 130 and the storage unit 120 may be functions of the cooking assistance application installed in the cooking assistance apparatus 100. 4 to 6, the user installs the cooking support apparatus 100 on the stand 200 and displays a display screen in which information related to cooking of the designated dish is superimposed on the moving image being captured by the imaging unit 114. Cooking is performed while viewing 112a (see FIGS. 4 (1), 5 (1), and 6 (1)).

  FIG. 4 is a diagram showing an outline of a procedure for preparing a material according to the embodiment. In the example shown in FIG. 4, a case where a dish D using eggs, zucchini, and carrots is designated as the designated dish is shown.

  First, when the dish D is designated, characters of eggs, zucchini, and carrots that are ingredients used for the dish D are displayed on the display screen 112a (see FIG. 4B). Thereafter, when imaging is started by the imaging unit 114, the determination unit 132 determines the material used for the dish D from the image captured by the imaging unit 114. In the example shown in FIG. 4, the moving image being imaged by the imaging unit 114 includes zucchini F1 and carrot F2 (see FIG. 4 (3)). Therefore, the notification unit 133 notifies that the ingredients required for the dish D are not included in the ingredients determined by the determination unit 132, that is, eggs are insufficient. For example, the notification unit 133 causes the speaker to output a message M1 with the content “No egg” to the speaker. In addition, the notification unit 133 superimposes and displays information for notifying that eggs are insufficient on the display screen 112a that displays a moving image being captured by the imaging unit 114. For example, among the eggs, zucchini, and carrot characters that are displayed on the display screen 112a by being superimposed on the moving image being captured by the image capturing unit 114, the underlining of the eggs alone indicates that the eggs are insufficient. Notice. The notification is not limited to underlining, but may be performed by changing the color or changing the character size.

  Thereafter, when the user adds an egg to the range imaged by the imaging unit 114, the moving image being imaged by the imaging unit 114 includes zucchini F1, carrot F2, and egg F3 (see FIG. 4 (4)). ). As a result, since all the ingredients required for cooking D have been prepared, the notification unit 133, for example, causes the speaker to output a message M2 having the content “materials have been prepared” to the speaker. In addition, by deleting the underline drawn on the character of the egg displayed superimposed on the moving image being imaged by the imaging unit 114 on the display screen 112a, it is notified that the eggs are aligned. In the above example, when all the materials are aligned in the same image, an example of determining that the materials are aligned is shown. However, when each material is sequentially imaged and all the materials are sequentially imaged, the material It may be determined that is complete. In addition, the notification unit 133, when a material unnecessary for the dish D, for example, green pepper is included in the moving image being captured by the imaging unit 114, the material unnecessary for the dish D among the materials determined by the determination unit 132, that is, Notify that peppers are included. For example, the notification unit 133 causes the speaker to output a message with the content “No bell pepper is necessary”.

  Further, the determination unit 132 may determine the amount of the material of the moving image by comparing the material of the moving image being imaged by the imaging unit 114 and the information on the material stored in the material information storage unit 121. . If the determination unit 132 cannot identify a material from the image captured by the imaging unit 114, the notification unit 133 displays a plurality of materials as options on the display screen 112a, and which material is included. May be selected by the user. In addition, when the determination unit 132 cannot identify the material from the image captured by the imaging unit 114, the notification unit 133 notifies the user that the direction and size of imaging performed by the imaging unit 114 are changed. May be.

  FIG. 5 is a diagram showing an outline of a procedure for processing the material according to the embodiment. In the example shown in FIG. 5, the case where the zucchini which is the material of the dish D is processed is shown.

  In the example illustrated in FIG. 5, the moving image being captured by the imaging unit 114 includes the zucchini F1 (see FIG. 5 (2)). Therefore, the notification unit 133 refers to the processing method of the material zucchini of the dish D stored in the material information storage unit 121, and notifies the information regarding the processing method for the zucchini F1. Specifically, the notification unit 133 causes the three auxiliary lines L1 to be superimposed and displayed on the display screen 112a that displays the zucchini F1 included in the moving image being captured (see FIG. 5B).

  In the example shown in FIG. 5, the case where three auxiliary lines L1 are displayed superimposed on the zucchini F1 in the display screen 112a is shown. However, the number of auxiliary lines according to the size of the material such as zucchini May be varied. Specifically, the determination unit 132 estimates the size from the material included in the moving image being captured by the imaging unit 114, and the notification unit 133 varies the number of auxiliary lines according to the estimated size. You may let them. Note that the determination unit 132 may appropriately use various image processing techniques as a technique for estimating the size of the material included in the moving image being captured by the imaging unit 114.

  For example, when the processing method of the material zucchini of the dish D stored in the material information storage unit 121 is stored as “1 cm thick and sliced”, if the length of the zucchini is 10 cm, nine auxiliary lines are superimposed. If the length of the zucchini is 5 cm, four auxiliary lines may be superimposed and displayed. In addition, when the material zucchini processing method of the dish D stored in the material information storage unit 121 is stored as “divided into four equal parts”, three auxiliary lines with equal intervals are provided according to the length of the zucchini. It may be displayed in a superimposed manner. Note that the notification unit 133 corrects the inclination of the material in the moving image being captured by the imaging unit 114, and creates and displays an appropriate auxiliary line.

  Further, the notification unit 133 may cause the speaker to output information regarding the processing method for the zucchini F1. For example, the notification unit 133 may cause the speaker to output a voice message “Please cut zucchini into 3 cm thick”. Further, the notification unit 133 may display a message of “Please cut zucchini in a thickness of 3 cm” on the display screen 112a. Thereby, the user can grasp | ascertain the content of cooking more correctly by audio | voice information and character information with a superimposed image. Moreover, when displaying the content of cooking by audio | voice information or character information, the auxiliary | assistant line made to superimpose may show the outline of how to cut.

  Next, the moving image being imaged by the imaging unit 114 includes the processed zucchini F11 that has been cut into circles (see FIG. 5 (3)). Therefore, the notification unit 133 refers to the processing method of the material zucchini of the dish D stored in the material information storage unit 121, and notifies the processing information for the processed zucchini F11 that has been cut into pieces. Specifically, the notification unit 133 displays the two auxiliary lines L2 superimposed on the display screen 112a that displays the processed zucchini F11 included in the moving image being captured (see FIG. 5C). ). In the example shown in FIG. 5, the zucchini F11 after being cut into circles is indicated by two auxiliary lines L2 that perform cutting in which the angles in the section of the ring cut are 90 ° each.

  In addition, the notification unit 133 may cause the speaker to output information related to the processing method for the zucchini F11. For example, the notification unit 133 may cause a speaker to output a message with the content “Please cut zucchini into four equal parts”. Further, the notification unit 133 may display a message “Please cut zucchini into four equal parts” on the display screen 112a. Thereby, the user can grasp | ascertain the content of cooking more correctly by audio | voice information and character information with a superimposed image.

  Next, the processed zucchini F12 that has been cut off is included in the moving image being imaged by the imaging unit 114 (see FIG. 5 (4)). Therefore, the notification unit 133 refers to the processing method of the material zucchini of the dish D stored in the material information storage unit 121, and when there is no information regarding the processing method for the processed zucchini F12 after being cut, You may notify that F12 is a final shape. For example, the notification unit 133 may cause the speaker to output a message having the content “Zucchini's final shape”. In addition, the notification unit 133 may display a message with the content “Zucchini's final shape” on the display screen 112a. Note that the processing method of the material of the dish D stored in the material information storage unit 121 is also used when eggs and carrots, which are other materials used for the dish D, are included in the moving image being captured by the imaging unit 114. Refer to and process.

  FIG. 6 is a diagram showing an outline of a procedure for making a dish according to the embodiment. In the example shown in FIG. 6, the case where the instrument C1 is used for cooking of the dish D is shown.

  In the example shown in FIG. 6, the moving image being imaged by the imaging unit 114 includes the appliance C1 used for cooking the dish D (see FIGS. 6 (2) to (4)). Therefore, the notification unit 133 refers to the work procedure of the dish D stored in the procedure information storage unit 122 and notifies the procedure of making the dish D using the appliance C1. Specifically, the notification unit 133 notifies the procedure No. 1, which is a procedure for putting the zucchini F12 after being cut into the appliance C1. For example, the notification unit 133 causes the auxiliary image FI12 to be superimposed and displayed on the display screen 112a that displays the instrument C1 included in the moving image being captured (see FIG. 6B). In the example illustrated in FIG. 6, the auxiliary image FI <b> 12 is a zucchini image cut into pieces. Thereby, the notification unit 133 visually notifies the user that the processed zucchini F12 cut into the appliance C1 is to be inserted.

  Further, the notification unit 133 may cause the speaker to output a procedure for making the dish D using the appliance C1. For example, the notification unit 133 may cause the speaker to output a message with the content “Please insert the chopped zucchini into the appliance C1”. In addition, the notification unit 133 may display a message “Please insert the zucchini that has been cut into the appliance C1” on the display screen 112a. Thereby, the user can grasp | ascertain the content of cooking more correctly by audio | voice information and character information with a superimposed image.

  Here, the procedure 1 is included in the moving image being imaged by the imaging unit 114 (see FIG. 6B). Therefore, the determination unit 132 determines that the procedure D of dish D has been performed. Thereafter, the moving image being imaged by the imaging unit 114 includes the instrument C1 into which the processed zucchini F12 that has been cut into pieces is inserted (see FIG. 6 (3)). Therefore, the notification unit 133 refers to the work procedure of the dish D stored in the procedure information storage unit 122 and notifies the procedure 2 which is the next work procedure of the procedure 1. For example, the notification unit 133 displays the auxiliary image on the display screen 112a that displays the appliance C1 included in the moving image being captured when the procedure 2 is a work procedure with a time designation such as “fire for 25 minutes”. T1 is superimposed and displayed (see FIG. 6 (3)). In the example illustrated in FIG. 6, the auxiliary image T <b> 1 indicates the remaining time specified for time. Thereby, the notification unit 133 visually notifies the user of the remaining time of the work procedure with the time designation. Further, the notification unit 133 may cause the speaker to output the remaining time of the procedure 2 as a sound. Thereby, the user can more accurately grasp the remaining time of the work procedure with time designation by using the audio information and the character information together with the superimposed image.

  When the time specified by the time has elapsed, the moving image being imaged by the imaging unit 114 includes the instrument C1 after completion of the procedure 2 (see FIG. 6 (4)). Therefore, the notification unit 133 refers to the work procedure of the dish D stored in the procedure information storage unit 122 and notifies the procedure No. 3 which is the next procedure after the procedure No. 2. Specifically, the notification unit 133 notifies the procedure No. 3, which is a procedure for putting the carrot F2 into the appliance C1. For example, the notification unit 133 causes the auxiliary image FI2 to be superimposed and displayed on the display screen 112a that displays the instrument C1 included in the moving image being captured (see FIG. 6 (4)). In the example shown in FIG. 6, the auxiliary image FI2 is a carrot image. Thereby, the notification unit 133 visually notifies the user that the carrot F2 will be put into the appliance C1.

  In addition, the notification unit 133 may cause the speaker to output procedure 3 by voice. For example, the notification unit 133 may cause the speaker to output a message “Please input carrot into the appliance C1”. In addition, the notification unit 133 may display a message “Please input carrots into the appliance C1” on the display screen 112a. Thereby, the user can grasp | ascertain the content of cooking more correctly by audio | voice information and character information with a superimposed image.

  After that, the notification unit 133 refers to the work procedure of the dish D stored in the procedure information storage unit 122, and notifies that cooking of the dish D has been completed when there is no work procedure next to the procedure 3. For example, the notification unit 133 may cause the speaker to output a message with the content “cooking of dish D has been completed”. Further, the notification unit 133 may display a message indicating that “cooking of the dish D has been completed” on the display screen 112a. Thereby, the user can grasp that cooking of the dish is completed.

[Processing by cooking support device]
Next, the process by the cooking assistance apparatus 100 which concerns on embodiment is demonstrated using FIGS. 7-9 is a flowchart for demonstrating the process sequence of the cooking assistance apparatus which concerns on embodiment. Specifically, FIG. 7 is a flowchart showing a processing procedure for preparing a material according to the embodiment. Moreover, FIG. 8 is a flowchart which shows the process sequence which processes the material which concerns on embodiment. Moreover, FIG. 9 is a flowchart which shows the process sequence which makes the dish which concerns on embodiment.

  First, with reference to FIG. 7, the process of preparing the ingredients for the designated dish will be mainly described. In addition, in FIG. 7, when the cooking assistance apparatus 100 is a smartphone or a tablet terminal in which the cooking assistance application is installed, the cooking assistance application is activated, and when it is a dedicated terminal, the power is turned on. It is assumed that when various types of information related to cooking are acquired from an external information processing apparatus, the process will be described assuming that the connection with the information processing apparatus has been confirmed.

  As illustrated in FIG. 7, when a dish is designated (Yes at Step S101), the reception unit 131 of the cooking support apparatus 100 acquires a recipe for the designated dish that is the designated dish (Step S102). For example, the reception unit 131 may display a list of dish names on the display screen 112a and allow the user to specify a dish from the displayed dish names. For example, the reception unit 131 acquires a recipe for a designated dish from an external information processing apparatus. The recipe of the designated dish mentioned here includes, for example, information about ingredients used for cooking stored in the material information storage unit 121, information about a procedure for making dishes stored in the procedure information storage unit 122, and the like.

  Next, the output unit 112 displays the ingredients used for the designated dish (step S103). For example, in the example shown in FIG. 4, characters of eggs, zucchini, and carrots, which are ingredients used for the dish D that is the designated dish, are displayed on the display screen 112a.

  When the imaging unit 114 has not started capturing a moving image (No at Step S104), the determining unit 132 repeats the determination of whether the imaging unit 114 has started capturing a moving image (Step S104). Further, when the imaging unit 114 starts capturing a moving image (Yes in step S104), the determination unit 132 determines whether there is an unnecessary material in the moving image being captured (hereinafter referred to as “captured image”). Is determined (step S105). When the determination unit 132 determines that there is an unnecessary material in the captured image (Yes at Step S105), the notification unit 133 notifies the output unit 112 of the unnecessary material (Step S106), and returns to Step S105. If the determination unit 132 determines that there is no unnecessary material in the captured image (No at Step S105), the determination unit 132 determines whether there is an insufficient material in the captured image (Step S107). When it is determined by the determination unit 132 that there is insufficient material in the captured image (Yes in step S107), the notification unit 133 notifies the shortage material by the output unit 112 (step S108), and the process returns to step S105.

  If the determination unit 132 determines that there is no insufficient material in the captured image (No at step S107), the process for preparing the material for the designated dish is completed, and the process proceeds to step S201 and subsequent steps shown in FIG. In addition, the cooking assistance apparatus 100 may transfer to the process shown in FIG. 8 according to a user's designation | designated after acquiring the recipe of a designated dish by step S102.

  Next, with reference to FIG. 8, the processing for processing the ingredients for the designated dish after the above-described process for preparing the ingredients for the designated dish in FIG. 7 will be mainly described. First, the determination unit 132 determines whether there is material in the captured image (step S201). If it is determined that there is no material in the captured image (No at Step S201), the determination unit 132 repeats the determination of whether there is material in the captured image (Step S201).

  If the determination unit 132 determines that there is material in the captured image (Yes in step S201), the determination unit 132 determines whether the material in the captured image has a final shape for cutting (step S202). When the determining unit 132 determines that the material in the captured image is not the final shape of the cutting method (No at Step S202), the notification unit 133 displays the cutting auxiliary line by the output unit 112 (Step S203).

  When the determination unit 132 determines that the material in the captured image has the final shape of the cutting method (Yes at Step S202), the determination unit 132 determines whether or not the processing of all the food materials (materials) has been completed ( Step S204). For example, the determination unit 132 may determine that all materials have been processed when a predetermined time has elapsed without a new material being included in the captured image. Moreover, the reception part 131 may receive the input which shows that the process of all the materials was completed from a user.

  If the determination unit 132 determines that the processing of all the materials has not been completed (No at Step S204), the determination unit 132 returns to the determination at Step S201. If the determination unit 132 determines that all the ingredients have been processed (Yes at Step S204), the process proceeds to Step S301 and the subsequent steps shown in FIG.

  Next, with reference to FIG. 9, the process of making the designated dish after the process of processing the ingredients of the designated dish shown in FIG. 8 will be mainly described.

  After setting the variable i to 0 (step S301), the determination unit 132 determines whether there is an instrument in the captured image (step S302). When it is determined that there is no instrument in the captured image (No at Step S302), the determination unit 132 repeats the determination of whether there is an instrument in the captured image (Step S302).

  When the determination unit 132 determines that there is an appliance in the captured image (Yes at Step S302), the notification unit 133 notifies the procedure i-th work content of the designated dish (Step S303). When the determination unit 132 determines that the i-th work content of the procedure of the designated dish has time specification (Yes in step S304), the notification unit 133 displays the remaining time based on the specified time by the output unit 112 ( Step S305). Thereafter, the determination unit 132 determines whether the specified time has elapsed (step S306).

  When it is determined that the time specified by the determination unit 132 has not elapsed (No at Step S306), the notification unit 133 updates the remaining time displayed based on the specified time by the output unit 112 (Step S305).

  On the other hand, when determining that the specified time has elapsed (Yes at Step S306), the determining unit 132 determines whether or not all the procedures have been completed (Step S307). If it is determined that the entire procedure has not been completed (No at Step S307), the determination unit 132 adds 1 to the variable i (Step S308), and then returns to Step S302 to perform processing. If it is determined by the determination unit 132 that all of the procedures have been completed (Yes at Step S307), the completion of cooking of the designated dish is notified (Step S309), and the process ends.

  In addition, when the determination unit 132 determines that the procedure i-th operation content of the designated dish does not specify time (No in step S304), the determination unit 132 determines whether the procedure i-th operation has been executed (step S310). If the determination unit 132 determines that the procedure i-th task has not yet been executed (No in step S310), the determination unit 132 repeats the determination of whether the procedure i-th task has been executed (step S310). The accepting unit 131 may accept an input indicating that the procedure i-th work has been executed from the user.

  If it is determined that the procedure i-th work has been executed (Yes at Step S310), the determination unit 132 determines whether all the procedures have been completed (Step S307). If it is determined that the entire procedure has not been completed (No at Step S307), the determination unit 132 adds 1 to the variable i (Step S308), and then returns to Step S302 to perform processing. If it is determined by the determination unit 132 that all of the procedures have been completed (Yes at Step S307), the completion of cooking of the designated dish is notified (Step S309), and the process ends. In addition, the cooking assistance apparatus 100 may designate time for all the cooking procedures and notify all the procedures based on the designated time.

[effect]
As described above, the cooking support apparatus 100 according to the embodiment includes a receiving unit 131 that receives a designation of cooking, a determination unit 132 that determines an object used for cooking from an image captured by the imaging unit 114, and a determination. A notification unit 133 for notifying information related to cooking of the designated dish, which is a dish whose designation has been received by the reception unit 131, based on the thing used for the dish determined by the unit 132.

  Thereby, the cooking assistance apparatus 100 which concerns on embodiment can assist cooking of cooking. Specifically, the cooking support device 100 displays a guide indicating the correct procedure or instructs by voice while comparing a captured image with a reference image on a cooking database (for example, a storage unit) at each stage of cooking. To do. Thereby, the user can understand that the operation | work in each step of cooking takes in hand, and can implement an operation | work appropriately according to an image.

  Moreover, in the cooking assistance apparatus 100 according to the embodiment, the notification unit 133 causes information related to cooking of the designated dish to be displayed on the display screen that displays the moving image being captured by the imaging unit 114.

  Thereby, the cooking assistance apparatus 100 can notify visually the information regarding cooking of cooking. Therefore, the cooking assistance apparatus 100 can notify the user of information related to cooking in a manner that is easier to grasp, and thus can support cooking. That is, the cooking assistance apparatus 100 can support a user on a captured image with a series of cooking operations, and can support the user without fail to complete cooking.

  Moreover, in the cooking assistance apparatus 100 which concerns on embodiment, the determination part 132 determines the material used for cooking from an image, and the notification part 133 is the material determined by the determination part 132 among the materials required for designated cooking. The information regarding the material not included in the information or the information regarding the material unnecessary for the designated dish among the materials determined by the determination unit 132 is notified.

  Thereby, since the cooking assistance apparatus 100 can notify the excess and deficiency of the material used for cooking as information regarding cooking, cooking can be supported.

  Moreover, in the cooking assistance apparatus 100 which concerns on embodiment, the determination part 132 determines the material used for a designated dish from an image, and the notification part 133 processes the material determined by the determination part 132 based on a designated dish. Informs about the method.

  Thereby, since the cooking assistance apparatus 100 can notify the processing method of the material used for cooking as information regarding cooking of cooking, cooking cooking can be supported.

  Moreover, in the cooking assistance apparatus 100 which concerns on embodiment, the notification part 133 notifies the information regarding how to cut the material based on a designated dish as a processing method with respect to an ingredient.

  Thereby, since the cooking assistance apparatus 100 can notify how to cut a material as a processing method of the material used for cooking, it can assist cooking of cooking.

  Moreover, in the cooking assistance apparatus 100 which concerns on embodiment, the determination part 132 determines the cooking utensil used for a designated dish from an image, and the notification part 133 is the cooking utensil determined by the determination part 132 based on a designated dish. Informs you about information about how to make a designated dish using.

  Thereby, since the cooking assistance apparatus 100 can notify the procedure which makes the dish which uses an instrument as information regarding cooking of a dish, it can support cooking of a dish.

(Configuration etc.)
Note that each component of each illustrated apparatus is functionally conceptual and does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution / integration of each device is not limited to that shown in the figure, and all or a part thereof may be functionally or physically distributed or arbitrarily distributed in arbitrary units according to various loads or usage conditions. Can be integrated and configured. Further, all or any part of each processing function performed in each device may be realized by a CPU and a program analyzed and executed by the CPU, or may be realized as hardware by wired logic.

  In addition, among the processes described in the present embodiment, all or part of the processes described as being automatically performed can be manually performed, or the processes described as being manually performed can be performed. All or a part can be automatically performed by a known method. In addition, the processing procedure, control procedure, specific name, and information including various data and parameters shown in the above-described document and drawings can be arbitrarily changed unless otherwise specified.

(program)
Moreover, the program which described the process which the cooking assistance apparatus 100 which concerns on the said embodiment performs in the language which can be performed by a computer can also be created. In this case, the same effect as the above-described embodiment can be obtained by the computer executing the program. Further, such a program may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read by the computer and executed to execute the same processing as in the above embodiment. Below, an example of the computer which performs a cooking assistance program is demonstrated.

  FIG. 10 is a diagram illustrating a computer that executes a cooking support program. As shown in FIG. 10, a computer 1000 includes, for example, a memory 1010, a CPU (Central Processing Unit) 1020, a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, a network Interface 1070. These units are connected by a bus 1080.

  The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM (Random Access Memory) 1012. The ROM 1011 stores a boot program such as BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to the hard disk drive 1090. The disk drive interface 1040 is connected to the disk drive 1100. A removable storage medium such as a magnetic disk or an optical disk is inserted into the disk drive 1100, for example. For example, a mouse 1110 and a keyboard 1120 are connected to the serial port interface 1050. For example, a display 1130 is connected to the video adapter 1060.

  Here, as shown in FIG. 10, the hard disk drive 1090 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. Each piece of information described in the above embodiment is stored in, for example, the hard disk drive 1090 or the memory 1010.

  Further, the cooking support program is stored in the hard disk drive 1090 as a program module in which a command executed by the computer 1000 is described, for example. Specifically, the described program module describing each process executed by the cooking support apparatus 100 described in the above embodiment is stored in the hard disk drive 1090.

  Further, data used for information processing by the cooking support program is stored as program data in, for example, the hard disk drive 1090. Then, the CPU 1020 reads out the program module 1093 and the program data 1094 stored in the hard disk drive 1090 to the RAM 1012 as necessary, and executes the above-described procedures. Note that the computer 1000 may acquire an image captured by an imaging device (not shown) wirelessly connected via the network interface 1070, for example. In addition, the computer 1000 may display an image captured by the imaging device or information for assisting cooking on a display terminal (not shown) that is wirelessly connected via the network interface 1070, for example. Further, the above-described imaging device and display terminal may be integrated.

  The program module 1093 and the program data 1094 related to the cooking support program are not limited to being stored in the hard disk drive 1090. For example, the program module 1093 and the program data 1094 are stored in a removable storage medium and read by the CPU 1020 via the disk drive 1100 or the like. May be issued. Alternatively, the program module 1093 and the program data 1094 related to the cooking support program are stored in another computer connected via a network such as a LAN (Local Area Network) or a WAN (Wide Area Network), and are transmitted via the network interface 1070. May be read by the CPU 1020.

DESCRIPTION OF SYMBOLS 100 Cooking assistance apparatus 111 Input part 112 Output part 113 Communication part 114 Imaging part 120 Storage part 121 Material information storage part 122 Procedure information storage part 130 Control part 131 Reception part 132 Determination part 133 Notification part

Claims (8)

A reception unit that accepts the designation of dishes;
A determination unit for determining an object used for cooking from an image captured by the imaging unit;
A notification unit for notifying information related to cooking of a designated dish, which is a dish whose designation is accepted by the accepting unit, based on an object used for the dish determined by the judging unit;
A cooking support apparatus comprising: The notification unit
Information on cooking of the designated dish is superimposed on a display screen that displays a moving image being captured by the imaging unit, and displayed.
The cooking assistance apparatus according to claim 1, wherein: The determination unit
Determine the ingredients used for cooking from the image,
The notification unit
Of the materials required for the designated dish, information on materials that are not included in the materials determined by the determination unit, or information on materials that are unnecessary for the specified dish among the materials determined by the determination unit,
The cooking assistance apparatus according to claim 1 or 2, wherein The determination unit
Determine the ingredients used for the designated dish from the image,
The notification unit
Based on the designated dish, the information on the processing method for the material determined by the determination unit is notified,
The cooking assistance apparatus according to any one of claims 1 to 3, wherein The notification unit
As a processing method for the material, to notify information on how to cut the material based on the specified dish,
The cooking assistance apparatus according to claim 4, wherein: The determination unit
Determine the cooking utensil used for the designated dish from the image,
The notification unit
Based on the designated dish, the information on the procedure for making the designated dish using the cooking utensil determined by the determination unit,
The cooking assistance apparatus according to any one of claims 1 to 5, wherein A cooking support method executed by a cooking support device,
A reception process for accepting the designation of a dish,
A determination step of determining an object used for cooking from the image captured by the imaging step;
A notification step of notifying information relating to cooking of a designated dish, which is a dish whose designation is accepted by the acceptance step, based on the thing used for the dish judged by the judgment step;
The cooking support method characterized by including.   The cooking assistance program for functioning a computer as a cooking assistance apparatus of any one of Claims 1-6.

Images