JP2011108156A - Device and method for instruction of cooking process - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technique that instructs a cooking process to a robot for executing a cooking task, using a simple method. <P>SOLUTION: A cooking process instruction device is provided for instructing the cooking process to the robot for executing a plurality of cooking tasks, including at least input of materials, mixing or reversing the materials and heating the materials. The device includes: a cooking process input unit for displaying an icon, corresponding to each cooking task and a time base, to arrange the icon, and for enabling a user to arrange the icon on the time base by operation of a pointing device; and a control unit for calculating a sequential order and timing for enabling the robot to execute the cooking tasks corresponding to the arranged icons and operation for enabling the robot to carry out the cooking tasks on the basis of kinds and positions of the icons arranged on the time base in the cooking process input unit, and for transmitting control instructions to the robot. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a technique for instructing a cooking process to a robot that performs a cooking task.

  Computer-controlled robots for general households have appeared on the market. For example, cleaning robots are used in many households. Research is also underway on robots that perform various household tasks such as clothing folding and dishwashing.

  The robot can be considered to be an automatic machine that faithfully executes a predetermined operation in accordance with an instruction given by a user, and an autonomous machine that has intelligence such as learning and determination.

  When using the robot as the former automatic machine, the user learns by associating the actions that can be performed by the robot with the instructions to be given to the robot to execute the actions, and the robot performs the user's desired task. Appropriate instructions must be given to the robot. As the robot functions become more complex and sophisticated, if the burden on the user side for learning the robot instruction system increases excessively, the user may not be able to use the advanced functions of the robot.

  For example, in a cooking system using a robot, it is necessary to cause the robot to perform complicated operations such as timing of input cooking of ingredients and control of thermal power. Therefore, in order to realize a user-friendly cooking system using a robot, it is important to provide an instruction interface that allows the robot to perform complex operations simply by giving a simple and intuitive instruction. It becomes a problem.

  Note that Non-Patent Document 1 is a known example of a cooking system using a robot.

Y. Nakauchi, T. Fukuda, K. Noguchi, T. Matsubara: Time Sequence Data Mining for Cooking Support Robot, Proc. Of 2005 IEEE International Symposium on Computational Intelligence in Robotics and Automation (CIRA2005), No. We-B3-4 , 2005

  The present invention has been made in view of the above circumstances, and an object thereof is to provide a technique capable of instructing a cooking process by a simple method to a robot that performs a cooking task.

In order to achieve the above object, a cooking process instruction apparatus according to the present invention includes:
A cooking process instruction device for instructing a cooking process to a robot that performs a plurality of cooking tasks including at least charging of ingredients, mixing or returning of ingredients, and heating of ingredients,
A cooking process input unit that displays icons corresponding to the cooking tasks and a time axis for arranging the icons, and allows a user to arrange the icons on the time axis by operating a pointing device;
Based on the types and positions of the icons arranged on the time axis in the cooking process input unit, the order and timing for causing the robot to execute cooking tasks corresponding to the arranged icons, and the cooking task to the robot A control unit that calculates an action for executing the control and sends a control command to the robot;
It is characterized by providing.

  According to the cooking process instruction apparatus of the present invention, a user can perform a desired cooking task by simply performing an intuitive operation of arranging icons corresponding to cooking tasks on the time axis displayed by the cooking process input unit. Control instructions for causing the robot to execute in order and timing can be given to the robot. Therefore, according to the cooking process instruction apparatus of the present invention, the user can cause the robot to execute a complicated cooking task necessary for completing the cooking only by giving a simple and intuitive instruction.

  As described above, according to the cooking process instruction apparatus of the present invention, the user can easily and intuitively instruct a cooking task to be executed by the robot. It is possible to easily change the instructions to the robot flexibly in accordance with the conditions that vary.

  For example, in order to harden or soften the boiled condition, the user may slow down or speed up the position of the icon corresponding to the heating task of the material on the time axis for the cooking process input unit All that is necessary is an intuitive operation. If it does so, a control part will calculate the suitable timing which should make a robot perform a heating task, and will send out the control command to a robot at a suitable timing. Accordingly, the user can cause the robot to perform an operation necessary for stroking the material harder or softly only by giving a simple and intuitive instruction.

  Further, according to the cooking process instruction apparatus of the present invention, the user can not only cause the robot to execute a recipe prepared in advance, but also can easily cause the robot to execute an original recipe created by the user. For example, when a recipe for curry is prepared in advance, the icon corresponding to the task to insert meat and potatoes is deleted from the time axis, and the icon corresponding to the task to input broccoli and maitake is replaced on the time axis instead. If it is arranged at an appropriate position, an original vegetable curry recipe can be easily created and the robot can execute the created vegetable curry recipe.

  In the cooking process instruction apparatus of the present invention, the cooking process input unit may enable not only the time axis but also the axis corresponding to the thermal power for the icon corresponding to the material heating task. By doing so, the user can cause the robot to perform a task of heating the material with a desired timing and heating by a simple and intuitive instruction. Further, for an icon corresponding to mixing of materials, not only a time axis but also an axis corresponding to mixing intensity may be made effective. By doing so, the user can make the robot execute a mixing task of the desired intensity, such as powerful mixing or mixing that is difficult to lose shape, according to the characteristics of the ingredients and the cooking phase, with simple and intuitive instructions Become. Also, instead of adding an axis for indicating the heating power and mixing intensity in this way, for example, select a desired icon from multiple heating task icons with different heating power and mixing intensity and place it on the time axis You may be able to do it. In any method, the user can cause the robot to execute a heating task or a mixing task having a desired intensity with simple and intuitive instructions.

Note that the present invention can also be understood as a cooking process instruction method including at least a part of the above processing, a program for causing an information processing apparatus to execute each step of the method, or a recording medium on which the program is recorded. Each of the above means and processes can be combined with each other as much as possible to constitute the present invention.

For example, a cooking process instruction method as one aspect of the present invention includes:
A cooking process instruction method for instructing a cooking process to a robot that performs a plurality of cooking tasks including at least charging of ingredients, mixing or returning ingredients, and heating of ingredients,
A cooking process input step of displaying icons corresponding to the cooking tasks and a time axis for arranging the icons, and allowing a user to arrange the icons on the time axis by operating a pointing device;
Based on the type and position of the icons arranged on the time axis in the cooking process input step, the order and timing for causing the robot to execute the cooking task corresponding to the arranged icons, and the robot performing the cooking task A control step of calculating an action for executing the control and sending a control command to the robot;
It is characterized by performing.

  According to the present invention, it is possible to instruct a cooking process in a simple manner to a robot that performs a cooking task.

The figure which shows the hardware constitutions of the cooking system of an Example. The figure which shows the function structure of the cooking system of an Example. The figure which shows an example of the recipe book used with the cooking system of an Example. 3 is a flowchart illustrating a processing procedure of the cooking system according to the first embodiment. The figure which shows an example of the timeline editor for inputting the cooking process displayed with the cooking system of an Example. The figure which shows an example of the foodstuff book used with the cooking system of an Example. 10 is a flowchart illustrating a processing procedure of the cooking system according to the second embodiment.

  Exemplary embodiments of the present invention will be described in detail below with reference to the drawings.

(Example 1)
FIG. 1 shows a hardware configuration of a cooking system provided with a cooking process instruction device according to an embodiment of the present invention. This cooking system includes an imaging device 1, robots 21, 22 and 23 for performing cooking tasks, a computer 3 as an information processing device, a heating device 4, a stage 5, and a pan 7 into which materials are charged. The imaging device 1, the robots 21, 22 and 23, and the heating device 4 are each connected to the computer 3 by wire or wirelessly.

The imaging device 1 is a small camera including an imaging device such as a CCD or CMOS, and is installed above the stage 5 so that the stage 5 and the pan 7 are held at an angle of view. In the present embodiment, the imaging device 1 is fixed so that the stage 5 and the optical axis of the imaging device 1 are almost orthogonal to each other. However, the configuration is not limited to this, and the imaging device 1 is set so that the stage 5 is looked down. And the imaging device 1
The angle of view and the direction of the optical axis may be variable.

  The robots 21 to 23 are robots that execute different cooking tasks.

The robot 21 is a robot that transports a dish on which a material is placed and throws the material into the pan 7. The robot 21 is a self-propelled robot including a moving mechanism 211 composed of two wheels and a drive motor, and a hand 213. The hand 213 is provided with a lifting mechanism that is expanded by one degree of freedom in the pitch direction by a servo motor and that can be lifted by catching the edge of the plate. By this lifting mechanism, the dish placed on the stage 5 is carried to the vicinity of the opening of the pan 7, where the material placed on the dish is dropped into the pan 7 by lifting the edge on the side opposite to the opening.

The robot 21 includes a microcontroller (not shown) that controls the moving mechanism 211 and the lifting mechanism of the hand 213 based on a control command from the computer 3. Operations such as “forward”, “reverse”, “right rotation”, “left rotation” of the moving mechanism 211 and “raising” and “lowering” of the lifting mechanism are pre-programmed in the microcontroller and correspond to each operation from the computer 3. The operation of the robot 21 can be controlled by sending a control command to perform.

A visual marker 212 on which a predetermined figure is drawn is attached to the upper part of the robot 21. This system uses the visual marker 212 to track the robot 21 (specifically, the position and orientation are recognized). FIG. 1 illustrates a state in which the robot 21 carries the plate 83 to the vicinity of the opening of the pan 7 and attempts to drop the material placed on the plate 83 into the pan 7.

The robot 22 is a robot that transports a container for storing seasonings and drops the seasonings into the pan 7. The robot 22 is a self-propelled robot including a moving mechanism 221 similar to the robot 21 and a hand 223. The hand 223 is attached with a rotation mechanism that is expanded by one degree of freedom in the roll direction and the yaw direction by a servo motor, can hold the container for containing the seasoning on the yaw axis, and can rotate the seasoning on the roll axis. ing. By this rotating mechanism, the seasoning container placed on the stage 5 is carried to the vicinity of the opening of the pan 7, and the seasoning container is rotated there to drop the seasoning into the pan 7.

The robot 22 includes a microcontroller (not shown) that controls the moving mechanism 221 and the rotating mechanism of the hand 223 based on a control command from the computer 3. The operation of the moving mechanism 221 similar to that of the robot 21 and the operations such as “grabbing”, “without” and “rotating” of the rotating mechanism are pre-programmed in the microcontroller, and control commands corresponding to each operation are sent from the computer 3 By doing so, the operation of the robot 22 can be controlled.

  A visual marker 222 similar to the robot 21 is attached to the robot 22. FIG. 1 illustrates a state in which the robot 22 carries the container of the seasoning 61 to the vicinity of the opening of the pan 7 and drops the seasoning 61 into the pan 7.

The robot 23 is a robot that carries cooking utensils and stirs the pot 7. The robot 23 is a self-propelled robot including a moving mechanism 231 similar to the robot 21 and a hand 233. The hand 233 has a cooking mechanism that is expanded by one degree of freedom in the pitch direction by a servo motor and can stir and return the ingredients in the pan 7 by attaching a cooking tool such as a ladle or frying. ing.

The robot 23 includes a microcontroller (not shown) that controls the moving mechanism 231 and the cooking mechanism of the hand 233 based on a control command from the computer 3. The operation of the moving mechanism 231 similar to that of the robot 21 and the operations such as “mixing” and “returning” of the cooking mechanism are pre-programmed in the microcontroller, and a control command corresponding to each operation is sent from the computer 3 The operation of the robot 23 can be controlled.

A visual marker 232 similar to the robot 21 is attached to the robot 23. FIG. 1 illustrates a state in which the robot 23 is trying to stir the ingredients in the pan 7 using a ladle 234 attached to the hand 233.

The heating device 4 is an IH cooking heater capable of controlling the start and end of heating and the heating intensity according to a control command from the computer 3. Any heating device that can be controlled by a control command from the computer 3 is not limited to an IH cooking heater.

The robots 21 and 22 of the present embodiment correspond to robots that perform a cooking task of inputting ingredients in the present invention. The robot 23 corresponds to a robot that performs a cooking task of mixing or returning ingredients in the present invention. The heating device 4 corresponds to a robot that performs a cooking task of heating materials in the present invention.

In addition, the embodiment of the robot that executes each cooking task is such as the robots 21 to 23 and the heating device 4 described above as long as the cooking task can be executed based on a control command from the computer 3. The form is not limited. For example, the robot that performs the heating task may be a self-propelled robot that enters the lower part of the pan 7 when the heating task is executed, as in the robots 21 to 23, or the robot that performs each cooking task. There is no need to be independent or self-propelled. For example, even a fixed robot having a multi-degree of freedom arm having a plurality of hands each provided with the above-described lifting mechanism, rotating mechanism, and cooking mechanism. good. In the present embodiment, an automatic machine that automatically executes a predetermined cooking task in the absence of a user in accordance with a control command from the computer 3 is included in the category of “robot”.

Further, in the cooking system of the present embodiment, a visual marker is attached to the self-propelled robot, an image of the visual marker is photographed by the imaging device 1, and the position and orientation of each self-propelled robot is determined using the photographed visual marker image. However, the method of recognizing the position and orientation of each robot in a configuration using a self-propelled robot is not limited to this. For example, a sensor for detecting the position and orientation of each robot moving on the stage 5 may be incorporated in the stage 5, and the position and orientation of each robot may be detected using detection data from the sensor. In such a configuration, it is not necessary to attach a visual marker to the robot, and the imaging apparatus 1 for photographing the visual marker is also unnecessary.

The stage 5 is a work space on which a dish on which a material to be put into the pan 7 is placed and a container for storing seasonings are placed, and the robots 21 to 23 perform work. The stage 5 is provided so that the height of the opening of the container 7 placed on the heating device 4 is substantially the same as the height of the surface of the stage 5. As a result, the robot 21 can put the material into the pan 7 by the operation of lifting one end of the plate and dropping the material placed on the plate into the pan 7. Of course, the stage 5 and the heating device 4 may have substantially the same height, and each robot may be provided with a mechanism for lifting a container containing a dish or seasoning to the height of the opening of the pan 7.

As shown in FIG. 1, in the present embodiment, a tag on which a code for identifying the type of material and the type of seasoning is attached is attached to the dish on which the material is placed and the container for storing the seasoning. In the example shown in FIG. 1, tags 91, 92, and 93 are attached to the dishes 81, 82, and 83, respectively, on which the material is placed, and tags 94 and 95 are attached to the containers that contain the seasonings 61 and 62, respectively. . In the present system, a tag is photographed by the imaging device 1, and a material and a seasoning are recognized using an image of the tag.

Note that the method by which the system recognizes the type of material and seasoning is not limited to the above-described method. For example, if the material can be recognized from the image of the food itself photographed by the imaging device 1, it is not necessary to use a tag. Further, if any sensor capable of detecting the material and recognizing the type of material is attached to the dish and the material can be recognized using the detection data from the sensor, it is not necessary to perform imaging with the imaging device 1.

FIG. 2 is a block diagram illustrating a functional configuration of the cooking system according to the present embodiment. The cooking system of the present embodiment includes an imaging device 1, a computer 3, robots 21 to 23, and a heating device 4 as its functions. The computer 3 functions as an image capturing unit 30, a tag recognition unit 31, a timeline editor 32 (corresponding to the cooking process input unit of the present invention), a robot control unit 33, and a heater control unit 34 (control unit of the present invention). Equivalent to). These functions are realized by the CPU of the computer 3 loading and executing a program stored in the storage device. As the computer 3, a general-purpose personal computer including a CPU, a memory, an auxiliary storage device, an input device including a pointing device such as a keyboard and a mouse, and a display device can be used. However, it is also preferable to use a dedicated computer or an on-board microcomputer mounted on a robot. It is also possible to configure a part or all of the program with a dedicated IC.

When the cooking system of this embodiment performs cooking, the user first selects a recipe to be cooked by the system from a recipe book prepared exclusively for the system as shown in FIG. Figure 3
Shows a recipe for pork soup as an example. As shown in Fig. 3, the pork soup recipe includes an image photo and illustration of pork soup 11, cooking time indicating the recommended order and timing of mixing ingredients and seasonings into the pan 7 and mixing in the pan 7 with cooking utensils Line 12, heating timeline 13 showing recommended timing and intensity of heating by heating device 4, photos and illustrations of ingredients and seasonings included in pork soup recipe (Daikon, carrot, burdock, potato 141, konnyaku 142, miso 143, Pork 144, salt 145) and tags 91-95 corresponding to each material and seasoning are included. The tag can be taken out from the recipe book, and the user attaches the tag taken out from the recipe book 10 to a dish or a seasoning container on which a material corresponding to the tag is placed. Then, a plate with a tag and a container for seasoning are arbitrarily arranged on the stage 5. Start up the system with this preparation,
When the recipe is designated and the start of cooking is instructed, the system automatically performs cooking according to the cooking timeline 12 and the heating timeline.

  Hereinafter, the process after starting the cooking system of the present embodiment will be described in detail.

<Processing flow>
FIG. 4 is a flowchart showing the processing flow of the cooking system of the present embodiment. When the system is activated, a timeline editor as shown in FIG. 5 is displayed (step S10). This timeline editor is an interface for allowing the user to input a cooking process.Outline, main screen 100, start button 101, stop button 102, pan confirmation screen 103, cooking timeline editor 120, heating timeline editor 130, It consists of a time bar 104. Input to the timeline editor can be basically performed by operating a pointing device (for example, mouse, trackpad, touch panel) provided in the computer 3.

On the main screen 100, an overhead image of the stage 5 taken by the imaging device 1 is displayed. As will be described later, in this overhead image, an icon corresponding to the material recognized by the system is displayed at the position of the dish on which the material is placed, and an icon corresponding to the seasoning recognized by the system is displayed at the position of the seasoning container. Is displayed. When the start button 101 is clicked, the system starts cooking, and when the stop button 102 is clicked, the system stops cooking. On the pan confirmation screen 103, an enlarged image of the contents of the pan 7 is displayed, and the user can confirm the state in the pan 7 on the timeline editor.

The cooking timeline editor 120 has a time axis on the horizontal axis, the order and timing for causing the robots 21 and 22 to execute the task of putting ingredients and seasonings into the pan 7, and the robot 23 using the cooking utensils in the pan 7. It is a screen which can display and edit the order and timing which perform the cooking task which mixes and returns. The cooking timeline editor 120 displays icons corresponding to ingredients and seasonings input by the material input task and seasoning input task, and icons corresponding to cooking tasks, and the user operates these icons by operating the pointing device. Can be moved to any position in the cooking timeline editor 120, deleted, or added.

The heating timeline editor 130 has a time axis of the same scale as the cooking timeline editor 120 on the horizontal axis, a heating intensity axis on the vertical axis, and the order and timing for performing the heating task by the heating device 4 and the heating intensity. It is a screen which can display and edit. The portion where the graph 105 exists indicates that the task of heating the pan 7 with the intensity corresponding to the height of the graph is executed by the heating device 4, and the portion where the graph 105 does not exist is not heated. Represents that.

The time bar 104 represents the position on the timeline of the cooking currently being performed by the system. When the system starts cooking by clicking the start button 101, the time bar 104 starts to move from the left end to the right, and when the right end is reached, the cooking of this recipe is completed.

  Next, a recipe to be cooked by the cooking system is designated by the computer 3 (step S20). The recipe can be designated by selecting from choices presented by the application of the cooking system or by downloading from a recipe site established on the Internet. In the above description of the preparation stage by the user, the tag is prepared in the recipe book, and the user explained the example in which the tag is taken out from the recipe book and attached to the dish or seasoning container. The tag may be acquired by printing the tag image included in the recipe data.

When the user designates a recipe, the data of the cooking timeline 12 and the heating timeline 13 included in the recipe are read into the timeline editor and displayed on the cooking timeline editor 120 and the heating timeline editor 130 of the timeline editor, respectively. (Step S30). When the pork soup recipe illustrated in Fig. 3 is specified, the cooking timeline editor 120
As shown in Fig. 5, the daikon, carrot, burdock and potato icons 1410, konjac icon 1420, miso icon 1430, pork icon 1440, salt icon 1450, and tama icon 2340 are arranged in the order and timing recommended by the recipe. Displayed. Also, the heating timeline editor 130 displays a graph 105 for heating at the recommended intensity and timing of the recipe, as shown in FIG.

When the initialization of the imaging device 1 is completed, the image capturing unit 30 captures the image of the stage 5 from the imaging device 1 and displays it on the main screen 100. The captured image is stored in the memory and transferred to the tag recognition unit 31 (step S40). Based on the tag read from the image, the tag recognition unit 31 recognizes the types of materials and seasonings, and the positions of the dishes and seasoning containers on which the materials are placed (step S50). The timeline editor 32 acquires the type of material and seasoning and the position of the dish and seasoning container from the tag recognition unit 31, and in the position of the dish and seasoning container in the image of the stage 5 displayed on the main screen 100, Icons corresponding to the ingredients and seasonings placed on the plate are displayed (step S60). As icons displayed on the main screen 100, the same icons as those displayed on the cooking timeline editor 120 are used. When the materials and seasonings recognized by the tag recognition unit 31 are insufficient for the necessary materials and seasonings specified by the recipe, the system may display a warning.

The user edits the order of ingredients and seasonings, the timing, and the amount of seasonings in the cooking timeline editor 120 as necessary, or edits the heating intensity and heating time in the heating timeline editor 130 ( Step S70). The amount of seasoning may be changed by, for example, turning the mouse wheel while pressing the salt icon 1450 displayed in the cooking timeline editor 120, or pressing + or-on the keyboard. . At this time, change the size of the icon according to the amount of seasoning, or overlay an icon with a small “+” or “-” on the icon to change the recommended seasoning amount of the recipe It may be displayed.

With the above processing, an instruction to be input by the user to the system to cause the robots 21 to 23 and the heating device 4 to cook the recipe is completed. When the user clicks the start button 101, the icon arrangement of the cooking timeline editor 120 and the graph shape of the heating timeline editor 130 are passed from the timeline editor 32 to the robot control unit 33 and the heater control unit 34. Based on the icon arrangement in the cooking timeline editor 120, the robot control unit 33 is configured to move the robots 21 and 22 to a dish or seasoning position where each material is placed, or to open the robot 23 in the pan 7 The route for moving to the vicinity of the part is calculated. Also,
The robot 21, 22 and 23 grab the dish and seasoning container, transport the dish and seasoning container to the vicinity of the opening of the pan 7, put the ingredients and seasoning into the pan 7, and mix in the pan 7 The operation and timing of the moving mechanism, the lifting mechanism, and the rotating mechanism are calculated to execute the above at an appropriate timing. Further, based on the graph in the heating timeline editor 130, the heater control unit 34 calculates a power control command value for causing the heating device 4 to heat the pan 7 at an appropriate timing and intensity (step S80). And based on a calculation result, a control command is sent to robots 21-23 and heating device 4 (Step S90). As a result, the ingredients and seasonings are charged and heated in the order and timing displayed on the cooking timeline editor 120 and the heating timeline editor 130, and the pork soup is automatically completed.

According to the cooking system of the present embodiment, even if the user does not know the cooking method, the user can simply place the ingredients and seasonings specified in the recipe at arbitrary positions on the stage and attach appropriate tags. It is possible to complete the cooking by properly performing complicated cooking tasks in ~ 23 and the heating device 4. In addition, the user can change the position on the time axis of icons corresponding to ingredients and seasonings in the cooking timeline editor 120, or change the shape of the graph in the heating timeline editor 130. You can also add your own arrangements for a given recipe simply by performing an operation. The computer 3 automatically performs the process of reflecting these changes made in the timeline editor to the actual actions (movement mechanisms, lifting mechanisms, rotation mechanisms, etc.) to be performed by the robots 21 to 23 and the heating device 4. Therefore, the user need not learn the command system of the microcontroller mounted on each robot in order to change the operations of the robots 21 to 23 and the heating device 4.

(Example 2)
In the cooking system of Example 1, it is possible to add arrangements to existing recipes that can be obtained from recipe books or the Internet with intuitive and simple operations, and it is easy to let the cooking system cook with the recipes after arrangement. According to the present invention, the cooking system can be configured so that the user can create an original recipe and easily cook according to the original recipe. You can also.

Such a cooking system uses an ingredient book in which main ingredients such as meat, seafood, vegetables, grains, seasonings, and tags corresponding to each ingredient are listed instead of the recipe book in the first embodiment. FIG. 6 shows an example of a food book. FIG. 6 shows an example of a vegetable page. This page includes photos and illustrations of ingredients (daikon 151, carrot 152, onion 153, lotus root 154) and tags with the codes corresponding to the ingredients (daikon tag 161, carrot tag 162, onion tag 163, lotus tag 164). As with the recipe book of the first embodiment, the food material data and tag image data may be acquired from a cooking system application, the Internet, or the like. In addition, a plurality of tags that are not assigned to the ingredients are prepared in the ingredient book so that tags can be assigned to the ingredients that do not exist in the ingredient book. The user captures one of the unassigned tags with the imaging device 1 and causes the system to recognize the data, and associates data such as a food name, a photograph, and an illustration assigned to the tag with the tag and stores them in the system.

The user prepares ingredients for the original recipe created by the user, places them on a plate, and places them on the stage 5. At this time, the ingredients to be put into the pan 7 at the same time are put on the same plate. However, this is due to the fact that there is only one robot 21 that performs the ingredient charging task.If there are multiple robots that execute the ingredient charging task, the ingredients to be put into the pan 7 at the same time are placed in multiple dishes. It is also possible to put them separately. The same applies to seasonings.

The user attaches tags corresponding to all ingredients and seasonings placed on the plate to each plate and seasoning container. When attaching multiple tags to one dish, attach the tags so that they do not overlap. The tag is an example of means for causing the system to recognize the type of food. As described in the first embodiment, the recognition method can be variously modified, such as a method using a sensor.

Further, the user attaches cooking utensils necessary for cooking operations such as mixing and returning executed in the original recipe to the hand 233 of the robot 23. If multiple cooking utensils are required, multiple robots that can be fitted with cooking utensils may be prepared and placed on stage 5, or you may not grab cooking utensils placed on stage 5 It is also possible to use a robot that has a mechanism that can change the cooking utensils as needed. When a plurality of cooking utensils are used, a tag may be attached to the cooking utensils in order to identify each cooking utensil. Of course, the cooking utensil may be recognized directly from the image of the cooking utensil photographed by the imaging device 1, or the cooking utensil may be recognized based on the sensor output by attaching a sensor to the cooking utensil.

If the system is started with such preparations made on stage 5, and the original recipe is input in the cooking timeline editor 120 and the heating timeline editor 130 to indicate the start of cooking, the system will start cooking the original recipe. Is automatically created.

  Hereinafter, the process after starting the cooking system of the present embodiment will be described in detail.

<Processing flow>
FIG. 7 is a flowchart showing the processing flow of the cooking system of the present embodiment. When the system is activated, the timeline editor shown in FIG. 5 is displayed (step S11).

When the initialization of the imaging device 1 is completed, the image capturing unit 30 captures the image of the stage 5 from the imaging device 1 and displays it on the main screen 100 of the timeline editor. The captured image is stored in the memory and transferred to the tag recognition unit 31 (step S21). The tag recognition unit 31 recognizes the types of ingredients, seasonings and cooking utensils placed on the stage 5 and their positions based on the tags read from the images (step S31). The timeline editor 32 includes a tag recognition unit 31.
The types of ingredients, seasonings, and cooking utensils and their positions are obtained from the image of stage 5 displayed on the main screen 100, the seasoning containers and the positions of the cooking utensils, and the ingredients, seasonings, and cooking utensils. A corresponding icon is displayed (step S41).

The user places the ingredients, seasonings, and cooking utensil icons displayed on the main screen 100 in the cooking timeline editor 120 by drag-and-drop operation, thereby placing the ingredients and seasonings into the pan 7 in the original recipe. In addition, the timing and the order and timing of cooking operations performed on the pot 7 by the cooking tool are expressed on the timeline editor (step S51). Further, a graph is drawn in the heating timeline editor 130, and the intensity and timing at which the pan 7 is heated in the original recipe is expressed on the timeline editor (step S61).

With the above processing, an instruction to be input by the user to the system to cause the robots 21 to 23 and the heating device 4 to cook the original recipe is completed. User starts button 101
When is clicked, the icon arrangement of the cooking timeline editor 120 and the graph shape of the heating timeline editor 130 are passed from the timeline editor 32 to the robot controller 33 and the heater controller 34. Based on the icon arrangement in the cooking timeline editor 120, the robot control unit 33 is configured to move the robots 21 and 22 to dishes or seasonings on which the ingredients are placed, or to open the robot 23 in the pan 7 The route for moving to the vicinity of the part is calculated. Also, grab the dish and seasoning container on the robots 21, 22 and 23, transport the dish and seasoning container to the vicinity of the opening of the pan 7, put the ingredients and seasoning into the pan 7, The operation and timing of the moving mechanism, the lifting mechanism, and the rotating mechanism for executing the mixing operation at an appropriate timing are calculated. Further, based on the graph in the heating timeline editor 130, the heater control unit 34 calculates a power control command value for causing the heating device 4 to heat the pan 7 at an appropriate timing and intensity (step S71). And based on a calculation result, a control command is sent to robots 21-23 and heating device 4 (Step S81). As a result, the ingredients and seasonings are charged and heated in the order and timing displayed on the cooking timeline editor 120 and the heating timeline editor 130, and the original recipe is automatically completed.

According to the cooking system of the present embodiment, the user can perform original and simple recipe operations by dragging and dropping the ingredients icon and the cooking utensil icon displayed on the main screen 100 to the cooking timeline editor 120. It is possible to cause the robots 21 to 23 and the heating device 4 to execute cooking tasks necessary for making the cooking at an appropriate timing. That is, according to the cooking system of the present embodiment, the user can easily perform a complicated cooking process including the execution order and execution timing of cooking tasks without learning the operations and command systems that can be executed by each robot. It becomes possible to give instructions to each robot.

Each of the above embodiments is merely an example of the present invention. The scope of the present invention is not limited to the above embodiment, and various modifications can be made within the scope of the technical idea. For example, although the case where one pan 7 is used as a container in which cooking is performed is illustrated in the above embodiment, a frying pan or the like may be used in addition to the pan, or a plurality of cooking containers may be used. Moreover, you may use the cooking container which incorporated the heating apparatus like a hotplate. When using a plurality of cooking containers, a plurality of heating devices may be used so that heating can be controlled independently for each cooking container. If it does so, it will also become possible to make not only pork soup but also a dish which requires a more complicated cooking process, and a plurality of dishes can be made simultaneously. In particular, when a person cooks, performing a plurality of dishes at the same time so that they can be put on the table at almost the same time is an advanced technique requiring skill. When the cooking system of the above-described embodiment is extended to allow the simultaneous progress of a plurality of dishes, the cooking timeline editor 120 of the timeline editor is simply used.
In addition, the number of heating timeline editors 130 may be increased by the number of dishes. The user repeats the work described in the above-described embodiment for dragging and dropping the ingredients icon and the cooking utensil icon displayed on the main screen to the cooking and heating timeline editor corresponding to each dish for the number of dishes. Just good. At that time, multiple dishes can be prepared just by performing simple and intuitive operations such as adjusting the icon arrangement and the shape of the heating graph in each cooking and heating timeline editor so that the completion timing of each dish is almost the same. It is possible to cause each robot to execute a sophisticated and complicated cooking task that is completed at almost the same timing.

1: Imaging device
21: Material input robot
211: Movement mechanism
212: Visual marker
213: Hand
22: Seasoning robot
221: Movement mechanism
222: Visual marker
223: Hand
23: Cooking robot
231: Movement mechanism
232: Visual marker
233: Hand
234: Cookware
3: Computer
30: Image capture unit
31: Tag recognition unit
32: Timeline editor
33: Robot controller
34: Heater controller
4: IH heater
5: Stage
61: Seasoning
62: Seasoning
7: Hot pot
81: dish
82: dish
83: dish
91: Tag
92: Tag
93: Tag
94: Tag
95: Tag
10: Recipe book
11: Cooking illustration
12: Cooking timeline
13: Heating timeline
141: Daikon, carrot, burdock, potato illustration
142: Konjac Illustration
143: Miso illustration
144: Pork illustration
145: Salt illustration
100: Main screen
101: Start button
102: Stop button
103: Pan confirmation screen
104: Time bar
120: Cooking timeline editor
130: Heating timeline editor
1410: Daikon, carrot, burdock, potato icons
1420: Konjac icon
1430: Miso icon
1440: Pork icon
1450: Salt icon
2340: Cookware icon
151: Daikon Illustration
152: Carrot illustration
153: Onion illustration
154: Lotus root illustration
161: Daikon tag
162: Carrot tag
163: Onion tag
164: Lotus tag

Claims (2)

A cooking process instruction device for instructing a cooking process to a robot that performs a plurality of cooking tasks including at least charging of ingredients, mixing or returning of ingredients, and heating of ingredients,
A cooking process input unit that displays icons corresponding to the cooking tasks and a time axis for arranging the icons, and allows a user to arrange the icons on the time axis by operating a pointing device;
Based on the types and positions of the icons arranged on the time axis in the cooking process input unit, the order and timing for causing the robot to execute cooking tasks corresponding to the arranged icons, and the cooking task to the robot A control unit that calculates an action for executing the control and sends a control command to the robot;
A cooking process instruction apparatus comprising: A cooking process instruction method for instructing a cooking process to a robot that performs a plurality of cooking tasks including at least charging of ingredients, mixing or returning ingredients, and heating of ingredients,
A cooking process input step of displaying icons corresponding to the cooking tasks and a time axis for arranging the icons, and allowing a user to arrange the icons on the time axis by operating a pointing device;
Based on the type and position of the icons arranged on the time axis in the cooking process input step, the order and timing for causing the robot to execute the cooking task corresponding to the arranged icons, and the robot performing the cooking task A control step of calculating an action for executing the control and sending a control command to the robot;
A cooking process instruction method comprising:

Images