JP7320370B2 - Server, system, method and program - Google Patents

Description

The present disclosure relates to terminal devices, systems, servers, and control methods thereof .

In recent years, the IoT of home appliances has progressed, and network-compatible devices such as smart home appliances are appearing. In order to save labor in cooking using such equipment, it is being studied to implement machine-readable recipe data and to control the equipment based on such recipe data.

In Non-Patent Document 1, in research on natural language processing that allows machines to process natural language, recipe sentences are classified into words representing cooking equipment and actions (operations related to cooking), and the relationship between words is estimated. A method of creating a graph (recipe data) with words as nodes and relationships between words as edges is described.

HIROKUNI MAETA, TETSURO SASADA, SHINSUKE MORI, "A FrameWork for Procedual Text Understanding", [ONLINE], IWPT, 2015 [Heisei Search for March 20, 30, Internet <URL: HTTP: // www. ar. media. kyoto-u. ac. jp/mori/research/public/maeta-IWPT15. pdf>

However, the recipe data described in Non-Patent Document 1 expresses the description of the recipe text, which is a natural language, as a graph as it is. Low and there is a risk that the recipe data will not be properly deciphered by the machine.

In addition, the recipe data described in Non-Patent Document 1 mainly describes actions to be taken by humans, and cooking appliances to be used may be specified. It is difficult to support various devices including new devices that will appear in the future.

SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to provide a terminal device, a system, a server, and a control method thereof that enable control of various devices using highly machine-readable recipe data.

In order to achieve the above object, a terminal device according to one aspect of the present disclosure provides a recipe from a user. A first receiving means for receiving an operation to search, and an operation for the first receiving means to search for recipes Accepts the operation to select a recipe from the list of recipes displayed in response to the acceptance of and an action that is an operation related to cooking necessary for the state transition of ingredients can be executed. an instruction means for instructing a capable cooking appliance to start an action; After a recipe is selected based on the operation received by the means, information about the selected recipe is and information about the cooking appliance to which the instruction is given by the instruction means. and according to the instruction by the instruction means, to the recipe transmitted by the transmission means an action based on the information about the cooking appliance and the information about the cooking appliance. It is characterized by being executed.

In order to achieve the above object, the terminal device according to one aspect of the present disclosure provides the second receiving means with Therefore, in response to the recipe being selected, the selected recipe is displayed in a user-readable state. It is characterized by further comprising control means for controlling to display.

In order to achieve the above object, the terminal device according to one aspect of the present disclosure provides the second receiving means with Therefore, in response to the recipe being selected, the selected recipe is displayed in a user-readable state. It is characterized by further comprising control means for controlling to display.

In order to achieve the above object, the terminal device according to one aspect of the present disclosure provides the second receiving means with Therefore, a button for controlling the cooking appliance is displayed according to the selection of the recipe. It is characterized by controlling to do.

In order to achieve the above object, a terminal device according to one aspect of the present disclosure includes: Recipes that are searched according to the received operation do not necessarily have to be executed in the recipe. The feature is that the cooking appliance that should perform the action that needs to be performed is not specified.

ADVANTAGE OF THE INVENTION According to this invention, the terminal device, system, server, and its control method which can control various apparatuses using highly machine-readable recipe data can be provided.

FIG. 4 is a diagram showing the data structure of recipes according to the first to third embodiments; 1 is a diagram showing the configuration of a device control system according to first to third embodiments; FIG. 3 is a diagram showing the hardware configuration of each device and each device included in the device control system according to the first to third embodiments; FIG. It is a figure which shows the functional block structure of the terminal device which concerns on 1st Embodiment. 3 is a diagram showing a functional block configuration of a device control server according to the first embodiment; FIG. It is a figure which shows the functional block structure of the apparatus which concerns on 1st Embodiment. It is a figure which shows the operation example of the apparatus control system which concerns on 1st Embodiment. It is a figure which shows the operation example of the apparatus control system based on the example of a change of 1st Embodiment. It is a figure which shows the functional block structure of the terminal device which concerns on 2nd Embodiment. It is a figure which shows the operation example of the apparatus control system which concerns on 2nd Embodiment. FIG. 11 is a diagram showing a functional block configuration of a device according to a third embodiment; FIG. It is a figure which shows the operation example of the apparatus control system which concerns on 3rd Embodiment. FIG. 11 is a diagram showing an operation example of a device control system according to a modification of the third embodiment;

A device control system according to an embodiment will be described with reference to the drawings. In the following description of the drawings, the same or similar parts are given the same or similar reference numerals.

<Overview of Embodiment>
A device control system according to an embodiment is a system for controlling devices. Devices to be controlled are cooking ingredients or devices related to cooking using the same. A device control system according to an embodiment acquires recipe data and decodes the acquired recipe data to perform processing for controlling devices. When a device other than the device (for example, a server device or a terminal device) decodes the recipe data, the process for controlling the device generates and transmits a command for controlling the device (hereinafter referred to as "control command"). may include processing to When the device itself decodes the recipe data, the processing for controlling the device may include performing actions according to the recipe data.

FIG. 1 is a diagram showing the data structure of a recipe according to the embodiment. As shown in FIG. 1, recipe data according to the embodiment is represented by a graph (directed graph) consisting of a plurality of nodes and edges between the nodes. In FIG. 1, nodes are indicated by circles and edges are indicated by arrows.

Multiple nodes define state transitions for each material. Edges represent actions required for state transitions between nodes. An action refers to a basic procedure constituting cooking, and for example, a procedure such as "cut" or "warm" corresponds to the action. Action types and their IDs are defined in advance, or IDs are given to action types ex post facto, a list of actions and their IDs is defined, and an action ID selected from this list is given to an edge. be. An ID may also be assigned to a state (intermediate node).

A plurality of nodes in the graph are a starting point of the graph and an ingredient node representing the ingredients of the dish, an end point of the graph and a cooking node representing the dish, and a state of the ingredients on the way to cooking. and intermediate nodes that represent Ingredients refer to materials for cooking, but may also include seasonings and materials that are discarded during the cooking process. Types of materials and their IDs are defined in advance, a list of materials and their IDs is defined, and material IDs selected from this list are assigned to material nodes. The state of the material refers to the state after the action is applied, for example, "material A cut into 3 equal parts", "material B cut into 3 cm squares", "warmed material C", etc. Corresponds to the state of the material.

In this way, the data structure of the recipe according to the embodiment expresses the state transition of each ingredient as a graph instead of expressing the description of the recipe text in natural language as a graph as it is. Each material is given a material ID that uniquely identifies the material, and each action required for state transition is given an action ID that uniquely identifies the action. Therefore, the machine can easily and reliably decode the recipe data based on the ID. In addition, since it is not necessary to specify in the recipe data which device should perform the action, it is easy to deal with various devices including new devices that will appear in the future. Furthermore, since each action is expressed as an edge, it is easy to extract each action from the recipe data.

Table 1 shows types of nodes in the recipe data according to the embodiment.

Each node can be given the attributes shown in Table 2, for example, as node attributes. Assume here that the node is a material node and that this material node represents "carrot".

Note that the "name" is information necessary for humans to create and decode the MRR, and does not necessarily have to be machine-readable.

Attributes shown in Table 3 can be assigned to each edge as edge attributes.

A specific example of each node will now be described. When creating a dish "salad", the dish node is "salad". Also, material nodes are, for example, "onion", "cucumber", "tomato", "ketchup", and "mayonnaise". Each material node is given an ID that uniquely identifies the material.

The material node "onion" is connected with the intermediate node "chopped onion" by the edge (action) "cut". Also, the material node "cucumber" is connected to the intermediate node "1 cm square cucumber" by the edge (action) "cut". Further, the material node "tomato" is connected to the intermediate node "1/2 tomato" by the edge (action) "cut". Furthermore, the material node "ketchup" and the material node "mayonnaise" are connected to the intermediate node "aurora sauce" by the edge (action) "mix". An ID that uniquely identifies the action is assigned to each of "cut" and "mix".

Then, the intermediate nodes 'chopped onion', '1 cm square cucumber' and '1/2 tomato' are connected to the cooking node 'salad' by the edge (action) 'Serve'. In addition, the intermediate node "aurora sauce" is connected to the dish node "salad" by the edge (action) "kake". An ID that uniquely identifies the action is assigned to each of "Serve" and "Sprinkle". Also, the order between the edges is set so that "put on" is performed after "put on". It should be noted that an action of "apply" may be used instead of "apply".

<First embodiment>
A first embodiment of a device control system using the recipe data structure described above will be described.

(System configuration according to the first embodiment)
A configuration of the device control system 1 according to the first embodiment will be described. FIG. 2 is a diagram showing the configuration of the device control system 1 according to the first embodiment.

As shown in FIG. 2 , the device control system 1 includes a device 100 , a terminal device 200 , a recipe distribution server 300 and a device control server 400 . The device 100 , the terminal device 200 , the recipe distribution server 300 and the device control server 400 communicate with each other via the communication network 10 . However, the device 100 may communicate directly with the terminal device 200 (for example, short-range wireless communication) without going through the communication network 10 .

The device 100 has a communication function and is related to cooking ingredients or cooking using the same. The device 100 may be a household electrical appliance used at home, or may be a commercial device. The device 100 may be any device as long as it is a device related to ingredients or cooking using the same. These include fryers, electric steamers, noodle machines, scales, and cooking robots.

The terminal device 200 is a device that has a communication function and provides a user interface. The terminal device 200 acquires and displays recipe data (recipe sentences, image data of dishes, etc.) from the recipe distribution server 300 according to user's operation. In addition, the terminal device 200 makes a request regarding the control of the device 100 to the device control server 400 according to the user's operation on the display recipe. The terminal device 200 is, for example, a mobile terminal (eg, tablet, smart phone, laptop, feature phone, portable game machine, electronic book reader, etc.). Alternatively, the terminal device 200 may be a television receiver (including Internet television), a PC (Personal Computer), a VR (Virtual Reality) terminal, an AR (Augmented Reality) terminal, or the like.

The recipe distribution server 300 accumulates display recipe data. In response to a request from the terminal device 200, the recipe distribution server 300 distributes to the terminal device 200 recipe data that matches the request from the stored recipe data. The terminal device 200 displays recipe data acquired from the recipe distribution server 300 . The recipe data distributed by the recipe distribution server 300 is mainly composed of human-readable recipe sentences (natural language), and is not machine-readable. Such recipe data is hereinafter referred to as HRR (Human Readable Recipe).

The device control server 400 accumulates device control recipe data corresponding to the recipe data accumulated by the recipe distribution server 300 . The device control recipe data has the above-described recipe data structure (see FIG. 1 and Tables 1 to 3) and is machine-readable. Such recipe data is hereinafter referred to as MRR (Machine Readable Recipe). The device control server 400 generates a control command based on the MRR in response to a request from the terminal device 200 and transmits the generated control command to the terminal device 200 . Specifically, the request from the terminal device 200 includes a recipe ID. The device control server 400 acquires the MRR corresponding to the recipe ID, decodes the acquired MRR, and transmits a control command based on the MRR to the terminal device 200 . The terminal device 200 transfers the control command received from the device control server 400 to the device 100 .

After transferring the control command to the device 100, the user of the terminal device 200 performs an action start operation on the device 100 for causing the device 100 to start an action according to the control command. For example, the device 100 is provided with a physical button for accepting an action start operation. Alternatively, the device 100 may display a software button for accepting an action start operation. Upon receiving the action start operation, the device 100 starts an action according to the control command.

(Hardware configuration)
A hardware configuration of each device and each device included in the device control system 1 will be described. FIG. 3 is a block diagram showing a hardware configuration example of each of the device 100, the terminal device 200, the recipe distribution server 300, and the device control server 400. As shown in FIG.

As shown in FIG. 3, each of the device 100, the terminal device 200, the recipe distribution server 300, and the device control server 400 includes a CPU 21, a RAM 22, a ROM 23, an auxiliary storage device 24, a communication module 25, and an input device. 26 and an output device 27 .

The CPU 21 reads and executes software (programs) from the RAM 22 and ROM 23 that constitute the main storage device. The RAM 22 is used as a work area for the CPU 21 . The auxiliary storage device 24 is composed of a hard disk, flash memory, or the like. The communication module 25 is a module that transmits and receives data by wired communication or wireless communication. The input device 26 is composed of a touch panel and/or a keyboard, etc., and receives user operations. The input device 26 may accept operations by voice input. The output device 27 is configured by a display or the like, and outputs (displays) various kinds of information.

By loading software onto hardware such as the CPU 21 and the RAM 22, the communication module 25, the input device 26, and the output device 27 are operated under the control of the CPU 21, and data is read from the RAM 22 and the auxiliary storage device 24. A series of functions in each device and each device is realized by writing.

Device 100 further includes various mechanisms 28 . Various mechanisms 28 are mechanisms for executing actions in the device 100, and are configured by, for example, a heating mechanism, a refrigerating/freezing mechanism, a cutting mechanism, a stirring mechanism, and the like.

(Functional block configuration of terminal device according to first embodiment)
A functional block configuration of the terminal device 200 according to the first embodiment will be described. FIG. 4 is a diagram showing the functional block configuration of the terminal device 200 according to the first embodiment.

As shown in FIG. 4 , terminal device 200 includes control unit 210 , communication unit 220 , storage unit 230 , operation unit 240 , and display unit 250 .

The control unit 210 includes a CPU 21 and controls operations of the terminal device 200 . The communication unit 220 includes the communication module 25 and performs communication via the communication network 10 . The communication unit 220 may have a function of directly communicating with the device 100 . The storage unit 230 includes a RAM 22, a ROM 23, and an auxiliary storage device 24, and stores various information and data. The operation unit 240 includes the input device 26 and receives user operations. The display unit 250 includes the output device 27 (display) and displays various information and data.

Control unit 210 includes HRR acquisition unit 211 , control request unit 212 , and command transfer unit 213 .

The HRR acquisition unit 211 acquires the HRR from the recipe distribution server 300 via the communication unit 220 according to the user's operation on the operation unit 240 . The HRR acquisition unit 211 may acquire the HRR of the recipe selected by the user from the recipe list displayed on the display unit 250, or may perform search processing on the recipe distribution server 30 using search conditions (keywords, etc.). HRRs that meet the conditions may be acquired by performing the above. The HRR acquisition unit 211 displays the acquired HRR on the display unit 250 . Along with the HRR, a button for transmitting a control command to the device 100 (hereinafter, “device control button”) may be displayed.

The control request unit 212 generates a control request requesting control of the device 100 in response to the user's operation of the device control button, and transmits the generated control request to the device control server 400 via the communication unit 220. do. The control request includes a recipe ID corresponding to the selected recipe (HRR). The control request may further include the device ID of the device 100 to be controlled.

Command transfer section 213 receives control commands transmitted from device control server 400 via communication section 220 . Command transfer section 213 transfers the received control command to device 100 via communication section 220 .

(Functional block configuration of device control server according to first embodiment)
A functional block configuration of the device control server 400 according to the first embodiment will be described. FIG. 5 is a diagram showing the functional block configuration of the device control server 400 according to the first embodiment.

As shown in FIG. 5 , the device control server 400 includes a control section 410 , a communication section 420 and a storage section 430 .

The control unit 410 includes the CPU 21 and controls the operation of the device control server 400 . The communication unit 420 includes the communication module 25 and performs communication via the communication network 10 . The storage unit 430 includes a RAM 22, a ROM 23, and an auxiliary storage device 24, and stores various information and data. The storage unit 430 also stores the MRR in association with the recipe ID.

The control unit 410 includes a request reception unit 411 , an MRR acquisition unit 412 and a processing unit 413 .

The request reception unit 411 receives control requests from the terminal device 200 via the communication unit 420 . The control request includes a recipe ID corresponding to the selected recipe. The control request may further include the device ID of the device 100 to be controlled.

The MRR acquisition unit 412 acquires from the storage unit 430 the MRR corresponding to the recipe ID included in the control request.

The processing unit 413 decodes the MRR acquired by the MRR acquisition unit 412 and performs processing for controlling the device 100 corresponding to the device ID included in the control request. The processing unit 413 includes an MRR decoding unit 413a, a command generation unit 413b, and a command transmission unit 413c.

The MRR decryption unit 413 a decrypts the MRRs acquired by the MRR acquisition unit 412 .

The command generation unit 413b extracts from the MRR an action (action ID) that can be executed by the device 100 corresponding to the device ID included in the control request, based on the result of MRR decoding by the MRR decoding unit 413a. Generate control commands that configure execution. For example, a list of executable actions (action IDs) is defined for each device ID. Such a list includes an ID representing "cut", an ID representing "baking", an ID representing "boil", and the like. The command generation unit 413b extracts from the MRR an action (action ID) that can be executed by the device 100 corresponding to the device ID included in the control request by referring to the list. Specifically, the list of action IDs executable by the device is compared with the action ID of each edge in the MRR, the matching action ID is extracted, and the action corresponding to the extracted ID is assigned to the device. can be judged.

The command transmission unit 413 c transmits the command generated by the command generation unit 413 b to the terminal device 200 via the communication unit 420 .

(Functional block configuration of the device according to the first embodiment)
A functional block configuration of the device 100 according to the first embodiment will be described. FIG. 6 is a diagram showing the functional block configuration of the device 100 according to the first embodiment.

As shown in FIG. 6 , the device 100 includes a control section 110 , a communication section 120 , a storage section 130 and an operation section 140 .

The control unit 110 includes a CPU 21 and controls operations of the device 100 . The communication unit 120 includes the communication module 25 and performs communication via the communication network 10 . The communication unit 120 may have a function of directly communicating with the terminal device 200 . The storage unit 130 includes a RAM 22, a ROM 23, and an auxiliary storage device 24, and stores various information and data. The operation unit 140 includes the input device 26 and receives user operations. The operation unit 140 has a physical button or software button for accepting an action start operation.

The control unit 110 has a command acquisition unit 111 and an action control unit 112 .

The command acquisition unit 111 acquires control commands transmitted from the terminal device 200 via the communication unit 120 .

After the control command is acquired by the command acquisition unit 111, the action control unit 112 controls to start an action corresponding to the control command in response to the operation of the operation unit 140 (action start button) by the user. do.

(Example of operation according to the first embodiment)
An operation example of the device control system 1 according to the first embodiment will be described. FIG. 7 is a diagram showing an operation example of the device control system 1 according to the first embodiment.

As shown in FIG. 7, in step S101, the terminal device 200 receives a recipe search operation from the user.

In step S<b>102 , the terminal device 200 transmits to the recipe distribution server 300 an HRR request requesting provision of a recipe (HRR).

In step S<b>103 , the recipe distribution server 300 reads the HRR corresponding to the HRR request from the terminal device 200 .

In step S<b>104 , the recipe distribution server 300 transmits the read HRR to the terminal device 200 . A recipe ID is associated with HRR.

In step S105, the terminal device 200 receives device control operations from the user.

In step S<b>106 , the terminal device 200 transmits a control request requesting control of the device 100 to the device control server 400 . A control request is associated with a recipe ID and a device ID.

In step S<b>107 , the device control server 400 reads (obtains) the MRR corresponding to the control request from the terminal device 200 .

In step S108, the device control server 400 decodes the read MRR.

In step S109, the device control server 400 extracts an action (action ID) executable by the device 100 from the MRR based on the decoded result of the MRR, and generates a control command for setting execution of the extracted action.

In step S<b>110 , the device control server 400 transmits the generated control command to the terminal device 200 .

In step S<b>111 , the terminal device 200 receives the control command from the device control server 400 and transfers the received control command to the device 100 .

In step S112, the device 100 receives an action start operation from the user.

In step S<b>113 , the device 100 executes an action corresponding to the control command acquired from the terminal device 200 .

<Modified example of the first embodiment>
In the first embodiment described above, an example in which the terminal device 200 transfers the control command from the device control server 400 to the device 100 has been described. However, the device control server 400 may transmit the control command to the device 100 without going through the terminal device 200 . In such a case, the destination information (address information) of the device 100 may be registered in the device control server 400 in advance, or the destination information (address information) of the device 100 may be sent to the device control server 400 by a control request from the terminal device 200. may be notified.

FIG. 8 illustrates an operation example of the device control system 1 according to this modified example. FIG. 7 is a diagram showing an operation example of the device control system 1 according to this modification.

As shown in FIG. 8, each process of steps S201 to S209 is the same as each process of steps S201 to S209 in FIG.

In step S<b>210 , the device control server 400 transmits the generated control command to the device 100 . The device 100 receives control commands from the device control server 400 .

In step S211, the device 100 receives an action start operation from the user.

In step S<b>212 , the device 100 executes an action corresponding to the control command acquired from the device control server 400 .

<Second embodiment>
Regarding the second embodiment, the differences from the first embodiment will be mainly described.

In the first embodiment described above, an example in which the device control server 400 decodes the MRR has been described. In contrast, in the second embodiment, the device control server 400 transmits (distributes) the read MRR to the terminal device 200 without decrypting the read MRR. The terminal device 200 decodes the MRR acquired from the device control server 400 .

(Functional block configuration of terminal device according to second embodiment)
A functional block configuration of the terminal device 200 according to the second embodiment will be described. FIG. 9 is a diagram showing the functional block configuration of the terminal device 200 according to the second embodiment.

As shown in FIG. 9, the terminal device 200 is common to the first embodiment in that it includes a control unit 210, a communication unit 220, a storage unit 230, an operation unit 240, and a display unit 250. FIG.

A control unit 210 according to the second embodiment includes an HRR acquisition unit 211 , a control request unit 212 , an MRR acquisition unit 214 and a processing unit 215 . The HRR acquisition unit 211 and the control request unit 212 are the same as in the first embodiment.

The MRR acquisition unit 214 acquires MRRs from the device control server 400 .

The processing unit 215 decodes the MRR acquired by the MRR acquisition unit 214 and performs processing for controlling the device 100 to be controlled. The processing unit 215 includes an MRR decoding unit 215a, a command generation unit 215b, and a command transmission unit 215c.

The MRR decryption unit 215a decrypts the MRR acquired by the MRR acquisition unit 214. FIG.

The command generation unit 215b extracts an action (action ID) that can be executed by the device 100 to be controlled from the MRR based on the MRR decoding result by the MRR decoding unit 215a, and generates a control command for setting execution of the extracted action. Generate. For example, a list of executable actions (action IDs) is defined for each device ID. ) from the MRR.

The command transmission unit 215c transmits the command generated by the command generation unit 215b to the device 100 via the communication unit 220. FIG.

(Example of operation according to the second embodiment)
An operation example of the device control system 1 according to the second embodiment will be described. FIG. 10 is a diagram showing an operation example of the device control system 1 according to the second embodiment.

As shown in FIG. 10, in step S301, the terminal device 200 receives a recipe search operation from the user.

In step S<b>302 , the terminal device 200 transmits to the recipe distribution server 300 an HRR request requesting provision of a recipe (HRR).

In step S<b>303 , the recipe distribution server 300 reads the HRR corresponding to the HRR request from the terminal device 200 .

In step S<b>304 , the recipe distribution server 300 transmits the read HRR to the terminal device 200 . A recipe ID is associated with HRR.

In step S305, the terminal device 200 receives a device control operation from the user.

In step S<b>306 , the terminal device 200 transmits a control request requesting control of the device 100 to the device control server 400 . A recipe ID is associated with the control request.

In step S<b>307 , the device control server 400 reads the MRR corresponding to the control request from the terminal device 200 .

In step S<b>308 , the device control server 400 transmits (distributes) the read MRR to the terminal device 200 .

In step S<b>309 , the terminal device 200 decodes the MRR acquired from the device control server 400 .

In step S310, the terminal device 200 extracts an action (action ID) executable by the device 100 from the MRR based on the decoded result of the MRR, and generates a control command for setting execution of the extracted action.

In step S<b>311 , the terminal device 200 transmits the generated control command to the device 100 .

In step S312, the device 100 receives an action start operation from the user.

In step S<b>313 , the device 100 executes an action corresponding to the control command acquired from the terminal device 200 .

<Third Embodiment>
The third embodiment will be described mainly with respect to the differences from the first embodiment.

In the first embodiment described above, an example in which the device control server 400 decodes the MRR has been described. In contrast, in the third embodiment, the device control server 400 transmits (distributes) the read MRR to the terminal device 200 without decrypting the read MRR. The terminal device 200 transfers the MRR from the device control server 400 to the device 100 . The device 100 decodes the MRR acquired from the terminal device 200 .

(Functional block configuration of the device according to the third embodiment)
A functional block configuration of the device 100 according to the third embodiment will be described. FIG. 11 is a diagram showing the functional block configuration of the device 100 according to the third embodiment.

As shown in FIG. 11, the device 100 is common to the first embodiment in that it includes a control unit 110, a communication unit 120, a storage unit 130, and an operation unit 140. FIG.

A control unit 110 according to the third embodiment includes an MRR acquisition unit 113 and a processing unit 114 .

The MRR acquisition unit 113 acquires MRRs from the terminal device 200 .

The processing unit 114 decodes the MRR acquired by the MRR acquisition unit 113 and performs processing for controlling the own device 100 . The processing unit 114 includes an MRR decoding unit 114a and an action control unit 114b.

The MRR decryption unit 114a decrypts the MRR acquired by the MRR acquisition unit 113, and extracts an action (action ID) that can be executed by the device 100 from the MRR based on the decryption result. For example, a list of executable actions (action IDs) is defined for each device ID. Extract from MRR.

After the action is extracted by the MRR decoding unit 114a, the action control unit 114b controls to start the action in response to the operation of the operation unit 140 (action start button) by the user.

(Example of operation according to the third embodiment)
An operation example of the device control system 1 according to the third embodiment will be described. FIG. 12 is a diagram showing an operation example of the device control system 1 according to the third embodiment.

As shown in FIG. 10, in step S401, the terminal device 200 receives a recipe search operation from the user.

In step S<b>402 , the terminal device 200 transmits an HRR request requesting provision of a recipe (HRR) to the recipe distribution server 300 .

In step S<b>403 , the recipe distribution server 300 reads the HRR corresponding to the HRR request from the terminal device 200 .

In step S<b>404 , the recipe distribution server 300 transmits the read HRR to the terminal device 200 . A recipe ID is associated with HRR.

In step S405, the terminal device 200 receives a device control operation from the user.

In step S<b>406 , the terminal device 200 transmits a control request requesting control of the device 100 to the device control server 400 . A recipe ID is associated with the control request.

In step S<b>407 , the device control server 400 reads the MRR corresponding to the control request from the terminal device 200 .

In step S<b>408 , the device control server 400 transmits (distributes) the read MRR to the terminal device 200 .

In step S<b>409 , the terminal device 200 transfers the MRR acquired from the device control server 400 to the device 100 .

In step S410, the device 100 decodes the MRR acquired from the terminal device 200, and extracts an action (action ID) executable by the device 100 from the MRR.

In step S411, the device 100 receives an action start operation from the user.

In step S412, the device 100 executes the action extracted from the MRR.

<Modified example of the third embodiment>
In the third embodiment described above, an example in which the terminal device 200 transfers the MRR from the device control server 400 to the device 100 has been described. However, the device control server 400 may transmit the MRR to the device 100 without going through the terminal device 200 . In such a case, the destination information (address information) of the device 100 may be registered in the device control server 400 in advance, or the destination information (address information) of the device 100 may be sent to the device control server 400 by a control request from the terminal device 200. may be notified.

FIG. 13 illustrates an operation example of the device control system 1 according to this modified example. FIG. 13 is a diagram showing an operation example of the device control system 1 according to this modification.

As shown in FIG. 13, each process of steps S501 to S507 is the same as each process of steps S401 to S407 in FIG.

In step S<b>508 , the device control server 400 transmits (distributes) the read MRR to the device 100 .

In step S509, the device 100 decodes the MRR acquired from the device control server 400, and extracts an action (action ID) executable by the device 100 from the MRR.

In step S510, the device 100 receives an action start operation from the user.

In step S511, the device 100 executes the action extracted from the MRR.

<Other embodiments>
Although the embodiments have been described above, the statements and drawings forming part of this disclosure should not be construed as limiting the invention. Various alternative embodiments, implementations and operational techniques will become apparent to those skilled in the art from this disclosure.

In the embodiment described above, an example in which the device control system 1 includes the terminal device 200 has been described. However, if the device 100 can communicate with the recipe distribution server 300 and the device control server 400 and has a user interface such as a touch panel, the terminal device 200 may be unnecessary. In such a case, the operation performed by the terminal device 200 in the first and third embodiments described above may be performed by the device 100 instead of the terminal device 200 .

In the embodiment described above, an example has been described in which the recipe distribution server 300 and the device control server 400 are separate server devices. However, the functions of the recipe distribution server 300 and the device control server 400 may be provided in one server device, and the recipe distribution server 300 and the device control server 400 may be integrated into one server device.

A program that causes a computer to execute each process in the device control system 1 may be provided. Also, the program may be recorded on a computer-readable medium.
A computer readable medium allows the installation of the program on the computer. Here, the computer-readable medium on which the program is recorded may be a non-transitory recording medium. The non-transitory recording medium is not particularly limited, but may be, for example, a recording medium such as CD-ROM or DVD-ROM.

1: Equipment control system 10: Communication network 21: CPU
22: RAM
23: ROM
24: Auxiliary storage device 25: Communication module 26: Input device 27: Output device 28: Mechanism 30: Recipe distribution server 110: Control unit 111: Command acquisition unit 112: Action control unit 113: MRR acquisition unit 114: Processing unit 114a: MRR decoding unit 114b: action control unit 120: communication unit 130: storage unit 140: operation unit 200: terminal device 210: control unit 211: HRR acquisition unit 212: control request unit 213: command transfer unit 214: MRR acquisition unit 215: Processing unit 215a: MRR decoding unit 215b: Command generation unit 215c: Command transmission unit 220: Communication unit 230: Storage unit 240: Operation unit 250: Display unit 300: Recipe distribution server 400: Device control server 410: Control unit 411: Request Reception unit 412 : MRR acquisition unit 413 : Processing unit 413a : MRR decoding unit 413b : Command generation unit 413c : Command transmission unit 420 : Communication unit 430 : Storage unit

Claims (10)

A system comprising a terminal device and a server,
The terminal device
a first receiving means for receiving an operation to search recipes from a user;
a second receiving means for receiving an operation to select a recipe from a list of recipes displayed in response to the first receiving means receiving an operation to search for recipes;
an instruction means for instructing a cooking appliance capable of executing an action, which is an operation related to cooking necessary for state transition of ingredients, to start the action;
transmitting means for, after a recipe is selected based on the operation received by said second receiving means, transmitting to said server information regarding said selected recipe and information regarding said cooking appliance for which said instruction is given by said instruction means; have
The server executes the cooking appliance corresponding to the information relating to the cooking appliance transmitted to the server by the transmitting means from the appliance control recipe data corresponding to the information relating to the recipe transmitted to the server by the transmitting means. extracting a possible action and sending a control command to the terminal device to cause the cooking appliance to initiate the extracted action;
wherein the terminal device transmits the control command transmitted to the terminal device by the server to the cooking appliance that performs the instruction by the instruction means;
wherein an action corresponding to a control command transmitted to the cooking appliance by the terminal device is executed in the cooking appliance in response to the instruction by the instruction means ;
The equipment control recipe data is represented by a graph consisting of a plurality of nodes and edges between the nodes,
The edge is given an action ID that uniquely identifies the action.
system. The terminal device further comprises control means for controlling the selected recipe to be displayed in a user-readable manner in response to the recipe being selected by the second receiving means. Item 1. The system according to item 1. The control means controls to display a button for controlling the cooking appliance in response to the recipe being selected by the second reception means ,
wherein said transmission means transmits information regarding said selected recipe and information regarding said cooking appliance to said server in response to operation of a button for controlling said cooking appliance. Item 3. The system according to item 2 . 1. The recipe retrieved according to the operation received by said first receiving means does not necessarily specify a cooking appliance for performing an action that must be executed in the recipe. 4. The system according to any one of 3 . In the server, in response to the selection of the recipe by the second receiving means, the recipe data for displaying the selected recipe on the terminal device is the recipe data for device control in a machine-readable form. The system according to any one of claims 1 to 4, characterized in that it is changed to The cooking equipment includes any one of a refrigerator, a microwave oven, an oven, a food processor, a mixer, a rice cooker, an electric pot, an electric fryer, an electric steamer, a noodle-making machine, a weighing scale, and a cooking robot. The system according to any one of claims 1-5 . the terminal device
a first receiving step of receiving an operation to search for recipes from a user;
a second receiving step of receiving an operation of selecting a recipe from the list of recipes displayed in response to receiving an operation of searching for recipes in the first receiving step;
an instruction step for instructing a cooking appliance capable of executing an action, which is an operation related to cooking necessary for state transition of ingredients, to start the action;
After a recipe is selected based on the operation received in the second receiving step, a transmitting step of transmitting to a server information about the selected recipe and information about the cooking appliance that performs the instruction in the instructing step. death,
The server executes the cooking appliance corresponding to the information relating to the cooking appliance transmitted to the server in the transmitting step from the appliance control recipe data corresponding to the information relating to the recipe transmitted to the server in the transmitting step. extracting a possible action and sending a control command to the terminal device to cause the cooking appliance to initiate the extracted action;
the terminal device transmitting the control command transmitted to the terminal device by the server to the cooking appliance that performs the instruction in the instruction step;
characterized in that an action corresponding to the control command transmitted to the cooking appliance by the terminal device is executed in the cooking appliance in response to the instruction in the instructing step ,
The equipment control recipe data is represented by a graph consisting of a plurality of nodes and edges between the nodes,
The edge is given an action ID that uniquely identifies the action.
Method. the server
After a recipe is selected from a list of recipes displayed on the terminal device based on an operation accepted by the terminal device, information about the selected recipe and actions related to cooking necessary for state transition of ingredients are executed. a receiving step of receiving from the terminal device information about cooking appliances that
An executable action of the cooking appliance corresponding to the information relating to the cooking appliance received in the receiving step is extracted from appliance control recipe data corresponding to the information relating to the recipe received in the receiving step , and the extracted action is used as the a generating step of generating a control command to cause the cooking appliance to start ;
a transmission step of transmitting the control command to the terminal device ;
and run
The equipment control recipe data is represented by a graph consisting of a plurality of nodes and edges between the nodes,
The edge is given an action ID that uniquely identifies the action.
Method. to the computer,
After a recipe is selected from a list of recipes displayed on the terminal device based on an operation accepted by the terminal device, information about the selected recipe and actions related to cooking necessary for state transition of ingredients are executed. a receiving step of receiving from the terminal device information about cooking appliances that
An executable action of the cooking appliance corresponding to the information relating to the cooking appliance received in the receiving step is extracted from appliance control recipe data corresponding to the information relating to the recipe received in the receiving step, and the extracted action is used as the a generating step of generating a control command to cause the cooking appliance to start;
a transmission step of transmitting the control command to the terminal device;
A program for executing
The equipment control recipe data is represented by a graph consisting of a plurality of nodes and edges between the nodes,
The edge is given an action ID that uniquely identifies the action.
program . After a recipe is selected from a list of recipes displayed on the terminal device based on an operation accepted by the terminal device, information about the selected recipe and actions related to cooking necessary for state transition of ingredients are executed. receiving means for receiving from the terminal device information about the cooking appliance that
An executable action of the cooking appliance corresponding to the information about the cooking appliance received by the receiving means is extracted from appliance control recipe data corresponding to the information about the recipe received by the receiving means, and the extracted action is used as the generating means for generating a control command to cause the cooking appliance to start;
a transmitting means for transmitting the control command to the terminal device;
a server comprising
The equipment control recipe data is represented by a graph consisting of a plurality of nodes and edges between the nodes,
The edge is given an action ID that uniquely identifies the action.
server .

Images