EP3608593B1

EP3608593B1 - Cooking system comprising an oven and an external computing means, and method of operating such system

Info

Publication number: EP3608593B1
Application number: EP18188387.7A
Authority: EP
Inventors: Volkmar Bunzel; Thomas Schulz; Thorben SCHIFFLER; Michele Toppano; Christoph Luckhardt; Klaus Wälzlein; Constantin KRAUSSE
Original assignee: Electrolux Appliances AB
Current assignee: Electrolux Appliances AB
Priority date: 2018-08-10
Filing date: 2018-08-10
Publication date: 2022-03-16
Anticipated expiration: 2038-08-10
Also published as: EP3608593A1; WO2020030462A1

Description

The present invention relates to a cooking system and to a method for operating a food processing oven.
In the field of cooking appliances several attempts were made to facilitate the cooking operation for a user by providing assistance via the internet, such as by providing recipe data from a central database.
In DE 10 2012 204229 A1 there is disclosed a device having a cooking chamber in which there is arranged a camera for recording an image of the food in the cooking chamber. A recognition unit is provided to recognize the food in the cooking chamber in dependence of the captured image, and a selection unit selects a cooking program from predetermined cooking programs in dependence of the captured image.
The system known from DE 10 2012 204 229 A1 is disadvantageous in that the food recognition requires sophisticated evaluation means, which requires a considerable amount of computation and data storage, which usually is not available in a household appliance. Thus, unless the household appliance is equipped with a fast processing device, which results in high costs of the household appliance, the quality and speed of the food recognition will not be satisfactory.
In WO 2016/179424 A1 there is disclosed a cooking system comprising an oven having an oven cavity that is accessible via an oven door, and an external computing means. The oven is provided with an optical sensor for obtaining picture data of the oven cavity; a door switch for identifying certain angles or the closing of the door, and communication means for providing picture data to the external computing means. The external computing means is adapted to perform a food recognition routine to determine food parameters based on the picture data provided from the oven, to select cooking parameters based on the determined food parameters, and to transmit the selected cooking parameters to the oven.
Further relevant prior art may be found in patent publication WO 2016/034295 A1 .
In view of the above problems, it is an object of the present invention to provide for a cooking system which provides for advances in terms of handling and user comfort at a good cost-benefit ratio.
In accordance with the present invention the above object is solved by a cooking system as it is defined in claim 1 and further by a method as it is defined in claim 7.
In the cooking system and the corresponding method suggested herein the oven is configured to automatically provide for a picture recognition which is triggered by the closing of the oven door, wherein in contrast to known systems the picture recognition is configured as rather basic picture recognition which merely aims at recognizing whether the oven cavity is empty or might contain a food item. Such basic picture recognition does not require sophisticated computation and hence can be implemented with comparable simple components both in terms of hardware and software. Such basic picture recognition thus can be implemented in a fast and effective manner at moderate costs.
The further detailed picture recognition which then is performed to identify the kind of food to be processed, which may include an evaluation of various parameters of the food being processed, such as size, shape, surface structure, color, etc., which parameters also may change during the cooking process, such as the degree of browning of the food, such detailed picture recognition is not performed at the oven but in an external computing means, for example a cloud-based server system.
Performing the detailed picture recognition in an external computing means which is not restricted to the inherent constraints of a stand-alone device allows achieving much more accurate results at a much higher speed and yet at lower costs. Thus, if the external computing means is for example a central server, such server not only can be provided with much higher computation capacity but further can be configured to have access to a large database of pictures of food of various kinds and at various stages of processing. Furthermore, particularly when the external computing means is configured as an adaptive self-learning system, such as a neural network system, configuring the external computing means as a server system allows the system to be trained by a plurality of users, which further increases the accuracy of the system.
Upon having performed a food recognition routine to determine, based on the picture data provided from the oven, food parameters of the food being processed, such as the type of food, and the size, shape and condition thereof, the external computing means selects cooking parameters based on the determined food parameters and transmits the selected cooking parameters to the oven.
The external computing means, such as a cloud based server system, thus can be provided with a database that contains cooking parameters that are specifically adapted for the particular oven, and which cooking parameters are designed to reflect the type and state of the particular food being processed, as well as personal preferences of a user, such as a degree of browning or the like.
Preferred embodiments of the present invention are defined in the dependent claims.
In preferred embodiments the communication means of the oven is adapted to communicate with a multi-purpose communication device, such as a mobile phone, a tablet, a laptop computer, or a personal computer, which provides for communication with the external computing means. In such embodiments the multi-purpose communication device can be employed as interfacing device so as to display settings and options to the user, and provide input to the oven and to the external computing means. In such embodiments the interfacing particularly can be implemented as an application that is executed on the multi-purpose communication device, so as to provide for a convenient means for the user to be presented with operation options, selected operating parameters and any further appropriate information as regards operation of the device. Providing for an application that is run on a multi-purpose communication device provides a comfortable means for controlling the cooking process, which application further can easily be updated via the internet so as to further improve the software.
The external computing means can be configured to transmit food data and/or cooking parameters to the user for confirmation by the user before proceeding to the step of selecting cooking parameters based on the determined food parameters and/or the step of transmitting the selected cooking parameters to the oven. Thus, before proceeding further with the operation program, the user may be presented with the determined food data, such as the type of food as determined in the food recognition process, cooking parameters that have been determined based on the determined type of food, which cooking parameters may comprise operating parameters of the oven, such as heating modes, temperature levels, durations, steam addition, and the like, or preparation parameters for the determined type of food such as a degree of cooking, such as rare or well-done, a degree of browning, a predetermined change in size, etc.
According to the invention, the oven comprises an interface which is configured for communication with the external computing means and/or the multi-purpose communication device.
Such oven interface can be configured, for example, to start a cooking process upon transmittal of selected cooking parameters to the oven, to assist in a configuration process, to initiate program routines, such as service operations, etc.
In preferred embodiments the external computing means is a cloud-based system, i.e. a remote server system that is accessible via the internet and which is configured to communicate with a plurality of users. In contrast to computation means that are provided in the oven as such, a cloud-based system basically is not restricted to any constraints in terms of computing capacity and data volume and hence allows implementing a much more accurate and fast picture recognition as a stand-alone device that is provided at the oven as such.
According to the inventive method for operating a food processing oven there is provided, after the step of performing a food recognition routine on the picture data to determine food data, a step of transmitting food data to the user for confirmation before proceeding to the step of selecting cooking parameters based on the determined food data.
In such embodiments the user thus is presented with an intermediate result of the overall evaluation process, such as the type of food to be processed, or suggested operating parameters, before these operating parameters are transmitted to the oven.
Similarly, the method of the present invention further may comprise a step of transmitting to the user a selection of cooking options and/or optional cooking parameters which is performed after the step of performing a food recognition routine on the picture data to determine food data, so as to first ask the user for confirmation and/or selection of further optional parameters before proceeding with selecting cooking parameters and transmitting the selected cooking parameters to the oven.
In order to allow the user to check the selected program and optional settings before starting a cooking operation, the method, after the step of transmitting the selected cooking parameters to the oven, may comprise a step of displaying the selected cooking parameters, wherein the method only proceeds to the step of operating the oven based on the transmitted cooking parameters upon confirmation by the user. Such confirmation by the user can be effected, for example, by providing a "Start" button either at the oven or at the multi-purpose communication device.
Preferred embodiments of the present invention are described below by reference of the drawings in which:

Figure 1 is a schematic perspective view of a domestic oven according to an embodiment of the present invention,
Figure 2 illustrates a schematic process diagram of an embodiment of a method in accordance with the present invention, and
Figure 3 illustrates the process diagram of Figure 3 as a flow diagram.

Figure 1 illustrates a schematic perspective view of a domestic cooking oven 10 which comprises a housing 12, an oven cavity 14 inside the housing 12, an oven door 16 for closing a front side of the oven cavity 14, and a front panel 22 which is arranged above the oven door 16. In the front panel 22 there are provided control and display elements for operation of the oven, such as a touchpad panel 24 and a control button 26.
The oven door 16 comprises a door window 18 arranged in a central portion of the oven door 16. A door handle 20 is attached at an outer side of the oven door 16 above the door window 18.
In a central region of the handle 20 there is provided a camera 28 having a camera lens at the rear side of the handle, which camera lens is arranged to have a viewing angle which covers basically the entire oven cavity 14.
While a camera or another optical sensor for monitoring the interior of the oven cavity 14 alternatively could be located within the oven cavity, providing the camera within the handle is advantageous because in this manner the camera is not exposed to the harsh environment of the oven atmosphere which may involve high temperatures and high humidity, but which also contains oven fumes and food particles or droplets.
Figure 2 illustrates a schematic process diagram for operation of the oven 10 shown in Figure 1, which process diagram is illustrated in Figure 3 as a flow diagram
In the embodiment of the method of the present invention illustrated in Figures 2 and 3, the method starts with the user 30 opening the oven door 16 so as to put a food item into the oven cavity 14 (step 1). The closing of the door is sensed by a door sensor (step 2), such as a switch which is provided in the door frame so as to be contacted by the door when the door is closed, or which is associated to the hinge to detect from the positioning of the hinge when the door is in the fully closed position.
Upon sensing a closing of the door, the camera 28 in the door handle 20 is activated to obtain picture data from within the oven cavity 14 (step 3). The obtained picture data then is evaluated in a processing unit of the oven 10 so as to determine by some basic picture recognition routine that the oven cavity 14 no longer is empty (step 4). In an exemplary embodiment, such basic picture recognition routine may comprise comparing a prestored picture of the empty oven cavity with the picture data obtained after closing the door, wherein it is determined that the oven cavity may contain a food item if the compared picture data does not correspond to each other. The software for conducting the basic picture evaluation routine can be stored in the memory of the oven controller or in an additional memory that is provided for such purpose.
In case that it is determined that the oven cavity no longer is empty the method proceeds to step 5, in which the obtained picture data is transmitted to an external computing means 32, which in Figure 2 is illustrated as a cloud-based server system which is accessible via internet. The obtained picture data is transmitted together with data for identifying the type of oven, such as a type name or model number. The necessary data for identifying the type of oven also can be stored at the cloud-based server system, such as by a registration process which the users performs once upon installing the oven. In such latter case the server can retrieve the required data based on the identification of the user, which identification anyway has to be provided so that the server can respond to the user.
Should it be determined during step 4 that the oven cavity 14 still is empty, then the program returns to step 1, i.e. waits for a detection of the closing of the door.
The external computing means 32 processes the picture data obtained from the oven in a picture recognition routine so as to determine food data including the type of food and various characteristics such as the size, volume, color, surface texture, etc. of the food (step 6). Based on the determined food characteristics and optionally based on settings or options selected by the user, the external computing means selects from a database operating parameters that specifically are designed for processing the particular food item in the particular oven (step 7) .
The operating parameters that have been determined by the external computing means then are transmitted to the user, which in the process scheme illustrated in Figure 2 is done with the aid of a mobile communication device 34, which in Figure 2 is illustrated as a tablet computer. The data is displayed on the tablet computer wherein optionally also a notification can be output, such as an acoustic tone, so as to alert the user that data has been transmitted (step 8).
The user now can check the displayed data so as to confirm that the food type and the suggested settings are correct, or to change settings or select additional options (step 9). The thus confirmed or adapted selections then are transmitted to the external computing means (step 10), which transmits corresponding operating parameters to the oven (step 11).
The oven then is operated in accordance with the transmitted operating parameters (step 13), either directly upon receipt of the operating parameters, or upon the user providing again for some confirmation that the operation of the oven 10 is to be started, such as by pressing the control button 26 which in this case acts as start button for the device (step 12).
It is to be noted that whereas in Figure 2 an embodiment is illustrated in which the oven directly communicates with the external computing means, i.e. the cloud-based server system, such communication usually will be effected via a network interface, such as a home communication system with which the oven communicates, such as via a LAN or WLAN connection. In a preferred embodiment of the method and system suggested herein, the communication between the oven and the external computing means is effected via a multipurpose communication device, in Figure 2 the tablet computer, which thus is operated not only as a user interface bus also as a communication interface for the oven.
Furthermore, whereas in the embodiment described above the determination of the operating parameters for conducting a cooking process in conformity with the recognized food and optional user preferences is done in the external computing means, this also could be done at least in part by the multipurpose communication deice, i.e. the tablet computer illustrated in Figure 2, which communicates with the external computing means. For example, whereas the detailed picture recognition may be effected in a cloud based served which is used for providing picture recognition for multiple users, the display and selection of further options, such as the suggesting of different recipes for the detected food or the display of further options for the cooking process, such as the degree of cooking, timer functions, warming functions to be conducted following a cooking process, could be done via an application that is run on the tablet computer, and which uses data that is stored on the tablet computer or which is obtained from the internet, such as from the external computing means.
Whereas in the above described embodiment the food data, oven parameters, and optional settings, is displayed at the tablet computer, such data also can be displayed directly at the touchpad panel 24 of the oven, which also can be used for displaying the recognized food type, recipes, suggested or selected oven parameters, and for making user selections, such as by displaying a list of options from which the user may chose.

Reference signs

10: oven
12: housing
14: oven cavity
16: oven door
18: door window
20: door handle
22: front panel
24: touchpad panel
26: control button
28: camera
30: user
32: external computing means
34: mobile communication device

Claims

A cooking system, comprising:
a) an oven (10) having an oven cavity (14) that is accessible via an oven door (16); and

b) an external computing means (32);
wherein the oven (10) is provided with
a1) at least one optical sensor (28) for obtaining picture data of the oven cavity (14);

a2) means for detecting door closing and triggering the optical sensor (28) to obtain picture data;

a3) processing means which are part of the oven (10) configured for performing a basic picture evaluation to determine that the oven cavity (14) no longer is empty;

a4) communication means for providing picture data to the external computing means (32); wherein the external computing means (32) is adapted to

b1) perform a food recognition routine to determine food parameters based on the picture data provided by the communication means;

b2) select cooking parameters based on the determined food parameters; and

b3) transmit the selected cooking parameters to the oven (10); wherein the oven (10) is configured to be operated based on the transmitted cooking parameters.
The cooking system of claim 1, wherein the communication means of the oven (10) is adapted to communicate with a multi-purpose communication device (34) which provides for communication with the external computing means (32).
The cooking system of claim 1, wherein the multi-purpose communication device (34) comprises a mobile phone, a tablet, a laptop computer, or a personal computer.
The cooking system of any of the preceding claims, wherein the external computing means (32) is configured to transmit food data and/or cooking parameters to the user (30) for confirmation by the user before proceeding to step b2 and/or step b3.
The cooking system of any of the preceding claims, wherein the oven (10) comprises an interface configured for communication with the external computing means (32) and/or the multi-purpose communication device (34).
The cooking system of any of the preceding claim 1, wherein the external computing means (32) is a cloud-based system.
Method for operating a food processing oven (10) and an external computing menas (32), according to any one of claims 1 to 6, having an oven cavity (14) that is accessible via an oven door (16), at least one optical sensor (28) for obtaining picture data from the oven cavity (14), and means for communicating with the external computing means (32), the method comprising:
a) monitoring closing of the door (16);

b) upon detecting closing of the door (16), obtaining picture data from within the oven cavity (14);

c) processing the picture data to determine whether the oven cavity (14) is empty by processing means of the oven (10);

d) upon determining that the oven cavity (14)is not empty, transmitting the picture data to the external computing means (32);

e) the external computing means (32) performing a food recognition routine on the picture data provided by the communication means to determine food data;

f) selecting cooking parameters based on the determined food data;

g) transmitting the selected cooking parameters to the oven (10); and

h) operating the oven (10) based on the transmitted cooking parameters.
The method of claim 7, further comprising after step (e) the step of transmitting food data to the user (30) for confirmation before proceeding to step (f).
The method of claim 7 or 8, further comprising after step (e) the step of transmitting to the user (30)a selection of cooking options and/or optional cooking parameters and proceeding to step (f) upon confirmation and/or selection by the user.
The method of any one of claims 7 to 9, further comprising after step (g) the step of displaying the selected cooking parameters and proceeding to step (h) upon confirmation by the user (30).