JP2022110855A - Cooker and learned model creating method - Google Patents

Abstract

To provide a cooker that can adjust the finishing state of a product to be heated and a learning model creating method.SOLUTION: A cooker 1 has a heating unit 30, an imaging unit 40, an estimation unit 11 and a heating control unit 12. The heating unit 30 heats a product to be heated. The imaging unit 40 captures an image of the product to be heated while the product to be heated is heated. The estimation unit 11 estimates a finishing state of the product to be heated during heating of the product to be heated on the basis of the result of capturing by the imaging unit 40. The heating control unit 12 controls the heating unit 30 according to the result of estimation by the estimation unit 11.SELECTED DRAWING: Figure 2

Description

The present invention relates to a heating cooker and a learned model generation method.

The cooking apparatus described in Patent Literature 1 includes heating means, an imaging device, and a central information processing device. The heating means heats food (object to be heated). The imaging device images the food before heating. The central information processing unit processes the captured image before heating into a finished image. Also, the imaging device captures an image of the food while it is being heated. The central information processing unit determines whether or not the image of the processed food matches the image of the food captured during heating. If it is determined that they match, the heating output is stopped.

JP-A-9-229373

In the cooking apparatus described in Patent Document 1, the central information processing unit estimates the burnt color of the image of the food before cooking and processes the image before heating. However, heating the food can cause the food to unexpectedly brown in color. In this case, the estimated burnt color differs from the actual burnt color, and the central information processing unit may continue heating output more than necessary.

SUMMARY OF THE INVENTION The present invention has been made in view of the above problems, and an object thereof is to provide a heating cooker and a learning model generation method that can adjust the finishing state of an object to be heated.

According to one aspect of the present application, a cooking device includes a heating section, an imaging section, an estimating section, and a heating control section. The heating unit heats the object to be heated. The imaging unit images the object to be heated while the object to be heated is being heated. The estimating unit estimates the finishing state of the object to be heated during heating of the object to be heated based on the imaging result of the imaging unit. The heating control section controls the heating section according to the estimation result of the estimation section.

According to another aspect of the present application, a method for generating a learned model includes the steps of acquiring learning data, and generating a learned model for estimating the finished state of an object to be heated by subjecting the learning data to machine learning. including. The learning data includes a learning object food image representing a food and state information representing a finished state of the food.

ADVANTAGE OF THE INVENTION According to this invention, the finishing state of to-be-heated material can be adjusted.

It is a block diagram showing a heating cooker concerning an embodiment of the present invention. It is a block diagram which shows the outline|summary of operation|movement of the heating cooker which concerns on this embodiment. 1 is a block diagram showing a learning device according to this embodiment; FIG. It is a figure which shows an example of the learning data which concerns on this embodiment. It is a figure which shows another example of the learning data which concerns on this embodiment. It is a figure which shows another example of the learning data which concerns on this embodiment. 4 is a block diagram showing input information and output information of a trained model according to this embodiment; FIG. 4 is a flow chart showing a method of generating a trained model according to the embodiment; It is a figure which shows the selection screen which the display part which concerns on this embodiment displays. It is a figure which shows an example of the imaging result of the imaging part which concerns on this embodiment. It is a figure which shows an example of the imaging result of the imaging part which concerns on this embodiment. It is a flow chart which shows the former stage of processing which a cooking-by-heating machine concerning this embodiment performs. It is a flow chart which shows the latter part of the processing which the cooking-by-heating machine concerning this embodiment performs. It is a block diagram which shows the heating cooker which concerns on the modification of this embodiment.

An embodiment of the present invention will be described with reference to the drawings. In the drawings, the same or corresponding parts are denoted by the same reference numerals, and description thereof will not be repeated.

First, the heating cooker 1 will be described with reference to FIG. FIG. 1 is a block diagram showing the heating cooker 1. As shown in FIG.

The heating cooker 1 is, for example, a microwave oven. As shown in FIG. 1, in the present embodiment, the heating cooker 1 includes a control unit 10, a storage unit 20, a heating unit 30, an imaging unit 40, an operation unit 50, a display unit 60, and a main unit. (not shown).

The main body includes a heating chamber and a lid. An object to be heated is placed in the heating chamber. The object to be heated is, for example, food. The lid is provided so as to be openable and closable with respect to the heating chamber.

The heating unit 30 heats an object to be heated placed in the heating chamber. In this embodiment, the heating unit 30 has, for example, a range heating function, a hot air circulation heating function, and a grill heating function. The microwave heating function is, for example, a function of cooking an object to be heated by radiating microwaves into the heating chamber. The hot air circulation heating function, for example, circulates hot air in the heating chamber to equalize the temperature in the heating chamber, or blows hot air directly onto the upper surface of the heated object to heat and cook the object. It is a function to The grill heating function is mainly a function of cooking an object to be heated by exposing the object to be heated to radiant heat. That is, the heating section 30 includes, for example, a microwave supply section and a heater.

The imaging unit 40 is, for example, a camera. Specifically, the imaging unit 40 is a camera configured by, for example, a CCD (Charge Coupled Device) digital television camera or the like. The imaging unit 40 is installed, for example, outside the heating chamber so as to face an opening formed in a wall surface of the heating chamber. The image capturing unit 40 captures an image of the object to be heated, for example, while the object to be heated is being heated by the heating unit 30 . In this embodiment, the imaging unit 40 images the object to be heated at predetermined time intervals. Specifically, for example, while the object to be heated is being heated by the heating unit 30, the imaging unit 40 takes an image of the object to be heated each time a predetermined time elapses. For example, the predetermined time is 3 seconds.

The storage unit 20 includes a storage device and stores computer programs such as software and various data. Specifically, the storage unit 20 includes a main storage device (eg, semiconductor memory) such as ROM (Read Only Memory) and RAM (Random Access Memory), and a solid state drive and/or hard disk drive. Auxiliary storage devices. The storage unit 20 may include removable media. The storage unit 20 is an example of a storage medium (eg, non-transitory computer-readable storage medium).

The storage unit 20 stores, for example, a learned model LM and a plurality of finished images MG.

In this embodiment, the trained model LM is constructed by machine-learning the learning data. Hereinafter, the learning data may be referred to as learning data TD in this specification. The learning data TD will be described later.

Also, in this embodiment, the plurality of finished images MG respectively correspond to a plurality of dishes (toast, sponge cake, etc.). Also, the plurality of finish images MG respectively correspond to a plurality of finish states for one dish. Specifically, for example, the plurality of finished images MG respectively correspond to toasts with different browning colors. Also, for example, the plurality of finished images MG correspond to sponge cakes with different swelling levels.

State information is associated with the finished image MG. The state information indicates the finishing state of the dish indicated by the finishing image MG. Hereinafter, in this specification, state information may be referred to as state information ST.

The display unit 60 is installed, for example, on the main body. The display unit 60 is, for example, a liquid crystal display. The display unit 60 displays various images. Specifically, for example, the display unit 60 displays a plurality of finished images MG.

The operation unit 50 includes, for example, operation keys provided in the cooking device 1 or a touch panel provided in the display unit 60 . In addition, the cooking-by-heating appliance 1 does not need to be equipped with a touch panel. Operation keys also include input devices such as a mouse and/or keyboard. The operation unit 50 is operated by the user and receives, for example, menu selection. Further, the operation unit 50 accepts, for example, selection of the finishing state of the object to be heated. The finished state of the object to be heated will be described later. The operation unit 50 is an example of a “reception unit”.

The control unit 10 includes processors such as a CPU (Central Processing Unit) and an MPU (Micro Processing Unit). Note that the storage unit 20 and the control unit 10 may be configured by an integrated circuit such as ASIC (Application Specific Integrated Circuit) or FPGA (Field-Programmable Gate Array). A processor is an example of a "computer."

The processor of the control unit 10 executes computer programs stored in the storage device of the storage unit 20 to control the storage unit 20, the heating unit 30, the imaging unit 40, the operation unit 50, and the display unit 60. The processor of the control unit 10 also functions as the estimation unit 11 and the heating control unit 12 by executing computer programs stored in the storage device of the storage unit 20 . That is, the control unit 10 includes an estimation unit 11 and a heating control unit 12 .

The estimating unit 11 estimates the finishing state of the object to be heated based on the imaging result of the imaging unit 40 while the object to be heated is being heated. The finished state of the heated object is, for example, the state of the heated heated object. Specifically, the estimating unit 11 estimates at least one of the color (brown color, burnt color, etc.) and shape (swelling degree, etc.) of the heated object.

The heating control section 12 controls the heating section 30 according to the estimation result of the estimation section 11 . For example, the heating control unit 12 stops the heating of the object to be heated by the heating unit 30 according to the estimation result of the estimation unit 11, or stops the heating of the object to be heated by the heating unit 30 after a certain period of time has elapsed. let

Next, an outline of the operation of the heating cooker 1 will be described with reference to FIGS. 1 and 2. FIG. FIG. 2 is a block diagram showing an overview of the operation of the heating cooker 1. As shown in FIG.

As shown in FIG. 2, first, the operation unit 50 accepts selection of one finished image MG from among multiple finished images MG. Then, the heating control section 12 acquires the state information ST of one finished image MG for which the operation section 50 has accepted the selection. That is, the heating control unit 12 receives the state information ST selected from the plurality of state information ST. Hereinafter, in this specification, the selected status information ST may be referred to as status information SA.

On the other hand, the estimating unit 11 inputs the cooking image HG, which is the imaging result of the imaging unit 40, to the learned model LM while the object to be heated is being heated. Specifically, the estimation unit 11 inputs the food image HG to the learned model LM while the object to be heated is being heated. The food image HG shows the object to be heated captured by the imaging unit 40 .

The learned model LM outputs state information ST of the input food image HG based on the input food image HG. Specifically, the learned model LM outputs the state information ST of the object to be heated shown in the food image HG during the heating of the object to be heated. Hereinafter, in this specification, the state information ST of the object to be heated may be referred to as state information SB.

The estimation unit 11 acquires the state information SB output by the learned model LM during heating of the object to be heated. That is, the estimation unit 11 estimates the finishing state of the object to be heated while the object to be heated is being heated, based on the food image HG. Therefore, the estimation unit 11 can accurately estimate the finishing state of the object to be heated. The estimation unit 11 then inputs the state information SB to the heating control unit 12 .

The heating control unit 12 controls the heating unit 30 according to the state information SA. That is, the heating control section 12 controls the heating section 30 according to the estimation result of the estimation section 11 . Therefore, the heating by the heating unit 30 can be continued or the heating by the heating unit 30 can be stopped depending on the finishing state of the object to be heated during heating. As a result, the finished state of the object to be heated can be adjusted.

Specifically, the heating control section 12 controls the heating section 30 according to the state information SA and the state information SB. That is, the heating control unit 12 controls the heating unit 30 according to the reception result of the operation unit 50 (corresponding to the state information SA) and the estimation result of the estimating unit 11 (corresponding to the state information SB). Therefore, the heating by the heating unit 30 can be continued or the heating by the heating unit 30 can be stopped so as to finish the object to be heated in a state desired by the user. As a result, the object to be heated can be heated so that the object to be heated is in a desired finished state. That is, the object to be heated can be heated appropriately for the user.

More specifically, in this embodiment, the heating control unit 12 determines whether or not the state information SA and the state information SB match. When the state information SA and the state information SB match, the heating control section 12 controls the heating section 30 to stop heating the object to be heated. As a result, the heating unit 30 stops operating. On the other hand, when determining that the state information SA and the state information SB do not match, the heating control section 12 controls the heating section 30 to continue heating the object to be heated.

Next, the learning device 100 that generates the trained model LM will be described with reference to FIG. FIG. 3 is a block diagram showing the learning device 100. As shown in FIG. Learning device 100 is, for example, a computer. The learning device 100 generates a trained model LM by performing machine learning on the learning data TD. Then, the learning device 100 outputs the learned model LM. The learned model LM estimates the finished state of the input food image HG as described above. The learning data TD is a target of machine learning by the learning device 100 .

As shown in FIG. 3, learning device 100 includes processing unit 110 and storage unit 120 .

The processing unit 110 includes processors such as CPU and GPU. The storage unit 120 includes a storage device and stores data and computer programs. The processor of the processing unit 110 executes computer programs stored in the storage device of the storage unit 120 to perform various processes. For example, the storage unit 120 may include a main storage device, an auxiliary storage device, and a removable medium, similar to the storage unit 20 of the heating cooker 1 . Storage unit 120 is, for example, a non-transitory computer-readable storage medium.

The storage unit 120 stores a learning program LP. The learning program LP is a program for executing a machine learning algorithm for finding certain rules out of a plurality of learning data TD and generating a trained model LM expressing the found rules.

The machine learning algorithm is not particularly limited as long as it is supervised learning, and examples thereof include decision tree, nearest neighbor method, naive Bayes classifier, support vector machine, or neural network. Thus, the trained model LM includes decision trees, nearest neighbors, naive Bayes classifiers, support vector machines, or neural networks. In the machine learning that generates the trained model LM, error backpropagation may be used.

For example, a neural network includes an input layer, one or more hidden layers, and an output layer. Specifically, the neural network is a deep neural network (DNN: Deep Neural Network), a recurrent neural network (RNN: Recurrent Neural Network), or a convolutional neural network (CNN: Convolutional Neural Network), and deep learning conduct. For example, a deep neural network includes an input layer, multiple hidden layers, and an output layer.

The processing unit 110 machine-learns a plurality of learning data TD based on the learning program LP. As a result, a certain rule is found out of the plurality of learning data TD, and a trained model LM is generated. That is, the learned model LM is constructed by machine learning the learning data TD. The storage unit 120 stores the learned model LM.

Specifically, the processing unit 110 finds a certain rule between the learning target dish image LG and the state information ST included in the learning data TD, and generates the learned model LM. The learning target food image LG is an explanatory variable, and the state information ST is an objective variable. In the present embodiment, the learning object food image LG indicates food. The state information ST indicates the finishing state of the food shown in the learning target food image LG. Hereinafter, the state information ST included in the learning data TD may be referred to as state information SC.

More specifically, the processing unit 110 calculates a plurality of learned parameters by performing machine learning on a plurality of learning data TD based on the learning program LP, and calculates one or more parameters to which the plurality of learned parameters are applied. Generate a trained model LM containing functions. A learned parameter is a parameter (coefficient) obtained based on the result of machine learning using a plurality of learning data TD.

The trained model LM operates a computer to receive input information and output output information. In other words, the trained model LM receives input information and outputs output information. Specifically, the learned model LM causes a computer to estimate the finished state of the food indicated by the input food image HG. In other words, the learned model LM estimates the finished state of the food indicated by the input food image HG.

Next, the learning data TD will be described with reference to FIGS. 4A to 4C. FIG. 4A shows an example of learning data TD. Each of FIGS. 4B and 4C shows another example of the learning data TD. Hereinafter, the learning data TD shown in FIG. 4A may be referred to as learning data TD1, the learning data TD illustrated in FIG. 4B may be referred to as learning data TD2, and the learning data TD illustrated in FIG. 4C may be referred to as learning data TD3.

4A to 4C illustrate the case where the dish is toast as an example. To facilitate understanding, the color of the toast is hatched. Specifically, the wider the hatching interval, the lighter the baked color, and the narrower the hatched interval, the darker the baked color.

In this embodiment, the learning data TD includes learning target dish images LG and state information ST. Further, in the present embodiment, for example, the user determines the finishing state of the dish shown in the learning target dish image LG. That is, the cooking image LG to be learned is labeled with the state information ST indicating the finishing state of the cooking determined by the user.

As shown in FIG. 4A, in this specification, the learning target food image LG of the learning data TD1 may be referred to as a food image LG1, and the state information ST of the learning data TD1 may be referred to as state information SC1. The browning (finished state) of the food image LG1 is judged by the user to be light, for example. Therefore, in the present embodiment, the food image LG1 is labeled with the state information SC1 indicating "thin".

Further, as shown in FIG. 4B, in this specification, the learning target food image LG of the learning data TD2 may be referred to as food image LG2, and the state information ST of the learning data TD2 may be referred to as state information SC2. . For example, the user determines that the browning (finishing state) of the food image LG2 is medium, neither light nor dark. Therefore, in this embodiment, the food image LG2 is labeled with the status information SC2 indicating "medium".

Furthermore, as shown in FIG. 4C, in this specification, the learning target food image LG of the learning data TD3 may be referred to as food image LG3, and the state information ST of the learning data TD3 may be referred to as state information SC3. . The browned color of the food image LG3 is determined by the user to be dark, for example. Therefore, in the present embodiment, the food image LG3 is labeled with the status information SC3 indicating "rich".

The status information ST may be represented by a character string such as "light", "medium" or "dark", or identification information (ID ).

Next, input information and output information of the trained model LM will be described with reference to FIG. FIG. 5 is a block diagram showing input information and output information of the trained model LM.

As shown in FIG. 5, the input information of the learned model LM is the food image HG, which is the imaging result of the imaging unit 40 . Based on the input food image HG, the learned model LM outputs the state information SB of the object to be heated indicated by the food image HG. In the example shown in FIG. 5, the learned model LM estimates that the toast indicated by the food image HG is moderately browned. Therefore, the trained model LM outputs status information SB indicating "medium".

Next, a learning method executed by the learning device 100 will be described with reference to FIGS. 3 and 6. FIG. FIG. 6 is a flowchart showing a learned model generation method according to this embodiment. As shown in FIG. 6, the learned model generation method includes steps S31 to S34.

As shown in FIGS. 3 and 6, in step S31, the processing unit 110 of the learning device 100 acquires learning data TD. For example, the processing unit 110 acquires learning data TD input by the user.

Next, in step S32, the processing unit 110 machine-learns the learning data TD based on the learning program LP.

Next, in step S33, the processing unit 110 determines whether or not the learning end condition is satisfied. A learning termination condition is a predetermined condition for terminating machine learning. The learning end condition is, for example, that the number of iterations reaches a specified number.

If a negative determination (No) is made in step S33, the process proceeds to step S31. As a result, machine learning is repeated.

On the other hand, if the determination in step S33 is affirmative (Yes), the process proceeds to step S34.

In step S34, the processing unit 110 outputs a model (one or more functions) to which a plurality of latest parameters (coefficients), that is, a plurality of learned parameters (coefficients) are applied, as a learned model LM. Then, the storage unit 120 stores the learned model LM.

As described above, the learned model LM is generated by the learning device 100 executing steps S31 to S34.

Next, the operation of the heating cooker 1 will be described with reference to FIGS. 7 to 9. FIG. FIG. 7 is a diagram showing a selection screen SG displayed by the display unit 60. As shown in FIG.

As shown in FIG. 7, first, the user operates the operation unit 50 to select the dish that he/she wants to cook. Then, the display unit 60 displays the selection screen SG. In this embodiment, the selection screen SG includes a plurality of finishing images MG. Specifically, the selection screen SG includes three finishing images MG.

The plurality of finish images MG correspond to the plurality of finish images for the dish selected by the user operating the operation unit 50, respectively. The user looks at the selection screen SG and selects one finish image MG from a plurality of finish images MG. As a result, the operation unit 50 accepts selection of the finishing image MG. Therefore, it is easier for the user to understand the finishing state than selecting one finishing state from among the finishing states represented by the character strings. As a result, the user's burden of selecting the finish state of the object to be heated can be reduced.

In this embodiment, the plurality of finishing images MG respectively correspond to a plurality of finishing states for one dish (toast). Specifically, each of the plurality of finished images MG has a different toasted color. Hereinafter, the plurality of finished images MG may be described separately from the finished image MG1, the finished image MG2, and the finished image MG3.

In the present embodiment, the finished image MG1 is associated with status information ST indicating "thin". Hereinafter, the state information ST associated with the finished image MG1 may be referred to as state information ST1. Similarly, the finished image MG2 is associated with status information ST indicating "medium". Hereinafter, the state information ST associated with the finished image MG2 may be referred to as state information ST2. Also, the finished image MG3 is associated with status information ST indicating "dark". Hereinafter, the state information ST associated with the finished image MG3 may be referred to as state information ST3. In the example shown in FIG. 7, the user has selected the finishing image MG2.

In this embodiment, when the display unit 60 displays the selection screen SG, the object to be heated may or may not be placed in the heating chamber. The object to be heated may be placed in the heating chamber by the time the operation unit 50 receives an instruction to start heating by the heating unit 30 .

In this embodiment, the object to be heated is placed in the heating chamber, and one finish image MG is selected from a plurality of finish images MG. tell it to start. That is, the operation unit 50 accepts selection of the finishing image MG and an instruction to start heating by the heating unit 30 .

In response to the operation unit 50 receiving the selection of the finish image MG and the instruction to start heating by the heating unit 30, the state information ST associated with the selected finish image MG and the heating by the heating unit 30 are started. An instruction is input to the heating control unit 12 . In the example shown in FIG. 7 , since the finished image MG2 is selected, the state information ST2 associated with the finished image MG2 is input to the heating control section 12 . The state information ST2 corresponds to the selected state information SA. Then, the heating control unit 12 controls the heating unit 30 so that the heating unit 30 starts heating. As a result, the heating unit 30 starts heating.

FIG. 8 is a diagram showing an example of the imaging result of the imaging unit 40. As shown in FIG. That is, FIG. 8 shows an example of the food image HG. Hereinafter, the food image HG shown in FIG. 8 may be referred to as a food image HG1. A cooking image HG1 shows the object to be heated captured by the imaging unit 40 after a predetermined time has passed since the heating unit 30 started heating the object to be heated.

The estimation unit 11 inputs the food image HG1 to the learned model LM. Based on the food image HG1, the learned model LM estimates, for example, that the finished state of the object to be heated is "thin". That is, the learned model LM outputs the state information SB indicating "thin". Then, the estimation unit 11 inputs to the heating control unit 12 the state information SB indicating “thin” output by the learned model LM.

The heating control unit 12 receives state information SA input via the operation unit 50 (that is, state information SA selected by the user) and state information SB input from the estimating unit 11 (that is, the state information SB that is estimated by the estimating unit 11). result), the heating unit 30 is controlled.

Specifically, in the examples shown in FIGS. 7 and 8, for example, the heating control unit 12 determines whether or not "medium" indicated by the state information SA and "thin" indicated by the state information SB match. do. The heating control unit 12 determines that the state information SA and the state information SB do not match, and controls the heating unit 30 to continue heating.

FIG. 9 is a diagram showing another example of the imaging result of the imaging unit 40. As shown in FIG. That is, FIG. 9 shows an example of the food image HG. Hereinafter, the food image HG shown in FIG. 9 may be referred to as a food image HG2. A cooking image HG2 shows an object to be heated captured by the imaging unit 40 when a predetermined time has passed since the imaging unit 40 captured the cooking image HG1 of FIG. That is, the imaging unit 40 images the object to be heated at predetermined time intervals. Therefore, the estimating section 11 can estimate the finishing state of the object to be heated every predetermined time. As a result, the heating control unit 12 can continue or stop the heating of the heating unit 30 every predetermined time, so that the finished state of the object to be heated can be adjusted more accurately.

The estimation unit 11 inputs the food image HG2 to the learned model LM. Based on the cooking image HG2, the learned model LM estimates, for example, that the finished state of the object to be heated is "medium". That is, the trained model LM outputs the state information SB indicating "medium". Then, the estimation unit 11 inputs to the heating control unit 12 the state information SB indicating “medium” output by the learned model LM.

The heating control unit 12 determines whether or not "medium" indicated by the state information SA matches "medium" indicated by the state information SB. The heating control unit 12 determines that the state information SA and the state information SB match, and controls the heating unit 30 to stop heating. Therefore, the object to be heated can be finished in the state selected by the user. As a result, the object to be heated can be heated so that the object to be heated is in a desired finished state. That is, the object to be heated can be appropriately heated.

In the present embodiment, for example, the imaging unit 40 images the object to be heated in response to the heating unit 30 stopping heating. That is, the imaging unit 40 images the object to be heated that has been heated by the heating unit 30 . Then, the display unit 60 displays the imaging result of the object to be heated that has been completely heated by the heating unit 30 . For example, the operation unit 50 can receive an instruction to register the imaging result of the object to be heated that has been completely heated by the heating unit 30 as a new finished image MG.

When the operation unit 50 accepts the instruction, the control unit 10 stores in the storage unit 20 the imaging result of the object to be heated that has been completely heated by the heating unit 30 as a new finished image MG corresponding to the dish selected by the user. Let At this time, the estimation unit 11 inputs the imaging result of the object to be heated that has been heated by the heating unit 30 to the learned model LM, and acquires the state information ST of the imaging result. Then, the control unit 10 causes the storage unit 20 to store the imaging result in association with the state information ST. Therefore, when the dish selected by the user is selected again, the display unit 60 can display the imaging result of the object to be heated that has been completely heated by the heating unit 30 . As a result, the user can select the finished image MG showing the desired state.

Here, for example, the control unit 10 causes the display unit 60 to display the food image HG, which is the imaging result when the object to be heated is being heated, in response to the imaging unit 40 imaging the object to be heated. may be controlled. That is, the display unit 60 may display the food image HG while the object to be heated is being heated. At this time, the display unit 60 preferably displays, for example, both the cooking image HG and the finishing image MG whose selection has been accepted by the operation unit 50 on the selection screen SG.

In this embodiment, the operation unit 50 stops the operation of the heating unit 30 when the display unit 60 displays both the imaging result when the object to be heated is being heated and the finished image MG. can accept instructions to Accordingly, the user can determine whether or not to continue heating the object by looking at the display section 60 . For example, when determining to continue heating the object to be heated, the user does not operate the operation unit 50 . On the other hand, when determining to stop heating the object to be heated, the user operates the operation unit 50 to input an instruction to stop heating. As a result, the heating of the object to be heated can be stopped when desired by the user. As a result, the object to be heated can be heated appropriately for the user.

Next, with reference to FIGS. 10 and 11, processing executed by the cooking device 1 will be described. FIG. 10 is a flow chart showing the first stage of the process executed by the heating cooker 1. As shown in FIG. FIG. 11 is a flow chart showing the latter stage of the process executed by the heating cooker 1. FIG. The processing executed by the cooking device 1 includes steps S1 to S13.

As shown in FIG. 10 , in step S<b>1 , the operation unit 50 receives a selection of the finishing image MG and an instruction to start heating by the heating unit 30 . That is, the operation unit 50 accepts selection of the finishing state associated with the finishing image MG. State information SA indicating the finishing state selected via the operation unit 50 is input to the heating control unit 12 .

In step S2, the heating control section 12 controls the heating section 30 to heat the object to be heated. As a result, the heating unit 30 heats the object to be heated.

In step S3, the control unit 10 controls the imaging unit 40 to image the object to be heated. As a result, the imaging unit 40 images the object to be heated.

In step S4, the control unit 10 controls the display unit 60 to display the cooking image HG, which is the imaging result of step S3. As a result, the display unit 60 displays the food image HG.

In step S<b>5 , the heating control unit 12 determines whether or not an instruction to stop the heating of the object to be heated by the heating unit 30 has been received. If a negative determination (No) is made in step S5, that is, if the user did not operate the operation unit 50 in step S4, the process proceeds to step S6. On the other hand, if an affirmative determination (Yes) is made in step S5, that is, if the user operates the operation unit 50 to input an instruction to stop heating in step S4, the process proceeds to step S11 in FIG. .

In step S6, the estimation unit 11 inputs the food image HG, which is the imaged result, to the learned model LM. Then, the process proceeds to step S7 in FIG.

As shown in FIG. 11, in step S7, the learned model LM outputs state information SB based on the cooking image HG input in step S6 of FIG.

In step S8, the estimation unit 11 acquires the state information SB output by the learned model LM. Then, the estimation unit 11 inputs the acquired state information SB to the heating control unit 12 .

In step S9, the heating control unit 12 determines the finishing state of the object to be heated according to the state information SA input in step S1 and the state information SB input in step S8. That is, the heating control unit 12 determines whether the state information SA and the state information SB match, and determines whether finishing (heating) of the object to be heated is completed.

If an affirmative determination (Yes) is made in step S9, that is, if the state information SA and the state information SB match, the process proceeds to step S11. On the other hand, if a negative determination (No) is made in step S9, that is, if the state information SA and the state information SB do not match, the process proceeds to step S10.

In step S10, the control unit 10 determines whether or not a predetermined time has passed since the imaging unit 40 captured an image of the object to be heated in step S3 of FIG. If a negative determination (No) is made in step S10, that is, if the predetermined time has not elapsed since the imaging unit 40 captured the image of the object to be heated in step S3, the process returns to step S10. On the other hand, when a positive determination (Yes) is made in step S10, the process proceeds to step S3 in FIG. That is, the imaging unit 40 images the object to be heated at predetermined time intervals.

In step S11, the heating control unit 12 controls the heating unit 30 to stop heating the object to be heated. As a result, the heating unit 30 stops heating.

In step S12, the control unit 10 controls the imaging unit 40 to image the object to be heated. As a result, the imaging unit 40 images the object to be heated.

In step S13, the control unit 10 controls the display unit 60 to display the imaging result of the imaging unit 40 in step S12. As a result, the display unit 60 displays the imaging result.

In step S<b>14 , the control unit 10 determines whether the operation unit 50 has received an instruction to register the cooking image HG representing the object to be heated that has been heated. If affirmative determination (Yes) is made in step S14, the process proceeds to step S15. On the other hand, if a negative determination (No) is made in step S14, the process ends.

In step S15, the control unit 10 causes the storage unit 20 to store the imaging result of the imaging unit 40 in step S12 as the finished image MG. Then the process ends.

(Modification)
Next, a heating cooker 1a according to a modification of the present embodiment will be described with reference to FIG. FIG. 12 is a block diagram showing a heating cooker 1a according to a modification of this embodiment. In the modified example, the cooking device 1a includes the communication unit 70, the communication unit 70 receives the selection of the finish image MG via the external terminal 2, and the storage unit 20a does not store the finish image MG. The modification mainly differs from the present embodiment in one point. Differences of the modified example from the present embodiment will be mainly described below.

A heating cooker 1a according to the modification includes a control section 10, a storage section 20a, a heating section 30, an imaging section 40, an operation section 50, a display section 60, and a communication section . The storage unit 20a stores, for example, the learned model LM. The communication unit 70 transmits and receives radio waves Bt to connect to the user's external terminal 2 to perform wireless communication. The external terminal 2 is, for example, a smart phone or a tablet terminal.

The communication unit 70 is, for example, a wireless communication module that performs wireless communication. In this embodiment, the communication unit 70 is, for example, a wireless communication module that performs short-range wireless communication. Near-field wireless communication is, for example, wireless communication with a communication distance of about several meters. Near-field wireless communication is, for example, Bluetooth (registered trademark), ZigBee (registered trademark), or communication conforming to WiFi standards. Communication conforming to the communication standard of Bluetooth (registered trademark) is, for example, BLE (Bluetooth (registered trademark) Low Energy).

The external terminal 2 is communicably connected to the server 3 via the network Ne. The network Ne is, for example, the Internet, LAN (Local Area Network), and Wi-Fi (registered trademark). Various communication devices are connected to the network Ne. Various communication devices are, for example, routers, bridges, access points, hubs and repeaters. In this embodiment, radio waves in the 2.4 GHz ISM (Industrial Scientific and Medical Band) band may be used for communication with the external terminal 2 and the server 3 .

In a modified example, the server 3 stores, for example, a plurality of finished images MG. The number of finished images MG stored in the server 3 is likely to be greater than the number of finished images MG stored in the storage unit 20 of the cooking device 1 of this embodiment. Also, the finishing image MG stored in the server 3 is associated with finishing information indicating the finishing state of the dish indicated by the finishing image MG.

The external terminal 2 receives a plurality of finished images MG from the server 3 via the network Ne. The user operates the external terminal 2 to select a desired finish image MG from among the plurality of finish images MG received by the external terminal 2 from the server 3 . Then, the external terminal 2 transmits the finishing image MG selected by the external terminal 2 to the heating cooker 1a via short-range wireless communication.

The communication unit 70 of the cooker 1 a receives the finishing image MG selected by the external terminal 2 from the external terminal 2 . Then, the estimating unit 11 estimates the finishing state of the object to be heated based on the food image HG obtained by the imaging unit 40 and the finished image MG received from the external terminal 2 . Therefore, the finished state of the object to be heated can be adjusted while selecting the desired finished image MG without storing the finished image MG in the storage unit 20a. As a result, the desired finish image MG may be selected from among many finish images MG, compared to the case where the finish image MG is stored in the storage unit 20a. You can adjust it according to your taste.

The embodiments of the present invention have been described above with reference to the drawings. However, the present invention is not limited to the above-described embodiments, and can be implemented in various aspects without departing from the gist of the present invention. Further, various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the above embodiments. For example, some components may be omitted from all components shown in the embodiments. In order to facilitate understanding, the drawings mainly show each component schematically, and the number of each component shown in the figure may differ from the actual number for convenience of drawing preparation. Also, each component shown in the above embodiment is an example and is not particularly limited, and various modifications are possible within a range that does not substantially deviate from the effects of the present invention.

(1) As described with reference to FIGS. 1 and 2, the estimating unit 11 inputs the food image HG representing the object to be heated to the learned model LM, and estimates the finishing state of the object to be heated. However, the estimation unit 11 may estimate the finishing state of the object to be heated by, for example, pattern matching the food image HG and the finishing image MG.

(2) As described with reference to FIG. 7, the selection screen SG displayed by the display unit 60 includes a plurality of finishing images MG. However, the selection screen SG may display a single finishing image MG. In this case, a different finish image MG may be displayed by the operation unit 50 receiving an operation from the user. Also, the selection screen SG may include, for example, an icon image instead of the finish image MG. In this case, the icon image includes, for example, a character string indicating the finishing state. Furthermore, the selection screen SG may include a character string indicating the finishing state instead of the finishing image MG.

(3) The order of steps S1 in FIG. 10 to step S15 in FIG. 11 can be changed as appropriate.

(4) As described with reference to FIG. 7, the selection screen SG displayed by the display unit 60 includes three finishing images MG. However, the selection screen SG may include one or two finish images MG, or may include four or more finish images MG. For example, when the selection screen SG includes four or more finish images MG, the four finish images MG may be associated with different state information ST or may be associated with the same state information ST. good.

(5) As described with reference to FIGS. 4A to 4C, each of the learning target food images LG of the learning data TD is labeled with three levels of state information SC. However, each of the cooking images LG to be learned may be labeled with two levels of state information SC, or may be labeled with four or more levels of state information SC.

(6) As described with reference to FIG. 12, the cooker 1a and the external terminal 2 communicated wirelessly by transmitting and receiving radio waves Bt. However, the cooking device 1a and the external terminal 2 may communicate with each other via the network Ne. Alternatively, the cooking device 1a may communicate with the server 3 via the network Ne.

INDUSTRIAL APPLICABILITY The present invention can be used in the fields of heating cookers and learned model generation methods, and has industrial applicability.

1 heating cooker 11 estimation unit 12 heating control unit 30 heating unit 40 imaging unit 50 operation unit 60 display unit 2 external terminal HG cooking image LM learned model MG finished image ST state information TD learning data

Claims (9)

a heating unit that heats an object to be heated;
an imaging unit that captures an image of the object to be heated while the object to be heated is being heated;
an estimating unit that estimates a finish state of the object to be heated during heating of the object based on the imaging result of the imaging unit;
and a heating control unit that controls the heating unit according to the estimation result of the estimation unit. further comprising a storage unit that stores a learned model constructed by machine learning the learning data,
The learning data includes a learning target food image representing a food and state information representing a finished state of the food,
The estimating unit inputs the food image, which is the imaging result, into the learned model,
The learned model outputs state information indicating a finishing state of the object to be heated based on the input food image,
The heating cooker according to claim 1, wherein the estimation unit acquires the state information output by the learned model. The heating cooker according to claim 1 or 2, wherein the imaging unit images the object to be heated at predetermined time intervals. further comprising a reception unit that receives selection of the finishing state of the object to be heated,
The heating cooker according to any one of claims 1 to 3, wherein the heating control unit controls the heating unit according to the reception result of the reception unit and the estimation result. further comprising a storage unit for storing a plurality of finished images,
The plurality of finished images respectively correspond to a plurality of finished states for one dish,
The finished image is associated with state information indicating the finished state,
The heating cooker according to claim 4, wherein the reception unit receives selection of the finishing image. further comprising a display unit that displays both the imaging result when the object to be heated is being heated and the finished image received by the reception unit;
The reception unit receives an instruction to stop the operation of the heating unit when the display unit displays both the imaging result when the object to be heated is heated and the finished image. The heating cooker according to claim 5. The image capturing unit captures an image of the object to be heated that has been heated by the heating unit,
7. The heating cooker according to claim 5 or 6, wherein the receiving unit receives from a user an instruction to register, as the new finished image, the imaging result of the object to be heated that has been heated by the heating unit. . further comprising a communication unit that receives the finished image selected by the external terminal from the external terminal;
The finished image is associated with state information indicating the finished state,
8. The estimating unit according to any one of claims 1 to 7, wherein the estimating unit estimates the finished state of the object to be heated based on the imaging result and the finished image received from the external terminal. heating cooker. obtaining training data;
generating a trained model for estimating the finished state of the object to be heated by machine learning the learning data,
The learned model generating method, wherein the learning data includes a learning target food image representing a food and state information representing a finished state of the food.

Images