JP2022109554A - Cooker and learned model generation method - Google Patents

Abstract

To provide a cooker and a learned model generation method that can reduce labor of a user operating the cooker.SOLUTION: A cooker 1 comprises a heating unit 30, an imaging unit 40, an estimating unit 11, and a heating control unit 12. The heating unit 30 heats an object to be heated. The imaging unit 40 images the object when the object is heated. The estimating unit 11 estimates the name of the object during heating of the object on the basis of an imaging result of the imaging unit 40. The heating control unit 12 controls the heating unit 30 depending on an estimating result of the estimating unit 11.SELECTED DRAWING: Figure 2

Description

The present invention relates to a heating cooker and a learned model generation method.

The cooking apparatus described in Patent Document 1 includes an operation input device as input means, heating means, an imaging device, and a central information processing device. The user sets the desired heating menu from the previously presented menus using the operation input device. The imaging device images the food before heating. If the heating menu selected by the operation input device is grill-type heating, the central information processing unit processes the captured image before heating into a finished image so as to add a burnt color to the surface of the food. . And a heating means heats food (to-be-heated material). The imaging device also captures an image of the food product while it is being heated. The central information processing unit determines whether or not the image of the processed food matches the image of the food imaged while being heated, and if it determines that they match, it stops the heating output.

JP-A-9-229373

However, in the cooking apparatus described in Patent Literature 1, the user needs to set a desired heating menu from menus presented in advance using the operation input device. Therefore, it is a burden for the user to set the heating menu.

SUMMARY OF THE INVENTION The present invention has been made in view of the above problems, and an object thereof is to provide a heating cooker and a learning model generation method that can reduce the trouble of a user operating the heating cooker.

According to one aspect of the present application, a cooking device includes a heating section, an imaging section, an estimating section, and a heating control section. The heating unit heats the object to be heated. The imaging unit images the object to be heated while the object to be heated is being heated. The estimating unit estimates the name of the object to be heated during heating of the object to be heated based on the imaging result of the imaging unit. The heating control section controls the heating section according to the estimation result of the estimation section.

According to another aspect of the present application, a method for generating a trained model includes the steps of acquiring learning data, and generating a trained model for estimating the name of an object to be heated by subjecting the learning data to machine learning. including. The learning data includes a learning target food image representing a food and name information representing the name of the food.

ADVANTAGE OF THE INVENTION According to this invention, the user's effort which operates a heating cooker can be reduced.

BRIEF DESCRIPTION OF THE DRAWINGS It is a block diagram which shows the heating cooker which concerns on Embodiment 1 of this invention. 4 is a block diagram showing an overview of the operation of the heating cooker according to Embodiment 1. FIG. 1 is a block diagram showing a learning device according to Embodiment 1; FIG. 4 is a diagram showing an example of learning data according to Embodiment 1; FIG. 4 is a diagram showing another example of learning data according to Embodiment 1; FIG. 4 is a diagram showing another example of learning data according to Embodiment 1; FIG. 4 is a diagram showing another example of learning data according to Embodiment 1; FIG. 4 is a diagram showing another example of learning data according to Embodiment 1; FIG. 4 is a diagram showing another example of learning data according to Embodiment 1; FIG. 4 is a block diagram showing input information and output information of a trained model according to Embodiment 1. FIG. 4 is a flow chart showing a method of generating a trained model according to Embodiment 1. FIG. FIG. 5 is a diagram showing an example of an imaging result of an imaging unit according to Embodiment 1; 4 is a diagram showing a table stored by a storage unit according to the first embodiment; FIG. 4A and 4B are diagrams showing an example of the operation of the heating unit according to the first embodiment; FIG. 4 is a flow chart showing processing executed by the heating cooker according to Embodiment 1. FIG. It is a block diagram which shows the heating cooker which concerns on Embodiment 2 of this invention. FIG. 10 is a diagram showing an example of a food image according to Embodiment 2; FIG. FIG. 10 is a diagram showing another example of a food image according to Embodiment 2; FIG. FIG. 10 is a diagram showing another example of a food image according to Embodiment 2; FIG. FIG. 10 is a diagram showing another example of a food image according to Embodiment 2; FIG. FIG. 10 is a diagram showing another example of a food image according to Embodiment 2; FIG. FIG. 10 is a diagram showing another example of a food image according to Embodiment 2; FIG. 10A and 10B are diagrams showing an imaging result of an imaging unit and an imaging result after processing according to the second embodiment; FIG. 9 is a flow chart showing processing executed by the cooking device according to Embodiment 2. FIG. FIG. 10 is a diagram showing learning data according to a modification of the first embodiment; FIG.

An embodiment of the present invention will be described with reference to the drawings. In the drawings, the same or corresponding parts are denoted by the same reference numerals, and description thereof will not be repeated.

First, the heating cooker 1 will be described with reference to FIG. FIG. 1 is a block diagram showing the heating cooker 1. As shown in FIG.

The heating cooker 1 is, for example, a microwave oven, an oven, a rice cooker, or an electric pot. Hereinafter, the present invention will be described by taking as an example the case where the heating cooker 1 is a microwave oven. As shown in FIG. 1, in Embodiment 1, the heating cooker 1 includes a control unit 10, a storage unit 20, a heating unit 30, an imaging unit 40, an operation unit 50, and a main unit (not shown). and

The main body includes a heating chamber. An object to be heated is placed in the heating chamber. The object to be heated is, for example, food.

The heating unit 30 heats an object to be heated placed in the heating chamber. In Embodiment 1, the heating unit 30 has, for example, a range heating function, a hot air circulation heating function, and a grill heating function. The microwave heating function is, for example, a function of cooking an object to be heated by radiating microwaves into the heating chamber. The hot air circulation heating function, for example, circulates hot air in the heating chamber to equalize the temperature in the heating chamber, or blows hot air directly onto the upper surface of the heated object to heat and cook the object. It is a function to The grill heating function is mainly a function of cooking an object to be heated by exposing the object to be heated to radiant heat. That is, the heating section 30 includes, for example, a microwave supply section and a heater.

The imaging unit 40 is, for example, a camera. Specifically, the imaging unit 40 is a camera configured by, for example, a CCD (Charge Coupled Device) digital television camera or the like. The imaging unit 40 is installed, for example, outside the heating chamber so as to face an opening formed in a wall surface of the heating chamber. The image capturing unit 40 captures an image of the object to be heated, for example, while the object to be heated is being heated by the heating unit 30 . In Embodiment 1, the imaging unit 40 images the object to be heated, for example, at predetermined time intervals. Specifically, for example, while the object to be heated is being heated by the heating unit 30, the imaging unit 40 takes an image of the object to be heated each time a predetermined time elapses. For example, the predetermined time is 2 seconds.

The storage unit 20 includes a storage device and stores computer programs such as software and various data. Specifically, the storage unit 20 includes a main storage device (eg, semiconductor memory) such as ROM (Read Only Memory) and RAM (Random Access Memory), and a solid state drive and/or hard disk drive. Auxiliary storage devices. The storage unit 20 may include removable media. The storage unit 20 is an example of a storage medium (eg, non-transitory computer-readable storage medium).

The storage unit 20 also stores, for example, the learned model LM and the table Tb.

In Embodiment 1, the trained model LM is constructed by machine-learning learning data. Hereinafter, the learning data may be referred to as learning data TD in this specification. The learning data TD will be described later.

Further, in the first embodiment, in the table Tb, each of the names of a plurality of dishes (pizza, toast, sponge cake, etc.) is associated with a heating time corresponding to the dish and a heating intensity corresponding to the dish. there is That is, the storage unit 20 stores the name of the dish in association with the heating time corresponding to the dish and the heating intensity corresponding to the dish. The heating intensity is, for example, the temperature or wattage when heating the object to be heated.

The operation unit 50 includes, for example, operation keys provided in the heating cooker 1 or a touch panel provided in a display unit such as a liquid crystal display. In addition, the heating cooker 1 does not need to be equipped with the display part. Operation keys also include input devices such as a mouse and/or keyboard. The operation unit 50 is operated by the user and receives an instruction to start heating by the heating unit 30, for example. The operation unit 50 is an example of a “reception unit”.

The control unit 10 includes processors such as a CPU (Central Processing Unit) and an MPU (Micro Processing Unit). Note that the storage unit 20 and the control unit 10 may be configured by an integrated circuit such as ASIC (Application Specific Integrated Circuit) or FPGA (Field-Programmable Gate Array). A processor is an example of a "computer."

The processor of the control unit 10 executes computer programs stored in the storage device of the storage unit 20 to control the storage unit 20 , the heating unit 30 , the imaging unit 40 and the operation unit 50 . Also, the processor of the control unit 10 executes computer programs stored in the storage device of the storage unit 20 and functions as the estimation unit 11 , the heating control unit 12 and the measurement unit 13 . That is, the control unit 10 includes an estimation unit 11 , a heating control unit 12 and a measurement unit 13 .

The estimation unit 11 estimates the name of the object to be heated based on the imaging result of the imaging unit 40 while the object to be heated is being heated.

The heating control section 12 controls the heating section 30 according to the estimation result of the estimation section 11 . For example, the heating control unit 12 causes the heating unit 30 to stop heating the object to be heated after a certain period of time has passed according to the estimation result of the estimation unit 11 . More specifically, the heating control unit 12 controls the heating unit 30 so that the object to be heated is heated for a heating time corresponding to the name of the object to be heated estimated by the estimating unit 11 .

The measurement unit 13 measures the time during which the object to be heated is heated. For example, the measuring unit 13 measures the time during which the object to be heated is heated from when the heating unit 30 starts heating the object to when the estimating unit 11 estimates the name of the object to be heated. .

Next, an outline of the operation of the heating cooker 1 will be described with reference to FIGS. 1 and 2. FIG. FIG. 2 is a block diagram showing an overview of the operation of the heating cooker 1. As shown in FIG.

As shown in FIG. 2 , first, the measurement unit 13 starts measuring the time during which the object to be heated is heated when the operation unit 50 receives an instruction to start heating by the heating unit 30 .

On the other hand, the estimating unit 11 inputs the cooking image HG, which is the imaging result of the imaging unit 40, to the learned model LM while the object to be heated is being heated. Specifically, the estimation unit 11 inputs the food image HG to the learned model LM while the object to be heated is being heated. The food image HG shows the object to be heated captured by the imaging unit 40 .

Based on the input food image HG, the learned model LM outputs name information MN indicating the name of the object to be heated indicated by the input food image HG. Specifically, the learned model LM outputs the name information MN of the object to be heated shown in the food image HG during the heating of the object to be heated.

Then, the estimation unit 11 acquires the name information MN output by the learned model LM during heating of the object to be heated. That is, the estimation unit 11 estimates the name of the object to be heated based on the food image HG during the heating of the object to be heated. Therefore, the estimation unit 11 can accurately estimate the name of the object to be heated. The estimation unit 11 then inputs the name information MN to the heating control unit 12 .

The heating control unit 12 controls the heating unit 30 according to the name information MN. That is, the heating control section 12 controls the heating section 30 according to the estimation result of the estimation section 11 . Therefore, the object to be heated can be heated according to the name of the object to be heated without the need for the user to select a heating method (menu) for the object to be heated by operating the operation unit 50 in advance. As a result, the user's trouble of operating the heating cooker 1 can be reduced.

Specifically, the heating control unit 12 determines the heating time and heating intensity according to the object to be heated according to the estimation result of the estimation unit 11 . Therefore, the object to be heated can be heated in accordance with the name of the object to be heated without the need for the user to operate the operation unit 50 in advance to select the heating time and the heating intensity for the object to be heated. As a result, the object to be heated can be heated with the heating time and heating intensity according to the name of the object to be heated, while reducing the user's trouble of operating the heating cooker 1 . That is, the object to be heated can be appropriately heated according to the name of the object to be heated.

Specifically, for example, when the object to be heated is toast, the estimation unit 11 estimates that the object to be heated is toast. On the other hand, for example, in the table Tb stored in the storage unit 20, the toast is associated with a heating time of 11 minutes and a heating intensity of 180°C. Therefore, the heating control unit 12 sets the time for heating the object to be heated to 11 minutes, and the heating intensity to heat the object to be heated to 180°C. Therefore, the heating control unit 12 controls the heating unit 30 so that the object to be heated is heated at 180° C. for 11 minutes, for example. As a result, the toast is properly baked.

More specifically, in Embodiment 1, the heating control unit 12 determines the remaining heating time for the object to be heated based on the determined heating time and the measurement result of the measuring unit 13 . Therefore, the heating control unit 12 controls the heating unit 30 so as to heat the object to be heated for the determined remaining heating time. As a result, the object to be heated can be prevented from being excessively heated by the heating unit 30 . That is, the object to be heated can be appropriately heated.

Next, the learning device 100 that generates the trained model LM will be described with reference to FIG. FIG. 3 is a block diagram showing the learning device 100. As shown in FIG. Learning device 100 is, for example, a computer. The learning device 100 generates a trained model LM by performing machine learning on the learning data TD. Then, the learning device 100 outputs the trained model LM. The learned model LM estimates the name of the object to be heated indicated by the input food image HG, as described above. The learning data TD is a target of machine learning by the learning device 100 .

As shown in FIG. 3, learning device 100 includes processing unit 110 and storage unit 120 .

The processing unit 110 includes processors such as CPU and GPU. The storage unit 120 includes a storage device and stores data and computer programs. The processor of the processing unit 110 executes computer programs stored in the storage device of the storage unit 120 to perform various processes. For example, the storage unit 120 may include a main storage device, an auxiliary storage device, and a removable medium, similar to the storage unit 20 of the heating cooker 1 . Storage unit 120 is, for example, a non-transitory computer-readable storage medium.

The storage unit 120 stores a learning program LP. The learning program LP is a program for executing a machine learning algorithm for finding certain rules out of a plurality of learning data TD and generating a trained model LM expressing the found rules.

The machine learning algorithm is not particularly limited as long as it is supervised learning, and examples thereof include decision tree, nearest neighbor method, naive Bayes classifier, support vector machine, or neural network. Thus, the trained model LM includes decision trees, nearest neighbors, naive Bayes classifiers, support vector machines, or neural networks. In the machine learning that generates the trained model LM, error backpropagation may be used.

For example, a neural network includes an input layer, one or more hidden layers, and an output layer. Specifically, the neural network is a deep neural network (DNN: Deep Neural Network), a recurrent neural network (RNN: Recurrent Neural Network), or a convolutional neural network (CNN: Convolutional Neural Network), and deep learning conduct. For example, a deep neural network includes an input layer, multiple hidden layers, and an output layer.

The processing unit 110 machine-learns a plurality of learning data TD based on the learning program LP. As a result, a certain rule is found out of the plurality of learning data TD, and a trained model LM is generated. That is, the learned model LM is constructed by machine learning the learning data TD. The storage unit 120 stores the trained model LM.

Specifically, the processing unit 110 finds a certain rule between the learning target dish image LG and the name information MN included in the learning data TD, and generates the learned model LM. The learning target food image LG is an explanatory variable, and the name information MN is an objective variable. In the first embodiment, the learning target food image LG represents food. The name information MN indicates the name of the dish shown in the learning target dish image LG.

More specifically, the processing unit 110 calculates a plurality of learned parameters by performing machine learning on a plurality of learning data TD based on the learning program LP, and calculates one or more parameters to which the plurality of learned parameters are applied. Generate a trained model LM containing functions. A learned parameter is a parameter (coefficient) obtained based on the result of machine learning using a plurality of learning data TD.

The trained model LM operates a computer to receive input information and output output information. In other words, the trained model LM receives input information and outputs output information. Specifically, the learned model LM causes the computer to estimate the name of the dish indicated by the input dish image HG. In other words, the learned model LM estimates the name of the dish indicated by the input dish image HG.

Next, the learning data TD will be described with reference to FIGS. 4A to 4F. FIG. 4A shows an example of learning data TD. Each of FIGS. 4B to 4F shows another example of the learning data TD. 4A is referred to as learning data TD1, the learning data TD illustrated in FIG. 4B is referred to as learning data TD2, the learning data TD illustrated in FIG. 4C is referred to as learning data TD3, and FIG. The learning data TD shown in FIG. 4E may be referred to as learning data TD4, the learning data TD illustrated in FIG. 4E may be referred to as learning data TD5, and the learning data TD illustrated in FIG. 4F may be referred to as learning data TD6.

In Embodiment 1, the learning data TD includes learning target dish images LG and name information MN. Further, in the first embodiment, for example, the user determines the name of the dish shown in the learning target dish image LG. That is, the learning object dish image LG is labeled with the name information MN indicating the name of the dish determined by the user. To make it easier to understand, hatching indicates the brown color of the food. Specifically, the wider the hatching interval, the lighter the baked color, and the narrower the hatched interval, the darker the baked color.

As shown in FIGS. 4A to 4F, the multiple learning data TD correspond to multiple dishes (eg, pizza, toast, etc.). Specifically, the plurality of learning data TD shown in FIGS. 4A to 4C correspond to pizza. A plurality of pieces of learning data TD shown in FIGS. 4D to 4F correspond to toast. In addition, each of the learning target food images LG of the plurality of learning data TD indicates the food at different times in the process of heating one food.

For example, each of learning data TD1 to learning data TD3 shown in FIGS. 4A to 4C indicates pizza at different times during the pizza heating process.

As shown in FIG. 4A, the pizza indicated by the learning target food image LG of the learning data TD1 is located before the pizza indicated by the learning target food image LG of the learning data TD2 and the pizza indicated by the learning target food image LG of the learning data TD3. Showing pizza in stages. In the first embodiment, the learning target food image LG of the learning data TD1 is labeled with name information MN indicating "pizza". In this specification, the name information MN indicating "pizza" may be referred to as name information MN1.

Also, as shown in FIG. 4B, the pizza indicated by the learning target food image LG of the learning data TD2 is different from the pizza indicated by the learning target food image LG of the learning data TD1 and the pizza indicated by the learning target food image LG of the learning data TD3. shows a pizza in between stages. Further, in the first embodiment, the learning target dish image LG of the learning data TD2 is labeled with the name information MN1 indicating "pizza".

Furthermore, as shown in FIG. 4C, the pizza indicated by the learning target food image LG of the learning data TD3 is higher than the pizza indicated by the learning target food image LG of the learning data TD1 and the pizza indicated by the learning target food image LG of the learning data TD2. A later stage pizza is shown. Further, in the first embodiment, the learning target food image LG of the learning data TD3 is labeled with the name information MN1 indicating "pizza".

Further, for example, each of learning data TD4 to learning data TD6 shown in FIGS. 4D to 4F indicates toast at different times in the process of heating the toast.

As shown in FIG. 4D, the toast indicated by the learning target food image LG of the learning data TD4 precedes the toast indicated by the learning target food image LG of the learning data TD5 and the toast indicated by the learning target food image LG of the learning data TD6. Indicates a staged toast. Further, in the first embodiment, the learning target food image LG of the learning data TD4 is labeled with name information MN indicating "toast". In this specification, the name information MN indicating "toast" may be referred to as name information MN2.

Also, as shown in FIG. 4E, the toast indicated by the learning target food image LG of the learning data TD5 is different from the toast indicated by the learning target food image LG of the learning data TD4 and the toast indicated by the learning target food image LG of the learning data TD6. Indicates a toast in the middle stage. Further, in the first embodiment, the learning target dish image LG of the learning data TD5 is labeled with the name information MN2 indicating "toast".

Furthermore, as shown in FIG. 4F, the toast indicated by the learning target food image LG of the learning data TD6 is higher than the toast indicated by the learning target food image LG of the learning data TD4 and the toast indicated by the learning target food image LG of the learning data TD5. Indicates a later stage toast. In the first embodiment, the learning target food image LG of the learning data TD6 is labeled with the name information MN2 indicating "toast".

The name information MN may be represented by a character string such as "pizza" or "toast", or may be represented by identification information (ID) corresponding to each of "pizza" and "toast". .

Next, input information and output information of the trained model LM will be described with reference to FIG. FIG. 5 is a block diagram showing input information and output information of the trained model LM.

As shown in FIG. 5, the input information of the learned model LM is the food image HG, which is the imaging result of the imaging unit 40 . Based on the input food image HG, the learned model LM outputs name information MN of the object to be heated indicated by the food image HG. In the example shown in FIG. 5, the learned model LM estimates that the name of the object to be heated indicated by the cooking image HG is "toast". Therefore, the trained model LM outputs name information MN indicating "toast".

Next, a learning method executed by the learning device 100 will be described with reference to FIGS. 3 and 6. FIG. FIG. 6 is a flowchart showing a learned model generation method according to the first embodiment. As shown in FIG. 6, the learned model generation method includes steps S41 to S44.

As shown in FIGS. 3 and 6, in step S41, the processing unit 110 of the learning device 100 acquires learning data TD. For example, the processing unit 110 acquires learning data TD input by the user.

Next, in step S42, the processing unit 110 machine-learns the learning data TD based on the learning program LP.

Next, in step S43, the processing unit 110 determines whether or not the learning end condition is satisfied. A learning termination condition is a predetermined condition for terminating machine learning. The learning end condition is, for example, that the number of iterations reaches a specified number.

If a negative determination (No) is made in step S43, the process proceeds to step S41. As a result, machine learning is repeated.

On the other hand, if the determination in step S43 is affirmative (Yes), the process proceeds to step S44.

In step S44, the processing unit 110 outputs a model (one or more functions) to which a plurality of latest parameters (coefficients), that is, a plurality of learned parameters (coefficients) are applied, as a learned model LM. Then, the storage unit 120 stores the learned model LM.

As described above, the learned model LM is generated by the learning device 100 executing steps S41 to S44.

Next, the operation of the heating cooker 1 will be described with reference to FIGS. 7 to 9. FIG. First, the operation unit 50 receives an instruction to start heating by the heating unit 30 while the object to be heated is placed in the heating chamber. Then, the heating control section 12 controls the heating section 30 so as to heat the object to be heated. As a result, the heating unit 30 starts heating the object to be heated. At this time, the heating unit 30 heats the object to be heated at a predetermined heating intensity, for example. In addition, the heating unit 30 starts heating the object to be heated, and the measurement unit 13 starts measuring the time during which the object to be heated is heated.

FIG. 7 is a diagram showing a food image HG that is the imaging result of the imaging unit 40. As shown in FIG. As shown in FIG. 7, the estimation unit 11 inputs the food image HG, which is the imaging result of the imaging unit 40, to the learned model LM. The learned model LM outputs name information MN representing "pizza" based on the input food image HG. Then, the estimation unit 11 acquires the name information MN output by the learned model LM.

FIG. 8 is a diagram showing a table Tb stored in the storage unit 20. As shown in FIG. As shown in FIG. 8, in table Tb, name information MN is associated with heating time Ht and heating intensity Hs. The heating control unit 12 determines the heating time Ht and the heating intensity Hs for the object to be heated according to the estimation result of the estimation unit 11, that is, the name information MN acquired by the estimation unit 11. FIG. For example, since the name information MN of the object to be heated indicated by the cooking image HG shown in FIG. is determined to be "250°C".

9A and 9B are diagrams showing an example of the operation of the heating unit 30. FIG. As shown in FIG. 9, the measurement result of the measuring unit 13 when the estimating unit 11 acquires the name information MN, that is, the heating time Ct of the object to be heated is 4 minutes. In other words, the name information MN is acquired based on the food image HG, which is the imaging result, when four minutes have passed since the heating unit 30 started heating the object to be heated. The heating control unit 12 sets the heating time Ht to "25 minutes" and the heating intensity Hs to "250°C". Therefore, the heating control unit 12 determines the remaining heating time Rt for the object to be heated based on the measurement result (heating time Ct) of the measuring unit 13 and the determined heating time Ht. Then, the heating control unit 12 controls the heating unit 30 so that the heating of the object to be heated ends when the remaining heating time Rt elapses after the estimating unit 11 acquires the name information MN. .

Next, with reference to FIG. 10, processing executed by the heating cooker 1 will be described. FIG. 10 is a flow chart showing the processing executed by the heating cooker 1 according to the first embodiment. The processing executed by the cooking device 1 includes steps S1 to S10.

As shown in FIG. 10 , in step S<b>1 , the operation unit 50 receives an instruction to start heating by the heating unit 30 . Then, the heating control section 12 controls the heating section 30 so as to heat the object to be heated. As a result, the heating unit 30 heats the object to be heated. In addition, the heating unit 30 starts heating the object to be heated, and the measurement unit 13 starts measuring the time during which the object to be heated is heated.

In step S2, the control unit 10 controls the imaging unit 40 to image the object to be heated. As a result, the imaging unit 40 images the object to be heated.

In step S3, the estimation unit 11 inputs the food image HG, which is the imaging result, to the learned model LM.

In step S4, the estimation unit 11 determines whether or not the learned model LM has output the name information MN based on the food image HG input in step S3. If a negative determination (No) is made in step S4, that is, if the learned model LM does not output the name information MN, or if the learned model LM outputs "unknown", the process proceeds to step S2. proceed to On the other hand, if an affirmative determination (Yes) is made in step S4, that is, if the trained model LM outputs the name information MN, the process proceeds to step S5. A case where the learned model LM does not output the name information MN and a case where the learned model LM outputs "unknown" will be described later.

In step S5, the estimation unit 11 acquires the name information MN output by the trained model LM. Then, the estimation unit 11 inputs the acquired state information SB to the heating control unit 12 .

In step S<b>6 , the heating control unit 12 determines the heating time Ht and the heating intensity Hs for the object to be heated according to the estimation result of the estimation unit 11 .

In step S7, the heating control unit 12 determines the remaining heating time Rt for the object to be heated based on the measurement result of the measuring unit 13 and the heating time Ht determined in step S6.

In step S8, the heating control unit 12 adjusts the heating intensity of the heating unit 30 based on the heating intensity Hs determined in step S6.

In step S9, the heating control unit 12 determines whether the remaining heating time Rt determined in step S7 has elapsed since the remaining heating time Rt was determined in step S7. If affirmative determination (Yes) is made in step S9, the process proceeds to step S10. On the other hand, if a negative determination (No) is made in step S9, the process returns to step S9.

In step S10, the heating control unit 12 controls the heating unit 30 to stop heating the object to be heated. As a result, the heating unit 30 stops heating.

(Embodiment 2)
Next, a heating cooker 1a according to Embodiment 2 will be described with reference to FIG. FIG. 11 is a block diagram showing a heating cooker 1a according to Embodiment 2. As shown in FIG. Embodiment 2 differs from Embodiment 1 mainly in that the heating cooker 1a includes a turntable 60 and the estimation unit 11a estimates the name of the object to be heated by pattern matching. In the following, differences of the second embodiment from the first embodiment will be mainly described.

The heating cooker 1 a includes a control section 10 a , a storage section 20 a , a heating section 30 , an imaging section 40 , an operation section 50 and a turntable 60 .

An object to be heated is placed on the turntable 60 . In Embodiment 2, the turntable 60 rotates, for example, at a substantially constant angular velocity while the heating unit 30 heats the object to be heated. That is, the turntable 60 rotates with the object to be heated placed thereon.

The control unit 10a includes processors such as CPU and MPU. The processor of the control unit 10 executes a computer program stored in the storage device of the storage unit 20a, and a motor for rotating the storage unit 20a, the heating unit 30, the imaging unit 40, the operation unit 50, and the turntable 60. (not shown).

The processor of the control unit 10 also functions as the estimation unit 11a, the heating control unit 12, the measurement unit 13, and the correction unit 14 by executing computer programs stored in the storage device of the storage unit 20a. That is, the control unit 10 includes an estimation unit 11a, a heating control unit 12, a measurement unit 13, and a correction unit 14. The estimation unit 11a and the correction unit 14 will be described later.

The storage unit 20a stores pattern matching data PD and a table Tb. The pattern matching data PD includes a plurality of food images MG and name information MN.

Next, the pattern matching data PD stored in the storage unit 20a will be described with reference to FIGS. 12A to 12F. FIG. 12A shows an example of pattern matching data PD. Each of FIGS. 12B to 12F shows another example of pattern matching data PD. Hereinafter, the pattern matching data PD shown in FIG. 12A is referred to as pattern matching data PD1, the pattern matching data PD illustrated in FIG. 12B is referred to as pattern matching data PD2, and the pattern matching data PD illustrated in FIG. The pattern matching data PD shown in FIG. 12D is denoted as pattern matching data PD3, the pattern matching data PD shown in FIG. 12D is denoted as pattern matching data PD4, the pattern matching data PD shown in FIG. 12E is denoted as pattern matching data PD5, and FIG. The pattern matching data PD shown may be referred to as pattern matching data PD6.

As shown in FIGS. 12A to 12F, the dish images MG of the plurality of pattern matching data PD respectively correspond to a plurality of dishes (eg, pizza, toast, etc.). Also, the plurality of food images MG each show the food at different times during the process of heating one food.

For example, each dish image MG of pattern matching data PD1 to pattern matching data PD3 shown in FIGS. 12A to 12C shows pizza at different times during the pizza heating process.

As shown in FIG. 12A, the pizza indicated by the dish image MG of the pattern matching data PD1 is located before the pizza indicated by the dish image MG of the pattern matching data PD2 and the pizza indicated by the dish image MG of the pattern matching data PD3. Showing pizza in stages. In the second embodiment, the dish image MG in the pattern matching data PD1 is labeled with the name information MN1 indicating "pizza".

Also, as shown in FIG. 12B, the pizza indicated by the dish image MG of the pattern matching data PD2 is different from the pizza indicated by the dish image MG of the pattern matching data PD1 and the pizza indicated by the dish image MG of the pattern matching data PD3. shows a pizza in between stages. In the second embodiment, the food image MG in the pattern matching data PD2 is labeled with the name information MN1 indicating "pizza".

Furthermore, as shown in FIG. 12C, the pizza indicated by the dish image MG of the pattern matching data PD3 is higher than the pizza indicated by the dish image MG of the pattern matching data PD1 and the pizza indicated by the dish image MG of the pattern matching data PD2. A later stage pizza is shown. In the second embodiment, the dish image MG in the pattern matching data PD3 is labeled with the name information MN1 indicating "pizza".

Also, for example, each of pattern matching data PD4 to pattern matching data PD6 shown in FIGS. 12D to 12F indicates toast at different times during the process of heating the toast.

As shown in FIG. 12D, the toast indicated by the dish image MG of the pattern matching data PD4 is an earlier toast than the toast indicated by the dish image MG of the pattern matching data PD5 and the toast indicated by the dish image MG of the pattern matching data PD6. Indicates a staged toast. In the second embodiment, the food image MG4 is labeled with name information MN2 indicating "toast".

Also, as shown in FIG. 12E, the toast indicated by the food image MG of the pattern matching data PD5 is different from the toast indicated by the food image MG of the pattern matching data PD5 and the toast indicated by the food image MG of the pattern matching data PD6. Indicates a toast in the middle stage. In the second embodiment, the dish image MG in the pattern matching data PD5 is labeled with name information MN2 indicating "toast".

Furthermore, as shown in FIG. 12F, the toast indicated by the food image MG of the pattern matching data PD6 is higher than the toast indicated by the food image MG4 of the pattern matching data PD4 and the toast indicated by the food image MG5 of the pattern matching data PD5. Indicates a later stage toast. In the second embodiment, the dish image MG6 in the pattern matching data PD6 is labeled with name information MN2 indicating "toast".

Next, with reference to FIG. 13, correction of the orientation of the imaging result will be described. FIG. 13 is a diagram showing a food image HG that is the imaging result of the imaging unit 40 and a food image HGA that is the imaging result after correction. As shown in FIG. 13, the case where the object to be heated is a pizza Pz will be described as an example. That is, in the example shown in FIG. 13, a pizza Pz is placed on the turntable 60 .

The correction unit 14 corrects the orientation of the food image HG based on the angular velocity ω when the turntable 60 rotates. The food image HGA is obtained by correcting the orientation of the food image HG by the correction unit 14 based on the angular velocity ω and the measurement result of the measurement unit 13 in which the object to be heated is heated. The measurement result of the measurement unit 13 is the time t during which the object to be heated is heated by the heating unit 30 . Specifically, the angle θ for correcting the orientation of the food image HG is calculated by θ=ω×t. The correction unit 14 corrects the food image HG by rotating the food image HG by an angle θ in the direction opposite to the direction in which the turntable 60 rotates.

The estimation unit 11a compares the corrected food image HGA with the food image MG stored in the storage unit 20a, and estimates the name of the object to be heated based on the comparison result. Therefore, the name of the object to be heated can be estimated with higher accuracy than when estimating the name of the object to be heated without correcting the orientation of the food image HG.

In this embodiment, the estimation unit 11a determines whether or not the matching rate indicated by the comparison result is equal to or greater than the threshold. The matching ratio indicates the ratio of matching between the object to be heated indicated by the food image HGA and the food image MG stored in the storage unit 20a. A threshold is, for example, 90%. That is, when the matching rate between the food image HGA and the food image MG is 90% or more, the estimation unit 11a estimates that the name of the object to be heated is the name information MN associated with the food image MG. Then, the heating control unit 12 controls the heating unit 30 according to the estimation result of the estimation unit 11a.

Next, with reference to FIG. 14, the process performed by the heating cooker 1a of Embodiment 2 will be described. FIG. 14 is a flow chart showing the process executed by the heating cooker 1a.

As shown in FIG. 14 , in step S<b>21 , the operation unit 50 receives an instruction to start heating by the heating unit 30 . Then, the heating control section 12 controls the heating section 30 so as to heat the object to be heated. As a result, the heating unit 30 heats the object to be heated. In addition, the heating unit 30 starts heating the object to be heated, and the measurement unit 13 starts measuring the time during which the object to be heated is heated.

In step S22, the control unit 10 controls the imaging unit 40 to image the object to be heated. As a result, the imaging unit 40 images the object to be heated.

In step S23, the correction unit 14 determines whether the turntable 60 is rotating. If affirmative determination (Yes) is made in step S23, that is, if the correction unit 14 determines that the turntable 60 is rotating, the process proceeds to step S24. on the other hand. If a negative determination (No) is made in step S23, that is, if the correction unit 14 determines that the turntable 60 is not rotating, the process proceeds to step S25. Here, the correction unit 14 may determine whether or not the turntable 60 is rotating by determining whether or not the motor that rotates the turntable 60 is being driven.

In step S24, the correction unit 14 corrects the food image HG, which is the imaging result of the imaging unit 40 in step S22, based on the angular velocity ω when the turntable 60 rotates and the measurement result of the measurement unit 13. .

In step S25, the estimating unit 11 compares the food image HG, which is the imaged result, with the food image MG. Note that when the food image HG, which is the imaged result, is corrected by the correction unit 14 in step S24, the estimation unit 11 compares the food image HGA, which is the imaged result after correction, with the food image MG.

In step S26, the estimation unit 11 determines whether or not the matching rate indicated by the comparison result in step S25 is equal to or greater than a threshold. If a negative determination (No) is made in step S26, that is, if the matching rate between the imaging result and the dish image MG is less than the threshold, the process proceeds to step S22. On the other hand, if an affirmative determination (Yes) is made in step S26, that is, if the matching rate between the imaging result and the food image MG is equal to or higher than the threshold, the process proceeds to step S27.

In step S27, the estimation unit 11 estimates the name of the object to be heated. Specifically, the name information MN associated with the food image MG whose matching rate is equal to or higher than the threshold is acquired.

In step S<b>28 , the heating control unit 12 determines the heating time Ht and the heating intensity Hs for the object to be heated according to the estimation result of the estimation unit 11 .

In step S29, the heating control unit 12 determines the remaining heating time Rt for the object to be heated based on the measurement result of the measuring unit 13 and the heating time Ht determined in step S6.

In step S30, the heating control unit 12 adjusts the heating intensity of the heating unit 30 based on the heating intensity Hs determined in step S29.

In step S31, the heating control unit 12 determines whether or not the heating time Rt has elapsed since the remaining heating time Rt was determined in step S29. If affirmative determination (Yes) is made in step S31, the process proceeds to step S32. On the other hand, if a negative determination (No) is made in step S31, the process returns to step S31.

In step S32, the heating control unit 12 controls the heating unit 30 to stop heating the object. As a result, the heating unit 30 stops heating.

(Modification)
Next, referring to FIG. 15, the heating cooker 1 according to the modification of the first embodiment will be described. FIG. 15 is a diagram showing learning data Tda machine-learned by the learned model LM of the heating cooker 1 according to the modification of the first embodiment. The modification mainly differs from the first embodiment in that one piece of learning data Tda includes a plurality of learning target food images LD. Differences of the modified example from the first embodiment will be mainly described below.

As shown in FIG. 15, one piece of learning data Tda includes a plurality of learning target dish images LD and name information MN. Specifically, the learning data Tda includes a plurality of learning target food images LD1 to LDNX. To make it easier to understand, the browning color of the food is indicated by hatching. Specifically, the wider the hatching interval, the lighter the baked color, and the narrower the hatched interval, the darker the baked color.

A plurality of learning object food images LD each show the food at different times during the process of heating the food. Specifically, each of the plurality of learning object food images LD indicates a food imaged at predetermined time intervals during the process of heating one food. In a variant, the predetermined time is, for example, 2 seconds. That is, the learning target cooking image LD learned by the learned model LM according to the modified example is a cooking video.

The imaging unit 40 according to the modification images the object to be heated, for example, at predetermined time intervals. Then, the estimating unit 11 inputs cooking videos, which are a plurality of imaging results captured at predetermined time intervals, to the learned model LM. That is, in the modified example, the input information of the trained model LM is the moving image captured by the imaging unit 40 . Based on the input cooking animation, the learned model LM outputs name information MN of the object to be heated indicated by the cooking animation.

The embodiments and modifications of the present invention have been described above with reference to the drawings. However, the present invention is not limited to the above-described embodiments, and can be implemented in various aspects without departing from the gist of the present invention. Various inventions can be formed by appropriately combining the plurality of constituent elements disclosed in the above embodiments. For example, some components may be omitted from all components shown in the embodiments. In order to facilitate understanding, the drawings mainly show each component schematically, and the number of each component shown in the figure may differ from the actual number for convenience of drawing preparation. Also, each component shown in the above embodiment is an example and is not particularly limited, and various modifications are possible within a range that does not substantially deviate from the effects of the present invention.

(1) As described with reference to FIG. 7, based on the food image HG input by the estimation unit 11, the learned model LM outputs name information MN indicating the name of the object to be heated indicated by the food image HG. did. However, if the trained model LM cannot estimate the name of the object to be heated based on the food image HG input by the estimation unit 11, the trained model LM does not have to output the name information MN, for example. , name information MN indicating "unknown" may be output. When the learned model LM does not output the name information MN, or when the learned model LM outputs the name information MN indicating “unknown”, the estimating unit 11 instructs the imaging unit 40 to image the object to be heated again. to control the imaging unit 40. Specifically, the estimating unit 11 controls the imaging unit 40 so that the object to be heated is imaged a predetermined time after the object to be heated is imaged by the imaging unit 40 last time. That is, until the estimation unit 11 estimates the name of the object to be heated, the imaging unit 40 takes an image of the object to be heated at predetermined time intervals. Therefore, it is possible to suppress heating of the object to be heated with heating intensity and heating time that are inappropriate for the object to be heated.

(2) As described with reference to FIG. 9, the heating control unit 12 determines the remaining heating time for the object to be heated based on the measurement result (heating time Ct) of the measuring unit 13 and the determined heating time Ht. Time Rt was determined. Then, the heating control unit 12 controls the heating unit 30 so that the heating of the object to be heated ends when the remaining heating time Rt elapses after the estimating unit 11 acquires the name information MN. . However, the heating control unit 12 may control the heating unit 30 so that the heating of the object to be heated is completed after a certain period of time has elapsed after the remaining heating time Rt has elapsed.

(3) The order of steps S1 to S10 in FIG. 10 can be changed as appropriate.

(4) The order of steps S21 in FIG. 14 to step S32 in FIG. 15 can be changed as appropriate.

(5) As described with reference to FIGS. 4A to 4C, for one dish, three learning target dish images LG are input to the learned model LM. However, as long as the trained model LM can estimate the name of the object to be heated, one or two learning target food images LG may be learned for one dish, or four or more learning target food images LG may be learned. may be learned.

(6) As described with reference to FIGS. 12 and 13, the cooker 1a according to Embodiment 2 includes the turntable 60. FIG. However, the heating cooker 1a does not have to include the turntable 60 as long as the heating cooker 1a heats the object to be heated. In this case, the correction unit 14 does not correct the imaging result.

(7) As described with reference to FIGS. 12 and 13, the estimating unit 11a of the heating cooker 1a according to the second embodiment calculates the food image HG that is the imaging result and the food image MG stored in the storage unit 20a. and the name of the object to be heated was estimated based on the comparison result. However, as long as the estimating unit 11a can estimate the name of the object to be heated, the estimating unit 11a calculates the difference between the food images HG, which are the results of a plurality of imaging, and the differences between the food images MG stored in the storage unit 20a. The name of the object to be heated may be estimated based on the comparison result. In this case, the difference between the plurality of food images MG stored in the storage unit 20a is the difference between the plurality of food images MG showing the food at different times in the process of heating one food.

INDUSTRIAL APPLICABILITY The present invention can be used in the fields of heat cookers and learned model generation methods.

1 heating cooker 11 estimation unit 12 heating control unit 13 measurement unit 14 correction unit 20 storage unit 30 heating unit 40 imaging unit 50 operation unit 60 turntable HG cooking image Ht heating time Hs heating intensity LM learned model LG learning target cooking image MG Finished image ST State information TD Learning data

Claims (7)

a heating unit that heats an object to be heated;
an imaging unit that captures an image of the object to be heated while the object to be heated is being heated;
an estimating unit for estimating the name of the object to be heated based on the imaging result of the imaging unit while the object to be heated is being heated;
and a heating control unit that controls the heating unit according to the estimation result of the estimation unit. further comprising a storage unit that stores a learned model constructed by machine learning the learning data,
The learning data includes a learning target food image representing a food and name information representing the name of the food,
The estimating unit inputs the food image, which is the imaging result, into the learned model,
The learned model outputs name information indicating the name of the object to be heated based on the input food image,
The heating cooker according to claim 1, wherein the estimating unit acquires the name information output by the learned model. 3. The heating cooker according to claim 2, wherein said learning target dish images of each of said plurality of said learning data respectively represent said dish at different times in the process of heating one said dish. further comprising a storage unit for storing a plurality of food images and the name of the food for one food,
the plurality of food images respectively show the food at different times in the process of heating the food,
The heating cooker according to claim 1, wherein the estimating unit compares the imaging result with the food image, and estimates the name of the object to be heated based on the comparison result. a turntable on which the object to be heated is placed and rotated;
a measurement unit that measures the time the object to be heated is heated;
a correction unit that corrects the orientation of the imaging result based on the angular velocity when the turntable rotates and the measurement result of the measurement unit;
The heating cooker according to claim 4, wherein the estimation unit estimates the name of the object to be heated based on the corrected imaging result. further comprising a storage unit that stores, in association with a name of a dish, a heating time corresponding to the dish and a heating intensity corresponding to the dish;
6. The heating control unit according to any one of claims 1 to 5, wherein the heating control unit determines the heating time and the heating intensity for the object to be heated according to the estimation result of the estimation unit. heating cooker. obtaining training data;
generating a trained model for estimating the name of the object to be heated by performing machine learning on the learning data;
The learned model generation method, wherein the learning data includes a learning target food image representing a food and name information representing a name of the food.

Images