WO2022055030A1

WO2022055030A1 - Artificial intelligence (ai)-based contactless qsc inspection solution system

Info

Publication number: WO2022055030A1
Application number: PCT/KR2020/016797
Authority: WO
Inventors: 박해선; 고형석; 박상록
Original assignee: 주식회사 에이브로스
Priority date: 2020-09-14
Filing date: 2020-11-25
Publication date: 2022-03-17
Also published as: KR102611452B1; KR20220035640A

Abstract

The present invention relates to an artificial intelligence (AI)-based contactless QSC inspection solution system capable of acquiring and analyzing, through a thermal imaging camera for various cooking utensils used for cooking in a kitchen, data such as a method of using the fire on a stove, temperature sensing of a pan or a wok on the stove, a temperature change after adding ingredients, a movement of a cook, and suitability of the ingredients, and through the analyzed data, enabling cooking with a cooking pattern that matches the recipe of a franchise store where a standardized recipe exists, such that the food can be provided with an optimized recipe. The system comprises: a kitchen cooker (200) that includes cooking utensils and a stove providing heat necessary for cooking food; a cooking pattern measuring device (300) that includes a thermal imaging camera, when food to be cooked and a number of servings of the food to be cooked are selected with respect to the food to be cooked in the kitchen cooker (200) and cooking starts, photographs, using the thermal imaging camera, a cooking timing for a cooking pattern including a cooking temperature and a cooking time of the corresponding food, measures time, and reports same to a cooking pattern analysis server (400); and the cooking pattern analysis server (400) that receives cooking pattern data measured by the cooking pattern measurement device (300), compares the measured cooking pattern data with cooking pattern data for each food corresponding to a store recipe for each food pre-stored with respect to the food to be cooked corresponding to the measured cooking pattern data, and transmits a cooking pattern solution according to a comparison result between the measured cooking pattern data and the cooking pattern data for each food to a smart phone of a cook using the cooking pattern measurement device (300) and the kitchen cooker (200) measured by the cooking pattern measurement device (300).

Description

Artificial intelligence (AI) based non-face-to-face QSC inspection solution system

The present invention relates to an artificial intelligence-based non-face-to-face QSC (Quality, Service, Cleanliness) inspection solution. It acquires and analyzes data such as pan and wok temperature sensing, temperature change after material input, chef movement and material suitability, etc. To provide an artificial intelligence (AI)-based non-face-to-face QSC inspection solution system that can provide food with an optimized recipe.

The most basic thing for human beings to enjoy life is to eat, and the most basic of human desires is appetite.

Food to sustain human life has evolved from simply heating it with fire to various cooking methods.

Such a cooking method is the best method for determining the taste and shape of a dish, and unique and unusual cooking recipes are being developed recently.

Korean Patent Application Laid-Open No. 2016-0107719 (Patent Document 1) discloses cooking utensils including a short-distance communication module for real-time transmission; Receive temperature, moisture, salinity, weight, stickiness, and acidity from the cooking utensil in real time, and refer to the real-time received temperature, moisture, salinity, weight, stickiness, and acidity to display the recipe of the food step by step through the speaker (speaker) and a smart terminal that outputs to a display (display), wherein the smart terminal does not stick in each step of the recipe of the food and the temperature, moisture, salinity, weight, acidity is a reference amount within a predetermined time range It is determined whether it is within the range, and as a result of the determination, the smart terminal outputs a voice to guide the temperature, moisture, salinity, weight, and acidity required in the step to be maintained within the time and reference amount range according to the recipe through the speaker, As a result of the determination, if the temperature, moisture, salinity, weight, and acidity required in the corresponding step meet the reference amount range within a predetermined time, the recipe based on the cooking data is configured to audio output the recipe of the next step through the speaker A provision system is disclosed.

Therefore, Patent Document 1 has an advantage in that the implementation of the recipe can be accurately implemented by being configured to feedback to the user through the smart terminal whether the recipe is being properly implemented through the data for each cooking process.

Korean Patent Application Laid-Open No. 2016-0116449 (Patent Document 2) includes the steps of displaying a list of materials received from a service server; updating the holding material list by inputting the changed holding material; and receiving and displaying a recommended recipe searched for from a service server based on a search word input from a user, wherein the recommended recipe is a final matching rate calculated based on a matching ratio of the retained ingredients to the essential ingredients of the recommended recipe A method of driving an application of a user terminal according to is disclosed, by providing an optimal cooking recipe that can be cooked within the ingredients currently held, minimizing the purchase of additional ingredients while ensuring the taste of the user's desired quality, and providing the user with the selected cooking recipe There are advantages to diversifying the types of recipes by evaluating and adding variant cooking recipes.

Also, recently, many stores with standardized recipes are emerging in the form of chain stores (franchises).

For example, McDonald's, Burger King, Hong Kong Banjeom, Gyodong Jjambbong, Jeonju bean sprout soup, etc.

However, in the case of hamburgers, with relatively simple ingredients and ingredients close to finished products, even part-time workers can achieve a certain taste through short-time training. Even if it is a chain store, there was a problem that the taste could be very different from one chain store to another. This is because the order in which the ingredients are added and the timing of non-adjustment play the biggest role, but there is a problem that the head office does not know exactly whether these timings are followed even during busy hours, which could cause customer dissatisfaction.

Therefore, the present invention is to solve the various disadvantages and problems of the prior art as described above, and for various cooking tools used for cooking in the kitchen, the method of using the fire of the cooker through the thermal imaging camera, the pan and the wok on the cooker Optimized by acquiring and analyzing data such as temperature sensing, temperature change after material input, chef movement, and suitability of ingredients, so that you can cook with a cooking pattern that matches the recipe in franchise stores where standardized recipes exist The purpose is to provide an artificial intelligence (AI) based non-face-to-face QSC inspection solution system that can provide recipe food.

In order to achieve the above object, the present invention provides a kitchen cooker 200 including a cooker and a cooker for providing heat necessary for cooking food; A cooking timing for a cooking pattern including a cooking temperature and a cooking time of the food, including a cooking temperature and a cooking time of the food, including a thermal imaging camera a cooking pattern measuring device 300 for photographing with a thermal imaging camera, measuring time, and reporting to the cooking pattern analysis server 400; and receiving the cooking pattern data measured by the cooking pattern measuring device 300 and comparing the cooked food corresponding to the measured cooking pattern data with the cooking pattern data for each food corresponding to the store recipe for each food stored in advance, and the A cook using the kitchen cooker 200 measured by the cooking pattern measuring device 300 and the cooking pattern measuring device 300 for a cooking pattern solution according to a comparison result of the measured cooking pattern data and the cooking pattern data for each food. It provides an artificial intelligence (AI)-based non-face-to-face QSC inspection solution system, characterized in that it includes; a cooking pattern analysis server 400 that is transmitted to a smartphone.

Here, the comparison between the measured cooking pattern data and the pre-stored cooking pattern data for each food uses a CNN algorithm, and the cooking pattern solution according to the comparison result uses an RNN algorithm.

In addition, the cooking pattern analysis apparatus 300 includes a thermal imaging camera for photographing a thermal image of a cooker, a thermal imaging camera 310 for photographing a thermal image from a cooking tool, a selection button unit 320 for the cooked food, A cooking pattern data collection unit 340 that collects thermal image data of the thermal imaging camera 310, photographing data or detection data from the selection button unit, and recipe data of cooked food are stored, and the cooking pattern data is collected A memory unit 350 in which the data collected in the unit 340 is stored, and a cooking pattern comparison unit ( 360), a communication unit 370 for transmitting cooking pattern data for cooked food collected by the cooking pattern data collection unit 340 to an external cooking pattern analysis server 400, and the plurality of thermal imaging cameras 310 and a control unit 380 for controlling the selection button unit 320, the cooking pattern data collection unit 340, the memory unit 350, the cooking pattern comparison unit 360, and the communication unit 370. do it with

Meanwhile, the cooking pattern analysis server 400 includes a communication unit 410 that communicates with the cooking pattern analysis device 300 through the communication network 500, and cooking pattern information for each food, member company information, member company kitchen cooker information, member company cooking pattern A database 420 in which analysis device information and member company cooking pattern data information are stored, a cooking pattern data collection unit 430 collecting cooking pattern data of a cook from the cooking pattern analysis device 300, Cooking pattern data classification unit 440 for classifying the section and time for each food through the CNN algorithm, and the cooking pattern information for each food in the database 420, the result classified through the CNN algorithm in the cooking pattern data classification unit 440 A cooking pattern data comparison unit 450 that extracts solution data, which is a solution, by predicting risk occurrence and differential discrimination through the RNN algorithm in comparison with the cooking pattern data comparison unit 450, and the solution data extracted from the cooking pattern data comparison unit 450 for corresponding cooking The pattern measuring device 300, the cooking pattern solution providing unit 460 and the communication unit 410, the database 420, the cooking pattern data collection unit 430, and the cooking pattern data collection unit 430 that transmit to the smart phone of the corresponding cook also compare It is characterized in that it is configured to include a control unit 490 for controlling the unit 440, the cooking pattern data determining unit 450 and the cooking pattern neural network (CNN) analysis unit 460 .

According to the present invention, there are the following effects.

First, it is possible to analyze the cooking patterns of the cooks by introducing a thermal imaging camera to food cooked using strong heat, acquiring cooking-related data of each cook, and comparing it with predetermined optimized cooking pattern data.

Second, it is possible to provide a non-face-to-face QSC (Quality, Service, Cleanliness) check solution using a CNN (Convolutional Neural Network) neural network based on artificial intelligence for a cook with a problem through comparison with a predetermined optimized cooking pattern. .

Third, by measuring the temperature of the cook using a thermal imaging camera, it is possible to prevent the recently prevalent Corona 19, and to check the health or rest time of the cook.

1 is a photograph showing object recognition through a thermal imaging camera.

2 and 3 are block diagrams for explaining the artificial intelligence (AI) based non-face-to-face QSC inspection solution system according to the first embodiment of the present invention.

4 is a block diagram illustrating an artificial intelligence (AI)-based non-face-to-face QSC inspection solution system according to a second embodiment of the present invention.

5 to 7 are diagrams for explaining cooking timing and solutions using a thermal imaging camera in an artificial intelligence (AI)-based non-face-to-face QSC inspection solution system according to the present invention.

A preferred embodiment of the present invention will be described in detail with reference to the accompanying drawings as follows.

In addition, the terms used in the present invention have been selected as widely used general terms as possible, but in specific cases, there are also terms arbitrarily selected by the applicant, and in this case, the meaning is described in detail in the description of the corresponding invention, so it is a simple term It is intended to clarify that the present invention should be understood as the meaning of the term, not the name. In addition, in describing the embodiments, descriptions of technical contents that are well known in the technical field to which the present invention pertains and are not directly related to the present invention will be omitted. This is to more clearly convey the gist of the present invention without obscuring the gist of the present invention by omitting unnecessary description.

1 is a photograph showing object recognition through a thermal imaging camera, and FIGS. 2 and 3 are block diagrams for explaining an artificial intelligence (AI) based non-face-to-face QSC inspection solution system according to the first embodiment of the present invention.

The artificial intelligence (AI)-based non-face-to-face QSC inspection solution system according to the first embodiment of the present invention uses object recognition using a thermal imaging camera as shown in FIG. The purpose of this is to provide a non-face-to-face QSC inspection solution based on artificial intelligence (AI) through measuring the temperature generated during the cooking process so that the cook can know whether the cook is cooking according to the standardized recipe.

For this purpose, the artificial intelligence (AI)-based non-face-to-face QSC inspection solution system according to the first embodiment of the present invention is a POS terminal 100, a kitchen cooker 200, and a cooking pattern measuring device ( 300) consists of

Here, the POS terminal 100 processes an order for food and performs payment.

The kitchen cooker 200 includes a cooking pot and cooking utensils (wok, frying pan, etc.) that provide thermal power necessary for cooking food.

The cooking pattern measuring device 300 includes a thermal imaging camera, and when the cooked food and cooked food are selected for the food cooked in the kitchen cooker 200 and cooking starts, the cooking temperature and cooking time of the corresponding food are included. The cooking timing for the cooking pattern is measured with a thermal imaging camera and time is compared with the cooking pattern of the pre-stored recipe so that the cook knows if it deviates from the preset value, and reports it to the cooking pattern analysis server of the head office. .

To this end, as shown in FIG. 3 , the cooking pattern measuring device 300 includes a plurality of thermal imaging cameras 310 , a selection button unit 320 including a cooked food selection button and a cooked serving selection button, and an electronic scale ( 330 ), an image camera 340 , a cooking pattern data collection unit 340 , a memory unit 350 , a cooking pattern comparison unit 360 , a communication unit 370 , and a control unit 380 .

Here, the plurality of thermal imaging cameras 310 include a thermal imaging camera (first thermal imaging camera) that takes a thermal image of a cooking sphere, and a thermal imaging camera (second thermal imaging) of a cooking tool, such as a wok or frying pan. camera), a thermal imaging camera that takes a thermal image of the cook's face (n-1 thermal imaging camera), and a thermal imaging camera that takes a thermal image of the cook's front (n-th thermal imaging camera). At this time, by photographing the cook's face, the cook can detect body abnormalities including the recent (2020) epidemic Corona 19 or the flu to ensure the safety of the cook and the customer, and take a thermal image of the cook's front. It is possible to detect whether a rest is necessary according to the health condition of the cook.

The selection button unit 320 including a cooked food selection button and a cooked serving selection button selects the type of cooked food and selects the number of servings.

Meanwhile, the electronic scale 330 may detect whether or not ingredients of a preset weight are properly contained according to the recipe of the corresponding cooked food.

The video camera 340 is installed on the upper surface of the kitchen cooker 200 to photograph the cooking process of the cooked food.

The cooking pattern data collection unit 340 collects thermal image data of a plurality of thermal imaging cameras 310 and photographing data or detection data from a cooked food selection button, a cooked serving selection button, an electronic scale, and the like.

The memory unit 350 stores recipe data of cooked food, and the collected data is stored in the cooking pattern data collection unit 340 .

The cooking pattern comparison unit 360 compares the cooking pattern data (cooking temperature, cooking time, etc. by the thermal imaging camera for each cooked food) collected by the cooking pattern data collection unit 340 with the recipe data of the memory unit 350 . . This will be described in more detail with reference to FIGS. 5 to 7,

The cooking pattern data collected by the cooking pattern data collection unit 340 through the communication unit 370 is transmitted to an external cooking pattern analysis server.

The control unit 380 includes a plurality of thermal imaging cameras 310 , a selection button unit 320 including a cooked food selection button and a cooked serving selection button, an electronic scale 330 , an image camera 340 , and cooking pattern data. It controls the collection unit 340 , the memory unit 350 , the cooking pattern comparison unit 360 , and the communication unit 370 , and provides time information when thermal images are taken by a plurality of thermal imaging cameras 310 . Of course, a separate timer unit for providing time information may be configured.

Meanwhile, an alarm unit 391 , a cooking pattern display unit 392 , and a cook input unit 393 may be further configured.

The alarm unit 391 sends an alarm to stop cooking when an infectious disease such as Corona 19, THAAD, influenza, cold, etc. Notifying, the cooking pattern display unit 392 allows the cook to compare the cooking pattern he or she cooks with the cooking pattern according to the recipe. Accordingly, the cook can change his or her wrong cooking habits, etc.

The cook input unit 393 is for distinguishing cooks when there are several cooks, and in this case, when there are also several kitchen cookers (plural units), the cooker's kitchen cooker also assigns an identification symbol (ID) to each cook. .

As shown in FIG. 4 , the artificial intelligence (AI)-based non-face-to-face QSC inspection solution system according to the second embodiment of the present invention is transmitted from the cooking pattern analysis device 300 through the communication network 500 in the cooking pattern analysis server 400 . This is to collect and analyze cooking pattern data.

To this end, the cooking pattern analysis server 400 includes a communication unit 410, a database 420, a cooking pattern data collection unit 430, a cooking pattern data classification unit 440, a cooking pattern data comparison unit 450, and a cooking pattern solution product. It is configured to include a study 460 , a cook body temperature analysis unit 470 , a cook health alarm unit 480 , and a control unit 490 .

The communication unit 410 communicates with the cooking pattern analysis apparatus 300 through the communication network 500 .

The database 420 includes food-specific cooking pattern information, member company information (industry, smartphone number, etc.), member company kitchen cooker information, member company cooking pattern analysis device information, member company cooking pattern data information, and chef body temperature information (before cooking, during cooking). etc are stored. also

The cooking pattern data collection unit 430 collects the cooking pattern data of the cook from the cooking pattern analysis apparatus 300 .

The cooking pattern data classification unit 440 is for classifying sections and times for cooking patterns for each food. For example, when heat is generated in the crater, it starts shooting, and stages S1, S2. ..Sn) are recorded and classified. For this, a CNN (Convolution Neural Network) algorithm is used.

The cooking pattern data comparison unit 450 compares the classification result according to the data learned through CNN in the cooking pattern data classification unit 440 with the cooking pattern information for each food in the database to predict risk occurrence and solve the difference through discrimination , that is, extract the solution data. For this, a Recurrent Neural Network (RNN) algorithm is used.

The cooking pattern solution providing unit 460 transmits the solution data extracted from the cooking pattern data comparing unit 450 to the corresponding cooking pattern measuring device 300 . In this case, by referring to the database, the fact that the solution data is transmitted to the manager terminal of the store may be transmitted as a text message, and the solution data may also be transmitted to the smart phone of the corresponding cook.

The cook's body temperature analysis unit 470 analyzes the cook's facial temperature and overall temperature detected by the thermal imaging camera, and when the set temperature is higher than the set temperature and the duration exceeds (eg, the facial temperature at the start of cooking is 38 degrees or higher, or , when the total body temperature is 40 degrees or higher for more than 30 minutes) Through the cook health alarm unit 480, the cooking stop alarm and break alarm are notified to induce customer safety for cooked food, health safety of the cook, etc. there is.

The control unit 490 includes a communication unit 410, a database 420, a cooking pattern data collection unit 430, a cooking pattern data comparison unit 440, a cooking pattern data determination unit 450, a cooking pattern neural network (CNN) analysis unit ( 460), the cook body temperature analysis unit 470 and the cook health alarm unit 480 are controlled,

5 is, for example, a cooking pattern according to the standard cooking timing of Champon determined by the head office, FIG. 6 is a cooking pattern through a thermal imaging camera and a timer when the cook cooks Champon, and FIG. 7 is a cooking pattern to be provided to the cook. To show an example of a solution, an artificial intelligence CNN engine is applied for image classification and definition of data acquired through a thermal imaging camera to derive data through image learning of a cook's cooking behavior, and through contrast with a reference image In case of analysis, more accurate problem identification and advice are possible.

Although the present invention has been described with the above examples, the present invention is not necessarily limited to these examples, and various modifications may be made within the scope without departing from the technical spirit of the present invention. Therefore, the examples disclosed in the present invention are not intended to limit the technical spirit of the present invention, but to explain, and the scope of the technical spirit of the present invention is not limited by these examples. The protection scope of the present invention should be interpreted by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present invention.

For various cooking tools used for cooking in the kitchen, data such as the method of using the fire on the stove, the temperature detection of the fan and the wok on the stove, the temperature change after input of ingredients, the movement of the cook and the suitability of the ingredients are acquired through the thermal imaging camera. An AI-based non-face-to-face QSC inspection solution system that can provide food with an optimized recipe by analyzing and analyzing can provide

Claims

A kitchen cooker 200 including a cooker and a cooker that provides heat necessary for cooking food;

A cooking timing for a cooking pattern including a cooking temperature and a cooking time of the food, including a cooking temperature and a cooking time of the food, including a thermal imaging camera a cooking pattern measuring device 300 for photographing with a thermal imaging camera, measuring time, and reporting to the cooking pattern analysis server 400; and

Receives the cooking pattern data measured by the cooking pattern measuring device 300 and compares the cooked food corresponding to the measured cooking pattern data with the cooking pattern data for each food corresponding to the store recipe for each food stored in advance, and the measurement The cooking pattern solution according to the comparison result of the prepared cooking pattern data and the cooking pattern data for each food is measured by the cooking pattern measuring device 300 and the cooking pattern measuring device 300. The cook using the kitchen cooker 200 is smart. Artificial intelligence (AI)-based non-face-to-face QSC inspection solution system, characterized in that it includes;
The method according to claim 1,

The comparison between the measured cooking pattern data and the pre-stored cooking pattern data for each food uses a CNN algorithm,

An artificial intelligence (AI) based non-face-to-face QSC inspection solution system, characterized in that the cooking pattern solution according to the comparison result uses an RNN algorithm.
The method according to claim 1,

The cooking pattern analysis device 300,

A thermal imaging camera that takes a thermal image of the cookware, a thermal imaging camera 310 that takes a thermal image of a cooking tool,

a selection button unit 320 for the cooked food;

a cooking pattern data collection unit 340 for collecting thermal image data of the thermal imaging camera 310 and photographing data or detection data from the selection button unit;

A memory unit 350 in which recipe data of cooked food is stored and the data collected in the cooking pattern data collection unit 340 is stored;

a cooking pattern comparison unit 360 that compares the cooking pattern data collected by the cooking pattern data collection unit 340 with the recipe data of the memory unit 350;

A communication unit 370 for transmitting the cooking pattern data for the cooked food collected by the cooking pattern data collection unit 340 to the external cooking pattern analysis server 400, and

A control unit for controlling the plurality of thermal imaging cameras 310 , a selection button unit 320 , a cooking pattern data collection unit 340 , a memory unit 350 , a cooking pattern comparison unit 360 , and a communication unit 370 . (380) Artificial intelligence (AI)-based non-face-to-face QSC inspection solution system, characterized in that it is configured to include.
The method according to claim 1,

The cooking pattern analysis server 400,

A communication unit 410 that communicates with the cooking pattern analysis device 300 through the communication network 500, cooking pattern information for each food, member company information, member company kitchen cooker information, member company cooking pattern analysis device information, and member company cooking pattern data information The stored database 420, the cooking pattern data collection unit 430 that collects the cooking pattern data of the cook from the cooking pattern analysis device 300, and the section and time for each food cooking pattern are classified through a CNN algorithm. A cooking pattern data classifying unit 440 that compares the results classified through the CNN algorithm in the cooking pattern data classifying unit 440 with the cooking pattern information for each food in the database 420 to determine risk occurrence through the RNN algorithm. A cooking pattern data comparison unit 450 for extracting solution data as a solution through prediction and difference discrimination, and a corresponding cooking pattern measuring device 300 for the solution data extracted from the cooking pattern data comparison unit 450, and a corresponding cook The cooking pattern solution providing unit 460 and the communication unit 410, the database 420, the cooking pattern data collection unit 430, the cooking pattern data comparison unit 440, and the cooking pattern data determination unit that also transmit to the smartphone of (450) Artificial intelligence (AI)-based non-face-to-face QSC inspection solution system, characterized in that it comprises a control unit 490 for controlling the cooking pattern neural network (CNN) analysis unit 460.