CN113095897A - Recipe-based food material pushing method, refrigerator, system and storage medium - Google Patents
Recipe-based food material pushing method, refrigerator, system and storage medium Download PDFInfo
- Publication number
- CN113095897A CN113095897A CN202010019304.8A CN202010019304A CN113095897A CN 113095897 A CN113095897 A CN 113095897A CN 202010019304 A CN202010019304 A CN 202010019304A CN 113095897 A CN113095897 A CN 113095897A
- Authority
- CN
- China
- Prior art keywords
- food material
- refrigerator
- weight
- food
- existing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 235000013305 food Nutrition 0.000 title claims abstract description 527
- 239000000463 material Substances 0.000 title claims abstract description 515
- 238000000034 method Methods 0.000 title claims abstract description 50
- 238000003860 storage Methods 0.000 title claims abstract description 16
- 238000004590 computer program Methods 0.000 claims description 12
- 238000012545 processing Methods 0.000 claims description 7
- 238000003709 image segmentation Methods 0.000 claims description 6
- 238000003702 image correction Methods 0.000 claims description 4
- 238000012216 screening Methods 0.000 claims description 3
- 238000004891 communication Methods 0.000 claims description 2
- 230000008569 process Effects 0.000 description 10
- 239000000843 powder Substances 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 230000008054 signal transmission Effects 0.000 description 5
- 244000291564 Allium cepa Species 0.000 description 3
- 244000273928 Zingiber officinale Species 0.000 description 3
- 235000006886 Zingiber officinale Nutrition 0.000 description 3
- 235000008397 ginger Nutrition 0.000 description 3
- 235000015277 pork Nutrition 0.000 description 3
- 235000007516 Chrysanthemum Nutrition 0.000 description 2
- 244000067456 Chrysanthemum coronarium Species 0.000 description 2
- 235000007871 Chrysanthemum coronarium Nutrition 0.000 description 2
- 244000189548 Chrysanthemum x morifolium Species 0.000 description 2
- 244000000626 Daucus carota Species 0.000 description 2
- 235000002767 Daucus carota Nutrition 0.000 description 2
- 235000003283 Pachira macrocarpa Nutrition 0.000 description 2
- 240000001085 Trapa natans Species 0.000 description 2
- 235000014364 Trapa natans Nutrition 0.000 description 2
- 235000008429 bread Nutrition 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 235000021186 dishes Nutrition 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 235000009165 saligot Nutrition 0.000 description 2
- 238000005303 weighing Methods 0.000 description 2
- 241000272525 Anas platyrhynchos Species 0.000 description 1
- 101100279436 Caenorhabditis elegans egg-2 gene Proteins 0.000 description 1
- 101100279442 Caenorhabditis elegans egg-6 gene Proteins 0.000 description 1
- 241000722731 Carex Species 0.000 description 1
- 235000015241 bacon Nutrition 0.000 description 1
- 239000008280 blood Substances 0.000 description 1
- 210000004369 blood Anatomy 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000010411 cooking Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000005520 cutting process Methods 0.000 description 1
- 235000013601 eggs Nutrition 0.000 description 1
- 238000005265 energy consumption Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 239000004615 ingredient Substances 0.000 description 1
- 238000005304 joining Methods 0.000 description 1
- 238000003064 k means clustering Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000011426 transformation method Methods 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/06—Buying, selling or leasing transactions
- G06Q30/0601—Electronic shopping [e-shopping]
- G06Q30/0631—Item recommendations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/06—Buying, selling or leasing transactions
- G06Q30/0601—Electronic shopping [e-shopping]
- G06Q30/0623—Item investigation
- G06Q30/0625—Directed, with specific intent or strategy
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/68—Food, e.g. fruit or vegetables
Abstract
The application relates to the field of smart home, and discloses a recipe-based food material pushing method, a refrigerator, a food material pushing system and a storage medium, wherein a camera is arranged in the refrigerator, and the method comprises the following steps: acquiring an image of the refrigerator food material acquired by the camera, and performing image recognition on the refrigerator food material to obtain the refrigerator food material; comparing the recipe food materials with the refrigerator food materials to obtain missing food materials and existing food materials; acquiring the weight of the existing food material, and calculating the difference value between the weight of the existing food material and the weight of the recipe food material; and determining the missing weight of the existing food material according to the difference, and sending the existing food material, the missing weight of the existing food material and the missing food material to a user.
Description
Technical Field
The application relates to the field of information extraction, in particular to a recipe-based food material pushing method, a refrigerator, a recipe-based food material pushing system and a storage medium.
Background
At present, some intelligent refrigerators in the market can recommend related recipes according to the identified food materials by performing image recognition on the food materials in the refrigerator, so that a user can cook the existing food materials, and the use of the user is facilitated.
However, after the user selects the recipe in advance, if the user cannot accurately memorize the existing food materials in the refrigerator, the user needs to purchase the food materials again according to the recipe before cooking. When part of food materials required in the menu selected by the user exist in the refrigerator, the food materials are wasted.
Therefore, how to enable the refrigerator to identify the missing food materials according to the menu and push the food materials to the user becomes a problem to be solved urgently.
Disclosure of Invention
The application provides a recipe-based food material pushing method, a refrigerator, a food material pushing system and a storage medium, so that the refrigerator can push missing food materials to a user according to the recipe to remind the user of purchasing the food materials.
In a first aspect, the application provides a recipe-based food material pushing method, a camera is arranged in a refrigerator, and the method includes:
acquiring a refrigerator food material image acquired by the camera, and performing image recognition on the refrigerator food material image to obtain a refrigerator food material;
comparing the recipe food materials with the refrigerator food materials to obtain missing food materials and existing food materials;
acquiring the weight of the existing food material, and calculating the difference value between the weight of the existing food material and the weight of the recipe food material;
and determining the missing weight of the existing food material according to the difference, and sending the existing food material, the missing weight of the existing food material and the missing food material to a user.
In a second aspect, the present application also provides a refrigerator, comprising: the device comprises a camera, a support tray, an electronic scale, a memory and a processor;
the camera is used for collecting refrigerator food material images in the refrigerator;
the supporting tray and the electronic scale are used for collecting the weight of refrigerator food materials in the refrigerator;
the memory is used for storing a computer program;
the processor is configured to execute the computer program and to implement the recipe-based food material pushing method according to the first aspect when the computer program is executed.
In a third aspect, the present application further provides a recipe-based food material pushing system, where the system includes: the refrigerator comprises a user terminal and a refrigerator, wherein the user terminal is in communication connection with the refrigerator;
the user terminal is used for sending menu information to the refrigerator;
the refrigerator is used for realizing the recipe-based food material pushing method according to the recipe information.
In a fourth aspect, the present application further provides a computer-readable storage medium storing a computer program, which when executed by a processor causes the processor to implement the recipe-based food material pushing method as described above.
The application discloses a recipe-based food pushing method, a refrigerator, a recipe-based food pushing system and a storage medium, food images in the refrigerator are collected through a camera arranged in the refrigerator, the refrigerator food images are subjected to image recognition to obtain refrigerator food, then the recipe food is compared with the refrigerator food to obtain missing food and existing food, a difference value between the weight of the existing food and the weight of the recipe food is calculated according to the weight of the existing food, the missing weight of the existing food is determined according to the difference value, the existing food, the missing weight of the existing food and the missing food are sent to a user, and the user is reminded of purchasing the food. The user selects the menu, the refrigerator judges whether food materials in the refrigerator meet the menu requirements or not according to the menu content, if not, the user is reminded to purchase the dishes, the refrigerator can automatically push the food materials needing to be purchased to the user according to the menu, and the use experience of the user is improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a schematic diagram of a recipe-based food material pushing system provided by the present application;
fig. 2 is a schematic structural view of a refrigerator;
fig. 3 is a schematic flow chart of a recipe-based food material pushing method according to an embodiment of the present application;
fig. 4 is a schematic flowchart illustrating a step of performing image recognition on the refrigerator food material to obtain the refrigerator food material in the embodiment of the present application;
FIG. 5 is a schematic flowchart of the steps for comparing recipe food materials with refrigerator food materials in the embodiment of the present application;
FIG. 6 is a schematic flow chart illustrating steps for obtaining the weight of an existing food material in the embodiment of the present application;
FIG. 7 is a numbered schematic view of a support tray in an embodiment of the present application;
fig. 8 is a schematic flow chart of another recipe-based food material pushing method according to the embodiment of the present application;
fig. 9 is a schematic block diagram of a structure of a refrigerator according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some, but not all, embodiments of the present application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The flow diagrams depicted in the figures are merely illustrative and do not necessarily include all of the elements and operations/steps, nor do they necessarily have to be performed in the order depicted. For example, some operations/steps may be decomposed, combined or partially combined, so that the actual execution sequence may be changed according to the actual situation.
It is to be understood that the terminology used in the description of the present application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in the specification of the present application and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
It should also be understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.
Referring to fig. 1, fig. 1 is a schematic view of a recipe-based food material pushing system according to the present application. The recipe-based food material pushing system in the present application will be described in detail below with reference to fig. 1.
As shown in fig. 1, the recipe-based food material push system 100 includes a refrigerator 110 and a user terminal 120. The refrigerator 110 is in signal connection with the user terminal 120, and specifically, the refrigerator 110 may be connected with the user terminal 120 through WiFi, bluetooth, internet of things, and the like.
The user terminal 120 may be an electronic device such as a mobile phone, a tablet computer, a notebook computer, a desktop computer, a personal digital assistant, and a wearable device.
The user terminal 120 is configured to present a menu list to the user, transmit a menu selected by the user to the refrigerator 110, and receive the missing food material and the weight of the missing food material transmitted by the refrigerator 110. For convenience of description, the user terminal 120 in the present application is a mobile phone.
After the user selects a menu on the user terminal 120, the user terminal 120 sends the menu selected by the user to the refrigerator, the refrigerator compares the menu food materials in the menu with the refrigerator food materials in the refrigerator to obtain the missing food material and the existing food material, obtains the weight of the existing food material after obtaining the existing food material, calculates the difference between the weight of the existing food material and the weight of the menu food material, and determines the missing weight of the existing food material according to the difference, for example, if the difference is a negative number, it is determined that the weight of the existing food material is smaller than the weight of the menu food material, the absolute value of the difference is the missing weight of the existing food material, and the existing food material, the missing weight and the missing food material are sent to the user terminal 120 together to remind the user of purchasing.
As shown in fig. 2, it is a schematic structural diagram of the refrigerator. The refrigerator 110 includes a refrigerator body 111 and a control device 112. The refrigerator body 111 is also provided therein with a camera 113, a support tray 114 and an electronic scale 115.
The camera 113 is arranged in the refrigerator body, the camera 113 is used for collecting food material images of food materials placed in the refrigerator, in the specific implementation process, the camera 113 can be arranged on a door of the refrigerator body so as to collect the food material images in the refrigerator, and the number of the cameras 113 can be one or multiple.
The supporting tray 114 is also arranged in the refrigerator body 111, a plurality of food material placing positions are divided on the supporting tray 114, an electronic scale 115 is arranged at the bottom of each food material placing position, the supporting tray 114 is used for placing food materials, and the electronic scale 115 is used for weighing the weight of the food materials placed on the supporting tray 114. In a specific implementation process, the supporting tray 114 may be divided into a plurality of grids, each grid is used as a food material placing position, and an electronic scale 115 is correspondingly arranged in each grid. Only one food material can be placed in each grid, so that the electronic scale 115 can weigh the weight of each food material, and the accuracy of weighing results is improved.
When the food material placing positions are divided, the supporting trays 114 can be equally divided, and the supporting trays 114 can be irregularly divided, so that the food material placing positions can be adapted to the sizes of different food materials.
The control device 112 includes an image recognition module, a processor, and a signal transmission module. The image recognition module is in signal connection with the camera 113 and is used for receiving food material images in the refrigerator collected by the camera 113 and carrying out image recognition on the food material images to obtain recognition results, namely refrigerator food materials in the refrigerator.
The processor is respectively communicated with the image identification module, the signal transmission module and the electronic scale 115, and the signal transmission module is used for receiving menu information sent by the user terminal and sending the menu information to the processor; the image recognition module sends the refrigerator food material inside the refrigerator to the processor, and at the same time, the electronic scale 115 sends the reading of the weight of the refrigerator food material to the processor. The processor compares the refrigerator food material and weight in the refrigerator with the recipe food material and weight in the recipe information to obtain the missing food material and the missing weight of the existing food material, and sends the obtained missing food material and the missing weight of the existing food material to the user terminal 120 through the signal transmission module.
The Processor may be a Central Processing Unit (CPU), or may be other general-purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA) or other Programmable logic device, a discrete Gate or transistor logic device, a discrete hardware component, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
In some embodiments, the control device 112 may also include a control module. When the user terminal 120 sends the menu selected by the user to the refrigerator 110, the control module controls the camera 113 to be turned on according to the received menu, and controls the camera 113 to be turned off after the refrigerator sends the missing food material and the missing weight of the existing food material to the user terminal 120. To reduce the power consumption of the camera 113.
It is understood that the recipe-based food material pushing system in fig. 1 and the above-mentioned naming of the components of the recipe-based food material pushing system are only for identification purposes, and do not limit the embodiments of the present application.
The recipe-based food material pushing method provided by the embodiment of the present application will be described in detail below based on the recipe-based food material pushing system in fig. 1.
Referring to fig. 3, fig. 3 is a schematic flow chart of a recipe-based food material pushing method according to an embodiment of the present application. The food material pushing method based on the menu can be applied to a refrigerator in a food material pushing system based on the menu, and by comparing the type and weight of the existing food materials in the refrigerator with the type and weight of the food materials in the menu, a user is reminded to purchase the lack of food materials, the food materials can be pushed to the user according to the menu, and the use of the user is facilitated.
Some embodiments of the present application will be described in detail below with reference to the accompanying drawings. The embodiments described below and the features of the embodiments can be combined with each other without conflict.
As shown in fig. 3, the recipe-based food material pushing method specifically includes: step S201 to step S204.
S201, acquiring the refrigerator food material image acquired by the camera, and performing image recognition on the refrigerator food material image to obtain the refrigerator food material.
The refrigerator food material image refers to a food material image of existing food materials in the refrigerator shot by a camera. The refrigerator acquires food material images of existing food materials in the refrigerator, which are acquired by the camera, and then performs image recognition on the food material images of the refrigerator, so that the food material types of the existing food materials in the refrigerator, namely the refrigerator food materials, are obtained.
In some embodiments, the recipe-based food material pushing method further includes: receiving menu information sent by a user terminal, and starting a camera according to the menu information.
Specifically, a user selects a menu through a user terminal, the menu selected by the user is sent to the refrigerator through the user terminal, and the refrigerator starts a camera according to received menu information so that the camera collects refrigerator food material images in the refrigerator. When the refrigerator receives menu information, the camera is opened, the camera is prevented from being normally opened, and the energy consumption of the camera and the refrigerator is reduced.
In some embodiments, the number of the cameras is multiple, the multiple cameras respectively collect multiple refrigerator food material images, please refer to fig. 4, and image recognition is performed on the refrigerator food material images to obtain the refrigerator food material, which specifically includes the following steps:
and S2011, respectively carrying out image correction on the plurality of refrigerator food material images to obtain a plurality of corrected refrigerator food material images.
Specifically, in order to enable the cameras to comprehensively shoot food material images in the refrigerator, the cameras can be arranged in a plurality of positions in the refrigerator respectively, each camera shoots partial areas in the refrigerator respectively, and then a plurality of images are spliced into a complete image by using an image splicing technology, so that the purpose of full coverage is achieved. For example, when there are multiple layers in the refrigerator, a camera may be disposed at each layer of the refrigerator for capturing an image of the refrigerator food material.
Because different cameras set up the difference in position, consequently the camera shooting angle is also different, and then leads to the different geometric deformation direction in the material image of a plurality of refrigerators that different cameras shot, in order to realize the image concatenation of material image of refrigerator, need carry out geometric correction respectively to material image of each refrigerator earlier, rectify the rectangle image that the camera was shot into trapezoidal image, carry out the image concatenation with other refrigerator material images after the cutting.
In a specific implementation process, the refrigerator food material image can be subjected to image deformation in a projection transformation mode, so that the refrigerator food material image is corrected.
S2012, image splicing is carried out on the corrected refrigerator food material images to obtain spliced refrigerator food material images.
Specifically, when a plurality of corrected refrigerator food material images are subjected to image splicing, matching of the images can be achieved by adopting a coordinate transformation method, and splicing is completed.
In some embodiments, for the overlapping area of the two refrigerator food material images, because the images are shot by different cameras at different times and different positions, and the images of the overlapping area have differences in brightness under the influence of light, a smooth transition method can be adopted to eliminate discontinuity at the connection position of the two refrigerator food material images.
S2013, performing image segmentation on the spliced refrigerator food material images to obtain images of the refrigerator food materials in the refrigerator food material images.
Because the spliced refrigerator food material images comprise various food materials in the refrigerator, the refrigerator food material images can be subjected to image segmentation to extract an individual image of each food material.
In a specific implementation process, the image may be segmented based on the texture features and the color features, specifically, the food materials of different colors are segmented by using the color information of the image, and then the food materials of different textures are segmented by using the texture information.
Firstly, extracting a texture characteristic image and a color characteristic image of an image, then carrying out image characteristic dimension reduction processing on the extracted texture characteristic image and color characteristic image to obtain a characteristic vector set of the image, finally obtaining a clustering center by using a clustering algorithm, and segmenting the image according to the characteristic value of the clustering center.
The color characteristic image is an image obtained by filtering and removing texture information, and the texture characteristic image is an image obtained by performing a Gaussian offset difference filter. In the image feature dimension reduction processing of the image, a PCA algorithm may be used. In clustering, a K-means clustering method may be used.
S2014, respectively carrying out image recognition on the images of the refrigerator food materials in the refrigerator food material images to obtain the refrigerator food materials included in the refrigerator food material images.
Specifically, when image recognition is performed, images may be firstly classified according to the extracted color features, then images of the same color are secondly classified according to texture features of the images, and finally recognition of different types of food materials is completed.
S202, comparing the menu food materials with the refrigerator food materials to obtain the missing food materials and the existing food materials.
The lack of food materials refers to food materials which are not available in the refrigerator and are needed in the menu, and the existing food materials refer to food materials which are available in the refrigerator and are also needed in the menu. Specifically, after receiving the recipe information, the refrigerator analyzes the received recipe information, thereby obtaining recipe food materials and recipe food material weights included in the recipe information. And then comparing the refrigerator food materials obtained through image recognition with the menu food materials, so as to obtain the missing food materials and the existing food materials.
In some embodiments, referring to fig. 5, comparing the recipe food material with the refrigerator food material may specifically include step S2021 and step S2022.
S2021, connecting the menu food materials with the refrigerator food materials at the left side. S2022, screening the recipe food materials according to the refrigerator food materials which are empty values.
The left connection means that all records of the left table (namely, the recipe food material) are displayed, and only records corresponding to the left table content are displayed in the right table (namely, the refrigerator food material). The positions where the right table records are insufficient, i.e. name matching is unsuccessful, are all shown as NULL. In a specific implementation process, the lack of food materials refers to the screened menu food materials, and the existing food materials refer to other food materials except the lack of food materials in the menu food materials.
For example, the recipe food materials are shown in table 1, and the refrigerator food materials are shown in table 2.
TABLE 1
| Serial number | Menu food material |
| 1 | Pork |
| 2 | Water chestnut |
| 3 | Carrot (Carex sativus L.) |
| 4 | Steamed bread |
| 5 | Egg |
| 6 | Scallion powder |
| 7 | Ginger powder |
TABLE 2
| Serial number | Refrigerator food material |
| 1 | Egg |
| 2 | Scallion powder |
| 3 | Ginger powder |
| 4 | Toast slice |
| 5 | Crowndaisy chrysanthemum |
| 6 | Mutton |
| 7 | Bacon |
The results obtained by left-joining the recipe food material and the refrigerator food material are shown in table 3:
table 3:
the food material with the empty value in the refrigerator food materials in the menu food materials is taken as the lack food material, and the food materials except the lack food material in the menu food materials are taken as the existing food materials. That is, the existing food materials are: eggs, scallion powder and ginger powder, and the lacking food materials are pork, water chestnut, carrot and steamed bread.
S203, acquiring the weight of the existing food material, and calculating the difference value between the weight of the existing food material and the weight of the recipe food material.
Specifically, after the existing food material and the missing food material are obtained, the weight of the existing food material is required to be obtained, so that whether the existing food material meets the recipe requirement or not is judged according to the obtained weight of the existing food material.
In a specific implementation process, a difference between the existing food material weight and the recipe food material weight is calculated, specifically, the difference is obtained by subtracting the recipe food material weight from the existing food material weight.
In some embodiments, a supporting tray may be further disposed in the refrigerator, the supporting tray is provided with a plurality of food material placing positions, each food material placing position is correspondingly provided with an electronic scale, please refer to fig. 6, and the step of obtaining the weight of the existing food material specifically includes steps S2031 to S2033.
S2031, obtaining the image of the existing food material.
Specifically, when the refrigerator food material is subjected to image recognition, the refrigerator food material image is subjected to image segmentation, and an image of each refrigerator food material in the refrigerator food material image is obtained. Then, the image of the existing food material is obtained from the image of each refrigerator food material, so that the placement position of the existing food material in the refrigerator can be determined conveniently.
S2032, performing image recognition on the image of the existing food material to determine a target food material placement position of the existing food material.
Specifically, the number of the supporting trays may be multiple, and the sizes of the food material placing positions on the supporting trays may also be different. In the specific implementation process, in order to determine the target food material placing position corresponding to the existing food material, the electronic scales corresponding to the food material placing position and the food material placing position can be numbered, and the number can be engraved on the supporting tray. For example, if there are three layers in the refrigerator and there is one support tray for each layer, as shown in fig. 7, the number of the support trays for each layer may be A, B, C in order, the number of the food material placing positions on each support tray may be A1 to A9, B1 to B6, and C1 to C7 in order, and the number of the electronic scales corresponding to the food material placing positions may be A1a to A9a, B1B to B6B, and C1C to C7C.
And carrying out image recognition on the obtained image of the existing food material, so as to recognize the number of the food material placing position from the image, and taking the food material placing position corresponding to the recognized number as a target food material placing position.
S2033, obtaining the reading of the electronic scale corresponding to the target food material placing position, and taking the reading of the electronic scale as the weight of the existing food material.
Specifically, after a target food material placing position is determined, an electronic scale reading corresponding to the number of the target food material placing position is obtained according to the number of the target food material placing position, and the electronic scale reading is used as the weight of the existing food material.
For example, when the number of the target food material placing location is C4, the number of the corresponding electronic scale is C4C, and the reading of the electronic scale is taken as the weight of the existing food material placed on the food material placing location with the number of C4.
S204, determining the missing weight of the existing food material according to the difference, and sending the existing food material, the missing weight of the existing food material and the missing food material to a user.
Specifically, if the difference between the weight of the existing food material and the weight of the recipe food material is a negative number, it indicates that the existing food material in the refrigerator is insufficient, and the absolute value of the difference is the missing weight of the existing food material, that is, the weight that the user needs to purchase again. After the missing weight of the existing food material is obtained, the existing food material, the missing weight of the existing food material and the missing food material are sent to the user together, so that the user can purchase the food material according to the needs.
The embodiment of the invention discloses a recipe-based food material pushing method, which includes the steps of setting a camera in a refrigerator to obtain food material images of the refrigerator, carrying out image recognition on the food material images of the refrigerator to obtain refrigerator food materials, then comparing the recipe food materials with the refrigerator food materials to obtain missing food materials and existing food materials, calculating a difference value between the weight of the existing food materials and the weight of the recipe food materials according to the weight of the existing food materials, taking an absolute value of the difference value as the missing weight of the existing food materials if the difference value is a negative number, and sending the existing food materials, the missing weight of the existing food materials and the missing food materials to a user to remind the user of purchasing the existing. The user selects the menu, the refrigerator judges whether food materials in the refrigerator meet the menu requirements or not according to the menu content, if not, the user is reminded to purchase the dishes, the refrigerator can automatically push the food materials needing to be purchased to the user according to the menu, and the use experience of the user is improved.
Referring to fig. 8, fig. 8 is a schematic flow chart of another recipe-based food material pushing method according to an embodiment of the present application. The food material pushing method based on the menu specifically comprises the following steps: step S301 to step S307.
S301, acquiring the refrigerator food material image acquired by the camera, and performing image recognition on the refrigerator food material to obtain the refrigerator food material.
The refrigerator food material image refers to a food material image of existing food materials in the refrigerator shot by a camera. The refrigerator acquires food material images of existing food materials in the refrigerator, which are acquired by the camera, and then performs image recognition on the food material images of the refrigerator, so that the food material types of the existing food materials in the refrigerator, namely the refrigerator food materials, are obtained.
S302, obtaining the weight of the refrigerator food material corresponding to the refrigerator food material, and establishing a food material list according to the refrigerator food material and the weight of the refrigerator food material.
Specifically, be provided with the support tray in the refrigerator, be provided with a plurality of edible material on the support tray and place the position, every edible material places the position and all corresponds and is provided with an electronic scale.
And according to the obtained reading of the electronic scale corresponding to the supporting tray where the refrigerator food material is located, the reading is the weight of the refrigerator food material corresponding to the refrigerator food material. After the weight of the refrigerator food material is obtained, a refrigerator food material list is established according to the refrigerator food material and the weight of the refrigerator food material.
For example, the list of refrigerator ingredients created is shown in table 4:
| serial number | Refrigerator food material | Weight of refrigerator food |
| 1 | Pork | 200g |
| 2 | Mutton | 300g |
| 3 | Duck blood | 100g |
| 4 | Crowndaisy chrysanthemum | 70g |
| 5 | Baby dish | 80g |
S303, comparing the menu food materials with the refrigerator food materials to obtain the missing food materials and the existing food materials.
Specifically, when the recipe food material is compared with the refrigerator food material, the comparison may be performed based on the refrigerator food material list, so as to obtain the missing food material and the existing food material. In a specific implementation process, a row of refrigerator food materials in the list of refrigerator food materials can be connected with the recipe food materials on the left.
S304, calculating the difference value between the weight of the existing food materials and the weight of the food materials in the menu.
Specifically, when calculating the difference between the existing food material weight and the recipe food material weight, the calculation may be performed based on the refrigerator food material list. In the specific implementation process, the existing food material weight corresponding to the existing food material is called from the refrigerator food material list, and then the difference value between the existing food material weight and the recipe food material weight is calculated based on the called existing food material weight.
S305, determining the missing weight of the existing food material according to the difference, and sending the existing food material, the missing weight of the existing food material and the missing food material to a user.
Specifically, if the difference between the weight of the existing food material and the weight of the recipe food material is a negative number, it indicates that the existing food material in the refrigerator is insufficient, and the absolute value of the difference is the missing weight of the existing food material, that is, the weight that the user needs to purchase again. After the missing weight of the existing food material is obtained, the existing food material, the missing weight of the existing food material and the missing food material are sent to the user together, so that the user can purchase the food material according to the needs.
S306, when the refrigerator door of the refrigerator is closed, the camera is opened to obtain the current refrigerator food material image collected by the camera.
Specifically, after the user opens the refrigerator door, when the refrigerator door closes again, the refrigerator sends the opening instruction to the camera promptly, opens the camera to obtain the current refrigerator food material image that the camera was gathered. After a user opens a refrigerator door, food materials are required to be taken out of the refrigerator or placed into the refrigerator according to high probability, so that when the refrigerator door of the refrigerator is closed, the camera is opened, and the current refrigerator food material image is collected. The current food material image of the refrigerator refers to the food material image in the refrigerator after the refrigerator door is opened and closed.
S307, performing image recognition on the current refrigerator food material image to obtain the current refrigerator food material.
Specifically, image recognition is carried out on the current refrigerator food material image to obtain the current refrigerator food material. In a specific implementation process, image recognition is performed on the current refrigerator food material image, specifically, image correction, image splicing, image segmentation and image recognition are performed on the current refrigerator food material image, and finally the current refrigerator food material is obtained.
S308, obtaining the weight of the food material corresponding to the current refrigerator food material, and updating the refrigerator food material list according to the current refrigerator food material and the weight of the food material corresponding to the current refrigerator food material.
Specifically, according to the obtained reading of the electronic scale corresponding to the support tray where the refrigerator food material is located, which is obtained correspondingly to the current refrigerator food material, the reading is the weight of the food material corresponding to the current refrigerator food material. And after the weight of the food materials is obtained, updating a refrigerator food material list according to the current refrigerator food materials and the corresponding food material weight. When a user needs to push the menu information, the refrigerator food material list can be directly inquired to judge the lack of food materials, and the corresponding speed of the refrigerator to the user is improved.
According to the recipe-based food material pushing method provided by the embodiment, the refrigerator food material list is established, wherein the refrigerator food material list comprises the refrigerator food materials and the weight of the refrigerator food materials, so that when the refrigerator identifies lack of food materials according to the recipe selected by a user, data can be directly called from the refrigerator food material list for data processing, the information processing amount of the refrigerator after receiving the recipe information is reduced, and the response speed of the refrigerator is improved. And when the refrigerator door is closed, the refrigerator food material list is updated, the refrigerator food materials are updated in time, and the response accuracy of the refrigerator is improved.
Referring to fig. 9, fig. 9 is a schematic block diagram of a refrigerator according to an embodiment of the present application. The refrigerator 110 includes a refrigerator body 111, a control device 112, a camera 113, a support tray 114, an electronic scale 115, and a storage 116. The control device 112, the camera 113, the electronic scale 115, and the memory 116 are connected by a bus, such as an I2C (Inter-integrated Circuit) bus.
Specifically, the control device 112 includes an image recognition module, a Processor, and a Signal transmission module, wherein the Processor may be a Micro-controller Unit (MCU), a Central Processing Unit (CPU), a Digital Signal Processor (DSP), or the like.
Specifically, the Memory 116 may be a Flash chip, a Read-Only Memory (ROM) magnetic disk, an optical disk, a usb disk, or a removable hard disk.
Wherein the processor is configured to run a computer program stored in the memory and to implement the following steps when executing the computer program:
acquiring a refrigerator food material image acquired by the camera, and performing image recognition on the refrigerator food material image to obtain a refrigerator food material; comparing the recipe food materials with the refrigerator food materials to obtain missing food materials and existing food materials; acquiring the weight of the existing food material, and calculating the difference value between the weight of the existing food material and the weight of the recipe food material; and determining the missing weight of the existing food material according to the difference, and sending the existing food material, the missing weight of the existing food material and the missing food material to a user.
In some embodiments, the processor is further configured to:
receiving menu information sent by a user terminal, and starting a camera according to the menu information.
In some embodiments, when implementing the image recognition on the refrigerator food material to obtain a refrigerator food material, the processor is configured to implement:
respectively carrying out image correction on the plurality of refrigerator food material images to obtain a plurality of corrected refrigerator food material images; carrying out image splicing on a plurality of corrected refrigerator food material images to obtain spliced refrigerator food material images; performing image segmentation on the spliced refrigerator food material images to obtain images of each refrigerator food material in the refrigerator food material images; and respectively carrying out image recognition on the images of the refrigerator food materials in the images of the refrigerator food materials to obtain the refrigerator food materials included in the images of the refrigerator food materials.
In some embodiments, the processor, when implementing the obtaining of the existing food material weight, is configured to implement:
acquiring an image of the existing food material; performing image recognition on the image of the existing food material to determine a target food material placement position of the existing food material; and acquiring the reading of the electronic scale corresponding to the target food material placing position, and taking the reading of the electronic scale as the weight of the existing food material.
In some embodiments, the processor, when enabling the comparing the recipe food material to the refrigerator food material, is to enable:
connecting the menu food materials with the refrigerator food materials on the left; and screening the menu food materials according to the refrigerator food materials which are empty.
In some embodiments, the processor is further configured to:
obtaining the weight of the refrigerator food material corresponding to the refrigerator food material, and establishing a food material list according to the refrigerator food material and the weight of the refrigerator food material.
In some embodiments, the processor is further configured to:
when a refrigerator door of the refrigerator is closed, the camera is opened to obtain a current refrigerator food material image acquired by the camera; performing image processing on the current refrigerator food material image to obtain a current refrigerator food material; and obtaining the weight of the food material corresponding to the current refrigerator food material, and updating the refrigerator food material list according to the current refrigerator food material and the weight of the food material corresponding to the current refrigerator food material.
An embodiment of the present application further provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, where the computer program includes program instructions, and the processor executes the program instructions to implement any recipe-based food material pushing method provided in the embodiment of the present application.
The computer-readable storage medium may be an internal storage unit of the refrigerator described in the foregoing embodiment, for example, a hard disk or a memory of the refrigerator. The computer readable storage medium may also be an external storage device of the refrigerator, such as a plug-in hard disk provided on the refrigerator, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like.
While the invention has been described with reference to specific embodiments, the scope of the invention is not limited thereto, and those skilled in the art can easily conceive various equivalent modifications or substitutions within the technical scope of the invention. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (10)
1. A food material pushing method based on a menu is characterized in that a camera is arranged in a refrigerator, and the method comprises the following steps:
acquiring a refrigerator food material image acquired by the camera, and performing image recognition on the refrigerator food material image to obtain a refrigerator food material;
comparing the recipe food materials with the refrigerator food materials to obtain missing food materials and existing food materials;
acquiring the weight of the existing food material, and calculating the difference value between the weight of the existing food material and the weight of the recipe food material;
and determining the missing weight of the existing food material according to the difference, and sending the existing food material, the missing weight of the existing food material and the missing food material to a user.
2. The food material pushing method according to claim 1, characterized in that the method further comprises:
receiving menu information sent by a user terminal, and starting a camera according to the menu information.
3. The food material pushing method according to claim 1, wherein the number of the cameras is multiple, and the multiple cameras respectively collect multiple refrigerator food material images; the image recognition of the refrigerator food material image is carried out to obtain the refrigerator food material, and the method comprises the following steps:
respectively carrying out image correction on the plurality of refrigerator food material images to obtain a plurality of corrected refrigerator food material images;
carrying out image splicing on a plurality of corrected refrigerator food material images to obtain spliced refrigerator food material images;
performing image segmentation on the spliced refrigerator food material images to obtain images of each refrigerator food material in the refrigerator food material images;
and respectively carrying out image recognition on the images of the refrigerator food materials in the images of the refrigerator food materials to obtain the refrigerator food materials included in the images of the refrigerator food materials.
4. The food material pushing method as claimed in claim 3, wherein a supporting tray is further arranged in the refrigerator, a plurality of food material placing positions are arranged on the supporting tray, and each food material placing position is correspondingly provided with an electronic scale; the acquiring of the weight of the existing food materials comprises the following steps:
acquiring an image of the existing food material;
performing image recognition on the image of the existing food material to determine a target food material placement position of the existing food material;
and acquiring the reading of the electronic scale corresponding to the target food material placing position, and taking the reading of the electronic scale as the weight of the existing food material.
5. The food material pushing method of claim 1, wherein the comparing the recipe food material with the refrigerator food material comprises:
connecting the menu food materials with the refrigerator food materials on the left;
and screening the menu food materials according to the refrigerator food materials which are empty.
6. The food material pushing method according to claim 1, characterized in that the method further comprises:
obtaining the weight of the refrigerator food material corresponding to the refrigerator food material, and establishing a food material list according to the refrigerator food material and the weight of the refrigerator food material.
7. The food material pushing method according to claim 6, further comprising:
when a refrigerator door of the refrigerator is closed, the camera is opened to obtain a current refrigerator food material image acquired by the camera;
performing image processing on the current refrigerator food material image to obtain a current refrigerator food material;
and obtaining the weight of the food material corresponding to the current refrigerator food material, and updating the refrigerator food material list according to the current refrigerator food material and the weight of the food material corresponding to the current refrigerator food material.
8. A refrigerator, characterized by comprising: the device comprises a camera, a support tray, an electronic scale, a memory and a processor;
the camera is used for collecting refrigerator food material images in the refrigerator;
the supporting tray and the electronic scale are used for collecting the weight of refrigerator food materials in the refrigerator;
the memory is used for storing a computer program;
the processor for executing the computer program and implementing the recipe based food material pushing method according to any one of claims 1 to 7 when executing the computer program.
9. A recipe-based food material pushing system, characterized in that the system comprises: the refrigerator comprises a user terminal and a refrigerator, wherein the user terminal is in communication connection with the refrigerator;
the user terminal is used for sending menu information to the refrigerator;
the refrigerator is used for realizing the recipe-based food material pushing method according to any one of claims 1 to 7 according to the recipe information.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program which, when executed by a processor, causes the processor to implement the recipe-based food material pushing method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010019304.8A CN113095897A (en) | 2020-01-08 | 2020-01-08 | Recipe-based food material pushing method, refrigerator, system and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010019304.8A CN113095897A (en) | 2020-01-08 | 2020-01-08 | Recipe-based food material pushing method, refrigerator, system and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113095897A true CN113095897A (en) | 2021-07-09 |
Family
ID=76663616
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010019304.8A Pending CN113095897A (en) | 2020-01-08 | 2020-01-08 | Recipe-based food material pushing method, refrigerator, system and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113095897A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114527682A (en) * | 2022-02-08 | 2022-05-24 | 海信（山东）冰箱有限公司 | Refrigerator and cooking control method |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103822430A (en) * | 2014-02-25 | 2014-05-28 | 四川长虹电器股份有限公司 | Method for detecting whether food in refrigerator is complete and refrigerator |
| CN105841434A (en) * | 2016-05-25 | 2016-08-10 | 海信（山东）冰箱有限公司 | Refrigerator and control method thereof |
| CN106032959A (en) * | 2015-03-13 | 2016-10-19 | 鸿富锦精密工业(武汉)有限公司 | Intelligent refrigerator |
| KR101754372B1 (en) * | 2016-05-26 | 2017-07-19 | 주식회사 원더풀플랫폼 | System and method of food reserves management |
-
2020
- 2020-01-08 CN CN202010019304.8A patent/CN113095897A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103822430A (en) * | 2014-02-25 | 2014-05-28 | 四川长虹电器股份有限公司 | Method for detecting whether food in refrigerator is complete and refrigerator |
| CN106032959A (en) * | 2015-03-13 | 2016-10-19 | 鸿富锦精密工业(武汉)有限公司 | Intelligent refrigerator |
| CN105841434A (en) * | 2016-05-25 | 2016-08-10 | 海信（山东）冰箱有限公司 | Refrigerator and control method thereof |
| KR101754372B1 (en) * | 2016-05-26 | 2017-07-19 | 주식회사 원더풀플랫폼 | System and method of food reserves management |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114527682A (en) * | 2022-02-08 | 2022-05-24 | 海信（山东）冰箱有限公司 | Refrigerator and cooking control method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111684368B (en) | Food preparation method and system based on ingredient identification | |
| US10803315B2 (en) | Electronic device and method for processing information associated with food | |
| JP2018528545A (en) | System and method for nutrition analysis using food image recognition | |
| EP3196758B1 (en) | Image classification method and image classification apparatus | |
| US10733479B2 (en) | System and method for providing a recipe | |
| CN109005368A (en) | A kind of generation method of high dynamic range images, mobile terminal and storage medium | |
| JP2011170745A (en) | Identification device of bread, and program for the same | |
| CN110674736A (en) | Method, device, server and storage medium for identifying freshness of food materials | |
| CN111563435A (en) | Sleep state detection method and device for user | |
| CN111080493A (en) | Dish information identification method and device and dish self-service settlement system | |
| JP2013145441A (en) | Object identification device | |
| CN112053428A (en) | Method and device for identifying nutritional information contained in food | |
| CN110969047A (en) | Method and device for identifying food materials and refrigerator | |
| CN110503498A (en) | A kind of order recommended method and device | |
| CN112902406B (en) | Air conditioner and/or fan parameter setting method, control device and readable storage medium | |
| CN113095897A (en) | Recipe-based food material pushing method, refrigerator, system and storage medium | |
| CN114266959A (en) | Food cooking method and device, storage medium and electronic device | |
| CN111104952A (en) | Method, system and device for identifying food types and refrigerator | |
| CN114549854A (en) | Method, system and equipment for acquiring overall space contour of food | |
| CN111568195A (en) | Brewed beverage identification method, device and computer-readable storage medium | |
| JP2018049584A (en) | Meal size estimation program, meal size estimation method, and meal size estimation apparatus | |
| CN111178274A (en) | Information prompting method and device, computer readable storage medium and electronic device | |
| CN112766020A (en) | Screen device identification method, screen device, menu recommendation system and storage medium | |
| CN110123125A (en) | Cook based reminding method, control device and cooking equipment | |
| WO2021082284A1 (en) | Baking mold specification detection method and apparatus, and kitchen appliance |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |