CN113932518B - Refrigerator and food material management method thereof - Google Patents
Refrigerator and food material management method thereof Download PDFInfo
- Publication number
- CN113932518B CN113932518B CN202110612822.5A CN202110612822A CN113932518B CN 113932518 B CN113932518 B CN 113932518B CN 202110612822 A CN202110612822 A CN 202110612822A CN 113932518 B CN113932518 B CN 113932518B
- Authority
- CN
- China
- Prior art keywords
- pinyin
- food material
- comparison table
- user
- trained
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
- F25—REFRIGERATION OR COOLING; COMBINED HEATING AND REFRIGERATION SYSTEMS; HEAT PUMP SYSTEMS; MANUFACTURE OR STORAGE OF ICE; LIQUEFACTION SOLIDIFICATION OF GASES
- F25D—REFRIGERATORS; COLD ROOMS; ICE-BOXES; COOLING OR FREEZING APPARATUS NOT OTHERWISE PROVIDED FOR
- F25D11/00—Self-contained movable devices, e.g. domestic refrigerators
-
- F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
- F25—REFRIGERATION OR COOLING; COMBINED HEATING AND REFRIGERATION SYSTEMS; HEAT PUMP SYSTEMS; MANUFACTURE OR STORAGE OF ICE; LIQUEFACTION SOLIDIFICATION OF GASES
- F25D—REFRIGERATORS; COLD ROOMS; ICE-BOXES; COOLING OR FREEZING APPARATUS NOT OTHERWISE PROVIDED FOR
- F25D29/00—Arrangement or mounting of control or safety devices
- F25D29/005—Mounting of control devices
-
- F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
- F25—REFRIGERATION OR COOLING; COMBINED HEATING AND REFRIGERATION SYSTEMS; HEAT PUMP SYSTEMS; MANUFACTURE OR STORAGE OF ICE; LIQUEFACTION SOLIDIFICATION OF GASES
- F25D—REFRIGERATORS; COLD ROOMS; ICE-BOXES; COOLING OR FREEZING APPARATUS NOT OTHERWISE PROVIDED FOR
- F25D2500/00—Problems to be solved
- F25D2500/06—Stock management
Abstract
The invention discloses a refrigerator, comprising: the box body comprises a plurality of storage chambers; the box door is arranged at the opening of the storage room, and a display screen is arranged on the outer side of the box door; an imaging device provided in the storage chamber; the controller is configured to: acquiring a picture to be trained shot by a camera device, and displaying the picture to be trained in the display screen; when a labeling operation of a user based on the pictures to be trained is received, inputting the labeled pictures to be trained to a cloud end, so that the cloud end inputs the pictures to be trained as a training set into an original image recognition model to update the original image recognition model; and acquiring the image recognition model updated by the cloud. The invention also discloses a refrigerator food material management method. By adopting the embodiment of the invention, a new training set can be formed by utilizing the food material information marked by the user, the trained new model is loaded on the refrigerator of the user, and each refrigerator product is provided with the unique image recognition model, so that the accuracy rate of food material recognition is improved.
Description
Technical Field
The invention relates to the field of refrigerator control, in particular to a refrigerator and a food material management method thereof.
Background
With the gradual development of refrigerator technology, refrigerators with food management functions have become the development trend of intelligent refrigerators. The refrigerator with food material management can utilize the camera to remotely check food materials stored in the refrigerator, so that a user can conveniently manage the food materials. In the existing refrigerator with food material management, a default image recognition model of the refrigerator is generally adopted to recognize food materials in the refrigerator, the default image recognition model cannot be updated according to user requirements, the food material purchased by each user is large in difference, and the default image recognition model may not cover the food materials required by the user, so that the food materials cannot be recognized.
Disclosure of Invention
The embodiment of the invention aims to provide a refrigerator and a food material management method thereof, which can form a new training set by utilizing food material information marked by a user, load a new training model on the refrigerator of the user, and mount each refrigerator product with a unique image recognition model so as to improve the accuracy of food material recognition.
To achieve the above object, an embodiment of the present invention provides a refrigerator including:
the box body comprises a plurality of storage chambers;
the box door is arranged at the opening of the storage chamber, and a display screen is arranged on the outer side of the box door;
the camera device is arranged in the storage chamber and used for identifying food materials in the storage chamber;
the controller is configured to:
acquiring a picture to be trained shot by the camera device, and displaying the picture to be trained in the display screen;
when a labeling operation of a user based on the pictures to be trained is received, inputting the labeled pictures to be trained to a cloud end, so that the cloud end inputs the pictures to be trained as a training set to an original image recognition model to update the original image recognition model;
and acquiring the image recognition model updated by the cloud.
As an improvement of the scheme, the labeling operation comprises food material definition operation and characteristic point labeling operation of the pictures to be trained by a user; the food material definition operation comprises pinyin input operation and handwriting input operation on the customized food material.
As an improvement of the above, the controller is further configured to:
responding to the pinyin input operation of a user on the custom food material, and acquiring pinyin to be converted input by the user;
acquiring a pre-established food material pinyin comparison table; wherein the food material pinyin comparison table comprises a plurality of food material names and corresponding pinyins thereof;
comparing the pinyin to be converted with the food material pinyin comparison table;
when the pinyin matched with the pinyin to be converted exists in the food material pinyin comparison table, the food material name corresponding to the pinyin to be converted is obtained from the food material pinyin comparison table and is used as the name of the custom food material.
As an improvement of the above solution, when there is no pinyin matching the pinyin to be converted in the food material pinyin-reference table, the controller is further configured to:
performing similarity calculation on the pinyin to be converted and the pinyin in the food material pinyin comparison table to obtain pinyin similarity of the pinyin to be converted and the pinyin;
and when the pinyin similarity is larger than a preset similarity threshold, acquiring a food material name corresponding to the pinyin to be converted from the food material pinyin comparison table as the name of the custom food material.
As an improvement of the above, the controller is further configured to:
responding to handwriting input operation of a user on the custom food material, and acquiring a handwriting name input by the user;
comparing the handwritten name with the food pinyin comparison table;
and when the food material names matched with the handwritten names exist in the food material pinyin comparison table, taking the handwritten names as the names of the custom food materials.
As an improvement of the above, when there is no food material name matching the handwritten name in the food material pinyin reference table, the controller is further configured to:
converting the handwriting name into pinyin to be processed;
comparing the pinyin to be processed with the food material pinyin comparison table;
when the pinyin matched with the pinyin to be processed exists in the food material pinyin comparison table, the food material name corresponding to the pinyin to be processed is obtained from the food material pinyin comparison table and is used as the name of the custom food material.
In order to achieve the above object, the embodiment of the present invention further provides a method for managing food materials in a refrigerator, including:
acquiring a picture to be trained which is shot by a camera device in a refrigerator, and displaying the picture to be trained in a display screen of the refrigerator;
when a labeling operation of a user based on the pictures to be trained is received, inputting the labeled pictures to be trained to a cloud end, so that the cloud end inputs the pictures to be trained as a training set to an original image recognition model to update the original image recognition model;
and acquiring the image recognition model updated by the cloud.
As an improvement of the scheme, the labeling operation comprises food material definition operation and characteristic point labeling operation of the pictures to be trained by a user; the food material definition operation comprises pinyin input operation and handwriting input operation on the customized food material.
As an improvement of the above solution, the method further includes:
responding to the pinyin input operation of a user on the custom food material, and acquiring pinyin to be converted input by the user;
acquiring a pre-established food material pinyin comparison table; wherein the food material pinyin comparison table comprises a plurality of food material names and corresponding pinyins thereof;
comparing the pinyin to be converted with the food material pinyin comparison table;
when the pinyin matched with the pinyin to be converted exists in the food material pinyin comparison table, the food material name corresponding to the pinyin to be converted is obtained from the food material pinyin comparison table and is used as the name of the custom food material.
As an improvement of the above solution, the method further includes:
when there is no pinyin matched with the pinyin to be converted in the food material pinyin comparison table, the method further includes:
performing similarity calculation on the pinyin to be converted and the pinyin in the food material pinyin comparison table to obtain pinyin similarity of the pinyin to be converted and the pinyin;
and when the pinyin similarity is larger than a preset similarity threshold, acquiring a food material name corresponding to the pinyin to be converted from the food material pinyin comparison table as the name of the custom food material.
Compared with the prior art, the refrigerator and the food material management method thereof can form a new training set by utilizing the food material information marked by the user, the trained new model is loaded on the refrigerator of the user, and each refrigerator product is provided with a unique image recognition model, so that the accuracy of food material recognition is improved. In addition, when the user needs to customize the food names, the pinyin input and output operation and the handwriting input operation of the user can be corrected, and the customized food names are ensured to be accurate.
Drawings
Fig. 1 is a flowchart of a controller in a refrigerator according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a table for comparing Pinyin of food materials according to an embodiment of the present invention;
FIG. 3 is a flow chart of a Pinyin input operation provided by an embodiment of the present invention;
FIG. 4 is a flow chart of a handwriting input operation provided by an embodiment of the present invention;
fig. 5 is a flowchart of a method for managing food materials of a refrigerator according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Referring to fig. 1, fig. 1 is a flowchart of a controller in a refrigerator according to an embodiment of the present invention, the refrigerator includes:
the box body comprises a plurality of storage chambers;
the box door is arranged at the opening of the storage chamber, and a display screen is arranged on the outer side of the box door;
the camera device is arranged in the storage chamber and used for identifying food materials in the storage chamber;
the controller is configured to:
s11, acquiring a picture to be trained shot by the camera device, and displaying the picture to be trained in the display screen;
s12, when a labeling operation of a user based on the pictures to be trained is received, inputting the labeled pictures to be trained to a cloud end, so that the cloud end inputs the pictures to be trained as a training set to an original image recognition model, and the original image recognition model is updated;
s13, acquiring an image recognition model updated by the cloud.
Specifically, after a user puts food materials into a storage room of the refrigerator, when a stored image recognition model needs to be updated, a camera is started to photograph the food materials, so that the pictures to be trained are obtained and displayed in the display screen. The user can carry out marking operation on the picture to be trained in the display screen, wherein the marking operation comprises food material definition operation and characteristic point marking operation of the picture to be trained by the user; the food material definition operation comprises pinyin input operation and handwriting input operation on the customized food material. After the user performs the labeling operation, the labeled picture to be trained is input to the cloud end, so that the cloud end inputs the picture to be trained as a training set to an original image recognition model to update the original image recognition model, and the cloud end is utilized to update the original image recognition model without occupying the resources of a refrigerator. After the cloud end updates the original image recognition model, the refrigerator acquires the updated image recognition model from the cloud end.
Optionally, during the labeling by the user, the controller is further configured to:
responding to the pinyin input operation of a user on the custom food material, and acquiring pinyin to be converted input by the user;
acquiring a pre-established food material pinyin comparison table; wherein the food material pinyin comparison table comprises a plurality of food material names and corresponding pinyins thereof;
comparing the pinyin to be converted with the food material pinyin comparison table;
when the pinyin matched with the pinyin to be converted exists in the food material pinyin comparison table, the food material name corresponding to the pinyin to be converted is obtained from the food material pinyin comparison table and is used as the name of the custom food material.
Further, when there is no pinyin in the food material pinyin-lookup table that matches the pinyin to be converted, the controller is further configured to:
performing similarity calculation on the pinyin to be converted and the pinyin in the food material pinyin comparison table to obtain pinyin similarity of the pinyin to be converted and the pinyin;
and when the pinyin similarity is larger than a preset similarity threshold, acquiring a food material name corresponding to the pinyin to be converted from the food material pinyin comparison table as the name of the custom food material.
For example, referring to fig. 2, the refrigerator stores a food material pinyin comparison table in advance, only examples of partial food material names are given in the figure, and the food material pinyin comparison table actually used should cover the food material names of all food materials in the market and the corresponding pinyins thereof, and the food material pinyin comparison table can be updated periodically.
Referring to fig. 3, when it is detected that the pinyin input operation is performed by the user, the pinyin to be converted is matched with the pinyin in the pinyin comparison table, and if the pinyin names matched with the pinyin to be converted exist in the pinyin comparison table, the pinyin names are obtained as the names of the custom foodstuff. If the food material names matched with the to-be-converted pinyin do not exist in the food material pinyin comparison table, fuzzy search is needed, a plurality of pinyins similar to the to-be-converted pinyin are found from the food material pinyin comparison table, then similarity calculation is conducted one by one, and when the similarity is larger than the similarity threshold, the food material names corresponding to the pinyins larger than the similarity threshold are obtained to serve as the names of the self-defined food materials. It can be understood that if two or more pinyins are greater than the similarity threshold, the name of the food material corresponding to the pinyin with greater similarity is obtained as the name of the custom food material. If the similarity is smaller than or equal to the similarity threshold, acquiring a food material name corresponding to pinyin with similarity directly approaching the similarity threshold as a custom food material name; or, at the moment, the pinyin to be converted input by the user is reserved and directly used as the name of the custom food; or when the similarity is smaller than or equal to the similarity threshold, the refrigerator sends out prompt information to remind the user to input again.
Illustratively, the similarity calculation between the pinyin A to be converted and the pinyin B in the food material pinyin comparison table satisfies the following formula:
wherein k is the number of syllables, and the number of syllables is less than 4 syllables in the Chinese phonetic alphabet, so that the situation of more than 4 syllables is not considered; n is the number of continuous k syllable matches of the character to be compared, and N is the total syllable number of the Pinyin B. The larger S is considered to be the higher the similarity.
Illustrating: the pinyin A to be converted is (pinguo), the pinyin B in the food material pinyin comparison table is (pingguo), and the pinyin comparison result refers to table 1.
TABLE 1 Pinyin comparison results
| A | p | i | n | g | u | o | |
| B | p | i | n | g | g | u | o |
When k=1, 1 consecutive syllables match p, i, n, g, u, o for a total of 6; when k=2, 2 consecutive syllables match pi, in, ng, gu, uo for a total of 5; when k=3, 3 consecutive syllables match pin, ing, guo for a total of 3; when k=4, the 4 consecutive syllables match ping 1 total. It is worth noting that, during each round of matching, the syllables matched in the Pinyin B are no longer involved in matching. Similarity of pinguo and pingguo
Calculating the similarity S of all the pinyins in the pinyin with conversion and the food material pinyin comparison table, if S >4, considering the word as correct pinyin, stopping calculation and converting into corresponding Chinese; if the similarity is not greater than 4, selecting the highest similarity for conversion or storing the original input content of the user, and adding the input content into a dictionary of the model to be used as a new food material for processing.
Optionally, during the labeling by the user, the controller is further configured to:
responding to handwriting input operation of a user on the custom food material, and acquiring a handwriting name input by the user;
comparing the handwritten name with the food pinyin comparison table;
and when the food material names matched with the handwritten names exist in the food material pinyin comparison table, taking the handwritten names as the names of the custom food materials.
Further, when there is no food material name in the food material pinyin reference table that matches the handwritten name, the controller is further configured to:
converting the handwriting name into pinyin to be processed;
comparing the pinyin to be processed with the food material pinyin comparison table;
when the pinyin matched with the pinyin to be processed exists in the food material pinyin comparison table, the food material name corresponding to the pinyin to be processed is obtained from the food material pinyin comparison table and is used as the name of the custom food material.
Referring to fig. 4, when it is detected that handwriting input operation is adopted by a user, a handwriting name is acquired, the handwriting name is matched with a food material name in the food material pinyin comparison table, and if the food material name matched with the handwriting name exists in the food material pinyin comparison table, the handwriting name is used as a custom food material name. If the food material pinyin comparison table does not contain the food material names matched with the handwriting names, converting the handwriting names into to-be-processed pinyin, wherein the handwriting names are possibly wrong handwriting of users, such as writing carrots into carrots, matching the to-be-processed pinyin with the pinyin in the food material pinyin comparison table, and if the matched pinyin can be found in the food material pinyin comparison table, acquiring the food material names corresponding to the to-be-processed pinyin from the food material pinyin comparison table as the names of the self-defined food materials. If the matched pinyin is not found in the food material pinyin comparison table, fuzzy search is needed at the moment, a plurality of pinyins similar to the pinyin to be processed are found from the food material pinyin comparison table, then similarity calculation is carried out one by one, and when the similarity is larger than the similarity threshold, the food material name corresponding to the pinyin larger than the similarity threshold is obtained to be used as the name of the custom food material. It can be understood that if two or more pinyins are greater than the similarity threshold, the name of the food material corresponding to the pinyin with greater similarity is obtained as the name of the custom food material. If the similarity is smaller than or equal to the similarity threshold, acquiring a food material name corresponding to pinyin with similarity directly approaching the similarity threshold as a custom food material name; or, at the moment, the pinyin to be converted input by the user is reserved and directly used as the name of the custom food; or when the similarity is smaller than or equal to the similarity threshold, the refrigerator sends out prompt information to remind the user to input again.
Compared with the prior art, the refrigerator disclosed by the embodiment of the invention can form a new training set by utilizing the food material information marked by the user, the trained new model is loaded on the refrigerator of the user, and each refrigerator product is provided with the unique image recognition model, so that the accuracy of food material recognition is improved. In addition, when the user needs to customize the food names, the pinyin input and output operation and the handwriting input operation of the user can be corrected, and the customized food names are ensured to be accurate.
Referring to fig. 5, fig. 5 is a flowchart of a method for managing food materials of a refrigerator according to an embodiment of the present invention, where the method for managing food materials of a refrigerator includes:
s21, acquiring a picture to be trained which is shot by a camera device in the refrigerator, and displaying the picture to be trained in a display screen of the refrigerator;
s22, when a labeling operation of a user based on the pictures to be trained is received, inputting the labeled pictures to be trained to a cloud end, so that the cloud end inputs the pictures to be trained as a training set to an original image recognition model, and the original image recognition model is updated;
s23, acquiring an image recognition model updated by the cloud.
Specifically, after a user puts food materials into a storage room of the refrigerator, when a stored image recognition model needs to be updated, a camera is started to photograph the food materials, so that the pictures to be trained are obtained and displayed in the display screen. The user can carry out marking operation on the picture to be trained in the display screen, wherein the marking operation comprises food material definition operation and characteristic point marking operation of the picture to be trained by the user; the food material definition operation comprises pinyin input operation and handwriting input operation on the customized food material. After the user performs the labeling operation, the labeled picture to be trained is input to the cloud end, so that the cloud end inputs the picture to be trained as a training set to an original image recognition model to update the original image recognition model, and the cloud end is utilized to update the original image recognition model without occupying the resources of a refrigerator. After the cloud end updates the original image recognition model, the refrigerator acquires the updated image recognition model from the cloud end.
Optionally, in the labeling process performed by the user in step S22, the food material management method further includes steps S221 to S226:
s221, responding to the pinyin input operation of the user on the self-defined food material, and acquiring pinyin to be converted input by the user;
s222, acquiring a pre-established food material pinyin comparison table; wherein the food material pinyin comparison table comprises a plurality of food material names and corresponding pinyins thereof;
s223, comparing the pinyin to be converted with the food pinyin comparison table;
s224, when the pinyin matched with the pinyin to be converted exists in the food material pinyin comparison table, acquiring the food material name corresponding to the pinyin to be converted from the food material pinyin comparison table as the name of the custom food material;
s225, when the pinyin matched with the pinyin to be converted does not exist in the food material pinyin comparison table, similarity calculation is carried out on the pinyin to be converted and the pinyin in the food material pinyin comparison table, so that the pinyin similarity of the two is obtained;
and S226, when the pinyin similarity is larger than a preset similarity threshold, acquiring a food material name corresponding to the pinyin to be converted from the food material pinyin comparison table as the name of the custom food material.
The refrigerator stores a food material pinyin comparison table in advance, only examples of partial food material names are given in the figure, the food material pinyin comparison table actually used should cover the food material names and corresponding pinyins of all food materials in the market, and the food material pinyin comparison table can be updated periodically.
Specifically, when it is detected that the pinyin input operation is adopted by the user, the pinyin to be converted is matched with the pinyin in the food material pinyin comparison table, and if the food material name matched with the pinyin to be converted exists in the food material pinyin comparison table, the food material name is obtained as the name of the custom food material. If the food material names matched with the to-be-converted pinyin do not exist in the food material pinyin comparison table, fuzzy search is needed, a plurality of pinyins similar to the to-be-converted pinyin are found from the food material pinyin comparison table, then similarity calculation is conducted one by one, and when the similarity is larger than the similarity threshold, the food material names corresponding to the pinyins larger than the similarity threshold are obtained to serve as the names of the self-defined food materials. It can be understood that if two or more pinyins are greater than the similarity threshold, the name of the food material corresponding to the pinyin with greater similarity is obtained as the name of the custom food material. If the similarity is smaller than or equal to the similarity threshold, acquiring a food material name corresponding to pinyin with similarity directly approaching the similarity threshold as a custom food material name; or, at the moment, the pinyin to be converted input by the user is reserved and directly used as the name of the custom food; or when the similarity is smaller than or equal to the similarity threshold, the refrigerator sends out prompt information to remind the user to input again.
Illustratively, the similarity calculation between the pinyin A to be converted and the pinyin B in the food material pinyin comparison table satisfies the following formula:
wherein k is the number of syllables, and the number of syllables is less than 4 syllables in the Chinese phonetic alphabet, so that the situation of more than 4 syllables is not considered; n is the number of continuous k syllable matches of the character to be compared, and N is the total syllable number of the Pinyin B. The larger S is considered to be the higher the similarity.
Illustrating: the pinyin A to be converted is (pinguo), the pinyin B in the food material pinyin comparison table is (pingguo), and the pinyin comparison result refers to table 1.
TABLE 1 Pinyin comparison results
| A | p | i | n | g | u | o | |
| B | p | i | n | g | g | u | o |
When k=1, 1 consecutive syllables match p, i, n, g, u, o for a total of 6; when k=2, 2 consecutive syllables match pi, in, ng, gu, uo for a total of 5; when k=3, 3 consecutive syllables match pin, ing, guo for a total of 3; when k=4, the 4 consecutive syllables match ping 1 total. It is worth noting that, during each round of matching, the syllables matched in the Pinyin B are no longer involved in matching. Similarity of pinguo and pingguo
Calculating the similarity S of all the pinyins in the pinyin with conversion and the food material pinyin comparison table, if S >4, considering the word as correct pinyin, stopping calculation and converting into corresponding Chinese; if the similarity is not greater than 4, selecting the highest similarity for conversion or storing the original input content of the user, and adding the input content into a dictionary of the model to be used as a new food material for processing.
Optionally, in the labeling process performed by the user in step S22, the food material management method further includes steps S321 to S326:
s321, responding to handwriting input operation of a user on the self-defined food material, and acquiring a handwriting name input by the user;
s322, comparing the handwritten name with the food pinyin comparison table;
s323, when the food material names matched with the handwritten names exist in the food material pinyin comparison table, the handwritten names are used as the names of the custom food materials.
S324, converting the handwritten name into pinyin to be processed when the food material name matched with the handwritten name does not exist in the food material pinyin comparison table;
s325, comparing the pinyin to be processed with the food material pinyin comparison table;
s326, when the pinyin matched with the pinyin to be processed exists in the food material pinyin comparison table, acquiring the food material name corresponding to the pinyin to be processed from the food material pinyin comparison table as the name of the self-defined food material.
Specifically, when the handwriting input operation adopted by the user is detected, a handwriting name is obtained, the handwriting name is matched with the food material names in the food material pinyin comparison table, and if the food material names matched with the handwriting name exist in the food material pinyin comparison table, the handwriting name is used as the name of the custom food material. If the food material pinyin comparison table does not contain the food material names matched with the handwriting names, converting the handwriting names into to-be-processed pinyin, wherein the handwriting names are possibly wrong handwriting of users, such as writing carrots into carrots, matching the to-be-processed pinyin with the pinyin in the food material pinyin comparison table, and if the matched pinyin can be found in the food material pinyin comparison table, acquiring the food material names corresponding to the to-be-processed pinyin from the food material pinyin comparison table as the names of the self-defined food materials. If the matched pinyin is not found in the food material pinyin comparison table, fuzzy search is needed at the moment, a plurality of pinyins similar to the pinyin to be processed are found from the food material pinyin comparison table, then similarity calculation is carried out one by one, and when the similarity is larger than the similarity threshold, the food material name corresponding to the pinyin larger than the similarity threshold is obtained to be used as the name of the custom food material. It can be understood that if two or more pinyins are greater than the similarity threshold, the name of the food material corresponding to the pinyin with greater similarity is obtained as the name of the custom food material. If the similarity is smaller than or equal to the similarity threshold, acquiring a food material name corresponding to pinyin with similarity directly approaching the similarity threshold as a custom food material name; or, at the moment, the pinyin to be converted input by the user is reserved and directly used as the name of the custom food; or when the similarity is smaller than or equal to the similarity threshold, the refrigerator sends out prompt information to remind the user to input again.
Compared with the prior art, the food material management method provided by the embodiment of the invention can form a new training set by utilizing the food material information marked by the user, and load the trained new model onto the user refrigerator, and each refrigerator product is provided with a unique image recognition model, so that the accuracy of food material recognition is improved. In addition, when the user needs to customize the food names, the pinyin input and output operation and the handwriting input operation of the user can be corrected, and the customized food names are ensured to be accurate.
While the foregoing is directed to the preferred embodiments of the present invention, it will be appreciated by those skilled in the art that changes and modifications may be made without departing from the principles of the invention, such changes and modifications are also intended to be within the scope of the invention.
Claims (4)
1. A refrigerator, comprising:
the box body comprises a plurality of storage chambers;
the box door is arranged at the opening of the storage chamber, and a display screen is arranged on the outer side of the box door;
the camera device is arranged in the storage chamber and used for identifying food materials in the storage chamber;
the controller is configured to:
acquiring a picture to be trained shot by the camera device, and displaying the picture to be trained in the display screen;
when a labeling operation of a user based on the pictures to be trained is received, inputting the labeled pictures to be trained to a cloud end, so that the cloud end inputs the pictures to be trained as a training set to an original image recognition model to update the original image recognition model; the marking operation comprises food material definition operation and characteristic point marking operation of the pictures to be trained by a user, wherein the food material definition operation comprises pinyin input operation and handwriting input operation of user-defined food materials;
acquiring an image recognition model updated by a cloud;
responding to the pinyin input operation of a user on the custom food material, and acquiring pinyin to be converted input by the user; acquiring a pre-established food material pinyin comparison table; wherein the food material pinyin comparison table comprises a plurality of food material names and corresponding pinyins thereof; comparing the pinyin to be converted with the food material pinyin comparison table; when the pinyin matched with the pinyin to be converted exists in the food material pinyin comparison table, acquiring a food material name corresponding to the pinyin to be converted from the food material pinyin comparison table as the name of the custom food material;
responding to handwriting input operation of a user on the custom food material, and acquiring a handwriting name input by the user; comparing the handwritten name with the food pinyin comparison table; when the food material names matched with the handwritten names exist in the food material pinyin comparison table, the handwritten names are used as the names of the custom food materials;
when the food material names matched with the handwritten names do not exist in the food material pinyin comparison table, converting the handwritten names into pinyin to be processed; comparing the pinyin to be processed with the food material pinyin comparison table; when the pinyin matched with the pinyin to be processed exists in the food material pinyin comparison table, the food material name corresponding to the pinyin to be processed is obtained from the food material pinyin comparison table and is used as the name of the custom food material.
2. The refrigerator of claim 1, wherein when there is no pinyin in the food material pinyin-lookup table that matches the pinyin to be converted, the controller is further configured to:
performing similarity calculation on the pinyin to be converted and the pinyin in the food material pinyin comparison table to obtain pinyin similarity of the pinyin to be converted and the pinyin;
and when the pinyin similarity is larger than a preset similarity threshold, acquiring a food material name corresponding to the pinyin to be converted from the food material pinyin comparison table as the name of the custom food material.
3. A method for managing food materials of a refrigerator, comprising:
acquiring a picture to be trained which is shot by a camera device in a refrigerator, and displaying the picture to be trained in a display screen of the refrigerator;
when a labeling operation of a user based on the pictures to be trained is received, inputting the labeled pictures to be trained to a cloud end, so that the cloud end inputs the pictures to be trained as a training set to an original image recognition model to update the original image recognition model; the marking operation comprises food material definition operation and characteristic point marking operation of the pictures to be trained by a user, wherein the food material definition operation comprises pinyin input operation and handwriting input operation of user-defined food materials;
acquiring an image recognition model updated by a cloud;
responding to the pinyin input operation of a user on the custom food material, and acquiring pinyin to be converted input by the user; acquiring a pre-established food material pinyin comparison table; wherein the food material pinyin comparison table comprises a plurality of food material names and corresponding pinyins thereof; comparing the pinyin to be converted with the food material pinyin comparison table; when the pinyin matched with the pinyin to be converted exists in the food material pinyin comparison table, acquiring a food material name corresponding to the pinyin to be converted from the food material pinyin comparison table as the name of the custom food material;
responding to handwriting input operation of a user on the custom food material, and acquiring a handwriting name input by the user; comparing the handwritten name with the food pinyin comparison table; when the food material names matched with the handwritten names exist in the food material pinyin comparison table, the handwritten names are used as the names of the custom food materials;
when the food material names matched with the handwritten names do not exist in the food material pinyin comparison table, converting the handwritten names into pinyin to be processed; comparing the pinyin to be processed with the food material pinyin comparison table; when the pinyin matched with the pinyin to be processed exists in the food material pinyin comparison table, the food material name corresponding to the pinyin to be processed is obtained from the food material pinyin comparison table and is used as the name of the custom food material.
4. The refrigerator food material management method as in claim 3, further comprising:
when there is no pinyin matched with the pinyin to be converted in the food material pinyin comparison table, the method further includes:
performing similarity calculation on the pinyin to be converted and the pinyin in the food material pinyin comparison table to obtain pinyin similarity of the pinyin to be converted and the pinyin;
and when the pinyin similarity is larger than a preset similarity threshold, acquiring a food material name corresponding to the pinyin to be converted from the food material pinyin comparison table as the name of the custom food material.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110612822.5A CN113932518B (en) | 2021-06-02 | 2021-06-02 | Refrigerator and food material management method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110612822.5A CN113932518B (en) | 2021-06-02 | 2021-06-02 | Refrigerator and food material management method thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113932518A CN113932518A (en) | 2022-01-14 |
| CN113932518B true CN113932518B (en) | 2023-08-18 |
Family
ID=79274268
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110612822.5A Active CN113932518B (en) | 2021-06-02 | 2021-06-02 | Refrigerator and food material management method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113932518B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114440524A (en) * | 2022-01-26 | 2022-05-06 | 海信（山东）冰箱有限公司 | Refrigerator and processing method of refrigerator food material data |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109460461A (en) * | 2018-11-13 | 2019-03-12 | 苏州思必驰信息科技有限公司 | Text matching technique and system based on text similarity model |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7478033B2 (en) * | 2004-03-16 | 2009-01-13 | Google Inc. | Systems and methods for translating Chinese pinyin to Chinese characters |
| CN104238991B (en) * | 2013-06-21 | 2018-05-25 | 腾讯科技（深圳）有限公司 | Phonetic entry matching process and device |
| CN103593062B (en) * | 2013-11-08 | 2016-09-28 | 北京奇虎科技有限公司 | A kind of data detection method and device |
| CN106683677B (en) * | 2015-11-06 | 2021-11-12 | 阿里巴巴集团控股有限公司 | Voice recognition method and device |
| CN108038509B (en) * | 2017-12-21 | 2020-08-04 | 合肥美的智能科技有限公司 | Image recognition method and device based on image recognition model |
| CN110164435A (en) * | 2019-04-26 | 2019-08-23 | 平安科技（深圳）有限公司 | Audio recognition method, device, equipment and computer readable storage medium |
| CN110569505B (en) * | 2019-09-04 | 2023-07-28 | 平顶山学院 | Text input method and device |
-
2021
- 2021-06-02 CN CN202110612822.5A patent/CN113932518B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109460461A (en) * | 2018-11-13 | 2019-03-12 | 苏州思必驰信息科技有限公司 | Text matching technique and system based on text similarity model |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113932518A (en) | 2022-01-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105814519B (en) | System and method for inputting image or label to electronic equipment | |
| CN108537176B (en) | Target barrage identification method and device, terminal and storage medium | |
| CN111782772A (en) | Text automatic generation method, device, equipment and medium based on OCR technology | |
| CN110363194A (en) | Intelligently reading method, apparatus, equipment and storage medium based on NLP | |
| US20080193015A1 (en) | Contextual input method | |
| US10304439B2 (en) | Image processing device, animation display method and computer readable medium | |
| CN108227565A (en) | A kind of information processing method, terminal and computer-readable medium | |
| TWI567569B (en) | Natural language processing systems, natural language processing methods, and natural language processing programs | |
| US20090189905A1 (en) | Apparatus and method of generating personal fonts | |
| US20140289238A1 (en) | Document creation support apparatus, method and program | |
| CN113932518B (en) | Refrigerator and food material management method thereof | |
| EP3872652A2 (en) | Method and apparatus for processing video, electronic device, medium and product | |
| US20220292133A1 (en) | Image retrieving method and apparatus, storage media and electronic device | |
| US8849034B2 (en) | System, method, and apparatus for triggering recognition of a handwritten shape | |
| CN107368181B (en) | Gesture recognition method and device | |
| CN110659398A (en) | Visual question-answering method based on mathematical chart data set | |
| CN111435595A (en) | Text normalization method and device | |
| US7911452B2 (en) | Pen input method and device for pen computing system | |
| CN111985184A (en) | Auxiliary writing font copying method, system and device based on AI vision | |
| CN107797981B (en) | Target text recognition method and device | |
| CN107783958B (en) | Target statement identification method and device | |
| US20230215066A1 (en) | Generating visual feedback | |
| WO2019170038A1 (en) | Target screen determining method and device, and storage medium | |
| CN104850819B (en) | Information processing method and electronic equipment | |
| US11735167B2 (en) | Electronic device and method for controlling the same, and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 266000 Haixin Avenue, Nancun Town, Pingdu City, Qingdao City, Shandong Province Applicant after: Hisense refrigerator Co.,Ltd. Address before: 266100 Songling Road, Laoshan District, Qingdao, Shandong Province, No. 399 Applicant before: HISENSE (SHANDONG) REFRIGERATOR Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |