CN113468910A - Article management method and device and intelligent refrigerator - Google Patents
Article management method and device and intelligent refrigerator Download PDFInfo
- Publication number
- CN113468910A CN113468910A CN202010236472.2A CN202010236472A CN113468910A CN 113468910 A CN113468910 A CN 113468910A CN 202010236472 A CN202010236472 A CN 202010236472A CN 113468910 A CN113468910 A CN 113468910A
- Authority
- CN
- China
- Prior art keywords
- article
- user
- item
- code scanning
- actual type
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000007726 management method Methods 0.000 title claims abstract description 85
- 238000000034 method Methods 0.000 claims abstract description 72
- 230000009471 action Effects 0.000 claims description 14
- 238000012937 correction Methods 0.000 claims description 8
- 238000012790 confirmation Methods 0.000 claims description 6
- 230000000007 visual effect Effects 0.000 claims description 3
- 230000008569 process Effects 0.000 abstract description 27
- 230000006399 behavior Effects 0.000 description 32
- 238000012545 processing Methods 0.000 description 24
- 238000010586 diagram Methods 0.000 description 18
- 239000000463 material Substances 0.000 description 15
- 235000013305 food Nutrition 0.000 description 11
- 238000004590 computer program Methods 0.000 description 8
- 238000004891 communication Methods 0.000 description 6
- 238000000605 extraction Methods 0.000 description 5
- 230000003993 interaction Effects 0.000 description 4
- 230000006870 function Effects 0.000 description 3
- 239000008267 milk Substances 0.000 description 3
- 210000004080 milk Anatomy 0.000 description 3
- 235000013336 milk Nutrition 0.000 description 3
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 235000019800 disodium phosphate Nutrition 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000012706 support-vector machine Methods 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 230000001960 triggered effect Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000003066 decision tree Methods 0.000 description 1
- 238000000151 deposition Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000002245 particle Substances 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
- F25—REFRIGERATION OR COOLING; COMBINED HEATING AND REFRIGERATION SYSTEMS; HEAT PUMP SYSTEMS; MANUFACTURE OR STORAGE OF ICE; LIQUEFACTION SOLIDIFICATION OF GASES
- F25D—REFRIGERATORS; COLD ROOMS; ICE-BOXES; COOLING OR FREEZING APPARATUS NOT OTHERWISE PROVIDED FOR
- F25D29/00—Arrangement or mounting of control or safety devices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06K—GRAPHICAL DATA READING; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
- G06K17/00—Methods or arrangements for effecting co-operative working between equipments covered by two or more of main groups G06K1/00 - G06K15/00, e.g. automatic card files incorporating conveying and reading operations
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Chemical & Material Sciences (AREA)
- Combustion & Propulsion (AREA)
- Mechanical Engineering (AREA)
- Thermal Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Cold Air Circulating Systems And Constructional Details In Refrigerators (AREA)
Abstract
The application provides an article management method, an article management device and an intelligent refrigerator, wherein the method comprises the following steps: acquiring an image sequence when a user manages articles in an article storage device, wherein images of the image sequence comprise hand image areas of the user; carrying out target tracking on the hand image area to obtain the management behavior of the user on the article and whether the user executes code scanning operation on the article; determining the actual type of the article according to whether the user performs code scanning operation on the article; pushing prompt information and updating a current stored item set of the item storage device, wherein the prompt information is used for indicating the actual type of the item and the management behavior of the user. The operation process of the user in managing the articles is greatly simplified, and the user experience is greatly improved.
Description
Technical Field
The application relates to an intelligent household appliance technology, in particular to an article management method and device and an intelligent refrigerator.
Background
With the continuous development of intelligent electric appliance technology, intelligent refrigerators appear. Compare with traditional refrigerator, parts such as camera, bar code scanning module, pronunciation collection module, display screen that intelligent refrigerator utilized setting on the refrigerator help the user to eat material and deposit, eat material and take out and eat material management process such as material to and, provide edible material processing mode inquiry etc. to the user.
In the prior art, a barcode scanning module can be arranged on the intelligent refrigerator, and the intelligent refrigerator scans barcodes of food materials based on the barcode scanning module on the refrigerator to manage the food materials. Specifically, before the user stores food materials into the intelligent refrigerator, the bar code scanning module of the intelligent refrigerator is triggered to start working in modes of voice or interface operation and the like, and the bar code scanning module scans information of the stored food materials and completes a food material storing process. Correspondingly, before a user takes out a certain food material from the intelligent refrigerator, firstly, the bar code scanning module is triggered to start work in modes of voice or interface operation and the like, and the information of the taken-out food material is scanned by the bar code scanning module and the food material taking-out process is completed.
However, the method in the prior art may cause a cumbersome operation process of food material management, which affects the use experience of the user.
Disclosure of Invention
The application provides an article management method and device and an intelligent refrigerator, and aims to solve the problem that the user experience is affected due to the fact that the food management process is complicated in the prior art.
In a first aspect, the present application provides an article management method, comprising:
acquiring an image sequence when a user manages articles in an article storage device, wherein images of the image sequence comprise hand image areas of the user and article image areas;
performing target tracking on the hand image area to obtain a management behavior of the user on the article and whether the user performs code scanning operation on the article, wherein the management behavior comprises storing the article in the article storage device or taking the article out of the article storage device;
determining the actual type of the article according to whether the user performs code scanning operation on the article;
pushing prompt information and updating a current stored item set of the item storage device, wherein the prompt information is used for indicating the actual type of the item and the management behavior of the user.
In a second aspect, the present application provides an article management device comprising:
the system comprises a processing module, a storage module and a display module, wherein the processing module is used for acquiring an image sequence when a user manages articles in an article storage device, and images of the image sequence comprise hand image areas and article image areas of the user; and the number of the first and second groups,
performing target tracking on the hand image area to obtain a management behavior of the user on the article and whether the user performs code scanning operation on the article, wherein the management behavior comprises storing the article in the article storage device or taking the article out of the article storage device; and the number of the first and second groups,
determining the actual type of the article according to whether the user performs code scanning operation on the article;
and the pushing module is used for pushing prompt information and updating the current stored item set of the item storage equipment, wherein the prompt information is used for indicating the actual type of the item and the management behavior of the user.
In a third aspect, the present application provides an intelligent refrigerator, comprising:
the camera is positioned at the top of the intelligent refrigerator and used for collecting images of a user during management of articles in the intelligent refrigerator; the bar code scanner is positioned in the visual field range of the camera; and the processor is used for identifying the article management behavior of the user according to the image collected by the camera and whether the user executes code scanning operation, selecting the actual type of the article based on the image identification mode according to whether the user executes the code scanning operation, and/or starting the bar code scanner, identifying the actual type of the article in the bar code identification mode, pushing prompt information and updating the current stored article set, wherein the prompt information is used for indicating the actual type of the article and the management behavior of the user.
In a fourth aspect, the present application provides an article storage device comprising:
a memory for storing program instructions;
a processor for calling and executing the program instructions in the memory to perform the method steps of the first aspect.
In a fifth aspect, the present application provides a readable storage medium having stored thereon a computer program for executing the method of the first aspect.
According to the article management method and device and the intelligent refrigerator, the management behaviors of the user can be automatically identified and whether the user executes code scanning operation or not by collecting the image of the user when managing articles and carrying out target tracking, the actual types of the articles managed by the user are automatically identified by using the corresponding method according to whether the user executes the code scanning operation or not, and then the information of the article management at this time is pushed to the user and the current stored article set is updated. In the process, the user can complete the article management and check the result of the article management without executing any additional operation, so that the operation process of the user in managing the articles is greatly simplified, and the experience of the user is greatly improved.
Drawings
In order to more clearly illustrate the technical solutions in the present application or the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.
FIG. 1a is an exemplary front view block diagram of an article storage device according to an embodiment of the present application;
FIG. 1b is a side view block diagram of an exemplary article storage device according to an embodiment of the present application;
fig. 2 is a schematic flowchart of an article management method according to an embodiment of the present application;
fig. 3 is a schematic flowchart of an article management method according to an embodiment of the present application;
FIG. 4 is a schematic view of a process for interacting with a user and updating a currently stored item set according to an operation result;
FIG. 5 is an exemplary diagram of an interface for successful logging;
FIG. 6 is an exemplary diagram of an interface during a retrieval failure;
FIG. 7 is a schematic flow chart illustrating a first method for identifying an actual type of an item when the user stores the item;
FIG. 8 is a schematic flow chart illustrating a first method for identifying the actual type of an item when the user removes the item;
FIG. 9 is a schematic flow chart illustrating a second method for identifying the actual type of an item when the user deposits the item;
FIG. 10 is a schematic flow chart illustrating a second method for identifying the actual type of an item when the user removes the item;
FIG. 11 is a schematic view of a process for obtaining management behavior by target tracking and whether a user performs a code scanning operation;
FIG. 12a is a diagram of an example of the recognition of a child action;
FIG. 12b is a diagram illustrating an example of the recognition of a return action;
FIG. 12c is a diagram illustrating an example of the recognition of the stop sub-action;
fig. 13 is a block diagram of an article management apparatus according to an embodiment of the present application;
fig. 14 is a schematic structural diagram of an article storage device 1400 according to an embodiment of the present disclosure.
Detailed Description
In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The embodiment of the application is applied to a scene that a user manages articles by using the article storage device, wherein the article storage device can be an intelligent refrigerator, an intelligent freezer and the like, and the user manages the articles by storing the articles into the article storage device and taking the articles out of the article storage device. In the embodiment of the application, when a user manages articles, the article storage device collects images when the user executes storage or taking out operation by using a camera installed on the article storage device, and selects to manage the articles based on an image identification mode and/or starts a bar code scanner on the article storage device to manage the articles in a bar code identification mode by using the method of the embodiment of the application. The method specifically comprises the following steps: when a user stores articles, the identified article information is pushed to the user, and the current existing article set is updated after the user confirms the article information; and when the user takes out the article, judging whether the result of the taking-out operation is successful, pushing corresponding prompt information, and updating the current existing article set according to the response information of the user.
Fig. 1a is an exemplary front view structural diagram of an article storage device according to an embodiment of the present application, and fig. 1b is an exemplary side view structural diagram of an article storage device according to an embodiment of the present application, as shown in fig. 1a and fig. 1b, taking an article storage device as an intelligent refrigerator as an example, a camera is disposed on the top of the intelligent refrigerator, and the field of view of the camera faces downward. The bar code scanner is positioned on the box body between the upper and lower refrigerating chambers and the temperature changing chamber. In addition, the bar code scanner can also be positioned at the outer side of the intelligent refrigerator, the door bodies or between the door bodies and the like. This application does not do the restriction to the mounted position of bar code scanner, only need make bar code scanner be in the field of vision within range of camera can. The camera can gather the image when the user manages article, and bar code scanner can scan the bar code on the article in order to acquire the information of article.
For convenience of description, the following embodiments are illustrated by taking the article storage device as an intelligent refrigerator. In addition, in some embodiments below, "article storage device" is simply referred to as "device".
Fig. 2 is a schematic flowchart of an article management method according to an embodiment of the present application, where an execution subject of the method is an article storage device. As shown in fig. 2, the method includes:
s201, acquiring an image sequence when a user manages articles in an article storage device, wherein images of the image sequence comprise hand image areas and article image areas of the user.
Illustratively, a camera arranged at the top of the intelligent refrigerator can collect multi-frame images in real time during operation of a user after the user opens a door body, and the multi-frame images form an image sequence. Each frame image in the image sequence at least comprises a hand of a user and an article held by the user, so that the image at least comprises a hand image area and an article image area.
S202, carrying out target tracking on the hand image area, and determining the management action of the user on the article and whether the user carries out code scanning operation on the article, wherein the management action comprises storing the article into an article storage device or taking the article out of the article storage device.
Optionally, the image sequence acquired by the camera includes movement information of the hand of the user when the user deposits or removes an article, so that target tracking is performed based on the hand image area of each image in the image sequence, and it can be known whether the operation performed by the user is depositing or removing and whether the user performs a code scanning operation.
The specific implementation of the target tracking for the hand image area will be described in detail in the following embodiments.
S203, determining the actual type of the article according to whether the user performs code scanning operation on the article.
By target tracking of the hand image area, the item storage device can automatically recognize whether the user has performed a code scanning operation, and based on this information, automatically recognize the actual kind of the item in a corresponding manner. Illustratively, if the user performs a code scanning operation, the actual type of the article is identified by a barcode identification method, and if the user does not perform a code scanning operation, the actual type of the article is identified by an image identification method.
The actual categories of items may include, for example: brand, specification, category, etc. For example, the actual type of an item is: 200ml of milk of brand A.
It should be noted that the step S202 may be completed before the step S203 is executed, or may be completed after the step S203 is executed. For example, it is first recognized by step S202 that the user performs the code scanning operation, at which point step S203 may be started to be performed, and at the same time, step S202 may continue to be performed to recognize the management behavior of the user.
S204, pushing prompt information and updating the current stored item set of the item storage device, wherein the prompt information is used for indicating the actual type of the item and the management behavior of the user.
The current stored item set of the item storage device comprises various items currently stored in the device, and the item storage device can display the items in the current stored item set for a user according to the needs of the user, so that the user can know whether to supplement certain items at any time.
In this step, after the article storage device identifies the management behavior of the user and the type of the article managed by the user this time, on one hand, the management behavior and the type of the article of this time can be pushed to the user, so that the user knows that the article storage device has identified the operation this time, and on the other hand, the article storage device further updates the currently stored article set according to the operation this time, so that the currently stored article set is consistent with the article actually stored in the device, that is, the accuracy of the currently stored article set is ensured. Illustratively, when a user stores a certain article a into the smart refrigerator, the smart refrigerator displays a prompt message of "you store article a" on its display screen and adds the information of article a to the currently stored article set, and when the user takes out a certain article B from the smart refrigerator, the smart refrigerator displays a prompt message of "you take out article B" on its display screen and deletes the information of article B from the currently stored article set.
In this embodiment, the article storage device collects an image of the user managing an article and performs target tracking, so that it is possible to automatically identify a management behavior of the user and whether the user performs a code scanning operation, and automatically identify an actual type of the article managed by the user using a corresponding method according to whether the user performs the code scanning operation, and then the article storage device pushes information of the article management of this time to the user and updates a currently stored article set. In the process, the user can complete the article management and check the result of the article management without executing any additional operation, so that the operation process of the user in managing the articles is greatly simplified, and the experience of the user is greatly improved.
When the user manages the article, the code scanning operation may be performed or not performed. If the user performs the code scanning operation, the article storage device can obtain the actual type of the article by using a bar code identification mode. If the user does not perform the code scanning operation, the actual type of the article can be obtained by using an image recognition mode. In actual implementation, the following two scenarios may occur: in the first scenario, the user performs a code scanning operation when storing an item, and does not perform a code scanning operation when retrieving the item. In the second scenario, the user does not perform the code scanning operation when storing an item, and performs the code scanning operation when taking out the item. In the two scenarios, when a user stores and takes out an article, the article storage device uses different identification modes, the identification results of the different identification modes for the same food material may be different, and a problem that the identified article type cannot be matched in the current stored article set when the article storage device updates the current stored article set when the user takes out the article may occur. In view of this problem, as an alternative embodiment, when the item storage device executes the above step S204, before updating the currently stored item set, the item storage device may first determine that the operation result of the management action of the user on the item is operation success or operation failure. For the deposit operation of the user, the successful operation means that the information of the article can be successfully added to the currently stored article set when the article is deposited. For the taking-out operation of the user, the successful operation means that the actual category of the identified item can be found from the current stored item set, and the failed operation means that the actual category of the identified item can be found from the current stored item set. For the deposit operation, as long as the item type is identified, the item type can be added into the current stored item set, so that the operation result in the deposit can be directly considered as the success of the operation. For the extraction operation, the operation result can be judged according to the actual type of the article and the currently stored article set through the following article association reasoning process. Specifically, if the currently stored item set includes an item that matches the identified item, the determination is successful.
Fig. 3 is a schematic flowchart of an article management method according to an embodiment of the present application, and as shown in fig. 3, the association inference process includes:
s301, it is queried whether there is an item in the current stored item set whose type is the same as the identified actual type, if yes, step 303 is executed, and if not, step 302 is executed.
For example, the same type of articles may mean that the brand, the specification and the category are the same. For example, if the actual type identified is milk of brand a of 200ml, and if there is an item of brand a of 200ml milk among the currently stored items, it is determined that the operation is successful, that is, the extraction is successful.
S302, querying whether there is an item in the current stored item set whose category is different from the identified actual category by less than a preset threshold, if so, performing step S303, and if not, performing step S304.
For example, the preset threshold may refer to one of brand, specification, and category.
The difference between the category in the stored item set and the identified actual category is smaller than a preset threshold, for example:
(1) the brands are the same, the specifications are different, and the categories are the same.
For example, if there is 300ml of brand a cola in the stored item set, and the identified actual type is 250ml of brand a cola, the difference between the two is considered to be less than the preset threshold value, and the operation is successful.
(2) Different brands, same specification and same category
For example, if there is 300ml of brand a cola in the stored item set, and the identified actual type is 250ml of brand a cola, the difference between the two is considered to be less than the preset threshold value, and the operation is successful.
In another example, the article a and the article B having a large difference in category division may be considered to have a difference in category smaller than a preset threshold value and operate successfully because the two articles have similar appearances although the two articles have a large difference in category.
And S303, determining that the operation is successful.
And S304, determining that the operation fails.
After the operation result of the management behavior of the user on the article is judged to be successful or failed by the process, interaction can be carried out with the user based on the operation result, and the current stored article set is updated, so that article management is completed.
Fig. 4 is a schematic flow chart illustrating interaction with a user and updating of a currently stored item set according to an operation result, and as shown in fig. 4, for a result of successful operation in the storing and retrieving, that is, successful storing and retrieving, a user may be prompted to confirm prompt information, and the set may be updated after the user confirms. For the result of the operation failure during the taking out, namely the taking out failure, the user can be prompted to correct the article types, and the set is updated after the user corrects the article types.
Optionally, if the operation result is that the operation is successful, updating the currently stored item set according to the confirmation information of the user on the prompt information. And if the operation result is operation failure, updating the actual type of the article according to the correction information of the user on the prompt information, and updating the currently stored article according to the actual type of the article.
The correction information of the user for the prompt information may be information for correcting the identified actual type, or information for correcting the currently stored item set.
In one example, when a user stores an article A, a code scanning operation is performed, the device identifies the article A as the article A in a barcode identification mode and adds the article A to a stored article set, when the user takes out the article A, the code scanning operation is not performed, the device identifies the article A as an article B in an image identification mode, at the moment, the user can correct the identified article B as the article A, and the device deletes the article A from the currently stored article set according to the article A corrected by the user.
In another example, a user does not perform a code scanning operation when storing an article a, the device identifies the article a as an article B in an image identification manner and adds the article B to a stored article set, the user performs a code scanning operation when taking out the article a, the device identifies the article a as the article a in a barcode identification manner, and at this time, the user can correct the article B recorded in the currently stored article set as the article a and then delete the article a from the currently stored article set.
When the article storage device interacts with the user, the article storage device can interact with the user through an interface of the article storage device, an application program interface on the terminal device bound with the article storage device, or voice interaction and the like. The following description will be given taking an example of interface interaction with an article storage device.
Fig. 5 is an exemplary diagram of an interface when the deposit is successful, as shown in fig. 5, the article storage device recognizes that the user has deposited the article a, and because the deposit is successful, the article storage device directly determines that the operation result is successful, that is, the deposit is successful, so that a prompt message "you have deposited the article a and ask you to confirm" is displayed on a screen of the article storage device, the user clicks a confirmation button below the prompt message to confirm, and the article storage device receives confirmation information of the user, so that the article a is added to the currently stored article set.
Fig. 6 is an exemplary view of an interface when the removal fails, and as shown in fig. 6, an item is item a, and the information stored at the time of the storage is "item a". When the user takes out the item, the item storage device recognizes that the user takes out the item B, and if the item B does not exist in the currently stored item set, the item storage device displays "do you take out the item B? If not, please correct the prompt message, the user clicks a correction button below the prompt message, an input box is displayed on a screen of the article storage device, the user inputs the actual type of the taken article in the input box, namely the correction information of the article A, and clicks a confirmation button below the input box, and the article storage device deletes the article A from the currently stored article set according to the correction information input by the user.
The following describes a process of determining the actual type of the item by the device according to whether the user performs the code scanning operation on the item in step S203.
The actual type of article may be determined in any of the following two ways.
In the first mode, the device determines a target identification mode according to whether a user executes a code scanning operation, wherein the target identification mode is a bar code identification mode or an image identification mode. The actual type of the article is identified using the target identification method.
The method selects a corresponding identification mode for identification processing according to whether the user executes code scanning operation, and can save calculation cost.
Specifically, if the target recognition mode is an image recognition mode, the device selects one frame of image in the image sequence, and performs image recognition on the article image area of the frame of image to obtain the actual type of the article. And if the target identification mode is a bar code identification mode, carrying out bar code identification by using a bar code scanner, and taking the result of the bar code identification as the actual type of the article.
Fig. 7 is a schematic flow chart of the first method for identifying the actual type of the item when the user deposits the item, as shown in fig. 7, the flow chart includes:
and S701, recognizing the code scanning operation of the user.
S702, judging whether the bar code scanner is started successfully, if so, executing the step S703, otherwise, executing the step S704.
And S703, setting the code scanning mark position to be 1.
S704, setting the code scanning mark position to 0.
S705, recognizing that the management behavior of the user is logging.
S706, determining whether the code scanning flag is 1, if yes, executing step S707, otherwise executing step S709.
And S707, identifying the actual type of the stored article by using a bar code identification mode.
And S708, setting the code scanning mark position to 0.
And setting the code scanning mark position to be 0 after the actual type of the article is obtained for the next identification process.
And S709, identifying the actual type of the stored article by using an image identification mode.
And S710, obtaining the actual type of the stored article.
Fig. 8 is a schematic flow chart of identifying the actual type of an article by using the first method when the user takes out the article, as shown in fig. 8, the flow chart includes:
s801, recognizing that the management behavior of the user is extraction.
S802, code scanning operation of the user is identified.
And S803, judging whether the starting of the bar code scanner is successful, if so, executing the step S804, otherwise, executing the step S805.
S804, setting the code scanning mark position to be 1.
S805, setting the code scanning mark position to 0.
S806, judging whether the code scanning flag bit is 1, if so, executing the step S807, otherwise, executing the step S809.
S807, the actual type of the taken article is identified by using a bar code identification mode.
And S808, setting the code scanning mark position to be 0.
And setting the code scanning mark position to be 0 after the actual type of the article is obtained for the next identification process.
And S809, identifying the actual type of the taken-out article by using an image identification method.
S810, obtaining the actual type of the taken-out article.
In the second mode, the device first performs image recognition on an article image area of one frame of image in the image sequence to obtain the selectable type of the article, and then determines the actual type of the article according to whether the user performs the code scanning operation and the selectable type of the article.
Specifically, if the user performs a barcode scanning operation, the selectable category is updated according to the result of the barcode recognition, and the updated selectable category is used as the actual category of the article. If the user does not perform a code scanning operation, the selectable categories of items are taken as the actual categories of items.
In the method, the type is firstly identified by using an image identification method when the identification process is started, so that the identification speed can be increased, and the user experience is further improved.
Fig. 9 is a schematic flow chart of identifying the actual type of an item by a second method when a user deposits the item, as shown in fig. 9, the flow chart includes:
and S901, identifying code scanning operation of a user.
S902, identifying the selectable types of the stored articles by using an image identification mode.
And S903, judging whether the bar code scanner is started successfully, if so, executing the step S904, otherwise, executing the step S905.
And S904, setting the code scanning mark position to be 1.
S905, setting the code scanning mark position to be 0.
And S906, identifying the management behavior of the user as logging.
S907 determines whether the code scanning flag is 1, if yes, step S908 is executed, otherwise step S911 is executed.
And S908, identifying by using a bar code identification mode.
And S909, setting the code scanning mark position to 0.
And setting the code scanning mark position to be 0 after the actual type of the article is obtained for the next identification process.
S910, updating the selectable categories by using the bar code identification result to obtain the actual categories.
And S911, taking the selectable types as actual types.
S912, obtaining the actual type of the stored article.
Fig. 10 is a schematic flow chart of identifying the actual type of an article by using the second method when the user takes out the article, and as shown in fig. 10, the flow chart includes:
s1001, recognizing that the management behavior of the user is extraction.
S1002, identifying the selectable types of the taken-out articles by using an image identification mode.
And S1003, recognizing the code scanning operation of the user.
And S1004, judging whether the bar code scanner is started successfully, if so, executing a step S1005, otherwise, executing a step S1006.
S1005, setting the code scanning mark position to 1.
S1006, setting the code scanning mark position to 0.
S1007, determining whether the code scanning flag is 1, if so, performing step S1008, otherwise, performing step S1011.
And S1008, identifying by using a bar code identification mode.
S1009, setting the code scanning mark position to 0.
And setting the code scanning mark position to be 0 after the actual type of the article is obtained for the next identification process.
And S1010, updating the selectable categories by using the bar code identification result to obtain the actual categories.
S1011, the selectable categories are used as actual categories.
S1012, the actual type of the taken-out article is obtained.
In the above two methods, when the barcode identification method is used to identify the type, based on the barcode scanner illustrated in fig. 1a and 1b, the barcode scanner scans the type of the article. The barcode scanner may be a Charge Coupled Device (CCD), a laser scanner, an optical pen scanner, or the like. The CCD scanner mainly uses a fixed light beam (usually a flood light source of a light emitting diode) to illuminate the whole bar code, reflects the bar code symbol onto the array of photosensitive elements, and recognizes the bar code symbol through photoelectric conversion. Some CCD scanners can recognize not only one-dimensional barcodes and row-by-row two-dimensional barcodes, but also matrix two-dimensional barcodes. The laser scanner is a scanner using laser as a light source. The laser scanner can scan at a long distance and has higher scanning precision. When the light pen scanner is used, a user needs to contact the light pen with the surface of a bar code, when a light spot emitted by the light pen sweeps across the bar code from left to right, part of light rays in the 'empty' part are reflected, and part of light rays in the 'bar' part are absorbed. After photoelectric conversion, the electric signal is amplified and shaped for decoder.
When the image recognition mode is used for recognizing the types, in one mode, the feature extractor and the classifier can be used for realizing the feature extraction, the learning training of the classifier and the test recognition of the image. The feature extractor may include a Histogram of Oriented Gradients (HOG), a Local Binary Pattern (LBP), a Deformable Part Model (DPM), and the like. The classifier may use Support Vector Machines (SVMs), Adaboost, decision trees, bayesian networks, neural networks, and the like. Alternatively, deep learning methods can be used for training and recognition, such as Fast-CNN, YOLO 9000, and the like.
The following describes the process of performing target tracking on the hand image area in step S202 to obtain the management behavior of the user on the article and whether the user performs a code scanning operation on the article.
Specifically, the device performs target tracking on the hand image area to obtain a hand movement track, and obtains a management behavior of the user and whether the user executes a code scanning operation according to the hand movement track.
Fig. 11 is a schematic diagram illustrating the process of obtaining management behavior by using object tracking and whether the user performs a code scanning operation, as shown in fig. 11, first, the device first identifies a hand image area in the image. Secondly, target tracking is carried out on the area, and three sub-actions of going, returning and stopping are identified according to the track tracked by the hand. Furthermore, the management behavior of the user can be identified according to the go and return sub-actions, and whether the user executes the code scanning operation or not can be identified according to the stay sub-action.
When the hand image area in the image is recognized, the image area including the hand and the article can be detected. The hand region detection method may be the above-described feature extractor and classifier detection method, or may be the above-described deep learning method.
When the target tracking is carried out, the target tracking of the hand image area can be carried out by using methods such as Kalman filtering, particle filtering and the like, and three sub-actions of going, returning and staying are identified according to the tracking track. Fig. 12a is a diagram illustrating an example of recognition of a go sub-action, fig. 12b is a diagram illustrating an example of recognition of a return sub-action, and fig. 12c is a diagram illustrating an example of recognition of a stay sub-action. As shown in fig. 12a, when the tracking track crosses the boundary between the refrigerator body and the door body from the direction of the refrigerator door body to the direction of the refrigerator body, the sub-action is go, and the management action is stored. As shown in fig. 12b, when the tracking track crosses the boundary between the refrigerator body and the door body from the refrigerator body direction to the refrigerator door body direction, the sub-action is back, and the management action is taken out. As shown in fig. 12c, when the length of time that the tracking track stays in the code scanning stay area exceeds the preset length of time, the subaction is determined as stay, and it is determined that the user has performed the code scanning operation.
Fig. 13 is a block diagram of an article management apparatus according to an embodiment of the present application, and as shown in fig. 13, the apparatus includes:
the processing module 1301 is configured to acquire an image sequence when a user manages an article in an article storage device, where an image of the image sequence includes a hand image area of the user and an article image area; target tracking is carried out on the hand image area, and management behaviors of the user on the article and whether the user performs code scanning operation on the article are obtained, wherein the management behaviors comprise storing the article into the article storage device or taking the article out of the article storage device; and determining the actual type of the item according to whether the user performs code scanning operation on the item.
A pushing module 1302, configured to push prompt information and update a current stored item set of the item storage device, where the prompt information is used to indicate an actual type of the item and a management behavior of the user.
As an optional implementation, the processing module 1301 is specifically configured to:
determining a target identification mode according to whether the user executes code scanning operation, wherein the target identification mode comprises a bar code identification mode or an image identification mode; and identifying the actual type of the article by using the target identification mode.
As an optional implementation, the processing module 1301 is specifically configured to:
and if the target identification mode is an image identification mode, carrying out image identification on the article image area of one frame of image of the image sequence to obtain the actual type of the article.
As an optional implementation, the processing module 1301 is specifically configured to:
and if the target identification mode is a bar code identification mode, taking the result of bar code identification as the actual type of the article.
As an optional implementation, the processing module 1301 is specifically configured to:
carrying out image recognition on the article image area of one frame of image of the image sequence to obtain the selectable type of the article; determining the actual type of the item according to whether the user performs a code scanning operation and the selectable type of the item.
As an optional implementation, the processing module 1301 is specifically configured to:
and if the user executes the code scanning operation, updating the selectable categories according to the result of the bar code identification, and taking the updated selectable categories as the actual categories of the articles.
As an optional implementation, the processing module 1301 is specifically configured to:
and if the user does not execute the code scanning operation, taking the selectable type of the article as the actual type of the article.
As an optional implementation manner, the pushing module 1302 is specifically configured to:
pushing the prompt information; and if the operation result of the management behavior of the user on the article is judged to be successful, updating the current stored article set according to the confirmation information of the user on the prompt information.
As an optional implementation manner, the pushing module 1302 is specifically configured to:
pushing the prompt information; and if the operation result of the user on the management behavior of the article is judged to be operation failure, updating the actual type of the article according to the correction information of the user on the prompt information, and updating the currently stored article according to the actual type of the article.
As an optional implementation, the processing module 1301 is further configured to:
and judging the operation result of the management action of the user on the article according to the actual type of the article and the current stored article set.
As an optional implementation, the processing module 1301 is specifically configured to:
and if the management action is to take the article out of the article storage device, and the current stored article set comprises the article matched with the article, determining that the operation result of the management action of the user on the article is successful.
The items matched with the items comprise the items with the same types as the actual types, or comprise the items with the differences between the types and the actual types smaller than a preset threshold value.
As an optional implementation, the processing module 1301 is specifically configured to:
carrying out target tracking on the hand image area to obtain a hand movement track; and obtaining the management behavior of the user and whether the user executes code scanning operation or not according to the hand movement track.
The article management device provided in the embodiment of the present application may perform the method steps in the above method embodiments, and the implementation principle and the technical effect are similar, which are not described herein again.
It should be noted that the division of the modules of the above apparatus is only a logical division, and the actual implementation may be wholly or partially integrated into one physical entity, or may be physically separated. And these modules can be realized in the form of software called by processing element; or may be implemented entirely in hardware; and part of the modules can be realized in the form of calling software by the processing element, and part of the modules can be realized in the form of hardware. For example, the determining module may be a processing element separately set up, or may be implemented by being integrated in a chip of the apparatus, or may be stored in a memory of the apparatus in the form of program code, and the function of the determining module is called and executed by a processing element of the apparatus. Other modules are implemented similarly. In addition, all or part of the modules can be integrated together or can be independently realized. The processing element described herein may be an integrated circuit having signal processing capabilities. In implementation, each step of the above method or each module above may be implemented by an integrated logic circuit of hardware in a processor element or an instruction in the form of software.
For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as: one or more Application Specific Integrated Circuits (ASICs), or one or more microprocessors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs), among others. For another example, when some of the above modules are implemented in the form of a processing element scheduler code, the processing element may be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor that can call program code. As another example, these modules may be integrated together, implemented in the form of a system-on-a-chip (SOC).
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the application to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.
The embodiment of the present application further provides an intelligent refrigerator, including:
the camera is positioned at the top of the intelligent refrigerator and used for collecting images of a user during management of articles in the intelligent refrigerator; the bar code scanner is positioned in the visual field range of the camera; and the processor is used for identifying the article management behavior of the user according to the image collected by the camera and whether the user executes code scanning operation, selecting the actual type of the article based on the image identification mode according to whether the user executes the code scanning operation, and/or starting the bar code scanner, identifying the actual type of the article in the bar code identification mode, pushing prompt information and updating the current stored article set, wherein the prompt information is used for indicating the actual type of the article and the management behavior of the user.
Fig. 14 is a schematic structural diagram of an article storage device 1400 according to an embodiment of the present disclosure. As shown in fig. 14, the article storage device may include: the system comprises a processor 141, a memory 142, a communication interface 143, a system bus 144, a camera 145 and a bar code scanner 146, wherein the memory 142, the communication interface 143, the camera 145 and the bar code scanner 146 are connected with the processor 141 through the system bus 144 and complete mutual communication, the memory 142 is used for storing computer execution instructions, the communication interface 143 is used for communicating with other equipment, and the camera 145 is used for collecting images when a user manages articles and sending the images to the processor 141. The barcode scanner 146 is used to scan a barcode on the item to obtain the actual type of the item and send it to the processor 141. The processor 141, when executing the computer program, implements the solution of the embodiment as described above with reference to fig. 2 to 12.
The system bus mentioned in fig. 14 may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The system bus may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus. The communication interface is used for realizing communication between the database access device and other equipment (such as a client, a read-write library and a read-only library). The memory may comprise Random Access Memory (RAM) and may also include non-volatile memory (non-volatile memory), such as at least one disk memory.
The processor may be a general-purpose processor, including a central processing unit CPU, a Network Processor (NP), and the like; but also a digital signal processor DSP, an application specific integrated circuit ASIC, a field programmable gate array FPGA or other programmable logic device, discrete gate or transistor logic, discrete hardware components.
Optionally, an embodiment of the present application further provides a storage medium, where instructions are stored in the storage medium, and when the storage medium is run on a computer, the storage medium causes the computer to execute the method according to the embodiment shown in fig. 2 to 12.
Optionally, an embodiment of the present application further provides a chip for executing the instruction, where the chip is configured to execute the method in the embodiment shown in fig. 2 to 12.
The embodiment of the present application further provides a program product, where the program product includes a computer program, where the computer program is stored in a storage medium, and the computer program can be read from the storage medium by at least one processor, and when the computer program is executed by the at least one processor, the method of the embodiment shown in fig. 2 to 12 can be implemented.
In the embodiments of the present application, "at least one" means one or more, "a plurality" means two or more. "and/or" describes the association relationship of the associated objects, meaning that there may be three relationships, e.g., a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone, wherein A and B can be singular or plural. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship; in the formula, the character "/" indicates that the preceding and following related objects are in a relationship of "division". "at least one of the following" or similar expressions refer to any combination of these items, including any combination of the singular or plural items. For example, at least one (one) of a, b, or c, may represent: a, b, c, a-b, a-c, b-c, or a-b-c, wherein a, b, c may be single or multiple.
It is to be understood that the various numerical references referred to in the embodiments of the present application are merely for convenience of description and distinction and are not intended to limit the scope of the embodiments of the present application.
It should be understood that, in the embodiment of the present application, the sequence numbers of the above-mentioned processes do not mean the execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiment of the present application.
Finally, it should be noted that: the above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present application.
Claims (10)
1. An item management method, comprising:
acquiring an image sequence when a user manages articles in an article storage device, wherein images of the image sequence comprise hand image areas of the user and article image areas;
performing target tracking on the hand image area to obtain a management behavior of the user on the article and whether the user performs code scanning operation on the article, wherein the management behavior comprises storing the article in the article storage device or taking the article out of the article storage device;
determining the actual type of the article according to whether the user performs code scanning operation on the article;
pushing prompt information and updating a current stored item set of the item storage device, wherein the prompt information is used for indicating the actual type of the item and the management behavior of the user.
2. The method of claim 1, wherein determining the actual type of the item based on whether the user performed a code scanning operation on the item comprises:
determining a target identification mode according to whether the user executes code scanning operation, wherein the target identification mode comprises a bar code identification mode or an image identification mode;
and identifying the actual type of the article by using the target identification mode.
3. The method of claim 2, wherein said identifying an actual type of said item using said object recognition approach comprises:
if the target identification mode is an image identification mode, carrying out image identification on the article image area of one frame of image of the image sequence to obtain the actual type of the article;
and/or the presence of a gas in the gas,
and if the target identification mode is a bar code identification mode, taking the result of bar code identification as the actual type of the article.
4. The method of claim 1, wherein determining the actual type of the item based on whether the user performed a code scanning operation on the item comprises:
carrying out image recognition on the article image area of one frame of image of the image sequence to obtain the selectable type of the article;
determining the actual type of the item according to whether the user performs a code scanning operation and the selectable type of the item.
5. The method of claim 4, wherein determining the actual type of the item based on whether the user performed a code scanning operation and the selectable types of items comprises:
if the user executes the code scanning operation, updating the selectable categories according to the result of the bar code identification, and taking the updated selectable categories as the actual categories of the articles;
and/or the presence of a gas in the gas,
and if the user does not execute the code scanning operation, taking the selectable type of the article as the actual type of the article.
6. The method of any of claims 1-5, wherein pushing the hint information and updating the current set of stored items of the item storage device comprises:
pushing the prompt information;
if the operation result of the user on the management behavior of the article is judged to be successful, updating the current stored article set according to the confirmation information of the user on the prompt information; and/or if the operation result of the user on the management behavior of the article is judged to be operation failure, updating the actual type of the article according to the correction information of the user on the prompt information, and updating the currently stored article according to the actual type of the article.
7. The method of claim 6, wherein before updating the actual type of the item according to the correction information of the user for the prompt message, further comprising:
and judging the operation result of the management action of the user on the article according to the actual type of the article and the current stored article set.
8. The method of claim 7, wherein determining the operation result of the user's management action on the item according to the actual type of the item and the current stored item set comprises:
if the management action is to take the article out of the article storage device, and the current stored article set comprises the article matched with the article, determining that the operation result of the management action of the user on the article is successful;
the items matched with the items comprise the items with the same types as the actual types, or comprise the items with the differences between the types and the actual types smaller than a preset threshold value.
9. The method of any one of claims 1-5, wherein the target tracking the hand image area to obtain the management behavior of the user on the item and whether the user performs a code scanning operation on the item comprises:
carrying out target tracking on the hand image area to obtain a hand movement track;
and obtaining the management behavior of the user and whether the user executes code scanning operation or not according to the hand movement track.
10. An intelligent refrigerator, comprising:
the camera is positioned at the top of the intelligent refrigerator and used for collecting images of a user during management of articles in the intelligent refrigerator; the bar code scanner is positioned in the visual field range of the camera;
and the processor is used for identifying the article management behavior of the user according to the image collected by the camera and whether the user executes code scanning operation, selecting the actual type of the article based on the image identification mode according to whether the user executes the code scanning operation, and/or starting the bar code scanner, identifying the actual type of the article in the bar code identification mode, pushing prompt information and updating the current stored article set, wherein the prompt information is used for indicating the actual type of the article and the management behavior of the user.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010236472.2A CN113468910A (en) | 2020-03-30 | 2020-03-30 | Article management method and device and intelligent refrigerator |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010236472.2A CN113468910A (en) | 2020-03-30 | 2020-03-30 | Article management method and device and intelligent refrigerator |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113468910A true CN113468910A (en) | 2021-10-01 |
Family
ID=77864899
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010236472.2A Pending CN113468910A (en) | 2020-03-30 | 2020-03-30 | Article management method and device and intelligent refrigerator |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113468910A (en) |
Citations (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20000039121A (en) * | 1998-12-11 | 2000-07-05 | 전주범 | Control method for indicating food storing state using barcode in refrigerator |
| CN102538379A (en) * | 2011-12-19 | 2012-07-04 | 四川长虹电器股份有限公司 | Refrigerator food information management device and method |
| CN102741874A (en) * | 2009-12-13 | 2012-10-17 | 因特伟特公司 | Systems and methods for purchasing products from a retail establishment using a mobile device |
| CN203588313U (en) * | 2013-09-30 | 2014-05-07 | 宁波高新区恒诺科技有限公司 | Intelligent supermarket management system based on Internet of Things technology |
| CN105184204A (en) * | 2015-07-10 | 2015-12-23 | 百度在线网络技术（北京）有限公司 | Article identification device, smart refrigerator and user terminal |
| CN105245828A (en) * | 2015-09-02 | 2016-01-13 | 北京旷视科技有限公司 | Item analysis method and equipment |
| CN106204573A (en) * | 2016-07-07 | 2016-12-07 | Tcl集团股份有限公司 | A kind of food control method and system of intelligent refrigerator |
| US20170255899A1 (en) * | 2016-03-02 | 2017-09-07 | Toshiba Tec Kabushiki Kaisha | System and method for managing inventory based on captured images |
| WO2018023445A1 (en) * | 2016-08-02 | 2018-02-08 | 步晓芳 | Interest display method in monitoring technique for shelf life of food, and refrigerator |
| CN108469148A (en) * | 2018-05-31 | 2018-08-31 | 上海理工大学 | The refrigerator of intelligent inventory management |
| CN108507270A (en) * | 2017-02-24 | 2018-09-07 | 九阳股份有限公司 | A kind of food materials of refrigerator determine method and food materials determining device |
| CN208284026U (en) * | 2018-06-03 | 2018-12-25 | 姚爱军 | A kind of lift smart shopper case |
| CN109353397A (en) * | 2018-09-20 | 2019-02-19 | 北京旷视科技有限公司 | Merchandise control methods, devices and systems and storage medium and shopping cart |
| CN110245887A (en) * | 2019-05-17 | 2019-09-17 | 顺丰科技有限公司 | A kind of article control method, system and storage medium |
| CN110608574A (en) * | 2019-09-16 | 2019-12-24 | 珠海格力电器股份有限公司 | Refrigerator and fresh-keeping control method thereof |
| CN110671881A (en) * | 2019-09-16 | 2020-01-10 | 珠海格力电器股份有限公司 | Refrigerator fresh-keeping control method and system |
| CN110671883A (en) * | 2019-09-16 | 2020-01-10 | 珠海格力电器股份有限公司 | Control method and system for automatically determining food material fresh-keeping time limit of refrigerator and refrigerator |
-
2020
- 2020-03-30 CN CN202010236472.2A patent/CN113468910A/en active Pending
Patent Citations (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20000039121A (en) * | 1998-12-11 | 2000-07-05 | 전주범 | Control method for indicating food storing state using barcode in refrigerator |
| CN102741874A (en) * | 2009-12-13 | 2012-10-17 | 因特伟特公司 | Systems and methods for purchasing products from a retail establishment using a mobile device |
| CN102538379A (en) * | 2011-12-19 | 2012-07-04 | 四川长虹电器股份有限公司 | Refrigerator food information management device and method |
| CN203588313U (en) * | 2013-09-30 | 2014-05-07 | 宁波高新区恒诺科技有限公司 | Intelligent supermarket management system based on Internet of Things technology |
| CN105184204A (en) * | 2015-07-10 | 2015-12-23 | 百度在线网络技术（北京）有限公司 | Article identification device, smart refrigerator and user terminal |
| CN105245828A (en) * | 2015-09-02 | 2016-01-13 | 北京旷视科技有限公司 | Item analysis method and equipment |
| US20170255899A1 (en) * | 2016-03-02 | 2017-09-07 | Toshiba Tec Kabushiki Kaisha | System and method for managing inventory based on captured images |
| CN106204573A (en) * | 2016-07-07 | 2016-12-07 | Tcl集团股份有限公司 | A kind of food control method and system of intelligent refrigerator |
| WO2018023445A1 (en) * | 2016-08-02 | 2018-02-08 | 步晓芳 | Interest display method in monitoring technique for shelf life of food, and refrigerator |
| CN108507270A (en) * | 2017-02-24 | 2018-09-07 | 九阳股份有限公司 | A kind of food materials of refrigerator determine method and food materials determining device |
| CN108469148A (en) * | 2018-05-31 | 2018-08-31 | 上海理工大学 | The refrigerator of intelligent inventory management |
| CN208284026U (en) * | 2018-06-03 | 2018-12-25 | 姚爱军 | A kind of lift smart shopper case |
| CN109353397A (en) * | 2018-09-20 | 2019-02-19 | 北京旷视科技有限公司 | Merchandise control methods, devices and systems and storage medium and shopping cart |
| CN110245887A (en) * | 2019-05-17 | 2019-09-17 | 顺丰科技有限公司 | A kind of article control method, system and storage medium |
| CN110608574A (en) * | 2019-09-16 | 2019-12-24 | 珠海格力电器股份有限公司 | Refrigerator and fresh-keeping control method thereof |
| CN110671881A (en) * | 2019-09-16 | 2020-01-10 | 珠海格力电器股份有限公司 | Refrigerator fresh-keeping control method and system |
| CN110671883A (en) * | 2019-09-16 | 2020-01-10 | 珠海格力电器股份有限公司 | Control method and system for automatically determining food material fresh-keeping time limit of refrigerator and refrigerator |
Non-Patent Citations (4)
| Title |
|---|
| SHOUMING QIAO; HONGZHEN ZHU; LIJUAN ZHENG; JIANRUI DING: "Intelligent Refrigerator based on Internet of Things", 2017 IEEE INTERNATIONAL CONFERENCE ON COMPUTATIONAL SCIENCE AND ENGINEERING (CSE) AND IEEE INTERNATIONAL CONFERENCE ON EMBEDDED AND UBIQUITOUS COMPUTING (EUC), 10 August 2017 (2017-08-10), pages 1 - 4 * |
| 余雷, 杨丽君, 伍艳泽, 余艳萍: "基于条形码技术的物流管理系统", 内蒙古科技与经济, no. 15, 25 March 2005 (2005-03-25) * |
| 方波;吕秀凤;程春明;李子胜;王波;: "基于动态拍照的食品图像识别技术研究", 家电科技, no. 04, 15 April 2016 (2016-04-15) * |
| 洪鑫: "基于用户体验的智能冰箱交互设计研究", CNKI优秀硕士学位论文全文库, 31 May 2018 (2018-05-31), pages 1 - 138 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Dendorfer et al. | Motchallenge: A benchmark for single-camera multiple target tracking | |
| US11978106B2 (en) | Method and non-transitory, computer-readable storage medium for deep learning model based product matching using multi modal data | |
| US11638490B2 (en) | Method and device for identifying product purchased by user and intelligent shelf system | |
| JP2021515297A (en) | Automatic sales methods and equipment, as well as computer-readable storage media | |
| JP2023534261A (en) | Barcode scanning based on gesture detection and analysis | |
| JP6618395B2 (en) | Apparatus, program, and method for predicting position of survey object by action value | |
| CN109033985B (en) | Commodity identification processing method, device, equipment, system and storage medium | |
| CN104268006A (en) | Keyboard and mouse script playback method and device | |
| CN104462530A (en) | Method and device for analyzing user preferences and electronic equipment | |
| WO2021072699A1 (en) | Irregular scan detection for retail systems | |
| CN112417970A (en) | Target object identification method, device and electronic system | |
| CN115131604A (en) | Multi-label image classification method and device, electronic equipment and storage medium | |
| CN117115571A (en) | Fine-grained intelligent commodity identification method, device, equipment and medium | |
| CN111126457A (en) | Information acquisition method and device, storage medium and electronic device | |
| WO2020107184A1 (en) | Model selection method and terminal | |
| CN113468910A (en) | Article management method and device and intelligent refrigerator | |
| CN112561954A (en) | Method and device for determining tracking track of target object and storage medium | |
| KR102519711B1 (en) | Method and system for recognizing barcodes based on images | |
| US20230235495A1 (en) | Usage dependent user prompting | |
| CN116129523A (en) | Action recognition method, device, terminal and computer readable storage medium | |
| EP3989105B1 (en) | Embedded device based detection system | |
| CN115017413A (en) | Recommendation method and device, computing equipment and computer storage medium | |
| CN115393755A (en) | Visual target tracking method, device, equipment and storage medium | |
| JP7108553B2 (en) | Planogram information generation device and planogram information generation program | |
| CN112905875A (en) | Offline interactive content recommendation method and system and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |