CN110762943B - Article display method and device and household appliance - Google Patents
Article display method and device and household appliance Download PDFInfo
- Publication number
- CN110762943B CN110762943B CN201810834839.3A CN201810834839A CN110762943B CN 110762943 B CN110762943 B CN 110762943B CN 201810834839 A CN201810834839 A CN 201810834839A CN 110762943 B CN110762943 B CN 110762943B
- Authority
- CN
- China
- Prior art keywords
- article
- image
- item
- model
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
- F25—REFRIGERATION OR COOLING; COMBINED HEATING AND REFRIGERATION SYSTEMS; HEAT PUMP SYSTEMS; MANUFACTURE OR STORAGE OF ICE; LIQUEFACTION SOLIDIFICATION OF GASES
- F25D—REFRIGERATORS; COLD ROOMS; ICE-BOXES; COOLING OR FREEZING APPARATUS NOT OTHERWISE PROVIDED FOR
- F25D29/00—Arrangement or mounting of control or safety devices
- F25D29/003—Arrangement or mounting of control or safety devices for movable devices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/20—Scenes; Scene-specific elements in augmented reality scenes
-
- F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
- F25—REFRIGERATION OR COOLING; COMBINED HEATING AND REFRIGERATION SYSTEMS; HEAT PUMP SYSTEMS; MANUFACTURE OR STORAGE OF ICE; LIQUEFACTION SOLIDIFICATION OF GASES
- F25D—REFRIGERATORS; COLD ROOMS; ICE-BOXES; COOLING OR FREEZING APPARATUS NOT OTHERWISE PROVIDED FOR
- F25D2500/00—Problems to be solved
- F25D2500/06—Stock management
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Chemical & Material Sciences (AREA)
- Combustion & Propulsion (AREA)
- Mechanical Engineering (AREA)
- Thermal Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Processing Or Creating Images (AREA)
- Cold Air Circulating Systems And Constructional Details In Refrigerators (AREA)
Abstract
The invention discloses a method and a device for displaying articles and household electrical appliance. Wherein, the method comprises the following steps: acquiring an article image in an accommodating space of equipment; inputting the article image into a first model for analysis, and obtaining article information contained in the containing space and a component of an article corresponding to the article information, wherein the first model is obtained by training a plurality of groups of data, and each group of data in the plurality of groups of data comprises: a first marker for indicating an item in the sample item image, a second marker for indicating a component of the item, and the sample item image; the article and/or component parts of the article are displayed. The refrigerator solves the technical problems that the user forgets the specific position stored in the refrigerator easily and the user cannot find the specific position conveniently along with the increase of the food stored in the refrigerator.
Description
Technical Field
The invention relates to the field of image recognition, in particular to a method and a device for displaying articles and household electrical appliances.
Background
With the improvement of the living standard of a user, the application of the refrigerator is more and more popular, various food materials and food are stored in the refrigerator when the user uses the refrigerator, and the freshness of the articles stored in the refrigerator is often forgotten when the user uses the food materials and food, so that the waste of the food or the food materials is caused; secondly, with the increase of food materials and food stored in the refrigerator, a user easily forgets the specific positions and the specific number of the food materials and the food, the time for the user to search for the needed food or the food materials is long, and the time of the user is wasted.
In view of the above problems, no effective solution has been proposed.
Disclosure of Invention
The embodiment of the invention provides a method and a device for displaying articles and household electrical appliance equipment, and at least solves the technical problems that as the number of foods stored in a refrigerator increases, a user easily forgets a specific position stored in the refrigerator, and the user is inconvenient to search.
According to an aspect of an embodiment of the present invention, there is provided a method for displaying an article, including: acquiring an article image in an accommodating space of equipment; inputting the article image into a first model for analysis, and obtaining article information contained in the containing space and a component of an article corresponding to the article information, wherein the first model is obtained by training a plurality of groups of data, and each group of data in the plurality of groups of data comprises: a first marker for indicating an item in the sample item image, a second marker for indicating a component of the item, and the sample item image; the article and/or component parts of the article are displayed.
Optionally, the first model is trained by: marking the sample article image by adopting a first mark to obtain a first marked image; splitting the sample article image to obtain a plurality of sub-images; marking the articles in the sub-images by using a second marker to obtain a plurality of second marked images; the first model is trained using the first labeled image and the plurality of second labeled images.
Optionally, the first model comprises a plurality of region models; before inputting the item image to the first model for analysis, the method further comprises: splitting the article image to obtain a plurality of sub-images; inputting the article image into the first model for analysis to obtain article information contained in the containing space and a component of an article corresponding to the article information, wherein the component comprises: respectively inputting a plurality of subimages into corresponding area models for analysis, and obtaining the component parts of the articles corresponding to the article information in the accommodating space, wherein the area models are obtained through training of a plurality of groups of data, and each group of data in the plurality of groups of data comprises: indicia for indicating components of the item and a sample item image; determining item information based on the obtained components of the item.
Optionally, inputting the article image into the first model for analysis, and after obtaining the article information contained in the containing space and the component of the article corresponding to the article information, determining the position information of the article corresponding to the article information in the containing space; and displaying the position information.
Optionally, determining the position information of the article in the accommodating space corresponding to the article information includes: inputting the article image into a second model for analysis, and obtaining the position information of the article corresponding to the article information in the accommodating space, wherein the second model is obtained by training a plurality of groups of data, and each group of data in the plurality of groups of data comprises: a marker indicating where the item is located in the sample image, and the sample image.
Optionally, after determining the position information of the article in the accommodating space corresponding to the article information, the method includes: and sending the position information to the mobile terminal.
Optionally, after obtaining the item information contained in the containing space and the component of the item corresponding to the item information, the method further includes: acquiring the placing time of the article information in the accommodating space; determining the time length between the placing time and the current time; comparing the duration with a threshold corresponding to the article information; and when the duration is greater than the threshold value, generating prompt information.
Optionally, the threshold corresponding to the article information is determined by: determining a threshold corresponding to each component of the article information; and taking the minimum value of the threshold values corresponding to the components as the threshold value corresponding to the article information.
According to another aspect of an embodiment of the present invention, there is provided a display device for an article, including: the acquisition device is used for acquiring an article image in the accommodating space of the equipment; the processing device inputs the article image into the first model for analysis, and obtains the article information contained in the containing space and the component of the article corresponding to the article information, wherein the first model is obtained by training a plurality of groups of data, and each group of data in the plurality of groups of data comprises: a first marker for indicating an item in the sample item image, a second marker for indicating a component of the item, and the sample item image; a display device for displaying an article and/or a component of an article.
According to another aspect of the embodiments of the present invention, there is provided a home appliance, including: the image acquisition device is used for acquiring an article image in the accommodating space of the equipment; the processor is used for inputting the article image into the first model for analysis to obtain article information contained in the containing space and a component of an article corresponding to the article information, wherein the first model is obtained through training of a plurality of groups of data, and each group of data in the plurality of groups of data comprises: a first marker for indicating an item in the sample item image, a second marker for indicating a component of the item, and the sample item image; a display screen for displaying the item and/or components of the item.
According to still another aspect of embodiments of the present invention, there is provided a storage medium including a stored program, wherein when the program is executed, a device on which the storage medium is located is controlled to perform the above method for displaying an article.
According to a further aspect of the embodiments of the present invention, there is provided a processor for executing a program, wherein the program executes the above method for displaying an article.
In the embodiment of the invention, the object image in the accommodating space of the equipment is acquired; inputting the article image into a first model for analysis, and obtaining article information contained in the containing space and a component of an article corresponding to the article information, wherein the first model is obtained by training a plurality of groups of data, and each group of data in the plurality of groups of data comprises: a first marker for indicating an item in the sample item image, a second marker for indicating a component of the item, and the sample item image; and displaying the item and/or component parts of the item. The identification of the object images in the accommodating space of the equipment and the display of the object positions are realized, and the technical problems that the specific positions stored in the refrigerator are easy to forget by a user and the user cannot conveniently find due to the increase of food stored in the refrigerator are solved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:
fig. 1 is a schematic structural diagram of another household appliance according to an embodiment of the present application;
FIG. 2 is a schematic flow chart diagram of a method of displaying an article according to an embodiment of the present application;
fig. 3 is a schematic structural diagram of a display device for an article according to an embodiment of the present application.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Fig. 1 is a schematic structural diagram of a home appliance according to an embodiment of the present application. As shown in fig. 1, the home appliance includes: image acquisition device 12, processor 14 and display screen 16, wherein:
and the image acquisition device 12 is used for acquiring an article image in the accommodating space of the equipment.
In an alternative embodiment, the device may be a refrigerator or a washing machine or the like. If the equipment is a refrigerator, a plurality of cameras can be arranged in the refrigerator to shoot pictures in the refrigerator and obtain an accommodating space, namely images of articles in the refrigerator; the articles can be food materials, fruits, cakes, etc. The image capturing device 12 may be a plurality of cameras provided in the refrigerator, and each camera may be labeled, for example, a camera for capturing an article image of a 1 st layer of the refrigerator is labeled as camera 1, and a camera for capturing an article image of a 2 nd layer of the refrigerator is labeled as camera 2.
The processor 14 is configured to input the article image into the first model for analysis, so as to obtain article information contained in the containing space and a component of an article corresponding to the article information, where the first model is obtained through training of multiple sets of data, and each set of data in the multiple sets of data includes: a first marker for indicating an item in the sample item image, a second marker for indicating a component of the item, and the sample item image.
In an optional embodiment, when the sample item image is an image of a strawberry cake, the data corresponding to the sample image includes: a first mark for indicating strawberry cake, a second mark for indicating strawberry and cake, respectively, and a mark for indicating the sample image itself.
The first model is trained by: marking the sample article image by adopting a first mark to obtain a first marked image; splitting the sample article image to obtain a plurality of sub-images; and marking the articles in the plurality of sub-images by using the second marker to obtain a plurality of second marker images.
In an alternative embodiment, if the sample item image is an image of strawberry cake, the strawberry cake is marked as "strawberry cake", resulting in a first marked image.
The strawberry cake image is split into a plurality of sub-images, namely an image of strawberry and an image of cake. And the image of the strawberry is marked as "strawberry", the image of the cake is marked as "cake", and both the image marked as "strawberry" and the image marked as "cake" are the second marked image.
The first model is trained using the first labeled image and the second labeled image.
In an alternative embodiment, the first model is trained by: marking the sample article image by adopting a first mark to obtain a first marked image; splitting the sample article image to obtain a plurality of sub-images; and marking the articles in the plurality of sub-images by using the second marker to obtain a plurality of second marker images.
In an alternative embodiment, if the sample object image is an image of a disk of potato red-cooked meat, the potato red-cooked meat is labeled as "a disk of potato red-cooked meat", resulting in a first labeled image.
The image of the potato braised meat is split into a plurality of sub-images, namely an image of the potato block and an image of the meat block. And the image of the potato piece is marked as "potato piece", the image of the meat piece is marked as "meat piece", the dish is marked as "dish", and the image marked as "potato piece", the image marked as "meat piece" and the image marked as "dish" are all the second marked images.
The first model is trained using the first labeled image and the second labeled image.
A display screen 16 for displaying the item and/or components of the item.
The display screen 16 may be disposed outside the housing of the refrigerator.
As shown in fig. 2, an article display method provided by an embodiment of the present application at least includes steps S202-S206:
step S202, an article image in the accommodating space of the equipment is acquired.
In an alternative embodiment, the device may be a refrigerator or a washing machine or the like. If the equipment is a refrigerator, a plurality of cameras can be arranged in the refrigerator to shoot pictures in the refrigerator and obtain an accommodating space, namely images of articles in the refrigerator; the articles can be food materials, fruits, cakes, etc.
Step S204, inputting the article image into the first model for analysis, and obtaining article information contained in the containing space and a component of an article corresponding to the article information.
Wherein, first model is obtained for training through multiunit data, and every group data in the multiunit data all includes: a first marker for indicating an item in the sample item image, a second marker for indicating a component of the item, and the sample item image;
in an optional embodiment, when the sample item image is an image of a strawberry cake, the data corresponding to the sample image includes: a first mark for indicating strawberry cake, a second mark for indicating strawberry and cake, respectively, and a mark for indicating the sample image itself.
The first model is trained by: marking the sample article image by adopting a first mark to obtain a first marked image; splitting the sample article image to obtain a plurality of sub-images; and marking the articles in the plurality of sub-images by using the second marker to obtain a plurality of second marker images.
In an alternative embodiment, if the sample item image is an image of strawberry cake, the strawberry cake is marked as "strawberry cake", resulting in a first marked image.
The strawberry cake image is split into a plurality of sub-images, namely an image of strawberry and an image of cake. And the image of the strawberry is marked as "strawberry", the image of the cake is marked as "cake", and both the image marked as "strawberry" and the image marked as "cake" are the second marked image.
The first model is trained using the first labeled image and the second labeled image.
In an alternative embodiment, the first model is trained by: marking the sample article image by adopting a first mark to obtain a first marked image; splitting the sample article image to obtain a plurality of sub-images; and marking the articles in the plurality of sub-images by using the second marker to obtain a plurality of second marker images.
In an alternative embodiment, if the sample object image is an image of a disk of potato red-cooked meat, the disk of potato red-cooked meat is labeled as "a disk of potato red-cooked meat", resulting in a first labeled image.
The image of the potato braised meat is split into a plurality of sub-images, namely an image of the potato block and an image of the meat block. And the image of the potato piece is marked as "potato piece", the image of the meat piece is marked as "meat piece", the dish is marked as "dish", and the image marked as "potato piece", the image marked as "meat piece" and the image marked as "dish" are all the second marked images.
The first model is trained using the first labeled image and the second labeled image.
In another optional embodiment, before inputting the image of the item to the first model for analysis, the method further comprises: and splitting the article image to obtain a plurality of sub-images. Wherein each sub-image may be a respective component of the item in the image of the item.
For example: the camera of equipment shoots the image of a pair of strawberry cake in the refrigerator, can be with the image split of this strawberry cake for strawberry image and cake image.
The object image is input to the first model for analysis to obtain the object information contained in the containing space and the component of the object corresponding to the object information, and the plurality of sub-images may be respectively input to the corresponding region models for analysis to obtain the component of the object corresponding to the object information in the containing space.
The first model comprises a plurality of area models, different area models being presettable in advance, possibly according to the use for identifying articles of different size and/or different shape. For example: the area model 1 is used to identify items like strawberries, grapes, and the area model 2 is used to identify items like plates. Wherein, the strawberry, the grape and the tray are components of the article, for example, the strawberry is a component of strawberry cake, the grape is a component of fruit salad, and the tray is a component of a tray of potato braised meat.
For example: after a camera in a refrigerator acquires a pair of images, the images can be processed according to a related algorithm, so that edge information of different articles contained in the images is acquired, the images are split into sub-images according to the edge information of the different articles, and after a strawberry cake image is acquired, the strawberry cake image is split into a sub-image 1 (an image containing the edge information of strawberries) and a sub-image 2 (an image containing the edge information of cakes) according to the edge information of strawberries in a strawberry cake photo and the edge information of cakes. The splitting rule may also identify different size and/or shape settings of the article according to each region model included in the first model. For example: if the region model is capable of identifying the grape bunch, each grape in the fruit tray is not required to be split into independent sub-images, and the grape bunch can be integrally used as one independent sub-image.
After the acquired article image is split into a plurality of sub-images, the sub-images are respectively input into corresponding area models for recognition, so that the content of each sub-image is determined according to the recognition result of each area model, namely the component of the article corresponding to the article information in the accommodating space is determined. For example, after an image of a strawberry cake is acquired, the split strawberry image is input to the area model 1, and the split cake image is input to the area model 2. So as to obtain the component of the article corresponding to the article information in the accommodating space, such as: the area model 1 and the area model 2 respectively output strawberry and cake; and determining the article information according to the obtained components of the article, namely determining the article information contained in the refrigerator as strawberry cake according to the strawberry and the cake.
Wherein, the region model is obtained through the training of multiunit data, and every group data in the multiunit data all includes: indicia for indicating components of the item and a sample item image.
In an alternative embodiment, before the image of the object in the accommodating space of the device is acquired, the object may be scanned three-dimensionally by a scanning device installed in the refrigerator, pre-recognized by the volume of the object, or pre-recognized by the color of the photographed object. The area models can be screened in the pre-recognition process, and the efficiency of article recognition is improved.
Inputting the article image into the first model for analysis, and after obtaining article information contained in the containing space and a component of an article corresponding to the article information, determining position information of the article corresponding to the article information in the containing space; the position information is displayed, so that the user can conveniently and quickly find the corresponding article.
In some embodiments of the present application, the position information of the article in the accommodating space corresponding to the article information may be determined by the following methods, but is not limited thereto: inputting the article image into a second model for analysis, and obtaining the position information of the article corresponding to the article information in the accommodating space, wherein the second model is obtained by training a plurality of groups of data, and each group of data in the plurality of groups of data comprises: a marker indicating where the item is located in the sample image, and the sample image. The position of the object in the sample image may be the layer number of the object in the sample image located in the whole refrigerator, or the left or right side or the inner or outer side of the known layer. For example, one of the sample images is a tomato on the left side of the second floor of the refrigerator, and the other is a potato on the right side of the third floor of the refrigerator.
In addition, in an alternative embodiment, a plurality of cameras may be provided in the refrigerator, and each camera may be labeled, for example, the camera for capturing the image of the article on the first layer of the refrigerator is labeled as camera 1, and the camera for capturing the image of the article on the second layer of the refrigerator is labeled as camera 2. And determining which layer of the refrigerator the article image shot by the article image is positioned according to the mark number of the camera shooting the article image.
And after the position information of the article corresponding to the article information in the accommodating space is determined, the position information is sent to the mobile terminal.
In addition, after the information of the articles accommodated in the accommodating space and the components of the articles corresponding to the information of the articles are obtained, the placing time of the information of the articles in the accommodating space is obtained; determining the time length between the placing time and the current time; comparing the duration with a threshold corresponding to the article information; and when the duration is greater than the threshold value, generating prompt information. The threshold corresponding to the article information is determined by the following method: determining a threshold corresponding to each component of the article information; and taking the minimum value of the threshold values corresponding to the components as the threshold value corresponding to the article information.
In an alternative embodiment, when the item information stored in the storage space is "strawberry cake", the item information corresponds to the item having the components of "strawberry" and "cake". When the strawberry cake is put into a refrigerator, the storage of the refrigerator records the placing time of the strawberry cake in the refrigerator as 10 am in 7 months and 1 day in 2018. And if the shelf life of the strawberries is 3 days and the shelf life of the cake is 2.5 days, determining that the shelf life of the strawberry cake is 2.5 days. In order to avoid the strawberry cake being out of date, the threshold corresponding to the strawberry cake may be set to 2 days, that is, 48 hours, the information of the items contained in the containing space obtained by the refrigerator is "strawberry cake", and after the components of the items corresponding to the item information are "strawberry" and "cake", if the time of currently collecting the image of the items is 10 o' clock in 7 months and 2 days in 2018, the time length between the placement time of the strawberry cake and the current time is 24 hours, and the 24 hours is less than 48 hours, so that the prompt information is not generated. If the time of the currently acquired article image is 10 o' clock in 7/3/2018, the time length between the strawberry cake placing time and the current time is 48.5 hours, and the 48.5 hours is more than 48 hours, the prompt information is generated. The prompt information is generated and can be sent to the mobile terminal to remind the user that the strawberry cake is about to expire soon, so that the user can eat the strawberry cake in time.
Step S206, displaying the articles and/or the components of the articles.
In particular, the items and/or components of the items may be displayed via a display screen disposed outside of the refrigerator housing.
In the embodiment of the invention, the object image in the accommodating space of the equipment is acquired; inputting the article image into a first model for analysis, and obtaining article information contained in the containing space and a component of an article corresponding to the article information, wherein the first model is obtained by training a plurality of groups of data, and each group of data in the plurality of groups of data comprises: a first marker for indicating an item in the sample item image, a second marker for indicating a component of the item, and the sample item image; and displaying the item and/or component parts of the item. The identification of the object images in the accommodating space of the equipment and the display of the object positions are realized, and the technical problems that the specific positions stored in the refrigerator are easy to forget by a user and the user cannot conveniently find due to the increase of food stored in the refrigerator are solved.
Fig. 3 is a schematic structural diagram of a display device for an article according to an embodiment of the present application. As shown in fig. 3, the apparatus includes: a collection device 32; a processing device 34; the device 36 is shown. Wherein:
the acquisition device 32 is used for acquiring an article image in the accommodating space of the equipment;
in an alternative embodiment, the device may be a refrigerator or a washing machine or the like. If the equipment is a refrigerator, a plurality of cameras can be arranged in the refrigerator to shoot pictures in the refrigerator and obtain an accommodating space, namely images of articles in the refrigerator; the articles can be food materials, fruits, cakes, etc. The capturing device 32 may be a plurality of cameras provided in the refrigerator, and each camera may be labeled, for example, a camera for capturing images of articles on the 1 st layer of the refrigerator is labeled as camera 1, and a camera for capturing images of articles on the 2 nd layer of the refrigerator is labeled as camera 2.
The processing device 34 inputs the article image into the first model for analysis, and obtains article information contained in the containing space and a component of an article corresponding to the article information, wherein the first model is obtained by training a plurality of sets of data, and each set of data in the plurality of sets of data includes: a first marker for indicating an item in the sample item image, a second marker for indicating a component of the item, and the sample item image.
In an optional embodiment, when the sample item image is an image of a strawberry cake, the data corresponding to the sample image includes: a first mark for indicating strawberry cake, a second mark for indicating strawberry and cake, respectively, and a mark for indicating the sample image itself.
The first model is trained by: marking the sample article image by adopting a first mark to obtain a first marked image; splitting the sample article image to obtain a plurality of sub-images; and marking the articles in the plurality of sub-images by using the second marker to obtain a plurality of second marker images.
In an alternative embodiment, if the sample item image is an image of strawberry cake, the strawberry cake is marked as "strawberry cake", resulting in a first marked image.
The strawberry cake image is split into a plurality of sub-images, namely an image of strawberry and an image of cake. And the image of the strawberry is marked as "strawberry", the image of the cake is marked as "cake", and both the image marked as "strawberry" and the image marked as "cake" are the second marked image.
The first model is trained using the first labeled image and the second labeled image.
In an alternative embodiment, the first model is trained by: marking the sample article image by adopting a first mark to obtain a first marked image; splitting the sample article image to obtain a plurality of sub-images; and marking the articles in the plurality of sub-images by using the second marker to obtain a plurality of second marker images.
In an alternative embodiment, if the sample object image is an image of a disk of potato red-cooked meat, the potato red-cooked meat is labeled as "a disk of potato red-cooked meat", resulting in a first labeled image.
The image of the potato braised meat is split into a plurality of sub-images, namely an image of the potato block and an image of the meat block. And the image of the potato piece is marked as "potato piece", the image of the meat piece is marked as "meat piece", the dish is marked as "dish", and the image marked as "potato piece", the image marked as "meat piece" and the image marked as "dish" are all the second marked images.
The first model is trained using the first labeled image and the second labeled image.
A display device 36 for displaying the item and/or component parts of the item.
The display device 36 may be a display provided on the refrigerator housing.
According to still another aspect of an embodiment of the present invention, there is provided a storage medium including a stored program, wherein when the program is executed, a device on which the storage medium is located is controlled to perform the above-mentioned method for displaying an article.
According to a further aspect of the embodiments of the present invention, there is provided a processor for executing a program, wherein the program executes the above-mentioned method for displaying an article.
In the above embodiments of the present invention, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In the embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units may be a logical division, and in actual implementation, there may be another division, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.
The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.
Claims (10)
1. A method of displaying an article, comprising:
acquiring an article image in an accommodating space of equipment;
inputting the article image into a first model for analysis, and obtaining article information contained in the containing space and a component of an article corresponding to the article information, wherein the first model is obtained through training of multiple groups of data, and each group of data in the multiple groups of data comprises: a first marker for indicating an item in a sample item image, a second marker for indicating a component of the item, and the sample item image;
displaying the item and/or a component of the item;
the first model comprises a plurality of region models; before inputting the item image to the first model for analysis, the method further comprises: splitting the article image to obtain a plurality of sub-images;
inputting the article image into a first model for analysis to obtain article information contained in the containing space and a component of an article corresponding to the article information, wherein the method comprises the following steps: respectively inputting the plurality of sub-images into corresponding area models for analysis to obtain the components of the articles corresponding to the article information in the accommodating space, wherein the area models are obtained through training of a plurality of groups of data, and each group of data in the plurality of groups of data comprises: indicia for indicating components of the item and the sample item image; determining the item information according to the obtained component of the item;
the first model is trained by: marking the sample article image with the first mark to obtain a first marked image; splitting the sample article image to obtain a plurality of sub-images; marking the articles in the sub-images by the second mark to obtain a plurality of second mark images; training the first model using the first labeled image and the plurality of second labeled images.
2. The method according to claim 1, wherein after inputting the article image into the first model for analysis and obtaining the article information contained in the containing space and the component of the article corresponding to the article information, the method further comprises:
determining the position information of the article corresponding to the article information in the accommodating space;
and displaying the position information.
3. The method of claim 2, wherein determining the position information of the article in the accommodating space corresponding to the article information comprises:
inputting the article image into a second model for analysis to obtain position information of an article corresponding to the article information in the accommodating space, wherein the second model is obtained through training of multiple groups of data, and each group of data in the multiple groups of data comprises: a marker for indicating where the item is located in the sample image, and the sample image.
4. The method according to claim 2, wherein after determining the position information of the article corresponding to the article information in the accommodating space, the method comprises:
and sending the position information to a mobile terminal.
5. The method of claim 1, wherein after obtaining the item information contained in the containing space and the component of the item corresponding to the item information, the method further comprises:
acquiring the placing time of the article information in the accommodating space; determining a duration between the placement time and the current time; comparing the duration with a threshold corresponding to the article information; and when the duration is greater than the threshold value, generating prompt information.
6. The method of claim 5, wherein the threshold value corresponding to the item information is determined by:
determining a threshold value corresponding to each component of the article information; and taking the minimum value of the threshold values corresponding to all the components as the threshold value corresponding to the article information.
7. An article display apparatus, comprising:
the acquisition device is used for acquiring an article image in the accommodating space of the equipment;
the processing device inputs the article image into a first model for analysis to obtain article information contained in the containing space and a component of an article corresponding to the article information, wherein the first model is obtained through training of multiple groups of data, and each group of data in the multiple groups of data comprises: a first marker for indicating an item in a sample item image, a second marker for indicating a component of the item, and the sample item image;
a display device for displaying the item and/or a component of the item;
the first model comprises a plurality of region models; the device is also used for splitting the article image to obtain a plurality of sub-images before inputting the article image to the first model for analysis;
the processing device is further configured to input the multiple sub-images into corresponding area models respectively for analysis, so as to obtain a component of the article corresponding to the article information in the accommodating space, where the area models are obtained through training of multiple sets of data, and each set of data in the multiple sets of data includes: indicia for indicating components of the item and the sample item image; determining the item information according to the obtained component of the item;
the first model is trained by: marking the sample article image with the first mark to obtain a first marked image; splitting the sample article image to obtain a plurality of sub-images; marking the articles in the sub-images by the second mark to obtain a plurality of second mark images; training the first model using the first labeled image and the plurality of second labeled images.
8. An appliance, comprising:
the image acquisition device is used for acquiring an article image in the accommodating space of the equipment;
the processor is configured to input the article image into a first model for analysis, so as to obtain article information contained in the containing space and a component of an article corresponding to the article information, where the first model is obtained through training of multiple sets of data, and each set of data in the multiple sets of data includes: a first marker for indicating an item in a sample item image, a second marker for indicating a component of the item, and the sample item image;
a display screen for displaying the item and/or components of the item;
the first model comprises a plurality of region models; the household appliance equipment is also used for splitting the article image to obtain a plurality of sub-images before inputting the article image to the first model for analysis;
the processor is further configured to input the plurality of sub-images into corresponding area models respectively for analysis, so as to obtain a component of the article corresponding to the article information in the accommodating space, where the area models are obtained through training of multiple sets of data, and each set of data in the multiple sets of data includes: indicia for indicating components of the item and the sample item image; determining the item information according to the obtained component of the item;
the first model is trained by: marking the sample article image with the first mark to obtain a first marked image; splitting the sample article image to obtain a plurality of sub-images; marking the articles in the sub-images by the second mark to obtain a plurality of second mark images; training the first model using the first labeled image and the plurality of second labeled images.
9. A storage medium, characterized in that the storage medium comprises a stored program, wherein when the program runs, the storage medium is controlled by a device to execute the method for displaying the article according to any one of claims 1 to 6.
10. A processor, characterized in that the processor is configured to run a program, wherein the program is configured to perform the method of displaying an article according to any one of claims 1 to 6 when running.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810834839.3A CN110762943B (en) | 2018-07-26 | 2018-07-26 | Article display method and device and household appliance |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810834839.3A CN110762943B (en) | 2018-07-26 | 2018-07-26 | Article display method and device and household appliance |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110762943A CN110762943A (en) | 2020-02-07 |
| CN110762943B true CN110762943B (en) | 2020-11-10 |
Family
ID=69327412
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810834839.3A Active CN110762943B (en) | 2018-07-26 | 2018-07-26 | Article display method and device and household appliance |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110762943B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111998606B (en) * | 2020-08-06 | 2022-04-05 | 厦门理工学院 | Information processing method and device for improving usability of storage cabinet |
Family Cites Families (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104061749B (en) * | 2013-03-18 | 2017-02-08 | 海尔集团公司 | Refrigerator food storage position recording method, device and terminal and refrigerator |
| US9626639B2 (en) * | 2014-09-26 | 2017-04-18 | Shyp, Inc. | Image processing and item transport |
| CN105868797B (en) * | 2015-01-22 | 2019-09-13 | 清华大学 | Identify network parameter training method, scenery kind identification method and the device of scenery type |
| CN105160332A (en) * | 2015-09-30 | 2015-12-16 | 刘毅 | Method for realizing food intake and heat management for diabetes by utilizing hyperspectral imaging technology |
| KR102545768B1 (en) * | 2015-11-11 | 2023-06-21 | 삼성전자주식회사 | Method and apparatus for processing metadata |
| CN105740820B (en) * | 2016-01-29 | 2020-01-03 | 魏志宇 | Method and system for acquiring picture for discrimination |
| US20170284733A1 (en) * | 2016-03-29 | 2017-10-05 | Teco Electric & Machinery Co., Ltd. | Remote food management system |
| US9990135B2 (en) * | 2016-09-23 | 2018-06-05 | Apple Inc. | Providing memory usage analysis by attributing memory allocations to development components |
| CN107133650A (en) * | 2017-05-10 | 2017-09-05 | 合肥华凌股份有限公司 | Food recognition methods, device and the refrigerator of refrigerator |
| CN107293098A (en) * | 2017-06-20 | 2017-10-24 | 北京京东尚科信息技术有限公司 | Method, device, terminal and the computer-readable medium of expired prompting |
-
2018
- 2018-07-26 CN CN201810834839.3A patent/CN110762943B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN110762943A (en) | 2020-02-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111080493B (en) | Dish information identification method and device and dish self-service settlement system | |
| CN108319996A (en) | Vegetable identification processing system and method, intelligent dining-table system | |
| CN108197635B (en) | Cooking mode display method and device and range hood | |
| CN110969047A (en) | Method and device for identifying food materials and refrigerator | |
| CN110610169B (en) | Picture marking method and device, storage medium and electronic device | |
| US20180211139A1 (en) | System and method for providing a recipe | |
| CN111584035A (en) | Menu recommendation method and device and refrigerator | |
| CN107918798B (en) | Food processing method and device | |
| CN111895720A (en) | Information interaction method and system of intelligent refrigerator and intelligent refrigerator | |
| CN110762943B (en) | Article display method and device and household appliance | |
| CN110675389A (en) | Food recommendation method, storage medium and intelligent household equipment | |
| CN112019800A (en) | Image sharing method and device, range hood and storage medium | |
| CN111323551A (en) | Food freshness prompting method and device and household appliance | |
| CN112419432A (en) | Method and device for controlling food in refrigerator, electronic equipment and storage medium | |
| CN109784131B (en) | Object detection method, device, storage medium and processor | |
| CN109636268B (en) | Goods monitoring system, method and device, electronic equipment and storage equipment | |
| CN116628338A (en) | Mobile terminal product recommendation method, system, computer and readable storage medium | |
| CN111159831A (en) | Method and device for predicting freshness of food materials and household appliance | |
| CN111104952A (en) | Method, system and device for identifying food types and refrigerator | |
| CN112183914B (en) | Evaluation method and device for dish items | |
| CN211854585U (en) | Refrigerator with food intelligent identification and positioning functions | |
| CN111415328B (en) | Method and device for determining article analysis data and electronic equipment | |
| CN114218415A (en) | Cooking recipe display method and device | |
| CN113566487A (en) | Food material storage system, refrigerator, food material management method and intelligent refrigerator system | |
| CN113239780A (en) | Food material determining method and device, electronic equipment, refrigerator and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |