CN116563833A

CN116563833A - Method and device for identifying article inventory, electronic equipment and storage medium

Info

Publication number: CN116563833A
Application number: CN202310451421.5A
Authority: CN
Inventors: 孙鑫; 金航; 陈冬君; 杨慧; 修竹文; 尚文超
Original assignee: Qingdao Haier Special Refrigerator Co Ltd; Qingdao Haier Smart Technology R&D Co Ltd; Haier Smart Home Co Ltd
Current assignee: Qingdao Haier Special Refrigerator Co Ltd; Qingdao Haier Smart Technology R&D Co Ltd; Haier Smart Home Co Ltd
Priority date: 2023-04-24
Filing date: 2023-04-24
Publication date: 2023-08-08

Abstract

The application relates to the technical field of image recognition, and discloses a method for recognizing an article inventory, which comprises the following steps: acquiring a picture to be identified; the picture to be identified is obtained by shooting a preset area, and the preset area is used for placing one or more articles; determining an image frame in a picture to be identified; the image frame is used for limiting all articles in the picture to be identified; and determining the inventory condition of the article according to the image frame. Shooting a preset area for placing articles to obtain a picture to be identified, determining an image frame for limiting all articles in the picture to be identified, and determining the inventory condition of the articles by utilizing the image frame. Since the manner of determining the inventory of the articles by image recognition is not affected by the weight change of the articles, the inventory condition of the articles can be determined more accurately than the manner of determining the inventory of the articles by weighing the articles. The application also discloses a device for identifying the inventory of the article, electronic equipment and a storage medium.

Description

Method and device for identifying article inventory, electronic equipment and storage medium

Technical Field

The present invention relates to the field of image recognition technology, for example, to a method and apparatus for recognizing an inventory of an article, an electronic device, and a storage medium.

Background

Typically, items are stored in the storage device. The user needs to regularly count the stock condition of the articles in the storage device. For example, in supermarkets, shops, etc. scenarios, merchants often place items in storage devices for sale. To understand the sales of items in a storage device, merchants are therefore often required to determine the inventory of items in the storage device.

In the process of implementing the embodiments of the present disclosure, it is found that at least the following problems exist in the related art:

the related technology mainly comprises the steps of arranging a weighing module on a storage device, weighing the articles through the weighing module to obtain the weight of the articles, and judging the inventory condition of the articles according to the weight of the articles. However, in the case where the weight of the article obtained by the weighing module is inaccurate, an inaccurate judgment of the inventory condition of the article may be caused. For example, in identifying the inventory of items in a freezer, if the weighing module frosts, it may not be possible to accurately obtain the weight of the items, or the weight of the items may change as the items in the freezer melt and solidify again. In this way, there may be errors in obtaining the inventory of the item based on the weight of the item, thereby reducing the accuracy of determining the inventory of the item.

It should be noted that the information disclosed in the foregoing background section is only for enhancing understanding of the background of the present application and thus may include information that does not form the prior art that is already known to those of ordinary skill in the art.

Disclosure of Invention

The following presents a simplified summary in order to provide a basic understanding of some aspects of the disclosed embodiments. This summary is not an extensive overview, and is intended to neither identify key/critical elements nor delineate the scope of such embodiments, but is intended as a prelude to the more detailed description that follows.

The embodiment of the disclosure provides a method and a device for identifying an article inventory, electronic equipment and a storage medium, which can improve the accuracy of identifying the article inventory condition.

In some embodiments, the method comprises: acquiring a picture to be identified; the picture to be identified is obtained by shooting a preset area, and the preset area is used for placing one or more articles; determining an image frame in a picture to be identified; the image frame is used for limiting all articles in the picture to be identified; and determining the inventory condition of the article according to the image frame.

In some embodiments, determining an image border in a picture to be identified includes: carrying out article identification on the picture to be identified to obtain identification information corresponding to the article; the identification information is used for representing the position identification of the object in the picture to be identified; and determining an image frame in the picture to be identified according to the identification information.

In some embodiments, performing article identification on a picture to be identified to obtain identification information corresponding to an article, including: carrying out article identification on the picture to be identified by using a first preset model to obtain the picture to be identified with the marking frame; the marking frame is used for marking the objects in the picture to be identified; and determining the labeling frame of the article as the identification information corresponding to the article.

In some embodiments, the annotation frame is a polygon; determining the image frame in the picture to be identified according to the identification information comprises the following steps: screening a first vertex, a second vertex, a third vertex and a fourth vertex from the labeling frame vertices of all the labeling frames; the first vertex is a point closest to the top left corner vertex of the picture to be identified, the second vertex is a point closest to the bottom left corner vertex of the picture to be identified, the third vertex is a point closest to the top right corner vertex of the picture to be identified, and the fourth vertex is a point closest to the bottom right corner vertex of the picture to be identified; and determining the image frame in the picture to be identified according to the first vertex, the second vertex, the third vertex and the fourth vertex.

In some embodiments, determining an image frame in the picture to be identified according to the first vertex, the second vertex, the third vertex, and the fourth vertex includes: connecting the first vertex, the second vertex, the third vertex and the fourth vertex to form a quadrilateral; and taking the quadrangle as an image frame.

In some embodiments, determining an item inventory condition from an image frame includes: acquiring the area of an image frame; acquiring the ratio of the area of the image frame to the preset area, and determining the ratio of the area of the image frame to the preset area as a target ratio; and determining the inventory condition of the article according to the target ratio.

In some embodiments, determining an item inventory condition based on the target ratio includes: matching the inventory condition corresponding to the target ratio from a preset database; and the preset database stores the corresponding relation between the target ratio and the stock condition.

In some embodiments, the apparatus for identifying an inventory of items includes a processor and a memory storing program instructions, the processor being configured to perform the method for identifying an inventory of items described above when the program instructions are executed.

In some embodiments, the electronic device includes the apparatus for identifying an inventory of items described above.

In some embodiments, the storage medium stores program instructions that, when executed, perform the method for identifying an inventory of items described above.

The method and device for identifying the inventory of the articles, the electronic equipment and the storage medium provided by the embodiment of the disclosure can realize the following technical effects:

shooting a preset area for placing articles to obtain a picture to be identified, determining an image frame for limiting all articles in the picture to be identified, and determining the inventory condition of the articles by utilizing the image frame. Since the manner of determining the inventory of the articles by image recognition is not affected by the weight change of the articles, the inventory condition of the articles can be determined more accurately than the manner of determining the inventory of the articles by weighing the articles.

The foregoing general description and the following description are exemplary and explanatory only and are not restrictive of the application.

Drawings

One or more embodiments are illustrated by way of example and not limitation in the figures of the accompanying drawings, in which like references indicate similar elements, and in which like reference numerals refer to similar elements, and in which:

FIG. 1 is a flow diagram of a method for identifying an inventory of items provided by an embodiment of the present disclosure;

FIG. 2 is a schematic diagram of the structure of an ice bin;

FIG. 3 is a schematic diagram of a picture to be identified with a label frame;

FIG. 4 is a flow diagram of another method for identifying an inventory of items provided by an embodiment of the present disclosure;

FIG. 5 is a schematic diagram of an apparatus for identifying inventory of items provided in an embodiment of the disclosure;

FIG. 6 is a schematic diagram of another configuration of an ice bin.

Detailed Description

So that the manner in which the features and techniques of the disclosed embodiments can be understood in more detail, a more particular description of the embodiments of the disclosure, briefly summarized below, may be had by reference to the appended drawings, which are not intended to be limiting of the embodiments of the disclosure. In the following description of the technology, for purposes of explanation, numerous details are set forth in order to provide a thorough understanding of the disclosed embodiments. However, one or more embodiments may still be practiced without these details. In other instances, well-known structures and devices may be shown simplified in order to simplify the drawing.

The terms first, second and the like in the description and in the claims of the embodiments of the disclosure and in the above-described figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate in order to describe embodiments of the present disclosure. Furthermore, the terms "comprise" and "have," as well as any variations thereof, are intended to cover a non-exclusive inclusion.

The term "plurality" means two or more, unless otherwise indicated.

In the embodiment of the present disclosure, the character "/" indicates that the front and rear objects are an or relationship. For example, A/B represents: a or B.

The term "and/or" is an associative relationship that describes an object, meaning that there may be three relationships. For example, a and/or B, represent: a or B, or, A and B.

The term "corresponding" may refer to an association or binding relationship, and the correspondence between a and B refers to an association or binding relationship between a and B.

In the embodiment of the disclosure, the execution subject of the method for identifying the inventory of the article is an electronic device, and the electronic device is a server, a computer, an ice chest or the like.

In some embodiments, where the electronic device is a server or computer, the server or computer is communicatively coupled to the ice bin via a connection to the Internet. The refrigerator is provided with an image acquisition device, and the image acquisition device is used for photographing a preset area to obtain a picture to be identified. The server or the computer obtains the picture to be identified through the image acquisition device on the refrigerator. The server or the computer identifies the picture to be identified so as to acquire the inventory condition of the articles in the preset area.

In some embodiments, where the electronic device is an ice bin. The refrigerator is provided with an image acquisition device, and the refrigerator uses the image acquisition device to photograph a preset area to obtain a picture to be identified. The refrigerator identifies the picture to be identified so as to acquire the inventory condition of the articles in the preset area.

Referring to fig. 1, an embodiment of the present disclosure provides a method for identifying an inventory of items, comprising:

step S101, an electronic device acquires a picture to be identified; the picture to be identified is obtained by shooting a preset area, and the preset area is used for placing one or more objects.

Step S102, the electronic equipment determines an image frame in a picture to be identified; the image frame is used for limiting all objects in the picture to be identified.

Step S103, the electronic equipment determines the inventory condition of the articles according to the image frame.

By adopting the method for identifying the inventory of the articles, which is provided by the embodiment of the disclosure, the image to be identified is obtained by shooting the preset area for placing the articles, then the image frames for limiting all the articles are determined in the image to be identified, and finally the inventory condition of the articles is determined by utilizing the image frames. Since the manner of determining the inventory of the articles by image recognition is not affected by the weight change of the articles, the inventory condition of the articles can be determined more accurately than the manner of determining the inventory of the articles by weighing the articles.

Further, the preset area is provided with a corresponding image acquisition device. The installation position and the lens inclination angle of the image acquisition device are set, so that the image acquisition device can take a picture of a corresponding preset area. Obtaining a picture to be identified, including: and shooting the corresponding preset area by using the image acquisition device to obtain a picture of the preset area. And determining the photo of the preset area as a picture to be identified.

The installation position and the inclination angle of the lens of the image acquisition device are unchanged, so that when the image acquisition device shoots a preset area, the shooting range is kept unchanged. Therefore, the area of the picture to be recognized obtained by the image pickup device is unchanged. Under the condition that the quantity of the articles stored in the preset area is changed, the relative position of the image representing the articles in the image to be identified can be correspondingly changed as the area of the shot image to be identified is unchanged. Therefore, the picture to be identified is obtained by shooting the preset area, and the image frame of the object is determined in the picture to be identified. And determining the inventory condition of the articles according to the image frames. The inventory of the articles is determined through image recognition and is not influenced by the weight change of the articles, so that the inventory condition of the articles can be determined more accurately.

The image acquisition device is provided with a self-heating device. And under the condition that the image acquisition device is covered with fog, the surface of the image acquisition device is heated by the self-heating device. Can get rid of the fog on the image acquisition device, make image acquisition device camera lens can keep dry, clean to the picture definition that waits to discern that makes image acquisition device shoot is higher.

In some embodiments, upon identification of items within the ice bin, the area within the ice bin where the items are stored is determined to be a preset area. Referring to fig. 2, fig. 2 is a schematic structural diagram. The ice chest 1 is provided with three storage devices, namely a first storage device, a second storage device and a third storage device. The first storage device corresponds to a first preset area 2 for storing articles, the second storage device corresponds to a second preset area 3 for storing articles, and the third storage device corresponds to a third preset area 4 for storing articles. The ice chest 1 is provided with three image capturing devices, a first image capturing device 5, a second image capturing device 6 and a third image capturing device 7, respectively. The first image acquisition device 5 is used for shooting an area corresponding to the first storage device for storing articles, and obtaining a picture to be identified corresponding to the first preset area 2. The second image acquisition device 6 is used for shooting an area corresponding to the second storage device for storing articles, and obtaining a picture to be identified corresponding to the second preset area 3. The third image acquisition device 7 is used for shooting an area corresponding to the third storage device for storing articles, and obtaining a picture to be identified corresponding to the third preset area 4. Therefore, each preset area is shot by the corresponding image acquisition device, so that the acquired picture to be identified corresponding to the preset area can be clearer and more complete, and the accuracy of identifying the inventory condition of the articles in the preset area is improved.

Further, determining an image frame in the picture to be identified includes: and carrying out object identification on the picture to be identified to obtain identification information corresponding to the object. The identification information is used for representing the position identification of the object in the picture to be identified. And determining an image frame in the picture to be identified according to the identification information. In this way, the image frame is determined according to the position identification of all the objects in the picture to be identified, so that all the objects in the picture to be identified can be limited in the image frame, namely, all the objects in the picture to be identified are contained in the image frame. Therefore, the condition of the articles in the preset area can be accurately determined according to the image frame.

Further, carrying out article identification on the picture to be identified to obtain identification information corresponding to the article, including: and carrying out object identification on the picture to be identified by using the first preset model to obtain the picture to be identified with the marking frame. The labeling frame is used for labeling the objects in the picture to be identified. And determining the labeling frame of the article as the identification information corresponding to the article. In this way, the labeling frame corresponding to the object is identified in the picture to be identified. Because the labeling frame marks the position information of the articles, all the articles in the image to be identified can be accurately identified, so that all the articles in the image to be identified can be limited in the image frame, and the inventory condition of the articles can be accurately determined.

Further, the first preset model is obtained by the following method, including: a plurality of first sample pictures are acquired, each first sample picture containing at least one sample item. Each sample article in each first sample picture is provided with a first sample label, and the first sample label is a marking frame for marking the sample article. Inputting each first sample picture into a preset SSD (single shottriboxDattribute) algorithm model for training to obtain a first preset model. The labeling frame for identifying the sample object is used for limiting the sample object, and the labeling frame for identifying the sample object is polygonal, such as rectangular. For example, the label box identifying the sample item is the smallest circumscribed rectangle of the sample item.

Further, after determining the inventory condition of the article according to the image frame, the method further comprises: and carrying out object identification on the picture to be identified to obtain name information corresponding to the object. In this way, the names of the articles in the inventory can be known while the inventory condition of the preset area is identified. The user can conveniently supplement the inventory for the absent articles.

Further, carrying out article identification on the picture to be identified to obtain name information corresponding to the article, including: and carrying out object identification on the picture to be identified by using the second preset model to obtain the picture to be identified with the name information. The name information is used for representing the names of the articles in the pictures to be identified. In this way, the names of the objects in the picture to be identified are identified through the second preset model, and the name information of the objects can be obtained. Therefore, the names of the articles in the inventory can be known while the inventory condition of the preset area is identified. The user can conveniently supplement the inventory for the absent articles.

Further, the second preset model is obtained by the following method, including: a plurality of second sample pictures are obtained, each second sample picture containing at least one sample article. The articles in each second sample picture are provided with a second sample label, the second sample label is name information for identifying the sample articles, each second sample picture is input into a preset residual error network algorithm model or a SENet (Squeeze-and-Excitation Networks) network algorithm model for training, and a second preset model is obtained.

Further, the labeling frame is polygonal. Determining the image frame in the picture to be identified according to the identification information comprises the following steps: and screening the first vertex, the second vertex, the third vertex and the fourth vertex from the labeling frame vertices of all the labeling frames. The first vertex is the nearest point to the top left corner vertex of the picture to be identified, and the second vertex is the nearest point to the bottom left corner vertex of the picture to be identified. The third vertex is the nearest point to the vertex of the upper right corner of the picture to be identified. The fourth vertex is the nearest point to the vertex of the right lower corner of the picture to be identified. And determining the image frame in the picture to be identified according to the first vertex, the second vertex, the third vertex and the fourth vertex. Thus, since the first vertex, the second vertex, the third vertex, and the fourth vertex are points closest to the upper left corner vertex, the lower left corner vertex, the upper right corner vertex, and the lower right corner vertex, respectively, of the picture to be recognized. Namely, the first vertex, the second vertex, the third vertex and the fourth vertex are the outermost vertices in all the vertices of the marked frames in the picture to be identified. And determining the image frame by using four vertexes at the outermost side of all vertexes of the marked frame in the picture to be identified. All the articles in the picture to be identified can be contained in the image frame, so that the inventory condition of the articles can be more accurately determined.

Further, the image frame is determined in the picture to be identified according to the first vertex, the second vertex, the third vertex and the fourth vertex. Comprising the following steps: and connecting the first vertex, the second vertex, the third vertex and the fourth vertex to form a quadrilateral. The quadrangle is taken as an image frame. In this way, the quadrangle obtained by connecting the most outside vertexes of all the vertexes of the marked frames in the picture to be identified is used as the image frame. All the articles in the picture to be identified can be contained in the image frame, so that the inventory condition of the articles can be more accurately determined.

In some embodiments, as shown in connection with fig. 3, fig. 3 is a schematic diagram of a picture to be identified with a label box. The circles in fig. 3 represent items, and the dashed boxes corresponding to the items in fig. 3 represent the label boxes of the corresponding items. In fig. 3, a quadrangle formed by connecting the first vertex 8, the second vertex 9, the third vertex 10 and the fourth vertex 11 is an image frame 12.

Optionally, determining the inventory condition of the article according to the image frame includes: and acquiring the area of the image frame. And acquiring the ratio between the area of the image frame and the preset area, and determining the ratio between the area of the image frame and the preset area as a target ratio. And determining the inventory condition of the article according to the target ratio. The area of the image frame is the area of the corresponding area of the image frame. The preset area is the area of the picture to be identified. Thus, the larger the area of the image frame is, the larger the occupation ratio of the image frame in the picture to be identified is reflected, and the more corresponding articles are stored. The smaller the area of the image frame, the smaller the occupancy rate of the image frame in the picture to be identified, and the smaller the corresponding object inventory. According to the image frame used for limiting the articles in the pictures to be identified, the inventory condition of the articles is determined, the articles are not affected by weight change of the articles, and therefore the inventory condition of the articles can be determined more accurately.

Further, the area of the image border is calculated by: respectively acquiring pixel point coordinates of a first vertex, a second vertex, a third vertex and a fourth vertex; the area of the image frame is obtained by calculating s=1/2 [ (x 1 x 2-x2 x 1) + (x 2 x 3-x3 x 2) + (x 3 x 4-x4 x 3) + (x 4 x 1-x1 x 4) ]. Wherein S is the area of the image frame. (x 1, y 1) is the pixel coordinates of the first vertex, (x 2, y 2) is the pixel coordinates of the second vertex, (x 3, y 3) is the pixel coordinates of the third vertex, and (x 4, y 4) is the pixel coordinates of the fourth vertex.

Further, an inventory of the item is determined based on the target ratio. Comprising the following steps: and matching the inventory condition corresponding to the target ratio from a preset database. The corresponding relation between the target ratio and the stock condition is stored in a preset database. Therefore, the corresponding relation between the target ratio and the inventory condition is stored in the preset database, so that inventory data corresponding to the target ratio is matched according to different ratios, and the inventory condition of the article can be determined more accurately and rapidly.

In some embodiments, the preset database stores the correspondence between the target ratio and the inventory condition. For example, where the target ratio is greater than or equal to 90%, the inventory of items is full, i.e., the inventory of items is 100% of the inventory remaining. In the case where the target ratio is 80% or more and 90% or less, the inventory condition of the article is 90% of the inventory remaining. In the case where the target ratio is 70% or more and 80% or less, the inventory condition of the article is 80% of the inventory remaining. In the case where the target ratio is greater than or equal to 60% and less than 70%, the inventory condition of the article is 70% of the inventory remaining. In the case where the target ratio is 50% or more and 60% or less, the inventory condition of the article is 60% of the inventory remaining. The target ratio and the inventory condition are pre-stored in a database, so that the inventory condition of the preset area can be quickly determined according to the target ratio. The larger the area of the image frame, the larger the ratio between the area of the corresponding image frame and the area of the picture to be identified. The more inventory of items that characterize the moment. The smaller the area of the image frame, the smaller the ratio between the area of the corresponding image frame and the area of the picture to be identified. The less inventory of items characterizing the moment. And calculating the ratio of the area of the image frame to the area of the picture to be identified. Because the area of the picture to be identified is unchanged, the object inventory condition can be determined by determining the ratio of the area of the image frame to the picture to be identified based on the area. And the inventory condition determined in this way is not affected by the weight change of the article. Thus, the determined inventory condition can be more accurate.

Further, after determining the inventory condition of the article according to the image frame, the method further comprises: and sending the inventory condition of the article to a preset user terminal. Thus, the user can conveniently know the inventory condition in the preset area. The user terminal comprises a smart phone, a tablet personal computer or a computer and the like.

Further, after obtaining the name information corresponding to the article, the method further comprises: and sending the inventory condition of the article and the name information of the article to a preset user terminal. Thus, the user can conveniently know the inventory condition in the preset area and the names of the remaining articles.

In a practical application scenario, a user stores articles to be sold, such as ice cream, etc., in an ice bin, and puts the ice bin storing the articles in a supermarket, a store, etc. to sell the articles in the ice bin. Such as the ice bin shown in fig. 2. Shooting the corresponding preset areas by utilizing the image acquisition devices at intervals of preset time intervals to obtain pictures to be identified corresponding to the preset areas. And respectively identifying each picture to be identified to obtain an article marking frame in each picture to be identified. And then obtaining the image frames in each picture to be identified. And determining the inventory condition of a preset area corresponding to each picture to be identified through each image frame. In this way, the manner of determining the inventory of the items by image recognition is not affected by the change in weight of the items, and therefore the inventory condition of the items can be determined more accurately than the manner of determining the inventory of the items by weighing the items. And simultaneously, respectively identifying each picture to be identified to obtain the name information of the object in each picture to be identified. And sending the inventory condition of each preset area and the name information of the articles to a preset user terminal. The method can facilitate users to know the inventory condition of each preset area and the names of the remaining articles in the refrigerator in time. Thus, when the inventory of the preset area in the refrigerator is less, the user can know the name of the missing article in time conveniently. And even if the user is at a place far away from the refrigerator, the user can know the article inventory condition of each preset area remotely and in real time. In this way, the user is prevented from personally going to the ice chest to view inventory. The method and the device provide convenience for users, save time for the users and improve the efficiency of knowing the inventory condition of the articles.

Optionally, determining the inventory condition of the article according to the image frame includes: and performing table lookup operation in a preset data table by utilizing the shape of the image frame, searching for an alternative frame shape matched with the shape of the image frame, and determining an alternative inventory condition corresponding to the alternative frame shape as an article inventory condition. The corresponding relation between the shape of the alternative frame and the alternative stock condition is stored in the preset data table. Therefore, the object inventory condition can be quickly found out by using the shape of the image frame in a table look-up mode.

Further, an alternative border shape matching the shape of the image border is found by: and comparing the shape of the image frame with all the alternative frame shapes in the preset data table to obtain the similarity between each alternative frame shape and the shape of the image frame. And determining the candidate frame shape with the largest similarity as the candidate frame shape matched with the shape of the image frame.

As shown in connection with fig. 4, an embodiment of the present disclosure provides another method for identifying an inventory of items, comprising:

step S201, an electronic device obtains a picture to be identified, the picture to be identified is obtained by photographing a preset area, and the preset area is used for placing one or more objects.

Step S202, the electronic equipment performs object identification on a picture to be identified by using a first preset model to obtain the picture to be identified with a labeling frame; the labeling frame is used for labeling the objects in the picture to be identified.

In step S203, the electronic device screens the first vertex, the second vertex, the third vertex, and the fourth vertex from the vertices of the labeling frames of all the labeling frames.

In step S204, the electronic device connects the first vertex, the second vertex, the third vertex and the fourth vertex to form a quadrilateral.

In step S205, the electronic device uses the quadrangle as an image frame.

In step S206, the electronic device obtains the area of the image frame.

In step S207, the electronic device obtains a ratio between the area of the image frame and the preset area, and determines the ratio between the area of the image frame and the preset area as a target ratio.

Step S208, the electronic equipment matches the stock condition corresponding to the target ratio from a preset database; the corresponding relation between the target ratio and the stock condition is stored in a preset database.

By adopting the method for identifying the inventory of the articles, which is provided by the embodiment of the disclosure, the image to be identified is obtained by shooting the preset area for placing the articles, then the image frames for limiting all the articles are determined in the image to be identified, and finally the inventory condition of the articles is determined by calculating the areas of the image frames. The larger the area of the image frame, the larger the occupation ratio of the image frame in the picture to be identified is reflected. The more inventory of items that characterize the moment. The smaller the area of the image frame, the smaller the occupation ratio of the image frame in the picture to be identified. The less inventory of items characterizing the moment. According to the image frame used for limiting the articles in the pictures to be identified, the inventory condition of the articles is determined, the articles are not affected by weight change of the articles, and therefore the inventory condition of the articles can be determined more accurately.

Optionally, a light emitting device is disposed in the preset area. Before obtaining the picture to be identified, the method further comprises: and obtaining the target lamplight color, and controlling the light-emitting device to emit light according to the target lamplight color. In an embodiment, the preset area is a storage area corresponding to the storage device, and a light emitting device is arranged at the bottom of the storage device. Wherein, the lighting device evenly distributed is in the bottom of storing device. For example, the Light emitting device is a Light-emitting diode (LED) lamp. And after the light-emitting device emits light, acquiring the picture to be identified. Therefore, when the bottom of the storage device is not completely shielded by the articles in the storage device, the acquired picture to be identified can contain the area of the light emitted by the light-emitting device.

Optionally, determining the inventory condition of the article according to the image frame includes: and acquiring the area of the image frame. And determining whether an illumination area exists in the image frame, wherein the illumination area is an area of light emitted by the characterization light-emitting device in the image frame in the picture to be identified. And under the condition that the illumination area exists in the image frame, acquiring the area of the illumination area in the image frame. Subtracting the area of the illumination area from the area of the image frame to obtain the area of the object area. And acquiring the ratio of the area of the article area to the preset area, and determining the ratio of the area of the article area to the preset area as a target ratio. And determining the inventory condition of the article according to the target ratio. In this way, in the case where there is an illumination area in the image frame, it can be determined that the illumination area is an area that is not occluded by the article, i.e., the illumination area characterizes an area without the article. Therefore, the area without the article area is removed from the area of the image frame, so that the area of the obtained article area is more accurate. And further, the accuracy of determining the inventory condition of the articles can be improved.

Further, obtaining the target light color includes: and acquiring a target picture, wherein the target picture is obtained by shooting a preset area, and the preset area is used for placing one or more objects. And carrying out image recognition on the target picture to obtain the color type in the target picture. And determining the light color of the target according to the color type in the target picture. The color of the target light is different from the color type in the target picture. Therefore, the light-emitting device emits light according to the target lamplight color, and the object area in the image frame of the picture to be identified can be obviously distinguished from the illumination area. Therefore, the area of the article area can be acquired more accurately, and the article inventory condition can be determined more accurately. For example, if the color types in the target picture include gray, red and yellow, the target light color is determined as follows: green or purple, etc.

Further, obtaining an area of an illumination area in the image frame includes: and acquiring the area of the picture to be identified and the image resolution of the picture to be identified. And acquiring the number of pixel points of an illumination area in an image frame of the picture to be identified. The area of the illuminated area within the image frame is obtained by calculating q=n×m/P. Wherein Q is the area of the illumination area in the image frame. N is the number of pixels of the illumination area in the image frame. M is the area of the picture to be identified. P is the image resolution of the picture to be identified.

As shown in connection with fig. 5, an embodiment of the present disclosure provides an apparatus 17 for identifying inventory of items, including a processor 13 and a memory 14 storing program instructions. Optionally, the apparatus may further include a communication interface (communication interface) 15 and a bus 16. The processor 13, the communication interface 15 and the memory 14 may communicate with each other via a bus 16. The communication interface 15 may be used for information transfer. The processor 13 may call program instructions in the memory 14 to perform the method for identifying an inventory of items of the above-described embodiments.

By adopting the device for identifying the inventory of the articles, which is provided by the embodiment of the disclosure, the image to be identified is obtained by shooting the preset area for placing the articles, then the image frames for limiting all the articles are determined in the image to be identified, and finally the inventory condition of the articles is determined by utilizing the image frames. Since the manner of determining the inventory of the articles by image recognition is not affected by the weight change of the articles, the inventory condition of the articles can be determined more accurately than the manner of determining the inventory of the articles by weighing the articles.

Further, the program instructions in the memory 14 described above may be implemented in the form of software functional units and may be stored in a readable storage medium when sold or used as a stand-alone product.

The memory 14 serves as a storage medium for storing a software program, an executable program, such as program instructions/modules corresponding to the methods in the embodiments of the present disclosure. The processor 13 executes the functional applications and data processing by running program instructions/modules stored in the memory 14, i.e. implements the method for identifying inventory of items in the above-described embodiments.

Memory 14 may include a storage program area that may store an operating system, at least one application program required for functionality, and a storage data area; the storage data area may store data created according to the use of the terminal device, etc. In addition, memory 14 may include high-speed random access memory, and may also include non-volatile memory.

The embodiment of the disclosure provides an electronic device, comprising: an electronic device body and the device for identifying the inventory of the articles. An apparatus for identifying an inventory of items is mounted to the electronic device body. The mounting relationship described herein is not limited to being placed inside the electronic device, but also includes mounting connections with other components of the electronic device, including but not limited to physical connections, electrical connections, or signal transmission connections, etc. Those skilled in the art will appreciate that the means for identifying an inventory of items may be adapted to a viable electronic device body, thereby enabling other viable embodiments.

As shown in connection with FIG. 6, in some embodiments, the electronic device is an ice bin. The ice bin comprises an ice bin body 18 and the above-described means 17 for identifying an inventory of items. A means 17 for identifying an inventory of items is mounted to the ice chest body.

The disclosed embodiments provide a storage medium storing program instructions. The program instructions, when executed, perform the method for identifying an inventory of items described above.

The computer readable storage medium may be a transitory computer readable storage medium or a non-transitory computer readable storage medium.

Embodiments of the present disclosure may be embodied in a software product stored on a storage medium, including one or more instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of a method according to embodiments of the present disclosure. And the aforementioned storage medium may be a non-transitory storage medium including: a plurality of media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-only memory (ROM), a random access memory (RAM, randomAccessMemory), a magnetic disk, or an optical disk, or a transitory storage medium.

The above description and the drawings illustrate embodiments of the disclosure sufficiently to enable those skilled in the art to practice them. Other embodiments may involve structural, logical, electrical, process, and other changes. The embodiments represent only possible variations. Individual components and functions are optional unless explicitly required, and the sequence of operations may vary. Portions and features of some embodiments may be included in, or substituted for, those of others. Moreover, the terminology used in the present application is for the purpose of describing embodiments only and is not intended to limit the claims. As used in the description of the embodiments and the claims, the singular forms "a," "an," and "the" (the) are intended to include the plural forms as well, unless the context clearly indicates otherwise. Similarly, the term "and/or" as used in this application is meant to encompass any and all possible combinations of one or more of the associated listed. Furthermore, when used in this application, the terms "comprises," "comprising," and/or "includes," and variations thereof, mean that the stated features, integers, steps, operations, elements, and/or components are present, but that the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof is not precluded. Without further limitation, an element defined by the phrase "comprising one …" does not exclude the presence of other like elements in a process, method or apparatus comprising such elements. In this context, each embodiment may be described with emphasis on the differences from the other embodiments, and the same similar parts between the various embodiments may be referred to each other. For the methods, products, etc. disclosed in the embodiments, if they correspond to the method sections disclosed in the embodiments, the description of the method sections may be referred to for relevance.

Those of skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. The skilled artisan may use different methods for each particular application to achieve the described functionality, but such implementation should not be considered to be beyond the scope of the embodiments of the present disclosure. It will be clearly understood by those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, which are not repeated herein.

In the embodiments disclosed herein, the disclosed methods, articles of manufacture (including but not limited to devices, apparatuses, etc.) may be practiced in other ways. For example, the apparatus embodiments described above are merely illustrative, and for example, the division of the units may be merely a logical function division, and there may be additional divisions when actually implemented, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. In addition, the coupling or direct coupling or communication connection shown or discussed with each other may be through some interface, device or unit indirect coupling or communication connection, which may be in electrical, mechanical or other form. The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to implement the present embodiment. In addition, each functional unit in the embodiments of the present disclosure may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. In the description corresponding to the flowcharts and block diagrams in the figures, operations or steps corresponding to different blocks may also occur in different orders than that disclosed in the description, and sometimes no specific order exists between different operations or steps. For example, two consecutive operations or steps may actually be performed substantially in parallel, they may sometimes be performed in reverse order, which may be dependent on the functions involved. Each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

Claims

1. A method for identifying an inventory of items, comprising:

acquiring a picture to be identified; the picture to be identified is obtained by shooting a preset area, and the preset area is used for placing one or more articles;

determining an image frame in a picture to be identified; the image frame is used for limiting all articles in the picture to be identified;

and determining the inventory condition of the article according to the image frame.

2. The method of claim 1, wherein determining an image border in the picture to be identified comprises:

carrying out article identification on the picture to be identified to obtain identification information corresponding to the article; the identification information is used for representing the position identification of the object in the picture to be identified;

and determining an image frame in the picture to be identified according to the identification information.

3. The method according to claim 2, wherein the identifying the picture to be identified by the article to obtain the identification information corresponding to the article, includes:

carrying out article identification on the picture to be identified by using a first preset model to obtain the picture to be identified with the marking frame; the marking frame is used for marking the objects in the picture to be identified;

and determining the labeling frame of the article as the identification information corresponding to the article.

4. A method according to claim 3, wherein the labelling frame is a polygon; determining the image frame in the picture to be identified according to the identification information comprises the following steps:

screening a first vertex, a second vertex, a third vertex and a fourth vertex from the labeling frame vertices of all the labeling frames; the first vertex is a point closest to the top left corner vertex of the picture to be identified, the second vertex is a point closest to the bottom left corner vertex of the picture to be identified, the third vertex is a point closest to the top right corner vertex of the picture to be identified, and the fourth vertex is a point closest to the bottom right corner vertex of the picture to be identified;

and determining the image frame in the picture to be identified according to the first vertex, the second vertex, the third vertex and the fourth vertex.

5. The method of claim 4, wherein determining an image frame in the picture to be identified based on the first vertex, the second vertex, the third vertex, and the fourth vertex comprises:

connecting the first vertex, the second vertex, the third vertex and the fourth vertex to form a quadrilateral;

and taking the quadrangle as an image frame.

6. The method of claim 1, wherein determining the inventory of the item based on the image frames comprises:

acquiring the area of an image frame;

acquiring the ratio of the area of the image frame to the preset area, and determining the ratio of the area of the image frame to the preset area as a target ratio;

and determining the inventory condition of the article according to the target ratio.

7. The method of claim 6, wherein determining the inventory condition of the item based on the target ratio comprises:

matching the inventory condition corresponding to the target ratio from a preset database; and the preset database stores the corresponding relation between the target ratio and the stock condition.

8. An apparatus for identifying an inventory of items, comprising a processor and a memory storing program instructions, wherein the processor is configured to perform the method for identifying an inventory of items of any one of claims 1 to 7 when the program instructions are executed.

9. An electronic device, characterized in that,

an electronic device body;

the apparatus for identifying an inventory of items of claim 8, mounted to the electronic device body.

10. A storage medium storing program instructions which, when executed, perform the method for identifying an inventory of items of any one of claims 1 to 7.