CN109886169B

CN109886169B - Article identification method, device, equipment and storage medium applied to unmanned container

Info

Publication number: CN109886169B
Application number: CN201910102825.7A
Authority: CN
Inventors: 李习华; 姚俊军; 高斌斌; 贾佳亚; 戴宇荣; 沈小勇
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2019-02-01
Filing date: 2019-02-01
Publication date: 2022-11-22
Anticipated expiration: 2039-02-01
Also published as: CN109886169A

Abstract

The embodiment of the application discloses an article identification method, an article identification device, article identification equipment and a storage medium, wherein the article identification method, the article identification device, the article identification equipment and the storage medium are applied to an unmanned container. The method comprises the following steps: identifying the sum of the picked and placed articles corresponding to the n times of article picking and placing actions executed by the user through a visual identification technology to obtain a visual identification result; detecting whether the gravity change value of the article corresponding to the visual identification result is matched with the total actual gravity change value corresponding to the unmanned container; if not, generating various prediction results; calculating the confidence degrees corresponding to the various prediction results; and determining the target prediction result with the highest confidence degree in the multiple prediction results as the sum of the picked and placed articles corresponding to the n-time article picking and placing behaviors. According to the method and the device, the multiple prediction results are generated, the confidence degrees corresponding to the multiple prediction results are calculated, the prediction result with the highest confidence degree in the multiple prediction results is selected and used as the final sum of the articles taken and placed, and the accuracy rate of the method and the device in identifying the categories of the articles taken by the user is guaranteed.

Description

Article identification method, device, equipment and storage medium applied to unmanned container

Technical Field

The embodiment of the application relates to the technical field of computers, in particular to an article identification method, an article identification device, article identification equipment and a storage medium applied to an unmanned container.

Background

Unmanned retail, such as unmanned supermarkets and unmanned containers, is a popular field which is recently emerging and has very broad prospects.

In the related art, an unmanned container generally adopts a dynamic visual recognition scheme to recognize the category of goods taken by a user. According to the dynamic visual identification scheme, a plurality of cameras are arranged on the cabinet body of the container, after the cameras collect identification images, the collected identification images are sent to the commodity detection module, and the commodity detection module identifies the commodity types according to the identification images.

However, due to the influence of factors such as illumination and camera angle, the dynamic visual recognition scheme in the related art has a low accuracy in recognizing the category of the goods taken by the user.

Disclosure of Invention

The embodiment of the application provides an article identification method, an article identification device and a storage medium, which are applied to an unmanned container and can be used for solving the problem of low accuracy in identifying the commodity category taken by a user in the related art. The technical scheme is as follows:

in one aspect, an embodiment of the present application provides an article identification method applied to an unmanned container, where the method includes:

identifying the sum of picked and placed articles corresponding to n times of article picking and placing actions executed in the unmanned container by a user through a visual identification technology to obtain a visual identification result, wherein n is a positive integer;

detecting whether the gravity change value of the article corresponding to the visual identification result is matched with the total actual gravity change value corresponding to the unmanned container;

if the article gravity change value corresponding to the visual identification result is not matched with the total actual gravity change value, generating a plurality of prediction results of the sum of the picked and placed articles corresponding to the n-time article picking and placing behaviors;

calculating the confidence degrees corresponding to the multiple prediction results;

and determining the target prediction result with the maximum confidence coefficient in the multiple prediction results as the sum of the picked and placed articles corresponding to the n times of article picking and placing behaviors.

In another aspect, an embodiment of the present application provides an article identification apparatus applied to an unmanned container, where the apparatus includes:

the result acquisition module is used for identifying the sum of the taken and placed articles corresponding to the n times of article taking and placing actions executed in the unmanned container by the user through a visual identification technology to obtain a visual identification result, wherein n is a positive integer;

the gravity detection module is used for detecting whether the article gravity change value corresponding to the visual identification result is matched with the total actual gravity change value corresponding to the unmanned container;

a result prediction module, configured to generate multiple prediction results of the total sum of the picked and placed articles corresponding to the n times of article picking and placing behaviors when the article gravity change value corresponding to the visual recognition result is not matched with the total actual gravity change value;

the confidence coefficient calculation module is used for calculating the confidence coefficient corresponding to each of the multiple prediction results;

and the result determining module is used for determining the target prediction result with the maximum confidence coefficient in the multiple prediction results as the sum of the picked and placed articles corresponding to the pick and place behaviors of the articles for n times.

In yet another aspect, an embodiment of the present application provides a computer device, which includes a processor and a memory, where the memory stores at least one instruction, at least one program, a set of codes, or a set of instructions, and the at least one instruction, the at least one program, the set of codes, or the set of instructions is loaded and executed by the processor to implement the method for identifying an article applied to an unmanned container as described in the above aspect.

In yet another aspect, the present application provides a computer-readable storage medium having at least one instruction, at least one program, a set of codes, or a set of instructions stored therein, the at least one instruction, the at least one program, the set of codes, or the set of instructions being loaded and executed by a processor to implement the method for identifying an item applied to an unmanned container as described in the above aspect.

In yet another aspect, the present application provides a computer program product, when being executed, for executing the method for identifying an article applied to an unmanned container described in the above aspect.

The beneficial effects brought by the technical scheme provided by the embodiment of the application at least comprise:

when the gravity change value of the article corresponding to the visual identification result is not matched with the total actual gravity change value corresponding to the unmanned container, multiple prediction results are generated, the confidence degrees corresponding to the multiple prediction results are calculated, the prediction result with the maximum confidence degree in the multiple prediction results is selected and used as the final sum of the articles taken and placed, and the accuracy rate of the article type taken by the user is guaranteed.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

FIG. 1 is a schematic illustration of an implementation environment provided by one embodiment of the present application;

FIG. 2 is a flow chart of an article identification method applied to an unmanned container provided by an embodiment of the present application;

FIG. 3 is a flow chart of an article identification method applied to an unmanned container according to another embodiment of the present application;

FIG. 4 is a schematic illustration of an image provided by an embodiment of the present application;

FIG. 5 is a timing diagram of an article identification method applied to an unmanned container according to an embodiment of the present application;

FIG. 6 is a schematic view of the structure of the identification apparatus applied to an unmanned container according to one embodiment of the present application;

FIG. 7 is a schematic view of the structure of the identification apparatus applied to an unmanned container according to another embodiment of the present application;

FIG. 8 is a schematic diagram of a computer device provided by one embodiment of the present application.

Detailed Description

To make the objects, technical solutions and advantages of the present application more clear, the following detailed description of the embodiments of the present application will be made with reference to the accompanying drawings.

Referring to fig. 1, a schematic diagram of an implementation environment provided by an embodiment of the present application is shown. The implementation environment may include: unmanned container 10 and computer equipment 20.

The unmanned container 10 includes a container body 101, a camera 102, and a gravity tray 103. The camera 102 is used for collecting images of the articles in the unmanned container 10, and the gravity tray 103 is used for measuring the total actual gravity change value corresponding to the unmanned container 10. Optionally, the unmanned container 10 is a 5-layer container, the first and third layers of containers have a camera 102 mounted on the left and right side of the container body 101, and a gravity tray 103 is mounted under each layer of container. The unmanned cargo container 10 supports 15 SKUs (Stock Keeping Unit), the operation of 15 categories of items.

The Computer device 20 refers to an electronic device having computing and processing capabilities, such as a PC (Personal Computer), a server, and the like. The server may be one server, a server cluster composed of a plurality of servers, or a cloud computing service center.

After the camera in the unmanned container 10 collects the image of the article, the image is sent to the computer device 20, and the computer device 20 identifies the image of the article to obtain a visual identification result.

The computer device 20 may be integrated in the unmanned container 10 or may be present independently of the unmanned container 10. The unmanned container 10 and the computer device 20 can communicate with each other through a network, which can be a wired network or a wireless network.

For convenience of description, in the following method embodiments, only the execution subject of each step is described as a computer device, but the method is not limited to this.

Referring to fig. 2, a flowchart of an article identification method applied to an unmanned container according to an embodiment of the present application is shown. The method may be performed by a computer device. The method may comprise the steps of:

step 201, identifying the sum of the picked and placed articles corresponding to the n times of article picking and placing actions executed by the user in the unmanned container through a visual identification technology to obtain a visual identification result, wherein n is a positive integer.

The visual recognition technology is a technology in which after a recognition image is collected by a camera, the collected recognition image is sent to a recognition model, and the recognition model recognizes the recognition image. In the embodiment of the application, the visual recognition technology is used for recognizing the sum of the picked and placed articles corresponding to the article picking and placing actions executed by the user in the unmanned container. The visual identification result is the result of the article taken and placed identified by the visual identification technology.

The user selects the articles in the unmanned container, for example, after the user takes one bottle of joy and one bottle of sprite from the unmanned container, the sprite is put back into the unmanned container, and one bottle of fruit juice is taken, then the user performs 3 times of article taking and placing actions in the unmanned container, and the total of the taken and placed articles corresponding to the 3 times of article taking and placing actions is one bottle of joy and one bottle of fruit juice.

And 202, detecting whether the gravity change value of the article corresponding to the visual identification result is matched with the total actual gravity change value corresponding to the unmanned container.

The gravity change value of the article corresponding to the visual identification result is the sum of the gravity values of the labels of all the articles taken and placed corresponding to the visual identification result.

The total actual gravity change value corresponding to the unmanned container is a difference value between the gravity value of the original article in the unmanned container and the gravity value of the remaining article in the unmanned container after the user takes and places the article. Optionally, the total actual gravity change value corresponding to the unmanned container may be obtained according to a measurement value corresponding to the gravity tray in the unmanned container.

In one example, when the article gravity change value corresponding to the visual recognition result is consistent with the total actual gravity change value corresponding to the unmanned container, the article gravity change value corresponding to the visual recognition result is matched with the total actual gravity change value corresponding to the unmanned container. For example, as a result of the visual recognition, the user takes a bottle of cola, and if the gravity value of the label of the bottle of cola is 5, the gravity change value of the article corresponding to the visual recognition is 5. The gravity value of the original articles in the unmanned container is 20, the gravity value of the remaining articles in the unmanned container after the user takes and places the articles is 15, and the total actual gravity change value corresponding to the unmanned container is 5. And matching the article gravity change value corresponding to the visual identification result with the total actual gravity change value corresponding to the unmanned container if the article gravity change value corresponding to the visual identification result is consistent with the total actual gravity change value corresponding to the unmanned container.

In another example, when the article gravity change value corresponding to the visual recognition result and the total actual gravity change value corresponding to the unmanned container differ by a preset value, the article gravity change value corresponding to the visual recognition result matches with the total actual gravity change value corresponding to the unmanned container, and the preset value is a gravity error value allowed by the article when the article leaves the factory. For example, if the user takes a bottle of cola as a result of the visual recognition, and the gravity value marked on the cola is 5, the gravity variation value of the article corresponding to the visual recognition is 5. The gravity value of the original articles in the unmanned container is 95, the gravity value of the remaining articles in the unmanned container after the user takes and places the articles is 91, and the total actual gravity change value corresponding to the unmanned container is 4. However, the actual gravity value of a bottle of cola corresponding to the visual identification result is 4, and the difference between the actual gravity value of the bottle of cola and the labeled gravity value of the bottle of cola is 1, but the difference value belongs to the gravity error value allowed by the cola when the cola leaves a factory, so that the gravity change value of the article corresponding to the visual identification result is matched with the total actual gravity change value corresponding to the unmanned container.

In step 203, if the gravity variation value of the article corresponding to the visual identification result is not matched with the total actual gravity variation value, a plurality of prediction results of the total of the picked and placed articles corresponding to the n-time article picking and placing behaviors are generated.

And generating various prediction results according to the visual identification result, the total actual gravity change value and the gravity value of each article provided by the unmanned container. The total gravity value corresponding to the generated various prediction results is matched with the total actual gravity change value. For example, if there is a coke, a sprite and a fruit juice in the unmanned container, the gravity value of the coke is 5, the gravity value of the sprite is 4, the gravity value of the fruit juice is 3, the visual recognition result is that the user has taken the coke, and the total actual gravity change value is 9, then a prediction result is generated that the user has taken a bottle of cola and a bottle of sprite.

And step 204, calculating the confidence corresponding to each of the multiple prediction results.

Confidence, which may also be referred to as confidence level or reliability, represents the probability that an overall parameter value falls within an interval of sample statistics. In the embodiment of the present application, the confidence level corresponding to each of the multiple prediction results refers to a probability that an article picked and placed by a user is a certain prediction result. The higher the confidence, the higher the possibility that the item picked up and placed by the user is the prediction result.

In the embodiment of the application, the confidence degree corresponding to the prediction result is calculated according to the confidence degree corresponding to each article in the prediction result. The confidence corresponding to each article is obtained according to at least one of the gravity value and the visual identification characteristic. Optionally, the confidence corresponding to each article in the prediction result is averaged to obtain the confidence corresponding to the prediction result.

And step 205, determining the target prediction result with the highest confidence degree in the multiple prediction results as the sum of the picked and placed articles corresponding to the n-time article picking and placing behaviors.

Optionally, the confidence degrees corresponding to the multiple prediction results are ranked from high to low, and the prediction result corresponding to the first confidence degree in the sequence is selected as the target prediction result.

The prediction result with the maximum confidence coefficient is used as the sum of the n times of article taking and placing behaviors, so that the accuracy of identifying the article types taken and placed by the user is ensured.

In summary, in the technical scheme provided by the embodiment of the application, when the gravity change value of the article corresponding to the visual identification result is not matched with the total actual gravity change value corresponding to the unmanned container, multiple prediction results are generated, confidence degrees corresponding to the multiple prediction results are calculated, the prediction result with the maximum confidence degree in the multiple prediction results is selected as the final sum of the taken and placed articles, and the accuracy of the type of the article taken by the identification user is ensured.

Tests show that after the method of the application is applied to the unmanned container, the accuracy rate of identifying that the user takes 1 commodity category is 100%, and the accuracy rate of identifying that the user takes 2 commodity categories is about 95%.

In addition, the visual identification technology provided by the embodiment of the application can identify the articles taken by the user and the articles put back by the user, namely, the article taking and putting behaviors of the user can be identified.

Please refer to fig. 3, which illustrates an article identification method applied to an unmanned cargo container according to another embodiment of the present application. The method may comprise the steps of:

step 301, identifying the sum of the picked and placed articles corresponding to the n times of article picking and placing actions executed by the user in the unmanned container through a visual identification technology to obtain a visual identification result, wherein n is a positive integer.

Step 302, detecting whether the gravity change value of the article corresponding to the visual identification result is matched with the total actual gravity change value corresponding to the unmanned container.

The introduction descriptions of steps 301 to 302 can refer to the above embodiments, and are not described herein again.

Step 303, if the gravity change value of the article corresponding to the visual identification result is matched with the total actual gravity change value, determining the visual identification result as the sum of the articles taken and placed corresponding to the article taking and placing behaviors of n times.

When the article gravity change value corresponding to the visual identification result is matched with the total actual gravity change value, the visual identification result is accurate, and the visual identification result can be determined as the sum of the articles taken and placed corresponding to the n times of article taking and placing behaviors.

And 304, if the article gravity change value corresponding to the visual identification result is not matched with the overall actual gravity change value, detecting whether the visual identification result and the overall actual gravity change value meet a preset fusion rule.

The preset fusion rule is a rule preset by a technician according to the behavior of a user. The technician can set multiple preset fusion rules according to the following principles:

1. the preset fusion rules do not conflict with each other.

2. Errors after the preset fusion rule is executed are acceptable, for example, guessing errors of the article categories can occur, but the guessed article categories are consistent with the prices of the article categories actually taken and placed by the user.

3. The more specific the preset fusion rule, the better.

4. And setting priority for the preset fusion rule, and not executing the second preset fusion rule when the first preset fusion rule is met.

Technicians can add different preset fusion rules according to the test conditions, so that some unreasonable conditions occurring in the article identification process can be rapidly solved. The preset fusion rule can be continuously adjusted and optimized in the test process, so that the final preset fusion rule is ensured to be correct. The preset fusion rule may be set in view of the number preferentially. For example, the total actual gravity change value is m times of the gravity value of the article corresponding to the visual recognition result, and m is a positive integer not greater than 6.

And 305, if the preset fusion rule is met, obtaining a total of the picked and placed articles corresponding to the n times of article picking and placing behaviors according to the preset fusion rule.

In one example, the preset fusion rule includes that the total actual gravity change value is an integral multiple of the gravity change value of the article corresponding to the visual recognition result. For example, if the visual recognition result is that the user takes one bottle of cola, the gravity value of the one bottle of cola is 5, the total actual gravity change value is 20, and 20 is four times of 5, the visual recognition result and the total actual gravity change value meet the preset fusion rule, and the total of the picked and placed articles corresponding to the article picking and placing behaviors is four bottles of cola.

In another example, the preset blending rule includes a gravity value of the designated article which is twice as large as a corresponding gravity change value of the visual recognition result when the article corresponding to the visual recognition result is the designated article. For example, the visual recognition result is that the user takes one bottle of cola, the cola has activities of buying one and sending one, the gravity value of one bottle of cola is 5, the total actual gravity change value is 10, and 10 is 2 times of 5, then the visual recognition result and the total actual gravity change value meet the preset fusion rule, and the sum of the picked and placed objects corresponding to the object picking and placing behaviors is two bottles of cola.

And step 306, if the preset fusion rule is not met, generating a plurality of prediction results of the sum of the picked and placed articles corresponding to the n-time article picking and placing behaviors.

Optionally, a plurality of prediction results are generated according to the visual recognition result, the total actual gravity change value and the gravity value of each article provided in the unmanned container.

For example, the unmanned counter contains cola, sprite, yogurt and fruit juice, the gravity value of the cola is 5, the gravity value of the sprite is 4, the gravity value of the yogurt is 4, the gravity value of the fruit juice is 3, the visual recognition result shows that the user takes the cola, the total actual gravity change value is 9, one prediction result shows that the user takes one bottle of cola and one bottle of sprite, and the other prediction result shows that the user takes one bottle of cola and one bottle of yogurt.

In a possible implementation manner, when the gravity change value of the article corresponding to the visual recognition result is not matched with the total actual gravity change value, it is not required to detect whether the visual recognition result and the total actual gravity change value satisfy the preset fusion rule, but a plurality of prediction results are generated directly according to the visual recognition result, the total actual gravity change value and the gravity value of each article provided in the unmanned container.

Step 307, for the ith prediction result in the multiple prediction results, generating a confidence corresponding to each article contained in the ith prediction result, wherein i is a positive integer.

Optionally, the confidence corresponding to each item is generated according to the following steps:

1. and for the jth item contained in the ith prediction result, determining the category to which the jth item belongs, wherein j is a positive integer.

Optionally, the article belongs to the following five categories:

the first category is articles which are obtained by identification in a single article taking and placing action through a visual identification technology and are matched with gravity;

the second category is objects which are obtained by identification in a single object picking and placing behavior through a visual identification technology and have unmatched gravity;

the third category refers to articles predicted based on gravity in a single article pick-and-place behavior;

the fourth category is articles identified in the n-time article picking and placing behaviors through a visual identification technology;

the fifth category refers to items that have not been identified by visual recognition techniques in the n pick and place activities.

2. And generating a confidence corresponding to the jth article according to the confidence endowing rule corresponding to the category to which the jth article belongs.

Optionally, this step comprises the following substeps:

2.1, if the jth article belongs to the first class, determining the sum of the visual identification confidence coefficient corresponding to the jth article and a first threshold value as the confidence coefficient corresponding to the jth article; wherein, the first category refers to the articles which are identified in the single article picking and placing action through the visual identification technology and are matched with the gravity.

In the embodiment of the present application, the visual recognition confidence corresponding to the jth article refers to a confidence output when the jth article is obtained through recognition by using a visual recognition technology.

The article corresponding to the single visual identification result is article A, and the gravity value of the article A is matched with the total actual gravity change value corresponding to the unmanned container, so that the article A belongs to the first category, which indicates that the possibility that the user takes the article A is very high, and therefore, the sum of the visual identification confidence coefficient corresponding to the article A and the first threshold value is used as the confidence coefficient corresponding to the article A.

2.2, if the jth article belongs to the second category, determining the difference between the visual identification confidence coefficient corresponding to the jth article and a second threshold as the confidence coefficient corresponding to the jth article; the second category refers to articles which are identified in a single article picking and placing behavior through a visual identification technology and are not matched with gravity.

And if the article corresponding to the single visual identification result is the article A, and the gravity value of the article A is not matched with the total actual gravity change value corresponding to the unmanned container, the article A belongs to the second category, which indicates that the possibility that the user takes the article A is not very high, and therefore, the difference between the visual identification confidence coefficient corresponding to the article A and the second threshold value is taken as the confidence coefficient corresponding to the article A.

2.3, if the jth article belongs to the third category, determining a third threshold as the confidence corresponding to the jth article; and the third category refers to articles predicted based on gravity in the single article taking and placing behavior.

And the article corresponding to the visual identification result is an article B, the sum of the gravity value of the article A and the gravity value of the article B is the total actual gravity change value corresponding to the unmanned container, which indicates that the user may take the article A and the article B, the article A belongs to the third category, and the third threshold is used as the confidence corresponding to the article A.

2.4, if the jth article belongs to the fourth category, determining the visual identification confidence corresponding to the jth article as the confidence corresponding to the jth article; the fourth category refers to articles identified in the n article pick-and-place behaviors by a visual identification technology.

And identifying the article A in the article taking and placing behaviors of 1 time by using the visual identification technology, wherein the article A belongs to the fourth category, and the visual identification confidence coefficient corresponding to the article A is used as the confidence coefficient corresponding to the article A.

2.5, if the jth article belongs to the fifth category, determining a fourth threshold as the confidence corresponding to the jth article; the fifth category refers to articles which are not identified in the n-time article picking and placing behaviors through a visual identification technology.

And if the article A is not identified in the 1-time article taking and placing behaviors by the visual identification technology, the article A belongs to the fifth category, and the fourth threshold is used as the confidence coefficient corresponding to the article A.

The first threshold, the second threshold, the third threshold and the fourth threshold are reasonable values obtained by a plurality of experiments.

And 308, calculating the average value of the confidence degrees corresponding to the articles contained in the ith prediction result to obtain the confidence degree corresponding to the ith prediction result.

For example, the first prediction result includes a sprite and a cola, the confidence corresponding to the sprite is 0.7, the confidence corresponding to the cola is 0.5, and the confidence corresponding to the first prediction result is (0.7 + 0.5)/2 =0.6.

And 309, determining a target prediction result with the highest confidence degree in the multiple prediction results as the sum of the picked and placed articles corresponding to the n-time article picking and placing behaviors.

In summary, in the technical scheme provided in the embodiment of the present application, the confidence corresponding to each article is generated by assigning a rule according to the confidence corresponding to the category to which each article belongs, and the confidence corresponding to each article is assigned according to the category to which each article belongs, so that the confidence corresponding to each article is more reasonable, and further the confidence corresponding to the prediction result is more reasonable, and finally the accuracy of picking and placing articles by the user is higher.

In another optional embodiment provided based on any one of the above embodiments, the visual recognition result may be obtained by:

firstly, identifying a pick-and-place article corresponding to a single article pick-and-place action executed by a user in an unmanned container through a visual identification technology.

The single article taking and placing behavior refers to a single article taking behavior or a single article returning behavior. The article taking and placing actions corresponding to the single article taking and placing actions refer to articles taken by a user at a time or articles put back by the user at a time. For example, a user takes a bottle of cola from an unmanned container, and through a visual recognition technique, the user is recognized to take a bottle of cola from the unmanned container.

Optionally, this step comprises the following substeps:

1. and carrying out article detection on the image shot by the camera in the unmanned container through the first article identification model.

The first item identification model is a model for identifying a category of goods. The first item identification model may be derived through machine learning and neural network models. An image with identification article information is input to a first article identification model, and the first article identification model can identify the article type corresponding to the image.

2. And if the first article identification model detects an article from the image, identifying the article through the first article identification model to obtain a first identification result and a confidence degree corresponding to the first identification result.

If there are 4 cameras in the unmanned container, and the frame rate corresponding to each camera is 120fps, the first item identification model needs to have an identification rate of 480 frames/second, so when the first item identification model detects an item from an image, the item is identified through the first item identification image, and the workload of computer equipment is saved.

Optionally, article identification is performed on the at least one image through the first article identification model to obtain articles corresponding to the at least one image and confidence degrees corresponding to the articles corresponding to the at least one image, when the articles corresponding to the at least one image are inconsistent, the article corresponding to the image with the highest confidence degree is selected as the first identification result, and the highest confidence degree is used as the confidence degree corresponding to the first identification result.

3. And carrying out article identification on the article area in the image through the second article identification model to obtain a second identification result and a confidence degree corresponding to the second identification result.

The second article identification model is a model for identifying the category of the commodity, and the identification accuracy of the second article identification model is greater than that of the first article identification model.

Optionally, article identification is performed on the at least one article area through the second article identification model to obtain articles corresponding to the at least one article area and confidence degrees corresponding to the articles corresponding to the at least one article area, when the articles corresponding to the at least one article area are inconsistent, the article corresponding to the article area with the highest confidence degree is selected as the second identification result, and the highest confidence degree is used as the confidence degree corresponding to the second identification result.

As shown in fig. 4, which is a schematic diagram of an image provided by an embodiment of the present application, the item in the image is a cola 41, the first item identification model identifies the image 40, and the second item identification model identifies the item area 42. Therefore, the recognition accuracy of the second article recognition model is greater than the recognition accuracy of the first article recognition model.

In addition to the item identification of the item area in the image by the second item identification model, the item identification of the item area in the image can be performed by a similar search mode. The step of identifying the article in the image by the similar retrieval mode is to compare the article area in the image with all article photo libraries in the unmanned container which are collected in advance, and if the article area in the image is similar to the images of the article A in all the article photo libraries, the article corresponding to the article area in the image is considered as the article A.

4. And if the confidence degree corresponding to the second recognition result is greater than the confidence degree corresponding to the first recognition result, determining the second recognition result as the pick-and-place article corresponding to the single article pick-and-place behavior.

5. And if the confidence corresponding to the second recognition result is smaller than the confidence corresponding to the first recognition result, determining the first recognition result as the pick-and-place article corresponding to the single article pick-and-place action.

The higher the confidence coefficient is, the more accurate the recognition result is, and the recognition result with the high confidence coefficient is used as the pick-and-place object corresponding to the final single pick-and-place action, so that the method is more reasonable.

And secondly, determining the sum of the picked and placed articles corresponding to the n times of article picking and placing behaviors according to the picked and placed articles corresponding to the single article picking and placing behaviors to obtain a visual identification result.

And accumulating the picked and placed articles corresponding to the article picking and placing behaviors obtained each time to obtain the picked and placed article sum corresponding to the article picking and placing behaviors obtained n times.

For example, the pick-and-place article corresponding to the first pick-and-place action is a bottle of cola, the pick-and-place article corresponding to the second pick-and-place action is a bottle of sprite, and the sum of the pick-and-place articles corresponding to the two pick-and-place actions is one bottle of cola and one bottle of sprite.

In summary, in the technical scheme provided by the embodiment of the application, when the first article identification model detects an article from the image, the article is identified, so that the workload of the computer device is reduced. The confidence degrees corresponding to the recognition results obtained by the two recognition models are compared, and the article corresponding to the maximum confidence degree is selected as the article taken and placed corresponding to the single article taking and placing action, so that the article taken and placed sum corresponding to the n-time article taking and placing actions is more reasonable, and the finally obtained visual recognition result is more accurate.

In this embodiment, the article identification model may be obtained by training of the computer device, or may be obtained by training of other devices, and then sent to the computer device.

When the article identification model is obtained by training of the computer equipment, the computer equipment firstly creates the article identification model; then obtaining a training sample, wherein the training sample comprises an image containing articles and marking information for marking the category and the position of each article in the image; and finally, training the article recognition model according to the training sample.

The computer device may construct the article identification model by using a structural model of any convolutional neural network, which is not limited in this embodiment. In a possible implementation manner, the computer device constructs the article recognition model by using an FSSD (Feature Fusion Single Shot multi box Detector) algorithm, and the FSSD algorithm can introduce more low-level features into a higher layer, so that although more accurate detection of a small target can be obtained, the calculation amount is larger.

Since the computational efficiency of the computer device is required to be high (for example, 480 frames/second), the number of channels (channels) of some layers in the FSSD algorithm can be cut to meet the requirement of computational efficiency.

Optionally, the embodiment of the present application further constructs an article identification model through a tracking algorithm, such as a maple algorithm. The stack algorithm is modified based on Correlation filtering, and combines the HOG (Histogram of Oriented Gradient) -KCF (kernel Correlation Filters) feature and the Color-KCF feature to track the item. The HOG characteristic is sensitive to deformation and motion blur, but has a good tracking effect on color change; in contrast, color features are more sensitive to Color, but track better for deformation and motion blur. Therefore, the fusion of the two can solve the problems of deformation, scale change, motion blur, color change and the like in most tracking processes. Therefore, the item identification model constructed by the repeat algorithm can also determine whether the user behavior is an item taking behavior or an item putting back behavior by generating a moving track of the item from the target image frame and the at least one historical image frame. For example, if the movement track of the article is from the inside to the outside of the unmanned container, the user behavior is determined to be an article taking behavior; and if the moving track of the article is from the outside to the inside of the unmanned container, determining that the user behavior is an article placing back behavior.

As shown in FIG. 5, it shows a timing chart of the article identification method applied to the unmanned container provided by one embodiment of the present application.

1: carrying out article detection on an image shot by a camera in the unmanned container through a first article identification model;

2: if the first article identification model detects an article from the image, performing article identification on the image through the first article identification model to obtain a first identification result and a confidence coefficient corresponding to the first identification result;

3: carrying out article identification on the article area in the image through a second article identification model to obtain a second identification result and a confidence coefficient corresponding to the second identification result;

4: if the confidence degree corresponding to the second recognition result is greater than the confidence degree corresponding to the first recognition result, determining the second recognition result as a pick-and-place article corresponding to the single article pick-and-place behavior;

5: if the confidence degree corresponding to the second recognition result is smaller than the confidence degree corresponding to the first recognition result, determining the first recognition result as a picked and placed article corresponding to the single article picking and placing action;

6: determining the sum of the picked and placed articles corresponding to the n times of article picking and placing actions according to the picked and placed articles corresponding to the single article picking and placing action to obtain a visual identification result;

7: detecting whether the gravity change value of the article corresponding to the visual identification result is matched with the total actual gravity change value corresponding to the unmanned container;

8: if the article gravity change value corresponding to the visual identification result is matched with the total actual gravity change value, determining the visual identification result as the sum of the picked and placed articles corresponding to the n times of article picking and placing behaviors;

9: if the article gravity change value corresponding to the visual identification result is not matched with the overall actual gravity change value, detecting whether the visual identification result and the overall actual gravity change value meet a preset fusion rule;

10: if the preset fusion rule is met, obtaining the sum of the picked and placed articles corresponding to the picking and placing behaviors of the articles for n times according to the preset fusion rule;

11: if the preset fusion rule is not met, generating a plurality of prediction results of the sum of the picked and placed articles corresponding to the n times of article picking and placing behaviors;

12: determining the category of the jth item contained in the ith prediction result, wherein j is a positive integer;

13: giving a rule according to the confidence corresponding to the category to which the jth article belongs to generate the confidence corresponding to the jth article;

14: calculating the average value of the confidence degrees corresponding to the articles contained in the ith prediction result to obtain the confidence degree corresponding to the ith prediction result;

15: and determining the target prediction result with the highest confidence degree in the multiple prediction results as the sum of the picked and placed articles corresponding to the n-time article picking and placing behaviors.

In a practical application scenario, a user purchases an article in an unmanned container. The unmanned container can be used for placing various commodities selected and purchased by a user, such as beverages, snacks, daily necessities and the like. For example, unmanned packing cabinets are divided into multiple layers of packing cabinets, each layer of packing cabinet is provided with goods for sale, and a gravity tray is arranged below each layer of packing cabinet. In addition, at least one camera is installed in the unmanned container, computer equipment with calculation and processing capabilities is integrated in the unmanned container, and a graphic code is arranged or displayed on the body of the unmanned container. Illustratively, a complete item purchase process is as follows:

1. and the user uses a mobile phone and other terminals to scan the graphic code for identity verification, binds the user account and unlocks the unmanned container. After the unmanned container is unlocked, the door of the unmanned container can be opened by a user.

2. And (4) opening the cabinet door by a user, executing at least one article taking and placing action, and selecting the articles to be purchased in the unmanned container.

3. In the process that the user selects the articles, the unmanned container can identify whether the article taking and placing behaviors of the user are taking or placing back the articles through the visual identification technology introduced above, gravity matching and other means, and identify the types of the articles taken or placed back, so that the total article taking and placing behaviors of the user during the opening period of the door of the unmanned container are obtained.

4. After the selection is finished, the user closes the cabinet door of the unmanned container.

5. After detecting that the cabinet door of the unmanned container is closed, the computer equipment calculates the total price of all the articles taken by the user and deducts the total price of all the articles taken by the user from the user account.

The following description is given by way of example of the application of unmanned cargo containers to unmanned supermarkets. At least one unmanned container is arranged in the unmanned supermarket. The unmanned supermarket is provided with an access control device, a camera is arranged in the access control device, face recognition can be carried out on a user, if the user is not a member of the unmanned supermarket, prompt information can be displayed on a panel of the access control device, the user is prompted to register a face to become a member of the unmanned supermarket, and account free withholding is opened. After the user successfully registers, the access control equipment is opened, and the user enters an unmanned supermarket; if the user is a member of the unmanned supermarket, after the camera of the access control equipment identifies the face of the user, the access control equipment is automatically opened, and the user enters the unmanned supermarket. A user selects commodities in an unmanned container, and computer equipment with computing and processing capabilities is integrated in the unmanned container. The unmanned container obtains the sum of the taken and placed articles corresponding to the article taking and placing actions executed in the unmanned container by the user through the means of visual recognition technology, gravity matching and the like, and sends the sum of the taken and placed articles to the settlement table. And the settlement table identifies the face of the user and the payment confirmation gesture of the user through the camera, and automatically deducts money from the user account. If the commodity is already settled, the access control equipment is opened, and the user can leave the unmanned supermarket.

The following are embodiments of the apparatus of the present application that may be used to perform embodiments of the method of the present application. For details which are not disclosed in the embodiments of the apparatus of the present application, reference is made to the embodiments of the method of the present application.

Refer to fig. 6, which illustrates an article identification apparatus applied to an unmanned container according to an embodiment of the present application. The device has the functions of realizing the method examples, and the functions can be realized by hardware or by hardware executing corresponding software. The apparatus 600 may include: a result acquisition module 610, a gravity detection module 620, a result prediction module 630, a confidence calculation module 640, and a result determination module 650.

The result obtaining module 610 is configured to identify, through a visual identification technology, a sum of picked and placed items corresponding to n times of article picking and placing actions performed by a user in the unmanned container to obtain a visual identification result, where n is a positive integer.

The gravity detection module 620 is configured to detect whether the gravity change value of the article corresponding to the visual identification result matches the total actual gravity change value corresponding to the unmanned container.

The result predicting module 630 is configured to generate multiple prediction results of the total sum of the picked and placed articles corresponding to the n times of article picking and placing behaviors when the article gravity change value corresponding to the visual recognition result is not matched with the total actual gravity change value.

The confidence coefficient calculating module 640 is configured to calculate confidence coefficients corresponding to the multiple prediction results.

The result determining module 650 is configured to determine the target prediction result with the largest confidence in the multiple prediction results as the sum of the picked and placed items corresponding to the n-time picking and placing actions.

To sum up, in the technical scheme provided by the embodiment of the application, when the article gravity change value corresponding to the visual identification result is not matched with the total actual gravity change value corresponding to the unmanned container, multiple prediction results are generated, confidence degrees corresponding to the multiple prediction results are calculated, the prediction result with the maximum confidence degree in the multiple prediction results is selected as the final sum of the taken and placed articles, and the accuracy rate of the type of the articles taken by the identification user is ensured.

Optionally, as shown in fig. 7, the confidence calculating module 640 includes: a confidence generating unit 641 and a confidence calculating unit 642.

The confidence generating unit 641 is configured to generate, for an ith prediction result in the multiple prediction results, a confidence corresponding to each item included in the ith prediction result, where i is a positive integer.

The confidence degree calculating unit 642 is configured to calculate an average value of confidence degrees corresponding to the articles included in the ith prediction result, so as to obtain a confidence degree corresponding to the ith prediction result.

Optionally, the confidence generating unit 641 includes: a category determination subunit 6411 and a confidence generation subunit 6412.

The category determining subunit 6411 is configured to determine, for a jth item included in the ith prediction result, a category to which the jth item belongs, where j is a positive integer.

The confidence coefficient generating subunit 6412 is configured to generate a confidence coefficient corresponding to the jth item according to the confidence coefficient assignment rule corresponding to the category to which the jth item belongs.

Optionally, the confidence generating subunit 6412 is configured to:

when the jth article belongs to a first category, determining the sum of the visual recognition confidence coefficient corresponding to the jth article and a first threshold as the confidence coefficient corresponding to the jth article; wherein the first category refers to articles identified in a single article pick-and-place behavior by the visual identification technique and matched by gravity;

when the jth article belongs to a second category, determining the difference between the visual identification confidence coefficient corresponding to the jth article and a second threshold value as the confidence coefficient corresponding to the jth article; the second category refers to articles which are obtained by identification in a single article taking and placing action through the visual identification technology and are not matched with gravity;

when the jth article belongs to a third category, determining a third threshold as the confidence corresponding to the jth article; the third category refers to articles obtained based on gravity prediction in a single article taking and placing behavior;

when the jth article belongs to a fourth category, determining the visual recognition confidence corresponding to the jth article as the confidence corresponding to the jth article; the fourth category refers to articles identified in the n times of article taking and placing behaviors through the visual identification technology;

when the jth article belongs to a fifth category, determining a fourth threshold as the confidence corresponding to the jth article; the fifth category refers to articles which are not identified in the n article taking and placing behaviors through the visual identification technology;

the visual identification confidence corresponding to the jth article refers to the confidence output when the jth article is identified and obtained through the visual identification technology.

Optionally, the result predicting module 630 is configured to generate the plurality of prediction results according to the visual recognition result, the total actual gravity change value, and the gravity value of each article provided in the unmanned container.

Optionally, the result determining module 650 includes: an item identification unit 651, and a result determination unit 652.

The article identification unit 651 is configured to identify, through the visual identification technology, the picked-and-placed article corresponding to the single article picking-and-placing action performed by the user in the unmanned container.

The result determining unit 652 is configured to determine, according to the picked and placed articles corresponding to the single article picking and placing behavior, a picked and placed article sum corresponding to the n-time article picking and placing behaviors, so as to obtain the visual identification result.

Optionally, the item identification unit 651 is configured to:

carrying out article detection on the image shot by the camera in the unmanned container through a first article identification model;

when the first article identification model detects an article from the image, the article identification is carried out on the image through the first article identification model to obtain a first identification result and a confidence coefficient corresponding to the first identification result;

carrying out article identification on the article area in the image through a second article identification model to obtain a second identification result and a confidence coefficient corresponding to the second identification result;

when the confidence corresponding to the second recognition result is greater than the confidence corresponding to the first recognition result, determining the second recognition result as a pick-and-place article corresponding to the single article pick-and-place behavior;

and when the confidence corresponding to the second recognition result is smaller than the confidence corresponding to the first recognition result, determining the first recognition result as the pick-and-place article corresponding to the single article pick-and-place behavior.

Optionally, the apparatus 600 further comprises: a first detection module 660 and an item guessing module 670.

The first detection module 660 is configured to detect whether the visual recognition result and the total actual gravity change value meet a preset fusion rule when the article gravity change value corresponding to the visual recognition result is not matched with the total actual gravity change value.

And the article guessing module 670 is configured to obtain a total of the picked and placed articles corresponding to the n times of the article picking and placing behaviors according to the preset fusion rule when the preset fusion rule is satisfied.

The result predicting module 630 is further configured to generate multiple prediction results of the total sum of the picked and placed items corresponding to the n times of the picking and placing actions when the preset fusion rule is not satisfied.

Optionally, the result determining module 650 is further configured to determine the visual recognition result as a sum of the picked and placed articles corresponding to the n times of picking and placing of the articles, when the article gravity variation value corresponding to the visual recognition result matches the total actual gravity variation value.

It should be noted that, when the apparatus provided in the foregoing embodiment implements the functions thereof, the division of each functional module is merely used as an example, and in practical applications, the function distribution may be completed by different functional modules according to needs, that is, the content structure of the device is divided into different functional modules, so as to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided by the above embodiments belong to the same concept, and specific implementation processes thereof are described in the method embodiments for details, which are not described herein again.

Referring to fig. 8, a block diagram of a computer device 800 according to an embodiment of the present application is shown. The computer device 800 refers to an electronic device having computing and processing capabilities, such as a PC, server, or the like. The computer apparatus 800 may be used to implement the method of article identification as applied to an unmanned container provided in the above embodiments.

Generally, the computer device 800 includes: a processor 801 and a memory 802.

The processor 801 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and so forth. The processor 801 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field Programmable Gate Array), PLA (Programmable Logic Array). The processor 801 may also include a main processor and a coprocessor, where the main processor is a processor for Processing data in an awake state, and is also called a Central Processing Unit (CPU); a coprocessor is a low power processor for processing data in a standby state. In some embodiments, the processor 801 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content required to be displayed on the display screen. In some embodiments, the processor 801 may further include an AI (Artificial Intelligence) processor for processing computing operations related to machine learning.

Memory 802 may include one or more computer-readable storage media, which may be non-transitory. Memory 802 can also include high-speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In some embodiments, a non-transitory computer readable storage medium in the memory 802 is used to store at least one instruction for execution by the processor 801 to implement the method of article identification applied to an unmanned container provided by the method embodiments herein.

In some embodiments, the terminal 800 may further include: a peripheral interface 803 and at least one peripheral. The processor 801, memory 802 and peripheral interface 803 may be connected by bus or signal lines. Various peripheral devices may be connected to peripheral interface 803 by a bus, signal line, or circuit board. Specifically, the peripheral device may include: at least one of a display 804, an audio circuit 805, a communication interface 806, and a power supply 807.

Those skilled in the art will appreciate that the configuration illustrated in FIG. 8 is not intended to be limiting of the computer device 800 and may include more or fewer components than those illustrated, or some components may be combined, or a different arrangement of components may be employed.

In an example embodiment, there is also provided a computer device comprising a processor and a memory having stored therein at least one instruction, at least one program, set of codes, or set of instructions. The at least one instruction, the at least one program, the set of codes or the set of instructions is configured to be executed by one or more processors to implement the above method of article identification as applied to an unmanned container.

In an exemplary embodiment, there is also provided a computer readable storage medium having stored therein at least one instruction, at least one program, a set of codes, or a set of instructions which, when executed by a processor of a computer device, implements the above-described method of item identification applied to an unmanned container.

Alternatively, the computer-readable storage medium may be a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

In an exemplary embodiment, there is also provided a computer program product for implementing the above-described method of item identification applied to an unmanned container when the computer program product is executed.

It should be understood that reference to "a plurality" herein means two or more. "and/or" describes the association relationship of the associated object, indicating that there may be three relationships, for example, a and/or B, which may indicate: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. In addition, the step numbers described herein only exemplarily show one possible execution sequence among the steps, and in some other embodiments, the steps may also be executed out of the numbering sequence, for example, two steps with different numbers are executed simultaneously, or two steps with different numbers are executed in a reverse order to the order shown in the figure, which is not limited by the embodiment of the present application.

It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.

The above description is only exemplary of the application and should not be taken as limiting the application, and any modifications, equivalents, improvements and the like that are made within the spirit and principle of the application should be included in the protection scope of the application.

Claims

1. An article identification method applied to an unmanned container, the method comprising:

identifying the sum of the picked and placed articles corresponding to the n times of article picking and placing actions executed by a user in the unmanned container by a visual identification technology to obtain a visual identification result, wherein n is a positive integer;

if the article gravity change value corresponding to the visual identification result is not matched with the total actual gravity change value, generating various prediction results of the sum of the taken and placed articles corresponding to the n times of article taking and placing behaviors;

2. The method of claim 1, wherein said calculating the confidence level for each of the plurality of predicted outcomes comprises:

for the ith prediction result in the multiple prediction results, generating a confidence coefficient corresponding to each article contained in the ith prediction result, wherein i is a positive integer;

and calculating the average value of the confidence degrees corresponding to all articles contained in the ith prediction result to obtain the confidence degree corresponding to the ith prediction result.

3. The method of claim 2, wherein the generating the confidence level for each item included in the i-th prediction comprises:

determining the category of a jth item contained in the ith prediction result, wherein j is a positive integer;

and generating a confidence corresponding to the jth article according to a confidence endowing rule corresponding to the category to which the jth article belongs.

4. The method according to claim 3, wherein the generating a confidence level corresponding to the jth item according to the confidence level assignment rule corresponding to the category to which the jth item belongs comprises:

if the jth article belongs to the first category, determining the sum of the visual identification confidence coefficient corresponding to the jth article and a first threshold value as the confidence coefficient corresponding to the jth article; wherein the first category refers to articles identified in a single article pick-and-place behavior by the visual identification technique and matched by gravity;

if the jth article belongs to the second category, determining the difference between the visual identification confidence coefficient corresponding to the jth article and a second threshold value as the confidence coefficient corresponding to the jth article; the second category refers to articles which are obtained by identification in a single article taking and placing action through the visual identification technology and are not matched with gravity;

if the jth article belongs to a third category, determining a third threshold as the confidence corresponding to the jth article; the third category refers to articles obtained based on gravity prediction in a single article pick-and-place behavior;

if the jth article belongs to a fourth category, determining the visual identification confidence corresponding to the jth article as the confidence corresponding to the jth article; the fourth category refers to articles identified in the n article pick-and-place behaviors by the visual identification technology;

if the jth article belongs to the fifth category, determining a fourth threshold as the confidence corresponding to the jth article; the fifth category refers to articles which are not identified in the n times of article taking and placing behaviors through the visual identification technology;

the visual identification confidence corresponding to the jth article refers to the confidence output when the jth article is obtained through identification by the visual identification technology.

5. The method of claim 1, wherein said generating a plurality of predicted results of the summation of the pick-and-place items corresponding to said n pick-and-place actions comprises:

and generating the multiple prediction results according to the visual identification result, the total actual gravity change value and the gravity value of each article provided in the unmanned container.

6. The method of claim 1, wherein the identifying, by a visual identification technology, a sum of the picked and placed items corresponding to the n times of picking and placing actions of the items performed by the user in the unmanned container to obtain a visual identification result comprises:

identifying a pick-and-place article corresponding to a single article pick-and-place action executed by the user in the unmanned container through the visual identification technology;

and determining the sum of the picked and placed articles corresponding to the n times of article picking and placing behaviors according to the picked and placed articles corresponding to the single article picking and placing behaviors to obtain the visual identification result.

7. The method of claim 6, wherein the identifying, by the visual recognition technique, the pick-and-place item corresponding to the single item pick-and-place action performed by the user in the unmanned container comprises:

if the first article identification model detects an article from the image, performing article identification on the image through the first article identification model to obtain a first identification result and a confidence degree corresponding to the first identification result;

identifying an article in the article area in the image through a second article identification model to obtain a second identification result and a confidence coefficient corresponding to the second identification result;

if the confidence corresponding to the second recognition result is greater than the confidence corresponding to the first recognition result, determining the second recognition result as a pick-and-place article corresponding to the single article pick-and-place behavior;

and if the confidence corresponding to the second recognition result is smaller than the confidence corresponding to the first recognition result, determining the first recognition result as the pick-and-place article corresponding to the single article pick-and-place behavior.

8. The method of any one of claims 1 to 7, wherein before generating the plurality of prediction results of the total pick-and-place item sum corresponding to the n pick-and-place actions, the method further comprises:

if the article gravity change value corresponding to the visual identification result is not matched with the overall actual gravity change value, detecting whether the visual identification result and the overall actual gravity change value meet a preset fusion rule or not;

if the preset fusion rule is met, obtaining the sum of the picked and placed articles corresponding to the n times of article picking and placing behaviors according to the preset fusion rule;

and if the preset fusion rule is not met, starting to execute the step of generating various prediction results of the sum of the picked and placed articles corresponding to the n times of article picking and placing behaviors.

9. The method according to any of the claims 1 to 7, wherein after the detecting whether the article gravity change value corresponding to the visual recognition result matches with the total actual gravity change value corresponding to the unmanned container, the method further comprises:

and if the article gravity change value corresponding to the visual identification result is matched with the total actual gravity change value, determining the visual identification result as the sum of the picked and placed articles corresponding to the n times of article picking and placing behaviors.

10. An article identification device applied to an unmanned container, the device comprising:

11. The apparatus of claim 10, wherein the confidence computation module comprises:

a confidence degree generating unit, configured to generate, for an ith prediction result of the multiple prediction results, a confidence degree corresponding to each item included in the ith prediction result, where i is a positive integer;

and the confidence coefficient calculation unit is used for calculating the average value of the confidence coefficients corresponding to the articles contained in the ith prediction result to obtain the confidence coefficient corresponding to the ith prediction result.

12. The apparatus of claim 11, wherein the confidence generating unit comprises:

a category determining subunit, configured to determine, for a jth item included in the ith prediction result, a category to which the jth item belongs, where j is a positive integer;

and the confidence degree generating subunit is used for generating the confidence degree corresponding to the jth article according to the confidence degree endowing rule corresponding to the category to which the jth article belongs.

13. The apparatus of claim 10, wherein the result determination module comprises:

the article identification unit is used for identifying the picked and placed articles corresponding to the single article picking and placing action executed by the user in the unmanned container through the visual identification technology;

and the result determining unit is used for determining the sum of the picked and placed articles corresponding to the n times of article picking and placing behaviors according to the picked and placed articles corresponding to the single article picking and placing behavior to obtain the visual identification result.

14. A computer device comprising a processor and a memory, the memory having stored therein at least one instruction, at least one program, a set of codes, or a set of instructions, the at least one instruction, the at least one program, the set of codes, or the set of instructions being loaded and executed by the processor to implement the method according to any one of claims 1 to 9.

15. A computer readable storage medium having stored therein at least one instruction, at least one program, a set of codes, or a set of instructions, which is loaded and executed by a processor to implement the method according to any one of claims 1 to 9.