CN111292466B - Sales counter and order generation method and system thereof - Google Patents

Sales counter and order generation method and system thereof Download PDF

Info

Publication number
CN111292466B
CN111292466B CN201811504746.0A CN201811504746A CN111292466B CN 111292466 B CN111292466 B CN 111292466B CN 201811504746 A CN201811504746 A CN 201811504746A CN 111292466 B CN111292466 B CN 111292466B
Authority
CN
China
Prior art keywords
sales counter
order
probability
selection list
updating
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201811504746.0A
Other languages
Chinese (zh)
Other versions
CN111292466A (en
Inventor
林金表
刘旭
徐卓然
万保成
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Jingdong Qianshi Technology Co Ltd
Original Assignee
Beijing Jingdong Qianshi Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Jingdong Qianshi Technology Co Ltd filed Critical Beijing Jingdong Qianshi Technology Co Ltd
Priority to CN201811504746.0A priority Critical patent/CN111292466B/en
Publication of CN111292466A publication Critical patent/CN111292466A/en
Application granted granted Critical
Publication of CN111292466B publication Critical patent/CN111292466B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07F—COIN-FREED OR LIKE APPARATUS
    • G07F11/00—Coin-freed apparatus for dispensing, or the like, discrete articles
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07F—COIN-FREED OR LIKE APPARATUS
    • G07F9/00—Details other than those peculiar to special kinds or types of apparatus
    • G07F9/006—Details of the software used for the vending machines

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Control Of Vending Devices And Auxiliary Devices For Vending Devices (AREA)

Abstract

The invention discloses a sales counter and an order generation method and system thereof. The order generation method comprises the following steps: acquiring weight change data of the articles in the sales counter; generating at least one article selection list according to the weight change data; acquiring image information of the selected article; updating the article selection list according to the image information; and generating an order according to the updated article selection list. The invention utilizes the gravity information of the articles in the sales counter and integrates the image recognition algorithm to carry out reasoning, can accurately recognize the articles selected by the user from the sales counter, and generates an accurate shopping order.

Description

Sales counter and order generation method and system thereof
Technical Field
The invention relates to the field of unmanned vending, in particular to a self-selection type vending cabinet and an order generation method and system thereof.
Background
At present, the retail industry is in great revolution, new technology has pushed the retail industry to the sharp wave of air opening, and the vending machine widely used in the market is widely matured, and although the payment means is broken through in recent years, the cost, the efficiency and the experience of the vending machine cannot keep pace with the demand of the times. Therefore, a self-service vending machine capable of realizing automatic settlement and self-service taking simultaneously is a mainstream development direction of the future retail industry.
In the prior art, there are three main methods for identifying a commodity purchased by a user and generating an order by a self-service vending machine:
firstly, RFID (radio frequency identification) technology is utilized, an RFID label is attached to each commodity, and commodity information is identified through an RFID reader-writer; this method manufacturing cost is high, and because every commodity all need paste the RFID label, has still increased the human cost of replenishment. In addition, the RFID reader-writer is difficult to identify whether the goods are inside or outside the cabinet door, so that errors are easy to occur;
secondly, the gravity sensor technology is utilized, and the identification of the change of the quantity of the commodities is realized through the change of the weight of the goods shelf; the mode cannot accurately identify commodity information and solve the problem of commodity misplacement between different shelf layers;
thirdly, by utilizing an image recognition technology, the taking action of a user and the change of the number of goods on a goods shelf can be recognized; however, it is difficult to accurately identify the category of the product during the process of taking the product by the user, especially for the product with little difference in appearance, such as water drink.
Disclosure of Invention
The invention provides a sales counter and an order generation method and system thereof, aiming at overcoming the defect that in the prior art, when an order generation mode of a vending machine is used for identifying commodities selected by a user, errors are easy to occur, and the order cannot be accurately generated.
The invention solves the technical problems through the following technical scheme:
an order generation method of a sales counter, the order generation method comprising the steps of:
acquiring weight change data of the articles in the sales counter;
generating at least one article selection list according to the weight change data;
acquiring image information of the selected article;
updating the article selection list according to the image information;
and generating an order according to the updated article selection list.
Preferably, the item selection list includes: selecting a probability;
generating at least one item selection list according to the weight change data, which specifically comprises the following steps:
calculating the similarity between the article selection list at the current moment and the article selection list at the previous moment;
calculating the selection probability according to the similarity;
the selection probability characterizes the matching probability of each item pick list with the actual picked item.
Preferably, after the step of generating the selection probability according to the similarity, the method further includes:
modifying the selection probability according to a weight coefficient.
Preferably, updating the item selection list according to the image information specifically includes:
And updating the selection probability according to the image information and based on a Bayesian inference algorithm.
Preferably, the image information includes: quantity of items and corresponding confidence;
updating the selection probability according to the image information and based on a Bayesian inference algorithm, specifically comprising:
updating the selection probability according to the quantity of the items and the confidence coefficient and based on a Bayesian inference algorithm.
Preferably, the step of generating an order according to the item selection list specifically includes:
and determining the item selection list with the maximum selection probability as the order.
Preferably, after the step of determining the item selection list with the highest selection probability as the order, the method further includes:
judging whether the maximum selection probability is smaller than a probability threshold value;
and when the judgment result is yes, prompting the article display prompt, and returning to the step of acquiring the image information of the article.
Preferably, the sales counter comprises: an electronic door lock;
before the step of acquiring the weight change of the articles in the sales counter, the method further comprises the following steps:
acquiring a shopping request; the shopping request includes user information;
and verifying the user information, and unlocking the electronic door lock when the verification is passed.
Preferably, after the step of generating an order according to the item selection list, the method further comprises:
judging whether the electronic door lock is locked or not;
if not, returning to the step of acquiring the weight change data of the articles in the sales counter;
if so, updating the inventory information of the sales counter according to the order; and/or generating payment information according to the order.
An order generating system for a sales counter, the order generating system comprising:
the weight sensor is used for acquiring weight change data of the articles in the sales counter;
the control module is used for generating at least one article selection list according to the weight change data;
the camera device is used for acquiring the image information of the selected article;
the control module is also used for updating the article selection list according to the image information and generating an order according to the updated article selection list.
Preferably, the item selection list includes: selecting a probability;
the control module specifically comprises:
the calculating unit is used for calculating the similarity between the article selection list at the current moment and the article selection list at the previous moment and calculating the selection probability according to the similarity;
The selection probability characterizes the matching probability of each item pick list with the actual picked item.
Preferably, the control module further comprises:
a modifying unit for modifying the selection probability according to a weight coefficient.
Preferably, the control module further comprises:
and the updating unit is used for updating the selection probability according to the image information and based on a Bayesian inference algorithm.
Preferably, the image information includes: quantity of items and corresponding confidence;
the updating unit is specifically configured to update the selection probability according to the quantity of the items and the confidence level and based on a bayesian inference algorithm.
Preferably, the control module further comprises:
and the selecting unit is used for determining the item selection list with the maximum selection probability as the order.
Preferably, the control module further comprises: a judging unit and a prompting unit;
the judging unit is used for calling the prompting unit when the maximum selection probability is judged to be smaller than the probability threshold;
the prompting unit is used for prompting article display reminding and calling the camera device.
Preferably, the sales counter comprises: an electronic door lock;
the order generating system further comprises:
The request acquisition module is used for acquiring a shopping request; the shopping request includes user information;
and the verification module is used for verifying the user information and unlocking the electronic door lock when the user information passes the verification.
Preferably, the order generation system further comprises: the system comprises a judgment module, an inventory updating module and/or a payment module;
the judging module is used for judging whether the electronic door lock is locked or not and calling the weight sensor when the judgment is negative; if so, calling the inventory updating module and/or the payment module;
the inventory updating module is used for updating the inventory information of the sales counter according to the order;
and the payment module is used for generating payment information according to the order.
A sales counter comprising an order generation system for a sales counter as claimed in any of the preceding claims.
The positive progress effects of the invention are as follows: the invention utilizes the gravity information of the articles in the sales counter and integrates the image recognition algorithm to carry out reasoning, can accurately recognize the articles selected by the user from the sales counter, and generates an accurate shopping order.
Drawings
FIG. 1 is a flowchart of an order generation method for a sales counter according to embodiment 1 of the present invention.
FIG. 2 is a schematic structural diagram of a sales counter according to embodiment 4 of the present invention.
FIG. 3 is a block diagram of an order generation system for a sales counter according to embodiment 4 of the present invention.
Detailed Description
The invention is further illustrated by the following examples, which are not intended to limit the scope of the invention.
Example 1
The embodiment provides an order generation method for a sales counter, wherein the sales counter is a self-selection type unmanned sales counter, and an electronic door lock is arranged on a door of the sales counter. The user can select the needed articles from the sales counter according to the actual requirements.
As shown in fig. 1, the order generation method of the present embodiment includes the following steps:
step 110, obtaining a shopping request.
Wherein the shopping request includes user information. Specifically, the user can establish communication connection with the sales counter through the mobile terminal, and send a shopping request to the sales counter through a shopping APP (application program).
And 120, verifying the user information by the sales counter, and unlocking the electronic door lock when the verification is passed.
At this time, the user can select the articles in the sales counter, and when the user takes out an article from the sales counter, the stock amount and weight of the articles stored in the sales counter change.
And step 130, acquiring weight change data of the articles in the sales counter.
In step 130, the weight sensor may be used to obtain the weight change data of the articles in the sales counter in real time or at intervals. If the sales counter comprises a plurality of goods placing layers, each goods placing layer is provided with a weight sensor, and weight change data, namely the weight m of the articles on all the goods placing layers at the current momenttThe weight m of the articles on all the goods placing layers at the last momentt-1The difference of (a).
Step 140, generating at least one item selection list according to the weight change data.
The list of the selected items is the list of the items possibly selected by the user, the list can be represented by a vector, if N kinds of items are included in the sales counter, an N-dimensional vector can be adopted
Figure BDA0001899150930000051
Representing a pick list of items, where xiThe number of the ith items selected by the user is represented, and i is more than or equal to 1 and less than or equal to N. By means of matrices
Figure BDA0001899150930000052
And representing all possible item selection lists at present, wherein M is the number of the possible item selection lists, and j is more than or equal to 1 and less than or equal to M.
For example, if the sales counter includes three articles A, B, C, the three articles weigh 50g, 100g, and 100g, respectively. When the weight sensor detects that the weight change data of the articles in the current sales counter is 100g, that is, the user takes 100g of the articles out of the sales counter, the article selection list of the user has three possibilities: 2 articles A; 1 item B; 1C article, corresponding to:
Figure BDA0001899150930000061
In this embodiment, the item selection list includes: a probability is selected. The selection probability is used for representing the matching probability of each item selection list and the item actually selected by the user, namely the probability of the item selection list, and the high probability indicates that the item is selected more frequently.
The following explains the calculation process of the selection probability:
step 140-1, calculating the similarity between the item selection list at the current moment and the item selection list at the previous moment.
There are many options for calculating the similarity, and in this embodiment, the number of elements in the matrix of the current item selection list that is the same as the number of elements in the matrix of the last item selection list is defined as the similarity.
For example, if the matrix of the current item selection list is:
Figure BDA0001899150930000062
the matrix of the item selection list at the previous moment is:
Figure BDA0001899150930000063
for XtEach column vector of
Figure BDA0001899150930000064
And Xt-1Each column vector of
Figure BDA0001899150930000065
The similarity is recorded as
Figure BDA0001899150930000066
That is to say
Figure BDA0001899150930000067
And
Figure BDA0001899150930000068
the number of the same elements. Wherein n is more than or equal to 1 and less than or equal to M, and M is more than or equal to 1 and less than or equal to M.
R21Represents X t-12 nd column vector of
Figure BDA0001899150930000069
And X t1 st column vector of
Figure BDA00018991509300000610
0,0]TThe same number of elements, R21R is calculated as 1mnAnd obtaining a similarity matrix R as follows:
Figure BDA0001899150930000071
for easy calculation, for similar momentsThe array R is normalized to sigmanRmnI.e. such that the sum of the elements in each column vector in R is 1. The normalized results were:
Figure BDA0001899150930000072
And 140-2, generating selection probability according to the similarity.
By vectors
Figure BDA0001899150930000073
Representing the selection probability of all the items to be picked, the selection probability vector at the current moment
Figure BDA0001899150930000074
I.e. the selection probability of the similarity matrix and the previous moment
Figure BDA0001899150930000075
The product of (a); when the user sends a purchase request but does not pick an item, the initial item pick list X0 is [0 ]]Selection probability
Figure BDA0001899150930000076
Unlocking the electronic door lock, wherein after a user selects an article for the first time, the selection probability (initial selection probability) of the last moment is pit-1＝[1]TThe selection probability at the current time is
Figure BDA0001899150930000077
For the convenience of calculation, pair
Figure BDA0001899150930000078
Normalization is carried out to obtain
Figure BDA0001899150930000079
In this embodiment, after step 140-2, the selection probability may be further modified according to the weighting factor. Wherein the weighting coefficients may be obtained empirically. For example, the weight coefficient of the selection probability corresponding to the item selection list which is more favored by the user is increased.
Step 150, image information of the selected item is obtained.
Wherein, the image information of the article can be acquired by the camera. It should be noted that the number of the cameras may be one or more. In order to save the production cost, one camera is adopted in the embodiment. The image information includes a category of the item and a quantity of the item. Specifically, the image detection method based on the convolutional neural network detects the image to obtain the image information, and adopts the vector
Figure BDA00018991509300000710
Recording the detection result of the current time, wherein yiIndicating the number of the detected ith product. The image detection method adopted in this step has various mature algorithms available for selection at present, such as fast Region-based Neural Networks (fast Convolutional Neural Networks), Solid State Disk (SSD) (Single Shot multi box Detector, Single scan Detector based on multiple detection frames), YOLO (young Only Look on, target detection system based on Single scan), and the like.
Or A, B, C, assuming that 1B item is detected in the current image, the detection vector is
Figure BDA0001899150930000081
And step 160, updating the article selection list according to the image information.
Specifically, the selection probability is updated in step 160 based on the image information. Preferably, the selection probability of the item selection list is updated based on a bayesian inference algorithm.
For XtEach column vector of
Figure BDA0001899150930000082
Calculating prior probabilities
Figure BDA0001899150930000083
That is, the article selection list is
Figure BDA0001899150930000084
When the image is detected
Figure BDA0001899150930000085
The probability of (c). The calculation formula of the prior probability can be set according to the actual situation. For example, suppose as long as
Figure BDA0001899150930000086
Each element of (1) is not less than
Figure BDA0001899150930000087
The corresponding position element can be detected with a certain probability p
Figure BDA0001899150930000088
Then under this assumption there are:
Figure BDA0001899150930000089
to be provided with
Figure BDA00018991509300000810
And
Figure BDA00018991509300000811
for example, XtOnly the second vector [0,1,0 ]]TAll elements are not less than
Figure BDA00018991509300000812
Elements of the corresponding position, then
Figure BDA00018991509300000813
Figure BDA00018991509300000814
Calculating posterior probability according to Bayes formula
Figure BDA00018991509300000815
I.e. image detection
Figure BDA00018991509300000816
When the article is selected from the list of articles
Figure BDA00018991509300000817
The probability of (c). From bayes' formula, there are:
Figure BDA00018991509300000818
wherein, pijAnd piiTo select a probability vector
Figure BDA00018991509300000819
Element of (5), can be a posterior probability vector
Figure BDA00018991509300000820
Indicating image detection
Figure BDA00018991509300000821
The probability that the user picks the list of all possible items.
Thus, if the calculation in step 140-2 yields
Figure BDA00018991509300000822
Then
Figure BDA00018991509300000823
Figure BDA00018991509300000824
Posterior probability
Figure BDA00018991509300000825
Further, the selection probability is performed based on the image informationUpdated, new selection probabilities
Figure BDA00018991509300000826
Figure BDA00018991509300000827
Where β is the update rate, and can be set according to the actual requirement, for example, β is set to 0.4, and then the result is obtained
Figure BDA0001899150930000091
Step 170, generating an order according to the updated item selection list.
In step 170, that is, in the updated item selection list, the item selection list with the highest selection probability is determined as the shopping order. If it is
Figure BDA0001899150930000092
Then pick the list of the articles
Figure BDA0001899150930000093
The determination is a shopping order, which indicates that the user actually picked 1 item B from the sales counter at that time.
In order to reduce the probability of image recognition error and further improve the accuracy, in this embodiment, after step 170, the method further includes:
Step 170-1, determine whether the maximum selection probability is less than a probability threshold.
The probability threshold may be set according to actual requirements, for example, set to 0.8.
If yes, go to step 170-2 in step 170-1; if not, go to step 180.
And step 170-2, prompting an article display reminder. And returns to step 150 to retrieve the image information of the selected item.
And step 180, judging whether the electronic door lock is locked.
If no, the user does not finish shopping, and the process returns to step 130. If yes, the user finishes shopping, and step 190 is executed.
Step 190, updating the inventory information of the sales counter according to the order; and/or generating payment information according to the order.
In the embodiment, all possible article selection lists are determined by utilizing the gravity information of the sales counter, the probability of the article selection lists is described by the selection probability and is used as a reliable criterion for judging whether the purchasing behavior of the user is normal, and the posterior probability is calculated by adopting a Bayesian formula by establishing the relation between the image recognition result and the article selection lists, so that the influence caused by the error of the image recognition result can be reduced, the articles selected by the user from the sales counter can be accurately verified, and the accuracy of a shopping order is improved.
Example 2
Embodiment 2 is basically the same as embodiment 1, except that in this embodiment, the confidence is further considered when updating the selection probability. When detecting an image, the detection result is expressed as
Figure BDA0001899150930000094
Where θ represents the confidence of the detection result output by fast RCNN, SSD, or YOLO. The following describes the updating method of the selection probability:
first calculating prior probability
Figure BDA0001899150930000101
If it is not
Figure BDA0001899150930000102
Is greater than
Figure BDA0001899150930000103
Element of the corresponding position, then
Figure BDA0001899150930000104
Make an initial
Figure BDA0001899150930000105
Or whether it is
Figure BDA0001899150930000106
And
Figure BDA0001899150930000107
by way of example, obtaining
Figure BDA0001899150930000108
Figure BDA0001899150930000109
Then by
Figure BDA00018991509300001010
The calculation of the posterior probability, the formula for calculating the posterior probability and the subsequent steps are similar to those in embodiment 1, and are not described herein again.
Example 3
In this embodiment, a plurality of cameras are arranged to acquire image information, and the plurality of cameras can be dispersedly arranged on the inner wall surface of a sales counter and above a cabinet door, which are beneficial to acquiring images of articles taken by a user, so as to improve the image detection precision.
The order generation method in this embodiment is basically the same as that in embodiment 2, except that the posterior probabilities of multiple cameras are fused in this embodiment to update the selection probability, specifically:
the detection result output by each camera is expressed as
Figure BDA00018991509300001011
K is more than or equal to 1 and less than or equal to q, and q represents the number of the cameras.
Calculating prior probability according to image information acquired by each camera in sequence
Figure BDA00018991509300001012
If it is used
Figure BDA00018991509300001013
Is greater than
Figure BDA00018991509300001014
Element of the corresponding position, then
Figure BDA00018991509300001015
Then to
Figure BDA00018991509300001016
Are normalized, i.e.
Figure BDA00018991509300001017
Then, the posterior probability is obtained, and the formula is as follows:
Figure BDA00018991509300001018
the posterior probability vector can then be used
Figure BDA00018991509300001019
Indicating detection in image information acquired by the k-th camera
Figure BDA00018991509300001020
The probability that the user picks the list of all possible items.
Finally, all the calculated values are compared
Figure BDA0001899150930000111
And (5) calculating an average value, and taking the average value as the overall posterior probability.
Example 4
The embodiment provides a sales counter, as shown in fig. 2-3, which comprises a cabinet door, a plurality of cargo layers 1, an electronic door lock 2 and an order generation system. Electronic lock installs on the cabinet door, puts the goods layer and is used for placing object article, and the quantity on putting the goods layer can set up by oneself according to actual demand. In this embodiment, the cabinet door can also be installed with the automatic door closer that helps the cabinet door close by oneself, still can install the display screen on the cabinet of selling goods.
Specifically, as shown in fig. 3, in this embodiment, the order generation system includes: a request acquisition module 31, a verification module 32, a weight sensor 33, a control module 34, a camera 35, a judgment module 36, an inventory update module 37, and a payment module 38. The weight sensor 33, the camera 35 and the determination module 36 are all electrically connected to the control module 32. The determination module 36 is also electrically connected to an inventory update module 37 and a payment module 38. The request acquisition module 31 and the electronic door lock 2 are both electrically connected to the verification module 32. The weight sensor 33 is arranged below each cargo layer; the camera device 33 is installed above the cabinet door of the sales counter, and the monitoring area is the front of the goods placing layer.
In this embodiment, the camera device 33 can be implemented by a camera. It should be noted that the number of the cameras may be one or more. In order to save the production cost, one camera is adopted in the embodiment.
The control module 34 of this embodiment specifically includes: a calculation unit 341, a correction unit 342, an update unit 343, and a selection unit 344.
When a user needs to buy goods through a sales counter, the user can establish communication connection with the sales counter through the mobile terminal and send a shopping request to the sales counter through a shopping APP (application program).
The request acquisition module 31 transmits the shopping request to the verification module 32 after acquiring the shopping request. Wherein the shopping request includes user information.
The authentication module 32 authenticates the user information and unlocks the electronic door lock 2 when the authentication is passed. At this time, the user can select the articles in the sales counter, and when the user takes out an article from the sales counter, the stock amount and weight of the articles stored in the sales counter change.
The weight sensor 33 is used for acquiring weight change data of the articles in the sales counter and sending the weight change data to the control module 34. Weight change data, i.e. the weight m of the articles on all the cargo layers at the present moment tThe weight m of the articles on all the goods placing layers at the last momentt-1The difference of (a).
The control module 34 is configured to generate at least one item selection list based on the weight change data.
The list of article selection, that is, the list of articles that the user may select, may be represented by a vector, if the sales counter includes N kinds of articlesThen an N-dimensional vector may be employed
Figure BDA0001899150930000121
Representing a pick list of items, where xiThe number of the ith items selected by the user is represented, and i is more than or equal to 1 and less than or equal to N. By means of matrices
Figure BDA0001899150930000122
And representing all possible item selection lists at present, wherein M is the number of the possible item selection lists, and j is more than or equal to 1 and less than or equal to M.
For example, if the sales counter includes three articles A, B, C, the three articles weigh 50g, 100g, and 100g, respectively. When the weight sensor detects that the weight change data of the articles in the current sales counter is 100g, that is, the user takes 100g of the articles out of the sales counter, the article selection list of the user has three possibilities: 2 articles A; 1 item B; 1C article, corresponding to:
Figure BDA0001899150930000123
in this embodiment, the item selection list includes: a probability is selected. The selection probability is used for representing the matching probability of each item selection list and the item actually selected by the user, namely the probability of the item selection list, and the high probability indicates that the item is selected more frequently.
The following explains the calculation process of the selection probability:
the calculation unit 341 calculates the similarity between the item selection list at the present time and the item selection list at the previous time, and calculates the selection probability based on the similarity.
There are many options for calculating the similarity, and in this embodiment, the number of elements in the matrix of the current item selection list that is the same as the number of elements in the matrix of the last item selection list is defined as the similarity.
For example, if the matrix of the current item selection list is:
Figure BDA0001899150930000124
the matrix of the item selection list at the previous moment is:
Figure BDA0001899150930000131
for XtEach column vector of
Figure BDA0001899150930000132
And Xt-1Each column vector of
Figure BDA0001899150930000133
The similarity is recorded as
Figure BDA0001899150930000134
That is to say
Figure BDA0001899150930000135
And
Figure BDA0001899150930000136
the number of the same elements. Wherein n is more than or equal to 1 and less than or equal to M, and M is more than or equal to 1 and less than or equal to M.
R21Represents X t-12 nd column vector of
Figure BDA0001899150930000137
And X t1 st column vector of
Figure BDA0001899150930000138
Figure BDA0001899150930000139
Same number of elements, R21R is calculated as 1mnAnd obtaining a similarity matrix R as follows:
Figure BDA00018991509300001310
to is coming toThe calculation is convenient, the similar matrix R is normalized, and sigma is obtainednRmnI.e. such that the sum of the elements in each column vector in R is 1. The normalized results were:
Figure BDA00018991509300001311
using vectors
Figure BDA00018991509300001312
Representing the selection probability of all the items to be picked, the selection probability vector at the current moment
Figure BDA00018991509300001313
I.e. the selection probability of the similarity matrix and the previous moment
Figure BDA00018991509300001314
The product of (a); when the user sends a purchase request but does not select an item, the initial item selection list X is used0＝[0]Selection probability
Figure BDA00018991509300001315
The electronic door lock is unlocked, and after the user selects the object for the first time, the selection probability at the current moment is
Figure BDA00018991509300001316
For the convenience of calculation, for
Figure BDA00018991509300001317
Normalization is carried out to obtain
Figure BDA00018991509300001318
In this embodiment, the selection probability may also be fine-tuned. Specifically, the modification unit 342 modifies the selection probability according to the weight coefficient. Wherein the weighting coefficients may be obtained empirically. For example, the selection probability corresponding to the item selection list more favored by the user may be increased according to the weighting factor.
The imaging device 35 is used to acquire image information of the selected article.
Wherein the image information includes a category of the item and a quantity of the item. Specifically, the image detection method based on the convolutional neural network detects the image to obtain the image information, and adopts the vector
Figure BDA00018991509300001319
Recording the detection result of the current time, wherein yiIndicating the number of the detected ith product. At present, various mature algorithms such as fast RCNN, SSD, YOLO network, etc. can be selected for the image detection method.
Or A, B, C, assuming that 1B item is detected in the current image, the detection vector is
Figure BDA0001899150930000141
The control module 34 updates the item selection list according to the image information and generates an order according to the item selection list.
Updating the item selection list, i.e. updating the selection probability of each item selection list. Specifically, the updating unit 343 updates the selection probability based on the bayesian inference algorithm according to the image information, and provides a possible implementation as follows:
for XtEach column vector of
Figure BDA0001899150930000142
Calculating prior probabilities
Figure BDA0001899150930000143
That is, the article selection list is
Figure BDA0001899150930000144
When the image is detected
Figure BDA00018991509300001423
The probability of (c). The calculation formula of the prior probability can be set according to the actual situation. For example, suppose as long as
Figure BDA0001899150930000146
Each element of (1) is not less than
Figure BDA00018991509300001422
The corresponding position element can be detected with a certain probability p
Figure BDA0001899150930000148
Then under this assumption there are:
Figure BDA0001899150930000149
to be provided with
Figure BDA00018991509300001410
And
Figure BDA00018991509300001411
for example, XtOnly the second vector [0,1,0 ]]TAll elements are not less than
Figure BDA00018991509300001424
Elements of the corresponding position, then
Figure BDA00018991509300001413
Figure BDA00018991509300001414
Calculating posterior probability according to Bayes formula
Figure BDA00018991509300001415
I.e. image detection
Figure BDA00018991509300001416
When the article is selected from the list of articles
Figure BDA00018991509300001417
The probability of (c). From bayes' formula, there are:
Figure BDA00018991509300001418
wherein, pijAnd piiTo select a probability vector
Figure BDA00018991509300001419
Element of (5), can be a posterior probability vector
Figure BDA00018991509300001420
Indicating image detection
Figure BDA00018991509300001425
The probability that the user picks the list of all possible items.
Thus, if the calculation unit calculates
Figure BDA0001899150930000151
Then
Figure BDA0001899150930000152
Figure BDA0001899150930000153
Posterior probability
Figure BDA0001899150930000154
Furthermore, after the selection probability is updated according to the image information, the new selection probability
Figure BDA0001899150930000155
Figure BDA0001899150930000156
Where β is the update rate, and can be set according to the actual requirement, for example, β is set to 0.4, thenTo obtain finally
Figure BDA0001899150930000157
Another possible implementation of updating the selection probability is provided below, in which the confidence is further considered to improve the accuracy of image detection, in which the detection result is expressed as
Figure BDA0001899150930000158
Wherein θ represents the confidence of the detection result output by fast RCNN, SSD, or YOLO, specifically:
first calculating prior probability
Figure BDA0001899150930000159
If it is not
Figure BDA00018991509300001522
Is greater than
Figure BDA00018991509300001521
Element of the corresponding position, then
Figure BDA00018991509300001512
Make an initial
Figure BDA00018991509300001513
Or whether it is
Figure BDA00018991509300001514
And
Figure BDA00018991509300001523
by way of example, obtaining
Figure BDA00018991509300001516
Figure BDA00018991509300001517
Then by
Figure BDA00018991509300001518
The posterior probability is calculated, and the subsequent steps are similar to the above-mentioned probability updating process without considering the confidence coefficient, and are not described herein again.
The selecting unit 344 determines the item picking list with the highest selection probability as the order. If it is
Figure BDA00018991509300001519
The selection unit selects the article list
Figure BDA00018991509300001524
The determination is a shopping order, which indicates that the user actually picked 1 item B from the sales counter at that time.
In order to reduce the probability of image recognition error and further improve the accuracy, in this embodiment, the control module further includes: a judging unit 345 and a prompting unit 346.
The judging unit 345 judges whether or not the maximum selection probability is smaller than the probability threshold value, and when judged yes, invokes the presentation unit 346. The probability threshold value can be set according to actual requirements, for example, set to 0.8.
The prompting unit 346 is used for prompting the article display reminder and calling the camera device 35 to retrieve the image information of the selected article.
The judging unit 345 calls the judging module 36 when judging that the maximum selection probability is not less than the probability threshold.
The judging module 36 is used for judging whether the electronic door lock is locked, and if not, the weight sensor 33 is called to monitor the weight information of the sales counter again, which indicates that the shopping of the user is not finished; if yes, indicating that the user is finished shopping, the inventory update module 37 and/or the payment module 38 are invoked.
The inventory updating module 37 is used for updating the inventory information of the sales counter according to the order; the payment module 38 is used to generate payment information according to the order.
In the embodiment, all possible article selection lists are determined by utilizing the gravity information of the sales counter, the probability of the article selection lists is described by the selection probability and is used as a reliable criterion for judging whether the purchasing behavior of the user is normal, and the posterior probability is calculated by adopting a Bayesian formula by establishing the relation between the image recognition result and the article selection lists, so that the influence caused by the error of the image recognition result can be reduced, the articles selected by the user from the sales counter can be accurately verified, and the accuracy of a shopping order is improved.
Example 5
Embodiment 5 is basically the same as embodiment 4, except that a plurality of cameras are provided for acquiring image information in this embodiment, and the plurality of cameras can be dispersedly disposed on the inner wall surface of a sales counter and above a cabinet door, which are favorable for acquiring an image of an article taken by a user, so as to improve the image detection precision.
In this embodiment, the updating unit 343 updates the selection probability in the following manner:
the detection result output by each camera is expressed as
Figure BDA0001899150930000161
K is more than or equal to 1 and less than or equal to q, and q represents the number of the cameras.
Calculating prior probability according to image information acquired by each camera in sequence
Figure BDA0001899150930000162
If it is not
Figure BDA0001899150930000163
Is greater than
Figure BDA0001899150930000164
Element of the corresponding position, then
Figure BDA0001899150930000165
To pair
Figure BDA0001899150930000166
Are normalized, i.e.
Figure BDA0001899150930000167
Then, the posterior probability is obtained, and the formula is as follows:
Figure BDA0001899150930000168
the posterior probability vector can then be used
Figure BDA0001899150930000169
Indicating detection in image information acquired by the k-th camera
Figure BDA00018991509300001610
The probability that the user picks the list of all possible items.
Finally, all the calculated values are compared
Figure BDA0001899150930000171
And (5) calculating an average value, and taking the average value as the overall posterior probability.
While specific embodiments of the invention have been described above, it will be appreciated by those skilled in the art that this is by way of example only, and that the scope of the invention is defined by the appended claims. Various changes and modifications to these embodiments may be made by those skilled in the art without departing from the spirit and scope of the invention, and these changes and modifications are within the scope of the invention.

Claims (17)

1. An order generation method for a sales counter, the order generation method comprising the steps of:
acquiring weight change data of the articles in the sales counter;
generating at least one article selection list according to the weight change data;
acquiring image information of the selected article;
updating the article selection list according to the image information;
generating an order according to the updated article selection list;
the item selection list comprises: selecting a probability;
generating at least one item selection list according to the weight change data, which specifically comprises the following steps:
calculating the similarity between the article selection list at the current moment and the article selection list at the previous moment;
calculating the selection probability according to the similarity;
the selection probability represents the matching probability of each item selection list and the actually selected item;
the step of updating the item selection list according to the image information specifically includes:
updating the selection probability according to the image information.
2. The sales counter order generation method of claim 1, further comprising, after the step of generating the selection probability based on the similarity,:
Modifying the selection probability according to a weight coefficient.
3. The method for generating an order for a sales counter according to claim 1, wherein updating the item selection list based on the image information specifically comprises:
and updating the selection probability according to the image information and based on a Bayesian inference algorithm.
4. The sales counter order generation method of claim 3, wherein the image information includes: quantity of items and corresponding confidence;
updating the selection probability according to the image information and based on a Bayesian inference algorithm, specifically comprising:
updating the selection probability according to the quantity of the items and the confidence coefficient and based on a Bayesian inference algorithm.
5. The method for generating an order for a sales counter according to claim 1, wherein the step of generating an order based on the item selection list specifically comprises:
and determining the item selection list with the maximum selection probability as the order.
6. The sales counter order generation method of claim 5, wherein after the step of determining as the order the list of item picks for which the selection probability is the maximum, further comprising:
judging whether the maximum selection probability is smaller than a probability threshold value;
And when the judgment result is yes, prompting the article display prompt, and returning to the step of acquiring the image information of the article.
7. The method of generating an order for a sales counter of claim 1, wherein the sales counter comprises: an electronic door lock;
before the step of acquiring the weight change of the articles in the sales counter, the method further comprises the following steps:
acquiring a shopping request; the shopping request includes user information;
and verifying the user information, and unlocking the electronic door lock when the verification is passed.
8. The sales counter order generation method of claim 7, further comprising, after the step of generating an order based on the item selection list:
judging whether the electronic door lock is locked or not;
if not, returning to the step of acquiring the weight change data of the articles in the sales counter;
if so, updating the inventory information of the sales counter according to the order; and/or generating payment information according to the order.
9. An order generating system for a sales counter, the order generating system comprising:
the weight sensor is used for acquiring weight change data of the articles in the sales counter;
the control module is used for generating at least one article selection list according to the weight change data;
The camera device is used for acquiring the image information of the selected article;
the control module is also used for updating the article selection list according to the image information and generating an order according to the updated article selection list; the item selection list includes: selecting a probability;
the control module specifically comprises:
the calculating unit is used for calculating the similarity between the article selection list at the current moment and the article selection list at the previous moment and calculating the selection probability according to the similarity;
the selection probability represents the matching probability of each item selection list and the actually selected item;
the control module is further configured to update the selection probability according to the image information.
10. The sales counter order generation system of claim 9, wherein the control module further comprises:
a modifying unit for modifying the selection probability according to a weight coefficient.
11. The sales counter order generation system of claim 9, wherein the control module further comprises:
and the updating unit is used for updating the selection probability according to the image information and based on a Bayesian inference algorithm.
12. The sales counter order generation system of claim 11, wherein the image information comprises: quantity of items and corresponding confidence;
The updating unit is specifically configured to update the selection probability according to the quantity of the items and the confidence level and based on a bayesian inference algorithm.
13. The sales counter order generation system of claim 9, wherein the control module further comprises:
and the selecting unit is used for determining the item selection list with the maximum selection probability as the order.
14. The sales counter order generation system of claim 13, wherein the control module further comprises: a judging unit and a prompting unit;
the judging unit is used for calling the prompting unit when the maximum selection probability is judged to be smaller than the probability threshold;
the prompting unit is used for prompting article display reminding and calling the camera device.
15. The order generating system for a sales counter of claim 9, wherein the sales counter comprises: an electronic door lock;
the order generating system further comprises:
the request acquisition module is used for acquiring a shopping request; the shopping request includes user information;
and the verification module is used for verifying the user information and unlocking the electronic door lock when the verification is passed.
16. The sales counter order generation system of claim 15, further comprising: the system comprises a judgment module, an inventory updating module and/or a payment module;
The judging module is used for judging whether the electronic door lock is locked or not and calling the weight sensor when the judgment result is no; if yes, calling the inventory updating module and/or the payment module;
the inventory updating module is used for updating the inventory information of the sales counter according to the order;
and the payment module is used for generating payment information according to the order.
17. A sales counter, characterized in that the sales counter comprises an order generation system for a sales counter according to any of claims 9-16.
CN201811504746.0A 2018-12-10 2018-12-10 Sales counter and order generation method and system thereof Active CN111292466B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811504746.0A CN111292466B (en) 2018-12-10 2018-12-10 Sales counter and order generation method and system thereof

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811504746.0A CN111292466B (en) 2018-12-10 2018-12-10 Sales counter and order generation method and system thereof

Publications (2)

Publication Number Publication Date
CN111292466A CN111292466A (en) 2020-06-16
CN111292466B true CN111292466B (en) 2022-06-07

Family

ID=71028959

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811504746.0A Active CN111292466B (en) 2018-12-10 2018-12-10 Sales counter and order generation method and system thereof

Country Status (1)

Country Link
CN (1) CN111292466B (en)

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US3504778A (en) * 1967-12-20 1970-04-07 Vendo Co Multiple price coin changer for variable price vending machines
CN106485567B (en) * 2016-09-14 2021-11-30 北京小米移动软件有限公司 Article recommendation method and device
CN108009560B (en) * 2016-11-02 2021-05-11 广州图普网络科技有限公司 Commodity image similarity category judgment method and device
CN108364422A (en) * 2018-02-24 2018-08-03 广州逗号智能零售有限公司 Self-service method and device
CN108389315B (en) * 2018-03-02 2021-07-20 北京京东尚科信息技术有限公司 Article identification method and apparatus, and computer-readable storage medium
CN108335408B (en) * 2018-03-02 2020-11-03 北京京东尚科信息技术有限公司 Article identification method, device and system for vending machine and storage medium
CN108875664B (en) * 2018-06-27 2022-07-05 北京京东尚科信息技术有限公司 Method and device for identifying purchased goods and vending machine

Also Published As

Publication number Publication date
CN111292466A (en) 2020-06-16

Similar Documents

Publication Publication Date Title
CN108875664B (en) Method and device for identifying purchased goods and vending machine
CN108335408B (en) Article identification method, device and system for vending machine and storage medium
CN108346221B (en) Unmanned vending equipment, vending method thereof and unmanned vending system
US10832311B2 (en) Method and medium for shopping in a physical store
CN109243096B (en) Intelligent container and method for determining commodity taking
CN109190705A (en) Self-service method, apparatus and system
US11651664B2 (en) Neural network classifier trained for purchasing differentiation
GB2593551A (en) Model-based machine-learning and inferencing
CN111524273B (en) Shipment detection method and device, unmanned vending equipment and storage medium
CN110363626B (en) Behavior mode monitoring method for unmanned container, electronic equipment and unmanned container
CN110164029A (en) Automatic vending machine and its control method
US20230037427A1 (en) Identifying barcode-to-product mismatches using point of sale devices and overhead cameras
CN110164036A (en) Automatic vending machine and its control method
CN111126990A (en) Automatic article identification method, settlement method, device, terminal and storage medium
CN109064631A (en) Kinds of goods recognition methods, vending method and selling system
CN111292466B (en) Sales counter and order generation method and system thereof
CN114743307A (en) Commodity identification method and device for intelligent container, electronic equipment and storage medium
CN113505767B (en) Cargo vehicle lane occupying operation behavior detection method and system based on cloud computing
CN112163600B (en) Commodity identification method based on machine vision
CN115661591A (en) Intelligent cabinet dynamic identification method and device
CN110910567A (en) Deduction method, device, electronic equipment, computer readable storage medium and container
CN110675106A (en) Unmanned container commodity identification method based on dynamic commodity inventory information
KR102416277B1 (en) Payment and Stock Control System for Unmanned Convenience Store
CN110647783A (en) Unmanned goods shelf payment detection method, device and system
JP7402898B2 (en) Apparatus and method for forming at least one ground truth database for an object recognition system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
TA01 Transfer of patent application right

Effective date of registration: 20210302

Address after: Room a1905, 19 / F, building 2, No. 18, Kechuang 11th Street, Daxing District, Beijing, 100176

Applicant after: Beijing Jingdong Qianshi Technology Co.,Ltd.

Address before: 101, 1st floor, building 2, yard 20, Suzhou street, Haidian District, Beijing 100080

Applicant before: Beijing Jingbangda Trading Co.,Ltd.

Effective date of registration: 20210302

Address after: 101, 1st floor, building 2, yard 20, Suzhou street, Haidian District, Beijing 100080

Applicant after: Beijing Jingbangda Trading Co.,Ltd.

Address before: 100086 8th Floor, 76 Zhichun Road, Haidian District, Beijing

Applicant before: BEIJING JINGDONG SHANGKE INFORMATION TECHNOLOGY Co.,Ltd.

Applicant before: BEIJING JINGDONG CENTURY TRADING Co.,Ltd.

TA01 Transfer of patent application right
GR01 Patent grant
GR01 Patent grant