CN111507702A

CN111507702A - Self-service shopping method for unmanned supermarket, computer readable storage medium and system

Info

Publication number: CN111507702A
Application number: CN201910173568.6A
Authority: CN
Inventors: 张红军
Original assignee: Heyuan Lianteng Industrial Co ltd
Current assignee: Heyuan Lianteng Industrial Co ltd
Priority date: 2019-03-07
Filing date: 2019-03-07
Publication date: 2020-08-07

Abstract

The invention provides a self-service shopping method of an unmanned supermarket, which is used for realizing self-service shopping of a user in a shopping mall and is characterized in that: the method comprises the following steps: s1, acquiring identity information; s2, recognizing the change of the commodity in the shop and matching the corresponding identity information; and S3, counting the commodities matched with the identity information, and acquiring the commodity information purchased corresponding to the identity information. The invention also provides a computer readable storage medium. The invention also provides a shopping system for the shopping mall.

Description

Self-service shopping method for unmanned supermarket, computer readable storage medium and system

[ technical field ] A method for producing a semiconductor device

The invention relates to the field of automatic shopping in markets, in particular to a self-service shopping method, a computer readable storage medium and a system for an unmanned supermarket.

[ background of the invention ]

The existing shopping mode needs that a user takes commodities to a cash register after selecting the commodities to be purchased in a supermarket, then the prices of the commodities are calculated one by a cashier, and finally the total prices of the commodities are obtained.

Further, as the manpower cost increases, the unmanned supermarket gets more and more popular, the existing supermarket self-service shopping system has high error frequency, low commodity identification speed and poor use experience, and therefore a new technical scheme for realizing self-service shopping in the unmanned supermarket is urgently needed to be provided.

[ summary of the invention ]

In order to overcome the existing problems, the invention provides an unmanned supermarket self-service shopping method, a computer readable storage medium and a system.

The technical scheme for solving the technical problem of the invention is to provide a self-service shopping method for an unmanned supermarket, which is used for realizing self-service shopping of a user in a shopping mall and is characterized in that: the method comprises the following steps: s1, acquiring identity information; s2, identifying the change information of the commodities in the shop and matching the corresponding identity information; and S3, counting the commodity types and the quantity matched with the identity information, and acquiring the commodity information purchased corresponding to the identity information.

Preferably, step S2 includes: step S21, recording the states of all commodities in the market; step S22, identifying the identity information captured when the commodity state changes and the changed commodity position; and step S23, matching the changed commodity with the corresponding identity information.

Preferably, step S22 includes: step S221, acquiring real-time video information in a mall; step S222, converting the video information into pictures by taking frames as units; step S223, comparing each frame of picture with the pictures in the step S21 in sequence, identifying the commodity change area and the changed commodity in the frame of picture when the comparison result is different, capturing the identity information close to the area, and matching the identity information with the changed commodity;

preferably, step S22 further includes: step S224, matching the changed commodity and the corresponding identity information.

Preferably, step S22 further includes: step S225, binding the identity information with the shopping cart; step S226, scanning the code of the commodity, acquiring the commodity information of each code-scanned commodity, and recording all the code-scanned commodities; step S227, acquiring a video after the code of the commodity is scanned, setting a shopping cart area in the video, and capturing the motion track of the commodity in the video to judge whether the commodity is put in the shopping cart; step S228: after the commodity is placed in the shopping cart, acquiring the weight change in the shopping cart, and judging whether the weight change in the shopping cart is consistent with the single weight of the placed commodity; and step S229: and after the weight change is judged to be consistent, calculating the total commodity price in the shopping cart according to the commodity information of the code-scanned commodity.

Preferably, step S22 further includes: in step S2210, a payment code is provided so that the user can pay for the merchandise in the shopping cart by scanning the payment code.

Preferably, step S227 includes the steps of: step S2271, converting the acquired video into pictures by taking each frame as a unit; step S2272, establishing a coordinate system on each picture by taking the same position of each picture as an origin; step S2273, establishing a shopping cart area in a coordinate system; and step S2274, recognizing the position of the commodity in the coordinate system in each picture, judging that the commodity is put into the shopping cart when the position of the commodity in the picture is in the shopping cart area, judging that the operation of the user is illegal when the position of the commodity cannot be found in the coordinate system, and giving an alarm to the shopping cart.

The present invention also provides a computer-readable storage medium characterized in that: the computer-readable storage medium has stored thereon a computer program, wherein the computer program is arranged to execute the shop shopping method according to any one of claims 1 to 7 when executed.

The invention also provides an unmanned supermarket self-service shopping system, which is characterized in that: the shopping system of the market comprises:

the identification module is used for acquiring identity information and identifying the change of commodities in a commercial place;

and the server is used for receiving the identity information acquired by the identification module and the change of the commodities in the store and matching the changed commodities with the identity information.

Preferably, the shopping system of the shopping mall further comprises a goods return module, an intelligent shelf and a shopping cart;

the goods returning module is used for returning the taken goods when the user loses the purchasing desire after taking the goods;

the intelligent goods shelf is placed in a shop to realize weighing of goods on the intelligent goods shelf, so that the intelligent goods shelf is combined with the identification module to verify the goods information of the goods taken by a user from the shop;

the shopping cart comprises a weighing module, a code scanning module and a shooting module; the weighing module is used for weighing the weight of the commodities in the shopping cart and transmitting the weight to the server, and the code scanning module is used for scanning the two-dimensional codes on the commodities to acquire commodity information and transmit the commodity information to the server. The camera module is used for shooting a motion video of the commodity after code scanning and transmitting the video to the server.

Compared with the prior art, the self-service shopping method, the computer readable storage medium and the system for the unmanned supermarket have the following advantages:

1. by the shopping method of the market, the self-service purchase of the commodities in the market can be realized, and the whole process of entering the market, purchasing the commodities, paying and leaving the market can be automatically completed without manual assistance.

2. By matching the identity information with the change of the commodities in the store, the commodities purchased in the store by each identity information can be acquired, so that the price of the purchased commodities is automatically generated, and the payment of a user is facilitated.

3. Through setting up the intelligent goods shelves that have the function of weighing, can acquire the weight of the commodity of being taken to verify the matching result of discernment, increase the precision of matching.

4. Through setting up the shopping cart, further verify the matching result to the degree of accuracy that further increases the matching avoids producing the error when pricing.

[ description of the drawings ]

Fig. 1 is a schematic flow chart of an unmanned supermarket self-service shopping system according to a first embodiment of the invention.

Fig. 2A is a schematic structural diagram of an intelligent shelf according to a second embodiment of the present invention.

Fig. 2B is an enlarged view a of fig. 2A.

Fig. 3 is a schematic block diagram of an unmanned supermarket self-service shopping system according to a third embodiment of the invention.

Fig. 4 is a schematic flow chart of a self-service shopping method in an unmanned supermarket according to a fourth embodiment of the invention.

Fig. 5A is a schematic flowchart of step S2 in fig. 4.

Fig. 5B is a flowchart of step S22 in fig. 5A.

Fig. 6 is a further flowchart of step S22 in fig. 5A.

Fig. 7 is a further flowchart of step S22 in fig. 5A.

Fig. 8 is a flowchart of step S226 in fig. 7.

Fig. 9 is a flowchart illustrating step S227 in fig. 7.

Fig. 10A is a further flowchart of step S22 in fig. 5A.

Fig. 10B is a flowchart illustrating step S228 in fig. 10A.

Fig. 11 is a further flowchart of step S22 in fig. 5A.

Reference numerals: 10. an unmanned supermarket self-service shopping system; 20. an identification module; 30. a server; 40. a return module; 50. an intelligent goods shelf; 60. a shopping cart; 51. a shelf body; 511. a first weighing device; 53. a second weighing device; 513. a reservoir region; 55. a payment code printing device; 57. a display device; 59. an automatic packaging mechanism; 591. a sealing unit; 593. a sensing unit; 595. an adjustment unit; 601. a weighing module; 603. a code scanning module; 605. a shooting module; 607. and a display module.

[ detailed description ] embodiments

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

Referring to fig. 1, a first embodiment of the present invention provides an unmanned supermarket self-service shopping system 10, which includes an identification module 20 and a server 30, wherein the server 30 stores commodity information of all commodities in a shopping mall. The identification module 20 is used for identifying the identity information of the user when the user enters the store, and transmitting the identified identity information to the server 30 for storage. Meanwhile, when the user enters a shopping mall for shopping, the shopping process which is recorded in real time by taking the identity information of each user as a label is recorded. And after the user finishes shopping, the commodity information taken from the shopping mall by each identity information is transmitted to the server 30 together, the server 30 counts the quantity of the commodities purchased by each identity information, and generates the total price of the purchased commodities according to the quantity, so that the user can pay for the purchased commodities, thereby finishing shopping.

It is understood that the commodity information includes the appearance characteristics of the commodity, the unit price of the commodity, and the unit weight of the commodity.

In some applications, the recognition module 20 may be a camera, which obtains facial features of each user when the user enters a store, and transmits the facial features to the server 30 for storage as identity information of each user. When the user purchases the commodity, the commodity taken from the store by each identity information is recorded in real time, and the information of the taken commodity and the identity information corresponding to the information are sent to the server 30 for storage. When the user finishes shopping, the server 30 calculates the shopping situation of each user in the shopping mall, that is, how many commodities each user purchases in total, according to the identity information and the commodity information corresponding to the identity information, and generates a total price according to the purchased commodities so that the user can pay, thereby finishing the shopping operation.

In some application scenarios, the identification module 20 may also be a card reader, and when the user enters a shopping mall, the user swipes an identity card, a bank card or other identity-bound document on the card reader, so that the identification module 20 can read the identity information of the user, and then transmit the identity information to the server 30. When a user enters a shopping mall for shopping, the identification module 20 reads the identity information of the user each time the user takes a commodity, and sends the taken commodity and the identity information corresponding to the taken commodity to the server 30 for storage, so that the commodity purchased by each user can be counted, and the user is helped to complete the shopping operation.

It can be understood that when the identification module 20 is a camera, the identity information of the user and the appearance characteristics of the goods to be taken can be directly and simultaneously obtained, so that the information of the goods to be taken by each user can be obtained. When the identification module 20 is a card reader, the information of the goods in each area in the market is pre-stored in the server 30, and the location where the user takes the goods in the market is identified according to the identity information of the user, so that the information of the goods taken corresponding to the identity information can be obtained.

It is understood that the identification module 20 can be disposed at any position in the mall, as long as it can identify the identity information of the user when the user enters the mall and record the process of taking goods by the user when the user shops.

It is understood that, in some other embodiments of the present invention, the identification module 20 may also be an identification module such as a fingerprint identification module, a MAC identification module, and the like, and the selection of the specific identification module and the identification method is determined based on the actual application scenario, which is not limited herein. For example, taking the MAC identification recognition module as an example, the operation principle is that when wifi of the mobile phone of the consumer is turned on, the MAC identification recognition device can recognize the MAC identification of the mobile phone of the consumer, so as to obtain the relevant information of the consumer.

In the present invention, the device module 20 is explained as a camera.

Further, market shopping system still includes return module 40, and return module 40 is used for when losing the desire to purchase again after the user takes commodity, and the commodity that will take returns to when carrying out payment operation after accomplishing the shopping, can not also add into the total price with the commodity price of returning, thereby make the condition that the commodity that produces does not correspond with the commodity of buying.

In some applications, the return module 40 is a container set at a specific location, for example, at the exit of a shopping mall or distributed at any location of the shopping mall. When the user does not want to purchase the goods after taking the goods, the user puts the goods into the goods returning module 40, identifies the information of the goods put into the goods returning module 40 and the corresponding identity information through the identification module 20, and then transmits the information to the server 30. When the user finishes the shopping operation, the server 30 subtracts the commodity information put into the return module 40 from the corresponding commodity information under the identity information, and finally generates a total price for purchasing the commodity.

It is understood that the information of the goods placed in the return module 40 by the user may be identified by the identification module 20, that is, the appearance characteristics of the goods are obtained through the camera.

Further, the return module 40 has a weighing function, and when a user puts a product into the return module 40, in addition to identifying the appearance characteristics of the product by using the identification module 20, verification can be performed by acquiring the single weight of the product put into the return module 40, so that the identification result is more accurate.

In some preferred embodiments of the present invention, the goods returning module 40 further includes a goods returning shelf, and the goods returning shelf is provided with a refrigerating chamber, and the goods to be kept fresh can be placed in the refrigerating chamber, so as to avoid the goods deterioration caused by placing at normal temperature, which brings loss to the operation of the shopping mall.

Referring to fig. 2A, a second embodiment of the present invention provides an intelligent shelf 50 that can be used in the first embodiment, and the intelligent shelf 50 is placed in a place to weigh the goods on the intelligent shelf 50, so that in combination with the identification module 20, the verification module verifies the goods information of the goods taken by the user from the place, and avoids the occurrence of identification errors when the identification module 20 identifies the appearance features of the goods to obtain the corresponding goods information. That is, when the user takes the goods from the smart shelf 50, in addition to identifying the appearance characteristics of the goods taken by the user through the identification module 20, further verification can be performed through the singles of the taken goods to increase the accuracy of identification.

Further, when the commodities are bulk commodities such as melon seeds, fruits and the like, the commodity singles cannot be directly compared with the commodity singles stored in the server 30 through the comparison between the commodity singles weighed by the intelligent shelf 50 and the commodity singles stored in the server 30 due to different commodity singles, so that the intelligent shelf 50 also has a bulk scale verification function.

Specifically, the smart shelf 50 includes a shelf body 51, and a first weighing device 511 and a second weighing device 53 provided on the shelf body 51. The shelf main body 51 is provided with at least one storage area 513, in this embodiment, it is described that the shelf main body 51 is provided with a plurality of storage areas 513 as an example, and each storage area 513 stores corresponding goods for the convenience of customers to purchase. Specifically, only products of the same type and price can be stored in each storage area 513.

The first weighing device 511 is disposed in each storage area 513 and directly below the bottom plate supporting the commodities, and is configured to sense first weight information of the corresponding commodities taken by the customer in each storage area 513. That is, each time a customer takes out one commodity in one storage area 513, the first weighing device 511 obtains first weight information due to the weight reduction, that is, information of the weight change in the storage area 513, by corresponding detection.

Optionally, the first weighing device 511 is a gravity sensor.

The second weighing devices 53 are provided on both sides of the shelf body 51, and when the customer has finished taking the goods from the storage area 513, the second weighing devices 53 can facilitate the customer to weigh the goods taken from the storage area 513 to obtain second weight information so as to verify the second weight information with the first weight information obtained by the first weighing device 511.

When the customer finishes purchasing the corresponding goods in each storage area 513 according to the purchasing rule set by the supermarket, the first weight information obtained by the first weighing device 511 is equal to the second weight information obtained by the second weighing device 53.

The supermarket sets the purchasing rules as follows: the customer can only pick the goods in the same storage area 513 to the second weighing device 53 for weighing at a time.

When a customer performs a wrong operation during shopping, that is, does not purchase goods according to the set purchasing rules, such as taking out goods from the plurality of storage areas 513 at the same time, the sum of the first weight information obtained by the first weighing device 511 is not matched with the second weight information obtained by the second weighing device 53, and when the above situation occurs, an alarm signal is generated.

In some application scenarios, when the user takes 3 apples out of one of the storage areas 513, the first weight information is the weight of the 3 apples, and then the 3 apples are weighed by placing the second weighing device 53, and the obtained second weight information is still the weight of the 3 apples, where the first weight information is equal to the second weight information. When a user has removed 2 apples from one of the storage areas 513 and 1 orange from the other storage area 513, the first weights obtained by the first weighing devices 511 on both storage areas 513 are 2 apples and one orange respectively. When the customer puts the taken goods on the second weighing device 53 for weighing, the obtained second weight information is the weight of 2 apples plus 1 orange, and the first weight information weighed by any of the other weighing devices 511 is different, at this time, the customer considers that the user does not purchase according to the rule.

It will be appreciated that when the user does not make a purchase according to the rules, the smart shelf 50 alerts the customer of the abnormal operation.

It is understood that the first weighing device 511 and the second weighing device 53 obtain the first weight information and the second weight information, transmit the first weight information and the second weight information to the server 30, and verify whether the first weight information and the second weight information are identical on the server 30.

It is understood that the unit price of the bulk commodity is the price under the set weight, and may be, for example, the price of each kilogram of the commodity or each kilogram of the commodity.

Further, as shown in fig. 2A, the smart shelf 50 further includes a payment code printing device 55, the payment code printing device 55 is disposed on the shelf main body 51, and when the first weight information is identical to the second weight information, the server 30 generates price information of the goods purchased by the user, and transmits the price information to the payment code printing device 55. The payment code printing device 55, upon receiving the price information, generates a payment code from the price information and prints the payment code so that the user can scan the payment code to make a payment.

It is understood that the payment code printing device 55 can print either a bar-type payment code or a two-dimensional code, and in other embodiments of the present invention, the payment code printing device 55 can print an RFID tag capable of recognizing commodity information directly on the commodity to be purchased.

Further, as shown in fig. 2A, the shelf main body 51 further includes a display device 57, the display device 57 is disposed on a side of the shelf main body 51 close to the storage area 513, and the server 30 sends the goods information of each storage area 513 to the display device 57 for displaying, so that the customer can know the unit price of the purchased goods and the total price of the purchased goods in real time when purchasing the goods.

It will be appreciated that the display device 57 may also display user identification information and advertising information. That is, when the user purchases the goods from the smart shelf 50, the server may also transmit the identification information recognized by the recognition module 20 to the display device 57.

It is understood that the display device 57 is a touch display device, and the user can click the display device 57 to operate, for example, click an advertisement place to watch specific content when the pushed advertisement is interested. It should be noted that, in the present embodiment, the position of the display device 57 is only used as an example, and in practical applications, the size and the specific setting position of the display device 57 may be adjusted based on practical requirements, which is not limited herein.

Further, the smart shelf 50 further includes an automatic sealing mechanism 59 provided on one side of the shelf body 51 in the arrangement direction of the storage areas 513 to seal the packing bags containing the goods when the goods are purchased by the customer. Preferably, the automatic packaging mechanism 59 is disposed adjacent to the second weighing device 53 or the payment code printing device 55, so as to facilitate timely sealing and attaching of payment codes to the weighed goods.

Referring to fig. 2B, the automatic packaging mechanism 59 includes a sealing unit 591 and a sensing unit 593. The sensing unit 593 is arranged adjacent to the sealing unit 591 so as to accurately seal the shopping bag according to the sensed change of the commodities placed in the shopping bag. Optionally, the sensing unit 593 is an infrared sensor to determine the sealing time by sensing information change of the goods placed in the shopping bag.

The automatic sealing mechanism 59 further includes an adjusting unit 595 and a driving unit (not shown), and the adjusting unit 595 and the driving unit can adjust the sealing position of the sealing unit 591 according to the commodity information.

Specifically, the adjusting unit 595 is connected to the sealing unit 591 and the sensing unit 593, and drives the sealing unit 591 and the sensing unit 593 to slide back and forth along the direction of the gravity of the object to be sealed and/or the opposite direction of the gravity through the driving unit; the sensing unit 593 is an infrared sensor; the sealing unit 591 seals the packaging bag when the sensing unit 593 senses the packaging bag.

It can be understood that the user can directly carry out payment operation on the scattered commodities by scanning the code payment code.

Referring to FIG. 3, a third embodiment of the present invention provides a shopping cart 60 that can be used in the first or second embodiment, wherein the shopping cart 60 is used for temporarily loading merchandise and cooperates with the identification module 20 to increase the accuracy of counting the purchased merchandise when the user purchases the merchandise.

Shopping cart 60 includes a weighing module 601, a code scanning module 603, and a camera module 605. The weighing module 601 is used for weighing the weight of the goods in the shopping cart 60 and transmitting the weight to the server 30, and the code scanning module 603 is used for scanning the two-dimensional code on the goods to acquire the information of the goods and transmitting the information to the server 30. The camera module 605 is used for shooting the motion video after the code scanning of the commodity and transmitting the video to the server 30. The server 30 is used for comparing whether the change of the weight information in the shopping cart 60 is consistent with the single weight of the commodities put into the shopping cart 60 or not, analyzing the motion track of the commodities in the video and counting the total price of the commodities in the shopping cart 60.

Specifically, the camera module 605 includes a plurality of cameras, and the plurality of cameras shoot the movement track of the commodity after scanning the code simultaneously to avoid increasing the shooting range, avoid shooting the dead angle.

Further, the shopping cart 60 further includes a display module 607, the display module 607 is a touch display device, and is disposed on the shopping cart 60 for displaying the commodity information in the shopping cart, and when the user needs to return the commodity in the shopping cart 60, the operation can be performed through the display module. Meanwhile, after the server 30 counts the total price of the goods in the shopping cart, the display module 607 can display the two-dimensional code, which is convenient for the user to pay by scanning the code.

It can be understood that the user can use the shopping cart when shopping in the shopping mall, and can also directly purchase without using the shopping cart. When the user does not use the shopping cart, when the user needs to leave the shopping mall after shopping is finished, the purchased commodities pass through the camera area one by one at the outlet of the shopping mall so as to verify the purchased commodities again. The specific point is that the camera shooting area can be composed of a camera and a conveyor belt, commodities are placed on the conveyor belt one by one, and shooting and confirmation are carried out one by one through the camera.

Referring to fig. 4, a fourth embodiment of the present invention provides a self-service shopping method for an unmanned supermarket, including the steps of:

s1, acquiring identity information;

s2, identifying whether the identity information is taken in the market; and

and S3, counting and accumulating the commodities taken by the identity information, and acquiring the information of the commodities purchased corresponding to the identity information.

Specifically, before entering a shopping mall, a user acquires identity information corresponding to each user, and after acquiring the identity information, the user enters the shopping mall to shop. When a user purchases goods, the user can take the goods, and the goods taken by each user in the shopping mall are accumulated, so that the total goods taken by the user can be obtained. After shopping is finished, the price to be paid is directly generated according to the commodity taken by the user, and the user can leave from a shopping mall after the payment is finished.

In some application scenarios, when a user enters a mall, the mall acquires the identity information of the user through modes of face recognition, identity information reading and the like. After the identity information is acquired, the entrance of the market is opened, and the user enters the inside of the market through the entrance to perform shopping operation. When a user wants to buy a certain commodity in a shopping mall, the user can take the commodity, the commodity is judged to be bought by the user, and when shopping is finished, the user only needs to pay the commodity at an outlet, and then the outlet of the shopping mall is opened so that the user can leave the shopping mall conveniently.

Furthermore, a display screen can be arranged at the outlet to display the related information of the user, and if alarm information appears, the display screen can display specific commodities which are problematic to purchase, so that the user can know and carry out corresponding processing in time.

It can be understood that the identity information can be obtained through the modes of face recognition, reading of a user identity card, binding of a user mobile phone and the like. Preferably, the acquisition is performed by means of face recognition.

Referring to fig. 5A, step S2 includes the steps of:

s21, recording the states of all commodities in the market;

s22, identifying the identity information captured when the commodity state changes and the changed commodity;

and S23, matching the changed commodity with the corresponding identity information.

Specifically, before the user enters the inside of the shopping mall, the state of the commodities in the shopping mall, that is, the position of each commodity in the shopping mall is recorded. When a user takes a commodity, the state of the commodity changes, that is, a certain commodity moves from the original position to another position. At this time, what the changed commodity is recognized, and the user and the identification information thereof closest to the changed position are matched with the commodity. That is, the article identified as being changed is taken by the user.

In some application scenarios, a user in a shopping mall sequentially and respectively takes a bottle of yogurt, a bag of instant noodles and a box of biscuits after entering the shopping mall, that is, the changed goods in the shopping mall are a bottle of yogurt, a bag of instant noodles and a box of biscuits. When the changes are captured, the user closest to the commodity area is judged to take the commodity, and after shopping is finished, the commodities purchased by the user are counted, namely a bottle of yoghourt, a bag of instant noodles and a box of biscuits.

It is understood that the end of the user's shopping is the location where the user reaches the exit area of the mall, and the size of the area may be determined according to the circumstances, and is not limited herein.

It can be understood that recording the states of all the commodities in the shopping mall means that all the commodity states are taken as pictures and stored before shopping is opened in the shopping mall.

It is understood that the matching is that the user identified as having the identity information purchased the changed goods.

Referring to fig. 5B, step S23 includes the steps of:

s221, acquiring real-time video information in a market;

s222, converting the video information into pictures by taking a frame as a unit;

and S223, sequentially comparing each frame of picture with the pictures in the step S21, identifying the commodity change area and the changed commodity in the frame of picture when the comparison result is different, capturing the nearest identity information close to the area, and matching the identity information with the changed commodity.

Specifically, after the initial picture of the commodity in the store is obtained in step S21, the user enters the store to start shopping. When a user enters a shopping mall for shopping, the inside of the shopping mall is monitored in real time through the camera, and shot videos are converted into pictures by taking frames as units. And sequentially comparing each frame of picture with the initial picture respectively according to the sequence, identifying the changed area in the picture when the compared picture is different from the initial picture, and identifying the specific changed commodity. And identifying the identity information of the user closest to the area in the frame of picture, finally matching the identity information with the changed commodity, and judging that the user purchases the changed commodity.

It is understood that the video is sorted in units of frames and then in units of time of the video, and when a change has been found in a preceding picture, the changed region is not compared in the comparison of the subsequent picture with the initial picture.

Referring to fig. 6, step S224 is further included after step S223 of the method, where the weight of the commodity that changes is obtained, and the commodity that matches the identity information is verified through weight.

Specifically, the step S223 of matching the product with the identity information specifically judges why the product has changed in an image recognition manner, but a single image recognition may generate an error in the recognition process, and the product information of the product cannot be accurately obtained. At this time, the matching accuracy can be improved by verifying the weight of the product.

In some application scenarios, goods in a store are placed on a shelf, and the shelf has a weighing function. When the user takes the goods on the goods shelf, the weight of the goods to be taken can be obtained by changing the weight of the goods shelf before and after taking. In combination with the commodity information of the commodity obtained by the image recognition in step S223, the commodity singles in the commodity information is compared with the weight of the removed commodity, if the comparison result is consistent, the matching result in step S223 is correct, otherwise, the matching result is wrong, the commodity is marked as an abnormal commodity, and the commodity is confirmed again when the user finishes payment.

Referring to fig. 7, after the step S224, the method further includes the steps of:

s225, binding the identity information with the shopping cart;

s226, scanning the code of the commodity, acquiring the commodity information of each code-scanned commodity, and recording all the code-scanned commodities; and

s227, the video after the code of the commodity is scanned is obtained, the shopping cart area in the video is set, and the motion track of the commodity in the video is captured to judge whether the commodity is placed in the shopping cart or not.

When a user shops in a shopping mall, the user usually takes goods and puts the goods into a shopping cart so as to carry the goods. The shopping cart is bound with the identity information of the user, the goods are scanned before being placed in the shopping cart, and then the goods are placed in the shopping cart, so that the step S224 of verification can be further performed. That is, after passing through the image recognition, the weight verification, and the code scanning verification, the product finally placed in the shopping cart must be the product purchased by the user.

In some application scenarios, a user may lose the desire to purchase the goods after scanning the codes and then drop the goods onto a shelf at will, or replace the goods after scanning the codes and place the goods without scanning the codes into a shopping cart, and these erroneous operations may affect the subsequent progress of shopping and increase the management cost of the supermarket.

In order to avoid the situation that the total price does not correspond to the time product in the shopping cart during payment, further control needs to be performed on the behavior of the user during autonomous shopping.

Specifically, the commodity information on the shelf is pre-stored in the database in advance, and when the user takes off the commodity to be purchased from the shelf, the specific numerical value of the commodity unit weight can be obtained. Then, the user scans the goods taken off the shelf to obtain the goods information of the goods, namely the appearance characteristics of the goods, the single weight information of the goods and the unit price of the goods. At this time, the motion track of the goods after code scanning is obtained, namely the motion route of the goods in the process of where the user puts the goods after code scanning. When the display commodity is put into the shopping cart, the weight of the display commodity is changed after the commodity is put into the shopping cart, if the weight change in the shopping cart is consistent with the single weight of the commodity taken away from the goods shelf, the display commodity is the same as the commodity taken away from the goods shelf, and a user can finish shopping only by paying for the total price generated according to the commodity information in the shopping cart after the shopping is finished. When the display commodity is not placed in the shopping cart, for example, the user randomly puts the code-scanned commodity back to a shelf or replaces the code-scanned commodity with other commodities, the shopping cart gives an abnormal alarm to remind the user of illegal operation.

Referring to fig. 8, step S226 includes step S2261, scanning the product and obtaining product information; and

step S2262, the unit prices of the goods passing the code scanning are accumulated.

Specifically, after the goods are taken off from the shelf and before the goods are put into the shopping cart, the goods need to be scanned to obtain the goods information of the goods, that is, the appearance characteristics, the single weight information and the unit price of the goods. In order to facilitate subsequent payment operation, the unit prices of the commodities are accumulated every time the user scans the codes, so that the total price of the commodities in the shopping cart is formed.

In some application scenarios, after scanning the code of the commodity and placing the commodity in the shopping cart, the user loses the desire to purchase the commodity, and needs to take out the commodity in the shopping cart, scan the code again, and then place the commodity back to the shelf or to a designated position. In this case, the unit price of the commodity obtained by scanning the code is a negative number, and the accumulated result is the total price minus the price of the commodity taken out, so as to ensure that the commodity in the final shopping cart is matched with the actually generated total price.

That is, when the user scans the product obtained from the outside of the shopping cart, the product unit price is a positive number, and when the user takes the product from the inside of the shopping cart, the code is scanned, the product unit price is a negative number.

It can be understood that whether the user takes out the goods from the shopping cart for code scanning can be determined by whether the total weight in the shopping cart is reduced. If the total weight of the commodities in the shopping cart is detected to be reduced and the user conducts code scanning operation on the commodities, the commodities are judged to be taken out from the shopping cart, and the unit price of the commodities obtained through code scanning is negative.

It can be understood that the total price of the items in the shopping cart is not limited to the accumulation of each code scanning, as long as the total price of the items in the shopping cart can be obtained finally.

Referring to fig. 9, step S227 includes step S2271, converting the acquired video into a picture in units of each frame;

step S2272, establishing a coordinate system on each picture by taking the same position of each picture as an origin;

step S2273, establishing a shopping cart area in a coordinate system; and

step S2274, identifying the position of the commodity in the coordinate system in each picture, judging that the commodity is put into the shopping cart when the position of the commodity in the picture is in the shopping cart area, judging that the operation of the user is illegal when the position of the commodity cannot be found in the coordinate system, and giving an alarm to the shopping cart.

The specific position selected as the origin is not limited, and can be adjusted based on the state of the actual shopping cart. It should be particularly noted that, the specific steps S2271-S2274 of the above step S3 are only used as examples of the present invention, and are not meant to be limitations of the present invention.

After the user takes the goods from the shelf and scans the codes, in order to prevent the user from not putting the goods into the shopping cart or putting the code-scanned goods into the shopping cart instead of other goods, the code-scanned goods need to be monitored, that is, the movement track of the goods needs to be monitored.

In some application scenarios, a user takes a commodity a with a unit price of 10 yuan, scans the code of the commodity a, puts the commodity a back after scanning the code, and takes a commodity B with a unit price of 20 yuan to put the commodity B into a shopping cart, wherein if the weight difference between the commodity a and the commodity B is small, the situation that the commodity a is bought at the price of 10 yuan may occur, and a loss may be caused to a shopping mall.

In order to avoid this situation, it is necessary to ensure that the scanned goods are not replaced by other goods. The specific implementation method comprises the steps of acquiring a video of a commodity after code scanning, capturing position change of the commodity in the video, and analyzing whether the code-scanned commodity is placed in a shopping cart or not through the position change. When the goods after code scanning are actually put into the shopping cart, the next operation can be normally carried out, and when the goods after code scanning are not put into the shopping cart and are out of the video monitoring range, the situation that goods replacement or other situations possibly occur is indicated, and at the moment, the shopping cart sends out an alarm to remind a user of abnormal operation.

It should be noted that, in the above steps, the interval time of each frame of picture can be selected as required to meet the requirements of different application scenarios, so that the monitoring accuracy can be ensured while the calculation amount is reduced as much as possible.

Specifically, after the acquired video is obtained, the video is converted into a picture form in a frame unit, and a coordinate system is established with the same position of each picture as an origin, for example, all the pictures are established with one corner as the origin. And then establishing a corresponding shopping cart area in the coordinate system, wherein the position of the shopping cart area in the coordinate system in each picture is the same as the position of the original point in each picture, and the commodity information obtained during code scanning comprises the appearance characteristics of the commodity, and the appearance characteristics serve as the judgment basis of the position of the commodity in the coordinate system, so that the position of the commodity in the coordinate system in each picture can be obtained. If the position of the commodity cannot be captured in any picture, namely the corresponding commodity is considered to be lost in the coordinate system, the commodity is out of the range of acquiring the video, and illegal operation may exist. When the captured commodities are not lost in the coordinate system in all the pictures corresponding to the videos, and the positions where the commodities are captured in the pictures are located in the shopping cart area, the commodities are judged to be placed in the shopping cart.

It can be understood that when a plurality of cameras with different angles are used for video shooting, pictures formed by the videos shot by each camera are separately judged, namely, the shopping cart areas are the same in the coordinate system set by the pictures formed by the videos shot by each camera.

For example, the motion track of the commodity is shot by using three cameras, namely a camera a, a camera B and a camera C, the shot video can be divided into three videos, namely a video a, a video B and a video C, and then a coordinate system is respectively established by using pictures formed by the three videos so as to judge whether the commodity is put into the shopping cart.

Referring to fig. 10A-10B, the self-service shopping method for an unmanned supermarket further includes step S228: after the commodity is placed in the shopping cart, acquiring the weight change in the shopping cart, and judging whether the weight change in the shopping cart is consistent with the single weight of the placed commodity; and

step S229: and after the weight change is judged to be consistent, calculating the total commodity price in the shopping cart according to the commodity information of the code-scanned commodity.

Specifically, after the user is verified to operate normally through the step S227, a weight verification is added, that is, the single weight of the code-scanned commodity is compared with the actually changed weight of the shopping cart, and then the total price in the shopping cart is calculated, so that the calculated total price is more accurate.

Specifically, step S228 further includes step S2281: acquiring the total weight of commodities in a shopping cart;

step S2282: subtracting the weight in the shopping cart before code scanning from the total weight obtained in the step S41 to obtain the change of the weight of the commodity; and

step S2283: and comparing the obtained weight change of the commodity with the commodity taken away from the goods shelf to judge whether the weight change of the commodity is consistent with the weight change of the commodity taken away from the goods shelf.

Specifically, as shown in fig. 10B, after determining that the goods are placed in the shopping cart after being scanned, the total weight of the goods in the shopping cart is obtained, and then the weight of the goods in the shopping cart before being scanned is subtracted from the total weight, so that the weight change of the goods in the shopping cart, that is, the weight of the goods placed in the shopping cart can be obtained. The weight is compared with the single weight of the commodity taken out of the shelf obtained in step S1, and if the result of the comparison is identical, it is determined that the commodity taken out of the shelf is identical to the commodity put in the shopping cart. Otherwise, the operation is judged to be abnormal, and the shopping cart gives an alarm.

By obtaining the comparison between the movement track of the commodities after code scanning and the weight of the goods shelf and the weight of the shopping cart, the final total price of the commodities in the shopping cart can be accurately obtained.

Referring to fig. 11, the self-service shopping method in unmanned supermarket further includes step S2210 of providing a payment code so that the user can pay for the goods in the shopping cart by scanning the payment code.

Specifically, after the total price of the commodities in the shopping cart is generated, the payment code is generated, and the user can scan the code payment code to pay for the commodities in the shopping cart.

It can be understood that the payment code generation can be that a display screen is arranged on the shopping cart to generate the payment code according to the summary, or that a two-dimensional code for payment is pasted on the shopping cart, and the user inputs the amount of money by himself after scanning the code.

It is understood that step S223 may be followed by step S224 and/or one of step S2210, step S225-step S229, step S224-step S229, step S225-step S2210, step S224-step S2210.

The fifth embodiment of the present invention also provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the self-service shopping method as described above when running.

According to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from the information transfer module via the communication section, and/or installed from a removable medium. The computer program, when executed by a Central Processing Unit (CPU), performs the above-described functions defined in the method of the present application. It should be noted that the computer readable medium described herein can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present application, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In this application, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

Computer program code for carrying out operations of the present application may be written in any combination of one or more programming languages, including AN object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages.

The computer-readable storage medium and the system provided by the invention have the advantages.

The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit of the present invention are intended to be included within the scope of the present invention.

Claims

1. A self-service shopping method of an unmanned supermarket is used for realizing self-service shopping of a user in a shopping mall, and is characterized in that: the method comprises the following steps:

s1, acquiring identity information;

s2, recognizing the change of the commodity in the shop and matching the corresponding identity information; and

and S3, counting the commodities matched with the identity information, and acquiring the commodity information purchased corresponding to the identity information.

2. The self-service shopping method for unmanned supermarket as claimed in claim 1, characterized in that: step S2 includes:

step S21, recording the states of all commodities in the market;

step S22, identifying the identity information captured when the commodity state changes and the changed commodity position; and

and step S23, matching the changed commodities with the corresponding identity information.

3. An unmanned supermarket self-service shopping method as claimed in claim 2, characterized in that: step S22 includes:

step S221, acquiring real-time video information in a mall;

step S222, converting the video information into pictures by taking frames as units; and

and step S223, comparing each frame of picture with the pictures in the step S21 in sequence, identifying the commodity change area and the changed commodity in the frame of picture when the comparison result is different, capturing the identity information close to the area, and matching the identity information with the changed commodity.

4. An unmanned supermarket self-service shopping method as claimed in claim 3, characterized in that: after step S223, the method further includes:

step S224, matching the changed commodities and the corresponding identity information and/or

In step S2210, a payment code is provided so that the user can pay for the merchandise in the shopping cart by scanning the payment code.

5. An unmanned supermarket self-service shopping method as claimed in claim 3 or 4, wherein: after step S223, the method further includes:

step S225, binding the identity information with the shopping cart;

step S226, scanning the code of the commodity, acquiring the commodity information of each code-scanned commodity, and recording all the code-scanned commodities; and

step S227, acquiring a video after the code of the commodity is scanned, setting a shopping cart area in the video, and capturing the motion track of the commodity in the video to judge whether the commodity is put in the shopping cart;

step S228: after the commodity is placed in the shopping cart, acquiring the weight change in the shopping cart, and judging whether the weight change in the shopping cart is consistent with the single weight of the placed commodity; and

6. An unmanned supermarket self-service shopping method as claimed in claim 5, characterized in that: step S229 further includes:

7. An unmanned supermarket self-service shopping method as claimed in claim 5, characterized in that: step S227 includes the steps of:

step S2271, converting the acquired video into pictures by taking each frame as a unit;

step S2273, establishing a shopping cart area in a coordinate system; and

8. A computer-readable storage medium characterized by: the computer-readable storage medium has stored thereon a computer program, wherein the computer program is arranged to execute the shop shopping method according to any one of claims 1 to 7 when executed.

9. The utility model provides an unmanned supermarket self-service shopping system which characterized in that: the shopping system of the market comprises:

10. An unmanned supermarket self-service shopping system as claimed in claim 9, wherein:

the shopping system of the shopping mall also comprises a goods returning module, an intelligent goods shelf and a shopping cart;