WO2019062018A1 - 商品自动结算方法、装置、自助收银台 - Google Patents

商品自动结算方法、装置、自助收银台 Download PDF

Info

Publication number
WO2019062018A1
WO2019062018A1 PCT/CN2018/077652 CN2018077652W WO2019062018A1 WO 2019062018 A1 WO2019062018 A1 WO 2019062018A1 CN 2018077652 W CN2018077652 W CN 2018077652W WO 2019062018 A1 WO2019062018 A1 WO 2019062018A1
Authority
WO
WIPO (PCT)
Prior art keywords
information
neural network
commodity
product
automatic
Prior art date
Application number
PCT/CN2018/077652
Other languages
English (en)
French (fr)
Inventor
陈子林
王良旗
Original Assignee
缤果可为（北京）科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 缤果可为（北京）科技有限公司 filed Critical 缤果可为（北京）科技有限公司
Publication of WO2019062018A1 publication Critical patent/WO2019062018A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q20/00—Payment architectures, schemes or protocols
    • G06Q20/08—Payment architectures
    • G06Q20/20—Point-of-sale [POS] network systems
    • G06Q20/208—Input by product or record sensing, e.g. weighing or scanner processing
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07G—REGISTERING THE RECEIPT OF CASH, VALUABLES, OR TOKENS
    • G07G1/00—Cash registers
    • G07G1/0036—Checkout procedures
    • G07G1/0045—Checkout procedures with a code reader for reading of an identifying code of the article to be registered, e.g. barcode reader or radio-frequency identity [RFID] reader
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07G—REGISTERING THE RECEIPT OF CASH, VALUABLES, OR TOKENS
    • G07G3/00—Alarm indicators, e.g. bells
    • G07G3/003—Anti-theft control

Definitions

  • the invention relates to a commodity automatic settlement method and device, and a self-service cash register, belonging to the technical field of image recognition.
  • the present invention provides an automatic product settlement method, which can accurately identify a product image through a neural network, and does not need to identify the product by any third-party identification, and the user only needs to place the purchased product on the desktop. Identification, the entire settlement process can be completed by the customer in the case of unattended service, which can effectively reduce operating costs.
  • Automatic product settlement methods including:
  • an image containing the merchandise comprising at least a first image to an Nth image of different angles and/or different depths of field;
  • the neural network-based network identification system includes a first neural network based on a regional convolutional neural network; the commodity automatic settlement method includes the steps of:
  • the first product information is output as the product information
  • the time interval after the payment is successful until the anti-theft code of the product is decoded does not exceed 0.3 seconds
  • the duration of the process of decoding the anti-theft code of the item does not exceed 0.3 seconds.
  • obtaining an image containing the item is at least a two-dimensional image.
  • N 2.
  • the step (a1) further includes the step of weighing the product to be inspected, and obtaining the total weight of the actually weighed product;
  • the step (b1) is: (b2) calculating the total weight of the commodity in the first commodity information, and obtaining differential data by comparing the total weight of the commodity with the actual weighing, and determining whether the differential data is less than or equal to a preset threshold:
  • the first product information is output as the product information
  • the neural network based identification system comprises a first neural network based on a regional convolutional neural network;
  • the commodity automatic settlement method comprises the steps of:
  • the first product information is output as the product information
  • the first product information is output as the product information
  • the method of determining whether the Nth product information is included in the first product information in the step (b1) and the step (b3) is determining whether the product type in the Nth product information is present in the first product information.
  • the method of determining whether the Nth item information is included in the first item information in the step (b1) and the step (b3) is: determining whether the quantity of the item in the item N item information is less than or equal to the item in the first item information. Quantity.
  • the method of determining whether the Nth item information is included in the first item information in the step (b1) and the step (b3) is: determining whether the quantity of each item in the item N item information is less than or equal to the first item information. The number of items in the market.
  • step (b1) and step (b3) are to determine whether the Nth item information is consistent with the first item information
  • the first product information is output as the product information
  • the Nth item information in the step (b1) and the step (b3) is consistent with the first item information, including the item type being consistent and the quantity of each item being consistent.
  • the preset threshold value in the step (b2) and the step (c3) is at least one of 0.1 g to 10 kg.
  • the preset threshold in the step (b2) and the step (c3) is the weight of the commodity with the smallest weight in the first commodity information.
  • the preset threshold in the step (b2) and the step (c3) is at least one of 10% to 80% of the weight of the smallest weight commodity in the first commodity information.
  • the feedback prompt in the step (b2) and the step (c3) includes at least one of a stacking prompt and an error report.
  • the number of items in the image containing the item is ⁇ 1.
  • the number of items in the image containing the product is from 1 to 1000.
  • the type of the item in the image containing the product is ⁇ 1.
  • the type of the product is 1 to 1000.
  • the neural network based identification system comprises a second neural network based on a regional convolutional neural network, and the neural network based identification system is obtained by a method comprising the following steps:
  • the second neural network is trained using the first set of images to obtain a first neural network.
  • the method of training the second neural network is a supervised learning method.
  • the method of training the second neural network is:
  • the third neural network is trained with the second image set to obtain a first neural network.
  • the second set of images includes an image of the merchandise that outputs the merchandise information via the neural network based recognition system.
  • the second neural network has an accuracy rate of 80% or more for the identification of the commodity.
  • the process of training the third neural network with the second set of images is not unsupervised.
  • the automatic product settlement method includes the following steps:
  • step (c1) when the determination result in the step (b1) is NO, identifying the difference product in the first product information and the Nth product information;
  • step (d1) acquiring the difference image set of the difference commodity in the step (c1), and intensively training the first neural network with the difference image set.
  • the automatic product settlement method includes the following steps:
  • the first product information is output as the product information
  • step (d3) when the determination result in the step (c3) is NO, identifying the difference commodity in the first product information and the Nth product information;
  • step (e3) acquiring the difference image set of the difference commodity in the step (d3), and intensively training the first neural network with the difference image set.
  • the automatic product settlement method includes the following steps:
  • step (d2) acquiring the collected image set of the identified item in the step (c2), and intensively training the first neural network with the collected image set.
  • the automatic product settlement method includes the following steps:
  • the first product information is output as the product information
  • the update frequency of the image containing the commodity obtained by the continuous update is not less than 10 times/second.
  • the update frequency of the image containing the merchandise is continuously updated from 10 times/second to 100 times/second.
  • the time to obtain the image containing the merchandise until the payment information is generated does not exceed 0.1 second.
  • the time from obtaining the image containing the merchandise to generating the payment information is from 0.001 second to 0.1 second.
  • the time interval from the successful payment to the decoding of the anti-theft code of the item does not exceed 0.1 second.
  • the time interval from the successful payment to the decoding of the anti-theft code of the item is 0.001 second to 0.1 second.
  • the duration of the process of decoding the anti-theft code for the item does not exceed 0.1 seconds.
  • the duration of the process of decoding the anti-theft code of the item is from 0.001 second to 0.1 second.
  • the method further includes: a verification step: determining whether the anti-theft code of the successful product is not decoded;
  • an alarm instruction is executed when the undecoded anti-theft code is detected when the product goes out.
  • the anti-theft code of the product is decoded, and the step of verifying whether the successful product anti-theft code is decoded is determined: whether the product anti-theft code is not decoded after the successful payment;
  • the alarm instruction is executed when the undecoded anti-theft code is detected when the product goes out.
  • the type and quantity of the current item are identified by the surveillance camera, and compared with the type and quantity of the product that is successfully paid, and the type and quantity of the unpaid item are displayed.
  • the result of the verification step is that all the decoding is successful
  • the type and quantity of the current item are identified by the surveillance camera, and compared with the type and quantity of the product that is successfully paid. Show the difference in some items and show that they are not paid.
  • an automatic commodity settlement apparatus comprising:
  • the image capturing unit is configured to obtain the first image to the Nth image that the product includes at least an angle and/or a depth of field, and the product is provided with an anti-theft code;
  • the identification information unit is configured to input the first image into the first neural network, the first neural network outputs the first commodity information; the Nth image is input to the first neural network, and the first neural network outputs the Nth commodity information;
  • the identification determining unit is configured to determine whether the Nth product information is included in the first product information; if the determination result is yes, the first product information is output as the to-be-detected product information; if the determination result is no, the feedback prompt is output;
  • a display unit configured to output commodity information and generate display payment information, determine whether the payment is successful, issue a decoding instruction, or display a feedback prompt;
  • a decoding unit configured to decode the anti-theft code
  • the imaging unit is connected to the identification information unit, the identification information unit is connected to the identification determination unit, the identification determination unit is connected to the display unit data, and the decoding unit is connected to the display unit.
  • the identification information unit and the identification determining unit are configured to perform commodity identification and determination according to the foregoing automatic commodity settlement method.
  • the commodity automatic settlement device further includes a verification unit, configured to determine whether the anti-theft code of the successful payment product is not decoded;
  • an alarm instruction is executed when the undecoded anti-theft code is detected when the product goes out.
  • the verification unit further includes a surveillance camera and a verification unit, wherein the verification unit is configured to identify the type and quantity of the current commodity by the surveillance camera, compare the type and quantity of the successfully paid product, and display the type and quantity of the unpaid commodity.
  • the surveillance camera is connected to the verification unit, and the verification unit is connected to the display unit.
  • the camera unit includes at least two cameras that capture product images of different angles and/or depths of field.
  • the camera unit includes at least 2 to 4 cameras that capture product images of different angles and/or depths of field.
  • the camera unit includes a first camera and a second camera
  • the first camera and the second camera respectively acquire product images from different angles.
  • a stage is included, the stage includes a weight sensor for measuring the total weight of the goods on the stage; and the weight sensor is connected to the identification information unit data.
  • a self-service checkout counter is provided, and the self-service checkout counter performs commodity identification using any of the above-described automatic item settlement methods.
  • a self-service checkout counter is provided, and the self-service checkout counter employs any of the above-described automatic item settlement devices.
  • the automatic product settlement method provided by the invention fully utilizes the neural network to identify the commodity, and judges the commodity information obtained from the obtained plurality of images, thereby avoiding the excessive recognition of the image recognition in the existing image recognition field, resulting in the recognition error. Rate, which improves recognition accuracy. At the same time, the entire settlement process can be completed without the cashier service, reducing operating costs.
  • the method for automatic settlement of goods provided by the present invention through the sustainable learning of deep learning, continuously improves the recognition accuracy of the method as the frequency of use increases.
  • the anti-theft code setting can also play the role of anti-theft.
  • the automatic product settlement method provided by the present invention can capture the product screen through the ordinary camera, thereby realizing rapid detection of the batch goods, and greatly reducing the cost and speed of the product identification.
  • the automatic product settlement method provided by the present invention can realize the low-cost and high-efficiency completion of product identification and settlement under the self-settlement scenario.
  • the automatic commodity settlement device provided by the present invention corrects the recognition result by neural network identification and multi-image comparison, and uses the recognition result to train the neural network system through the obtained image set through the deep learning system. Continuously improve its recognition accuracy and achieve efficient and accurate self-checkout.
  • FIG. 1 is a schematic block diagram showing the flow of an automatic settlement method for goods in a first preferred embodiment of the present invention
  • FIG. 2 is a schematic block diagram showing the flow of an automatic settlement method for goods in a second preferred embodiment of the present invention
  • FIG. 3 is a schematic block diagram showing the flow of an automatic settlement method for goods in a third preferred embodiment of the present invention.
  • FIG. 4 is a schematic block diagram showing the flow of an automatic settlement method for goods in a fourth preferred embodiment of the present invention.
  • Figure 5 is a schematic block diagram showing the flow of an automatic settlement method for goods in a fifth preferred embodiment of the present invention.
  • FIG. 6 is a schematic block diagram showing the flow of an automatic settlement method for goods in a sixth preferred embodiment of the present invention.
  • Figure 8 is a schematic block diagram showing the structure of an automatic commodity settlement device provided by the present invention.
  • FIG. 9 is a timing diagram of the present invention for providing an automatic settlement method for an item to an unattended convenience store of a self-service checkout counter.
  • the anti-theft code can be various types of barcodes or AM tags with anti-theft effects.
  • the decoding step can be degaussing.
  • an automatic settlement method for goods provided by the present invention includes:
  • an image containing the merchandise comprising at least a first image to an Nth image of different angles and/or different depths of field;
  • the neural network-based recognition system includes a first neural network based on a regional convolutional neural network; the commodity automatic settlement method includes the steps of:
  • the first product information is output as the product information
  • the automatic product settlement method provided by the invention is mainly used for self-service shopping after self-service acquisition of settlement commodity information in an unattended environment.
  • the method fully utilizes the neural network to identify the commodity, and judges the commodity information obtained from the obtained multiple images, thereby avoiding the excessive recognition of the image recognition in the existing image recognition field, resulting in the recognition error rate and improving the recognition accuracy.
  • the user can be reminded by the feedback prompt, so that the identification error can be corrected only by adjusting the product to be identified, without repeatedly scanning the code or trying many times.
  • various types of payment information such as payment information such as a two-dimensional code, may be generated, or may be a product information list, so that the customer can check the product information and make a payment.
  • the product is bulk decoded after payment, the product can be obtained. If the payment is not successful, the decoding step is not entered and the step of executing the display payment information is returned.
  • the anti-theft code can be set on the product to achieve self-checkout with high efficiency and anti-theft. It can be used in a variety of scenarios, such as canteens and unattended stores. No manual maintenance is required, which reduces manual use and reduces operating costs.
  • the feedback prompt here includes at least one of a stacking prompt and an error report.
  • the method may be used for the type and quantity of the goods to be processed, and may be, for example, the number of items to be detected in the image containing the item to be inspected ⁇ 1.
  • the number of items to be detected in the image containing the product to be detected is 1 to 1000.
  • the type of the product to be detected in the image containing the product to be inspected is ⁇ 1.
  • the type of the product to be tested is 1 to 1000.
  • the judged product information includes the product type or the number of each product. Determine whether the product type and/or the quantity of the product are consistent.
  • the automatic product settlement method provided by the invention is used for settlement in an unattended environment, and only needs to use an ordinary camera with network networking function to realize accurate identification of goods.
  • the first image is a frontal image of the item to be inspected.
  • the main image for recognition the accuracy of recognition can be improved.
  • N the recognition accuracy of the neural network can be improved. It is beneficial to improve the accuracy of subsequent recognition results.
  • the step (a1) further comprises the steps of weighing the product to be inspected to obtain the total weight of the commodity actually weighed; and the step (b1) is (b2) calculating the total weight of the commodity in the first product information. Comparing with the total weight of the actually weighed goods, the difference data is obtained, and it is judged whether the difference data is less than or equal to a preset threshold: if the judgment result is yes, the first commodity information is output as the commodity information to be detected; if the judgment result is no, the output is output. Feedback tips. At the same time, for the obtained product information, the obtained product can be corrected by analyzing the weight of the product included in the product information, thereby improving the accuracy of the image recognition result.
  • the neural network-based identification system includes a first neural network based on a regional convolutional neural network; the commodity automatic settlement method includes the steps of: (a3) inputting the first image into the first neural network, the first neural network Outputting the first product information; inputting the Nth image into the first neural network, the first neural network outputting the Nth product information; (b3) determining whether the Nth product information is included in the first product information; if the judgment result is yes, The first product information is output as the product information to be detected; if the determination result is no, the subsequent steps are performed; (c3) calculating the total weight of the goods in the first product information, and comparing with the total weight of the actually weighed goods to obtain differential data, and determining Whether the difference data is less than or equal to the preset threshold: if the judgment result is yes, the first commodity information is output as the commodity information to be detected; if the determination result is no, the feedback prompt is output.
  • the preset threshold here may be at least one of 0.1 g to 10 kg.
  • the preset threshold may also be the weight of the smallest item in the first item information.
  • the preset threshold may also be at least one of 10% to 80% of the weight of the smallest weight commodity in the first item information.
  • the method of determining whether the Nth product information is included in the first product information in the step (b1) and the step (b3) is to determine whether the product type in the Nth product information is present in the first product information.
  • the method of determining whether the Nth item information is included in the first item information in the step (b1) and the step (b3) is: determining whether the quantity of each item in the item N item information is less than or equal to the first item information. The number of items.
  • the step (b1) and the step (b3) are to determine whether the Nth item information is consistent with the first item information; if the determination result is yes, the first item information is output as the item information to be detected; , then perform the next steps.
  • the Nth item information in the step (b1) and the step (b3) is consistent with the first item information, including the same item type and the quantity of each item are consistent.
  • the neural network-based identification system comprises a second neural network based on a regional convolutional neural network
  • the neural network-based identification system is obtained by a method comprising the steps of: obtaining a first image set of each multi-angle image of the commodity to be inspected Using the first set of images to train the second neural network to obtain a first neural network.
  • the obtained result can be used to train the first neural network, thereby realizing deep learning automatic system error correction, and the recognition accuracy of the neural recognition system automatically increases as the number of identified commodities increases. It can be done according to the existing method. Training with multi-angle images of the products to be inspected can improve the recognition accuracy of the neural network-based recognition system in response to occlusion of goods.
  • the method of training the second neural network is a supervised learning method.
  • the method for training the second neural network is: using supervised learning, training the second neural network with the first image set to obtain a third neural network; obtaining a second image set of the product image to be detected; training with the second image set The third neural network obtains the first neural network.
  • the second image set includes an image of the item to be detected that outputs the item information to be detected via the neural network based identification system.
  • the recognition accuracy of the second neural network to be detected is 80% or more.
  • the process of training the third neural network of the second image set is unsupervised learning. It can be done according to the existing method.
  • the automatic commodity settlement method comprises the steps of:
  • the error correction capability of the system can be further improved by training the first neural network with the difference image set. At the same time, this operation can also be used in the method shown in FIG.
  • the method for automatically clearing goods includes steps:
  • This step can also be applied to the method shown in FIG. 3, which is not described here.
  • the detection result is no, the first commodity information in the case of multiple unrecognized situations is collected and used to train the first neural network, thereby improving the recognition capability of the first neural network for the unrecognizable situation.
  • the method further includes a verification step of: determining whether the anti-theft code of the successfully-paid product is not decoded; if the determination result is yes, determining whether the type and quantity of the product change, and if the determination result is no change, then the product is again The anti-theft code is decoded; if the judgment result is that all the decoding is successful, the alarm instruction is executed when the undecoded anti-theft code is detected when the product goes out.
  • the method of verifying the decoded product is further included. After verification, if the undecoded product is found, the product information and the type and quantity of the goods to be settled are checked again. In case the customer decodes the unsettled goods. After all decoding again, the products that have been paid but not decoded are decoded, and the inconvenience caused by undecoding to the customer is reduced, and the settlement efficiency is improved. At the same time, in the case of unsettled, since the anti-theft code is not decoded, the customer cannot take the merchandise away from the store. And there is a danger of triggering an alarm, which can better avoid the problem of stolen goods.
  • the type and quantity of the current item are identified by the surveillance camera, and compared with the type and quantity of the product that is successfully paid, and the type and quantity of the unpaid item are displayed.
  • the surveillance camera here can be a camera for anti-theft in all corners of the store, or a camera for obtaining surveillance images on the settlement counter.
  • the method for self-service settlement of goods comprises the steps of: the customer enters the unattended store, selects the goods, places them in the settlement device, acquires the product image, and identifies the product information, and the identification acquisition method is as described above. This is not exhaustive.
  • the payment information is generated based on the product information, and the customer makes a payment accordingly, and then determines whether the payment is successful. If the payment is successful, the decoding operation is performed, and if the payment is not successful, the payment information is continuously displayed to the customer. After decoding, check whether the degaussing is performed. At this time, the decoding result can be prompted again, and the customer checks it by himself. If not all decoded, return to decode again.
  • the anti-theft alarm device obtains the signal of the anti-theft code to determine whether there is an undecoded product. If the decoding has been completed, the shopping process ends. If there are still undecoded products, then alarm and close the gate. In case the customer takes out the unsettled goods.
  • the components, instruments and equipment used in the present invention are commercially purchased and used without modification.
  • the instruments and equipment used are those recommended by the manufacturer.
  • the automatic product settlement method when used, places the product to be detected on the stage, and N cameras are arranged around the product to be detected.
  • the images of the respective angles of the products to be detected are obtained by N cameras, and are respectively recorded as P1, P2, . . . , PN.
  • the camera located directly above the stage is the main camera, and is recorded as the first camera.
  • the image acquired by the camera is the first image P1.
  • P1, P2, . . . PN are uploaded to the local identification server or the cloud identification server, and each picture is identified, and the identified product information is respectively recorded as R1, R2, . . . RN, and the product information includes the product.
  • R2 second product information
  • R1 first product information
  • R1 is output as the commodity information to be detected
  • the total weight of the commodity in R1 is calculated, and the absolute value of the result obtained by subtracting the total weight of the commodity from the actual weighing is used as the difference data to determine whether the difference data is less than or equal to a preset threshold:
  • R1 is output as the product information to be detected, and the product information list including the category, quantity, and price of the product is output;
  • an automatic commodity settlement apparatus including:
  • the image capturing unit 100 is configured to obtain at least an image from the first image to the Nth image having different angles and/or depths of field, and the commodity is provided with an anti-theft code;
  • the identification information unit 210 is configured to input the first image into the first neural network, the first neural network outputs the first commodity information; the Nth image is input to the first neural network, and the first neural network outputs the Nth commodity information;
  • the identification determining unit 220 is configured to determine whether the Nth product information is included in the first product information; if the determination result is yes, the first product information is output as the to-be-detected product information; if the determination result is no, the feedback prompt is output ;
  • the display unit 300 is configured to output commodity information and generate display payment information, determine whether the payment is successful, issue a decoding instruction, or display a feedback prompt;
  • a decoding unit 400 configured to decode an anti-theft code
  • the imaging unit 100 is connected to the identification information unit, the identification information unit is connected to the identification determination unit 220, the identification determination unit 220 is connected to the display unit 300, and the decoding unit 400 is connected to the display unit 300.
  • the self-service settlement device After obtaining the image by the camera unit 100, the self-service settlement device provided by the camera unit 100 can obtain various information of the product, and cooperate with the anti-theft code on the product, and generate payment information according to the product information, and the payment is completed.
  • the self-decoding can take the goods away, and the whole settlement process can be completed without the help of the service personnel, and the recognition accuracy is high. Store operating costs are low.
  • the identification information unit and the identification determination unit 220 are configured to perform product identification and determination according to the above-described automatic item settlement method.
  • the commodity automatic settlement device further includes a verification unit, configured to determine whether the anti-theft code of the successful payment product is not decoded;
  • an alarm instruction is executed when the undecoded anti-theft code is detected when the product goes out.
  • the verification unit further comprises a surveillance camera and a verification unit, wherein the verification unit is configured to identify the type and quantity of the current commodity by the surveillance camera, compare the type and quantity of the successfully paid product, and display the type and quantity of the unpaid commodity.
  • the surveillance camera is connected to the verification unit, and the verification unit is connected to the display unit 300.
  • the camera unit 100 includes two common web cameras, two fixers that can adjust any angle, a continuous computer that can run image uploads, and a high-precision weight sensor.
  • the main workflow is: running an image capture program on the computer, which can upload the image captured by the two cameras at the same time to the remote server, and the remote server will return the recognition result.
  • the cost of this solution is extremely low, and the working computer only needs the most basic configuration.
  • the camera unit 100 includes 2-4 fixed-lens high-definition cameras, a corresponding number of adjustable angle fixers, a high-precision weight sensor, and a computer with a memory card of more than 2G.
  • the main workflow is to run an image capture program on the computer, which can identify the image captured by the two cameras at the same time locally.
  • the automatic commodity settlement device can detect in batches (low cost scheme), and obtain images of the goods to be detected from different angles by using a plurality of ordinary cameras.
  • a plurality of different angle cameras can solve the problem of occlusion caused by the difference in placement angle and item height in the same 2D picture. Basically, three cameras can achieve the information needed to be identified without dead angles. In the case of a suitable camera position, two cameras can also achieve better results.
  • the camera unit 100 includes a first camera and a second camera;
  • the first camera and the second camera respectively acquire product images from different angles.
  • the automatic commodity settlement device comprises a stage, and the stage comprises a weight sensor for measuring the total weight of the goods on the stage;
  • the weight sensor is electrically connected to the identification unit, and the total weight of the goods on the stage is input to the identification unit.
  • the present invention combines the weight sensor to correct the image recognition result, and obtains the weight of the item in the recognition result and the weight sensor in the identification device to be actually weighed. If not, the feedback item is in a stacked state.
  • a self-service checkout counter uses the aforementioned automatic item settlement method for product identification.
  • the self-help can be either unattended or under the supervision of a loss prevention officer. You only need to make a settlement operation for the customer.
  • the customer can complete the calculation process efficiently and accurately, and the entire equipment cost is low, and no electronic label is needed.
  • a self-service checkout counter is provided, and the self-service checkout counter adopts the aforementioned automatic commodity settlement device.
  • the self-help here can be either unattended or under the supervision of a loss prevention officer. You only need to make a settlement operation for the customer.
  • Fig. 9 is a timing chart showing an embodiment of an unmanned convenience store for a self-service checkout counter by the automatic commodity settlement device of the present invention. It can also be used as an implementation example of the self-service checkout counter provided by the present invention.
  • an automatic item settlement apparatus including any of the above-described automatic item settlement methods is used, and the shopping steps of the customer in the unattended convenience store are as follows:
  • the stage senses a weight of >0, triggering the automatic item settlement device to start the item identification program;
  • the camera captures the goods on the stage, obtains the product picture, and encodes the product picture Base64 to the image recognition server for image recognition;
  • the order processing interface is requested to generate an order
  • a stacking prompt is displayed on the operation interface, prompting the customer to move the product, so that the camera can capture the goods stacked on the lower layer; the camera is re-imaged. Shooting the goods on the stage, obtaining a new product picture... until the differential data is less than or equal to the preset threshold, requesting to generate an order from the order processing interface;
  • the order processing interface receives the generated order request, issues a payment QR code string, and generates a payment two-dimensional code on the operation interface;
  • the customer scans the payment QR code
  • the message SOCKET sends a message of successful payment to decode the goods on the stage;
  • the message SOCKET sends a face recognition message to the secure channel
  • the customer carries the goods through a secure passage including a detecting device. If no undecoded label is detected, the door is opened and the customer walks out of the unattended convenience store; if an undecoded tag is detected, an unpaid warning is issued and the door is not opened.
  • the anti-theft code of the product is decoded, and the step of verifying whether the payment of the product security code is successful is included: whether the product anti-theft code is not decoded after the successful payment.
  • the result of the verification step is that all the decoding is successful, when the undecoded anti-theft code is detected when the product goes out, the type and quantity of the current product are identified by the surveillance camera, and compared with the type and quantity of the successfully paid product, the difference is partially Show and show unpaid.
  • the customer carries the goods through a secure passage including a detecting device. If no undecoded label is detected, the door is opened and the customer walks out of the unattended convenience store; if an undecoded tag is detected, an unpaid warning is issued and the door is not opened.

Abstract

一种商品自动结算方法、装置、自助收银台，该方法包括：获得含有商品的图像，商品设有防盗码；将含有商品的图像输入基于神经网络的识别系统，基于神经网络的识别系统输出商品信息；由输出的商品信息生成支付信息；支付成功后对商品的防盗码解码。充分利用神经网络对商品进行识别，并对所得多幅图像所得商品信息进行判断，避免了现有图像识别领域过度依赖图像识别，导致的识别误差率，提高了识别准确性。同时整个结算过程可无收银员服务的情况下完成，降低了经营成本。

Description

商品自动结算方法、装置、自助收银台 技术领域
本发明涉及一种商品自动结算方法、装置、自助收银台，属于图像识别技术领域。
背景技术
为提高社区的生活便利度，在很多社区周边建设有24小时营业的超市，上述的营业超市多为连锁超市，在实际运营时，需要派驻2至3个人来运营上述类型的超市，上述的作业人员中，至少需要一名人员用来收银，另外的几名作业人员用来理货或者其他作业，并且一个社区超市在实际运营时，还需要派驻另外几名作业人员用来夜班作业。
一方面需要较多的人工来管理及收银，造成经营成本较高，而且通过人工收银的方式，计算准确率较低，常会产生货款对不上的问题。
发明内容
为解决上述技术问题，本发明提供一种商品自动结算方法，该方法通过神经网络对商品图像进行准确识别，不需要借助任何第三方标识识别商品，用户只要将选购商品放在桌面即可实现识别，整个结算过程可以在无人服务的情况下，由顾客自助完成，能有效降低经营成本。
商品自动结算方法，包括：
获得含有商品的图像，商品设有防盗码；
将含有商品的图像输入基于神经网络的识别系统，基于神经网络的识别系统输出商品信息；
由输出的商品信息生成支付信息；
支付成功后对商品的防盗码解码；
获得含有商品的图像至少包括不同角度和/或不同景深的第一图像至第N图像；
基于神经网的络识别系统包括基于区域卷积神经网络的第一神经网络；商品自动结算方法包括步骤：
(a1)将第一图像输入第一神经网络，第一神经网络输出第一商品信息；将第N图像输入第一神经网络，第一神经网络输出第N商品信息；
(b1)判断第N商品信息是否包含在第一商品信息中；
如判断结果为是，则将第一商品信息作为商品信息输出；
如判断结果为否，则输出反馈提示。
可选地，在支付成功前持续更新获得含有商品的图像；
获得含有商品的图像至生成支付信息的时间不超过0.5秒。
可选地，支付成功后至对商品的防盗码解码的时间间隔不超过0.3秒；
对商品的防盗码解码的过程的持续时间不超过0.3秒。
可选地，获得含有商品的图像至少为二维图像。
可选地，N≥2。
可选地，获得含有商品的图像至少包括不同角度和/或不同景深的第一图像至第N图像；N＝2～4。
可选地，步骤(a1)中还包括称量待检测商品重量的步骤，得到实际称量的商品总重量；
步骤(b1)为：(b2)计算第一商品信息中商品总重量，与实际称量的商品总重量对比得到差分数据，判断差分数据是否小于等于预设阈值：
如判断结果为是，则将第一商品信息作为商品信息输出；
如判断结果为否，则输出所述反馈提示。
可选地，基于神经网络的识别系统包括基于区域卷积神经网络的第一神经网络；商品自动结 算方法包括步骤：
(a3)将第一图像输入第一神经网络，第一神经网络输出第一商品信息；将第N图像输入第一神经网络，第一神经网络输出第N商品信息；
(b3)判断第N商品信息是否包含在第一商品信息中；
如判断结果为是，则将第一商品信息作为商品信息输出；
如判断结果为否，则执行后续步骤；
(c3)计算第一商品信息中商品总重量，与实际称量的商品总重量对比得到差分数据，判断差分数据是否小于等于预设阈值：
如判断结果为是，则将第一商品信息作为商品信息输出；
如判断结果为否，则输出反馈提示。
可选地，步骤(b1)和步骤(b3)中判断第N商品信息是否包含在第一商品信息中的方法为，判断第N商品信息中的商品种类是否均存在于第一商品信息中。
可选地，步骤(b1)和步骤(b3)中判断第N商品信息是否包含在第一商品信息中的方法为，判断第N商品信息中的商品数量是否小于等于第一商品信息中的商品数量。
可选地，步骤(b1)和步骤(b3)中判断第N商品信息是否包含在第一商品信息中的方法为，判断第N商品信息中的每种商品的数量是否小于等于第一商品信息中的商品数量。
可选地，步骤(b1)和步骤(b3)为判断第N商品信息是否与第一商品信息一致；
如判断结果为是，则将第一商品信息作为商品信息输出；
如判断结果为否，则执行后续步骤。
可选地，步骤(b1)和步骤(b3)中第N商品信息是否与第一商品信息一致，包括商品种类一致和每种商品的数量一致。
可选地，步骤(b2)和步骤(c3)中预设阈值为0.1g至10kg中的至少一个数值。
可选地，步骤(b2)和步骤(c3)中预设阈值为第一商品信息中重量最小的商品重量。
可选地，步骤(b2)和步骤(c3)中预设阈值为第一商品信息中重量最小的商品重量的10％至80％中的至少一个数值。
可选地，步骤(b2)和步骤(c3)中反馈提示包括堆叠提示、错误报告中的至少一种。
可选地，含有商品的图像中商品的数量≥1。
可选地，含有商品的图像中商品的数量为1～1000。
可选地，含有商品的图像中商品的种类≥1。
可选地，商品的种类为1～1000。
可选地，基于神经网络的识别系统包括基于区域卷积神经网络的第二神经网络，基于神经网络的识别系统由包括以下步骤的方法得到：
获得每件商品多角度图像的第一图像集；
使用第一图像集训练第二神经网络，得到第一神经网络。
可选地，训练第二神经网络的方法为监督学习方法。
可选地，训练第二神经网络的方法为：
采用监督学习，使用第一图像集训练第二神经网络，得到第三神经网络；
获得商品图像的第二图像集；
用第二图像集训练第三神经网络，得到第一神经网络。
可选地，第二图像集包括经基于神经网络的识别系统输出商品信息的商品的图像。
可选地，第二神经网络对商品的识别准确率为80％以上。
可选地，用第二图像集训练第三神经网络的过程未无监督学习。
可选地，商品自动结算方法，包括步骤：
(a1)将第一图像输入第一神经网络，第一神经网络输出第一商品信息；将第N图像输入 第一神经网络，第一神经网络输出第N商品信息；
(b1)判断第N商品信息是否包含在第一商品信息中；
(c1)步骤(b1)中判断结果为否时，识别第一商品信息与第N商品信息中差异商品；
(d1)获取步骤(c1)中的差异商品的差异图像集，用差异图像集强化训练第一神经网络。
可选地，商品自动结算方法，包括步骤：
(a3)将第一图像输入第一神经网络，第一神经网络输出第一商品信息；将第N图像输入第一神经网络，第一神经网络输出第N商品信息；
(b3)判断第N商品信息是否包含在第一商品信息中；
如判断结果为是，则将第一商品信息作为商品信息输出；
如判断结果为否，则执行后续步骤；
(c3)计算第一商品信息中商品总重量，与实际称量的商品总重量对比得到差分数据，判断差分数据是否小于等于预设阈值：
(d3)步骤(c3)中判断结果为否时，识别第一商品信息与第N商品信息中差异商品；
(e3)获取步骤(d3)中的差异商品的差异图像集，用差异图像集强化训练第一神经网络。
可选地，商品自动结算方法，包括步骤：
(a2)将第一图像输入第一神经网络，第一神经网络输出第一商品信息；
(b2)计算第一商品信息中商品总重量，与实际称量的商品总重量对比得到差分数据，判断差分数据是否小于等于预设阈值：
(c2)收集步骤(b2)中判断结果为否时，识别第一商品信息中的商品；
(d2)获取步骤(c2)中的识别商品的收集图像集，用收集图像集强化训练第一神经网络。
可选地，商品自动结算方法，包括步骤：
(a3)将第一图像输入第一神经网络，第一神经网络输出第一商品信息；将第N图像输入第一神经网络，第一神经网络输出第N商品信息；
(b3)判断第N商品信息是否包含在第一商品信息中；
如判断结果为是，则将第一商品信息作为商品信息输出；
如判断结果为否，则执行后续步骤；
(c3)计算第一商品信息中商品总重量，与实际称量的商品总重量对比得到差分数据，判断差分数据是否小于等于预设阈值：
(d3)收集步骤(c3)中判断结果为否时，识别第一商品信息中的商品；
(e3)获取步骤(d3)中的识别商品的收集图像集，用收集图像集强化训练第一神经网络。
可选地，持续更新获得含有商品的图像的更新频率不小于10次/秒。
可选地，持续更新获得含有商品的图像的更新频率为10次/秒至100次/秒。
可选地，获得含有商品的图像至生成支付信息的时间不超过0.1秒。
可选地，获得含有商品的图像至生成支付信息的时间为0.001秒至0.1秒。
可选地，支付成功后至对商品的防盗码解码的时间间隔不超过0.1秒。
可选地，支付成功后至对商品的防盗码解码的时间间隔为0.001秒至0.1秒。
可选地，对商品的防盗码解码的过程的持续时间不超过0.1秒。
可选地，对商品的防盗码解码的过程的持续时间为0.001秒至0.1秒。
可选地，所述解码步骤后，还包括验证步骤：判断支付成功商品的防盗码是否未解码；
如判断结果为是，则判断商品种类和数量是否有变化，如判断结果为无变化，则再次对所述商品的防盗码解码；
如判断结果为已全部解码成功，则在商品出门时检测到未解码防盗码时，执行报警指令。
可选地，支付成功后对商品的防盗码解码后，还包括对支付成功的商品防盗码是否解码进行验证的步骤：成功支付后是否有商品防盗码未解码；
如是，则判断商品种类和数量是否有变化，如无变化，则再次对商品防盗码解码。
可选地，如验证步骤的结果是已全部解码成功，则在商品出门时检测到未解码防盗码时，执行报警指令。
可选地，所述检测到未解码防盗码步骤后，通过监控摄像头识别当前商品的种类和数量，并与支付成功的商品种类和数量进行对比，并显示未支付商品的种类和数量。
可选地，如验证步骤的结果是已全部解码成功，则在商品出门时检测到未解码防盗码时，通过监控摄像头识别当前商品的种类和数量，并与支付成功的商品种类和数量对比，将差异部分商品展示并显示未支付。
根据本发明的又一方面，提供一种商品自动结算装置，包括：
摄像单元，用于获取商品至少包括角度和/或景深不同的第一图像至第N图像，商品设有防盗码；
识别信息单元，用于将第一图像输入第一神经网络，第一神经网络输出第一商品信息；将第N图像输入第一神经网络，第一神经网络输出第N商品信息；
识别判断单元，用于判断第N商品信息是否包含在第一商品信息中；如判断结果为是，则将第一商品信息作为待检测商品信息输出；如判断结果为否，则输出反馈提示；
显示单元，用于输出商品信息并生成显示支付信息，判断支付是否成功后发出解码指令，或显示反馈提示；
解码单元，用于对防盗码进行解码；
摄像单元与识别信息单元相连接，识别信息单元与识别判断单元相连接，识别判断单元与显示单元数据连接，解码单元与显示单元控制连接。
可选地，识别信息单元和识别判断单元，用于按前述商品自动结算方法进行商品识别和判断。
可选地，商品自动结算装置还包括验证单元，用于判断支付成功商品的防盗码是否未解码；
如判断结果为是，则判断商品种类和数量是否有变化，如判断结果为无变化，则再次对商品的防盗码解码；
如判断结果为已全部解码成功，则在商品出门时检测到未解码防盗码时，执行报警指令。
可选地，验证单元还包括监控摄像头和核对单元，核对单元用于通过监控摄像头识别当前商品的种类和数量，并与支付成功的商品种类和数量进行对比，并显示未支付商品的种类和数量，监控摄像头与核对单元相连接，核对单元与显示单元相连接。
可选地，摄像单元包括至少两个获取不同角度和/或景深的商品图像的摄像头。
可选地，摄像单元包括至少2至4个获取不同角度和/或景深的商品图像的摄像头。
可选地，摄像单元包括第一摄像头和第二摄像头；
第一摄像头和第二摄像头分别从不同角度获取商品图像。
可选地，包括载物台，载物台含有重量传感器，用于测量载物台上商品的总重量；重量传感器与识别信息单元数据连接。
根据本发明的又一方面，提供一种自助收银台，自助收银台采用上述任一商品自动结算方法进行商品识别。
根据本发明的又一方面，提供一种自助收银台，自助收银台采用上述任一商品自动结算装置。
本发明的有益效果包括但不限于：
(1)本发明所提供的商品自动结算方法，充分利用神经网络对商品进行识别，并对所得多幅图像所得商品信息进行判断，避免了现有图像识别领域过度依赖图像识别，导致的识别误差率，提高了识别准确性。同时整个结算过程可无收银员服务的情况下完成，降低了经营成本。
(2)本发明所提供的商品自动结算方法，通过深度学习的可持续性学习，随着使用频率的增加不断提高该方法的识别准确性。防盗码的设置还能起到防盗的效果。
(3)本发明所提供的商品自动结算方法，通过普通摄像头抓取商品画面，可实现批量商品的快速检测，大幅降低了商品识别的成本和速度。
(4)本发明所提供的商品自动结算方法，可实现自助结算场景下，低成本、高效率的完成商品识别和结算。
(5)本发明所提供的商品自动结算装置，通过神经网络识别和多图像比对，实现对识别结果的校正，并利用识别结果，通过深度学习系统通过所得图像集对神经网络系统进行训练，不断提高其识别准确性，实现高效，准确的自助结算。
附图说明
图1是本发明第一优选实施例中商品自动结算方法流程示意框图；
图2是本发明第二优选实施例中商品自动结算方法流程示意框图；
图3是本发明第三优选实施例中商品自动结算方法流程示意框图；
图4是本发明第四优选实施例中商品自动结算方法流程示意框图；
图5是本发明第五优选实施例中商品自动结算方法流程示意框图；
图6是本发明第六优选实施例中商品自动结算方法流程示意框图；
图7是本发明一种实施方式中商品自动结算方法中的解码流程图；
图8是本发明提供的商品自动结算装置结构示意框图；
图9是本发明提供商品自动结算方法应用于自助收银台的无人便利店的时序示意图。
具体实施方式
下面结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整的描述，显然，所描述的实施例是本发明的一部分实施例，而不是全部实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动的前提下所获得的所有其他实施例，都应属于本发明保护的范围。
本发明中，防盗码可以为各类具有防盗效果的条形码或AM标签。解码步骤可以为消磁。
参见图1，本发明提供的商品自动结算方法，包括：
获得含有商品的图像，商品设有防盗码；
将含有商品的图像输入基于神经网络的识别系统，基于神经网络的识别系统输出商品信息；
由输出的商品信息生成支付信息；
支付成功后对商品的防盗码解码；
获得含有商品的图像至少包括不同角度和/或不同景深的第一图像至第N图像；
基于神经网络的识别系统包括基于区域卷积神经网络的第一神经网络；商品自动结算方法包括步骤：
(a1)将第一图像输入第一神经网络，第一神经网络输出第一商品信息；将第N图像输入第一神经网络，第一神经网络输出第N商品信息；
(b1)判断第N商品信息是否包含在第一商品信息中；
如判断结果为是，则将第一商品信息作为商品信息输出；
如判断结果为否，则输出反馈提示。
本发明提供的商品自动结算方法主要用于无人值守环境下，自助获取结算商品信息后进行自助购物。该方法充分利用神经网络对商品进行识别，并对所得多幅图像所得商品信息进行判断，避免了现有图像识别领域过度依赖图像识别，导致的识别误差率，提高了识别准确性。当无法获取准确的商品信息是，可以通过反馈提示提醒用户，从而仅需通过调整待识别商品即可纠正识别错误，无需反复扫码或多次尝试。获取商品信息后，生成各类支付信息，例如二维码等支付信息，也可以为商品信息列表，以便于顾客核查商品信息并进行支付。支付后对商品进行批量解码后，即可获得商品。如果未支付成功，则不进入解码步骤，并返回执行显示支付信息步骤。
商品上设置防盗码能高效防盗实现自助结算，可运用于多种场景下，如食堂、无人值守各类店铺结算时均可使用。无需人工值守，降低了人工使用量，降低运营成本。
此处的反馈提示包括堆叠提示、错误报告中的至少一种。该方法可以用于处理的商品种类和数量不限，例如可以为含有待检测商品的图像中待检测商品的数量≥1。含有待检测商品的图像中待检测商品的数量为1～1000。含有待检测商品的图像中待检测商品的种类≥1。待检测商品的种类为1～1000。判断的商品信息包括商品种类或每种商品的数量。判断商品种类和/或商品数量是否一致。本发明提供的商品自动结算方法，用于无人值守环境下的结算时，仅需使用普通具有网络联网功能的摄像头即可实现对商品的准确识别。
优选的，第一图像为待检测商品的正面图像。以此作为主要图像进行识别，能提高识别的准确率。
优选的，获得含有待检测商品的图像至少包括不同角度和/或不同景深的第一图像至第N图像；N＝2～4。通过获取多角度图像，能提高神经网络的识别准确性。有利于提高后续识别结果的准确性。
参见图2，优选的，步骤(a1)中还包括称量待检测商品重量的步骤，得到实际称量的商品总重量；步骤(b1)为(b2)计算第一商品信息中商品总重量，与实际称量的商品总重量对比得到差分数据，判断差分数据是否小于等于预设阈值：如判断结果为是，则将第一商品信息作为待检测商品信息输出；如判断结果为否，则输出反馈提示。同时对于所获取的商品信息，还可通过对商品信息中所包含的商品重量进行分析，对所得结果进行校正，从而提高图像识别结果的准确性。
参见图3，优选的，基于神经网络的识别系统包括基于区域卷积神经网络的第一神经网络；商品自动结算方法包括步骤：(a3)将第一图像输入第一神经网络，第一神经网络输出第一商品信息；将第N图像输入第一神经网络，第一神经网络输出第N商品信息；(b3)判断第N商品信息是否包含在第一商品信息中；如判断结果为是，则将第一商品信息作为待检测商品信息输出；如判断结果为否，则执行后续步骤；(c3)计算第一商品信息中商品总重量，与实际称量的商品总重量对比得到差分数据，判断差分数据是否小于等于预设阈值：如判断结果为是，则将第一商品信息作为待检测商品信息输出；如判断结果为否，则输出反馈提示。
通过连用种类和商品信息作为校正参数，能更好对所得结果进行纠正，提高商品的识别准确度。此处的预设阈值可以为0.1g至10kg中的至少一个数值。预设阈值还可以为第一商品信息中重量最小的商品重量。预设阈值还可以为第一商品信息中重量最小的商品重量的10％至80％中的至少一个数值。
优选的，步骤(b1)和步骤(b3)中判断第N商品信息是否包含在第一商品信息中的方法为，判断第N商品信息中的商品种类是否均存在于第一商品信息中。
优选的，步骤(b1)和步骤(b3)中判断第N商品信息是否包含在第一商品信息中的方法为，判断第N商品信息中的每种商品的数量是否小于等于第一商品信息中的商品数量。
优选的，步骤(b1)和步骤(b3)为判断第N商品信息是否与第一商品信息一致；如判断结果为是，则将第一商品信息作为待检测商品信息输出；如判断结果为否，则执行后续步骤。
优选的，步骤(b1)和步骤(b3)中第N商品信息是否与第一商品信息一致，包括商品种类一致和每种商品的数量一致。
优选的，基于神经网络的识别系统包括基于区域卷积神经网络的第二神经网络，基于神经网络的识别系统由包括以下步骤的方法得到：获得每件待检测商品多角度图像的第一图像集；使用第一图像集训练第二神经网络，得到第一神经网络。通过使用第二神经网络，能将所得结果用于训练第一神经网络，从而实现深度学习化的自动系统纠错，随着识别商品数量的提高，该神经识别系统的识别准确性自动升高。按现有方法进行即可。以待检测商品的多角度图像进行训练，能提高基于神经网络的识别系统在应对商品被遮挡时的识别准确性。
优选的，训练第二神经网络的方法为监督学习方法。
优选的，训练第二神经网络的方法为：采用监督学习，使用第一图像集训练第二神经网络，得到第三神经网络；获得待检测商品图像的第二图像集；用第二图像集训练第三神经网络，得到第一神经网络。
优选的，第二图像集包括经基于神经网络的识别系统输出待检测商品信息的待检测商品的图像。
优选的，第二神经网络对待检测商品的识别准确率为80％以上。优选的，第二图像集训练第三神经网络的过程为无监督学习。按现有方法进行即可。
参见图4，优选的，商品自动结算方法，包括步骤：
(c1)或(d3)步骤(b1)或(c3)中判断结果为否时，识别第一商品信息与第N商品信息中差异商品；
(d1)或(e3)获取步骤(c1)或(d3)中的差异商品的差异图像集，用差异图像集强化训练第一神经网络。
通过收集判断结果为否时，第N商品信息中存在的差异商品并获取其图像集，通过以该差异图像集训练第一神经网络，能进一步提高该系统的纠错能力。同时该操作也可以用于如图3所示的方法中。
参见图5，优选的，商品自动结算方法，包括步骤：
(c2)或(d3)收集步骤(b2)或(c3)中判断结果为否时，识别第一商品信息中的商品；
(d2)或(e3)获取步骤(c2)或(d3)中的识别商品的收集图像集，用收集图像集强化训练第一神经网络。
该步骤也可以用于如图3所示的方法，在此不累述。当检测结果为否时，通过对多次无法识别情况下的第一商品信息进行收集，并将其用于训练第一神经网络，从而提高第一神经网络对无法识别情况的识别能力。
优选的，解码步骤后，还包括验证步骤：判断支付成功商品的防盗码是否未解码；如判断结果为是，则判断商品种类和数量是否有变化，如判断结果为无变化，则再次对商品的防盗码解码；如判断结果为已全部解码成功，则在商品出门时检测到未解码防盗码时，执行报警指令。
为了防止大批量购买时，解码不完全的问题出现，还包括对解码商品进行验证的步骤，经过验证后，如果发现未解码商品，再次核对商品信息与待结算商品的种类和数量是否有差异，以防顾客将未结算商品进行解码。经过再次全部解码，从而将已支付而未解码的商品进行解码，减少由于未解码给顾客带来的操作不便，提高结算效率。同时对于未结算的情况下，由于防盗码未解码，顾客无法将商品带离商店。并存在触发报警的危险，能较好的避免商品被盗问题的出现。
优选的，检测到未解码防盗码步骤后，通过监控摄像头识别当前商品的种类和数量，并与支付成功的商品种类和数量进行对比，并显示未支付商品的种类和数量。
此处的监控摄像头，可以为设置于商店各个角落用于防盗的摄像头，也可以为结算台上用于获取监控图像的摄像头。通过对比监控图像中商品信息与已支付商品信息，可以有效防止顾客将未结账商品夹带带出的问题。起到一定程度的防盗作用。
参见图7，本发明提供的商品自助结算方法，包括以下步骤：顾客进入无人值守商店，挑选货物后放置于结算装置中，获取商品图像并对商品信息进行识别，识别获取方法按前述，在此不累述。根据商品信息生成支付信息，顾客据此进行支付，之后判断是否支付成功，如果支付成功，则进行解码操作，如果没有支付成功，继续向顾客显示支付信息。解码后，核对是否消磁，此时可以通过再次提示解码结果，有顾客自行检查。如果未全部解码则返回再次解码。如果已经全部解码则此时顾客可准备出门。出门时，由防盗报警装置获取防盗码的信号，判断是否存在未解码的产品，如果已经解码完成，则购物过程结束。如果还有未解码的产品，则报警并关闭大门。以防顾客将未结算商品带出。
如无特殊说明，本发明所用元器件、仪器设备均来自商业购买，未经改装直接使用，所用仪器设备采用厂家推荐的方案和参数。
参见图6，本发明提供的商品自动结算方法，使用时，待检测商品放置于载物台上，N个摄像头围绕待检测商品环绕设置。通过N个摄像头获取待检测商品各个角度的图像，分别记为P1、P2.....PN。N个摄像头中，位于载物台正上方的摄像头为主摄像头，记为第一摄像头，该摄像头所获取的图像即为第一图像P1。
将P1、P2......PN上传到本地识别服务器或云端识别服务器，对各张图片进行识别，识别出的商品信息分别记为R1、R2....RN，商品信息中包括商品的类别信息和数量信息，其中，主摄像头的识别结果R1为第一商品信息，其他摄像头的识别结果R2......RN分别为第二商品信息......第N商品信息；
以两个摄像头为例，判断R2(第二商品信息)是否包含在R1(第一商品信息)中；
如果判断结果为是，则将R1作为待检测商品信息输出；
如判断结果为否，则计算R1中商品的总重量，其与实际称量的商品总重量相减所得结果的绝对值作为差分数据，判断差分数据是否小于等于预设阈值：
如判断结果为是，则将R1作为待检测商品信息输出，输出商品信息包含商品的类别、数量和价格的商品信息列表；
如判断结果为否，则显示堆叠提示或错误报告信息。
参见图8，本发明的另一方面还提供了一种商品自动结算装置，包括：
摄像单元100，用于获取商品至少包括角度和/或景深不同的第一图像至第N图像，商品设有防盗码；
识别信息单元210，用于将第一图像输入第一神经网络，第一神经网络输出第一商品信息；将第N图像输入第一神经网络，第一神经网络输出第N商品信息；
识别判断单元220，用于判断第N商品信息是否包含在第一商品信息中；如判断结果为是，则将第一商品信息作为待检测商品信息输出；如判断结果为否，则输出反馈提示；
显示单元300，用于输出商品信息并生成显示支付信息，判断支付是否成功后发出解码指令，或显示反馈提示；
解码单元400，用于对防盗码进行解码；
摄像单元100与识别信息单元相连接，识别信息单元与识别判断单元220相连接，识别判断单元220与显示单元300数据连接，解码单元400与显示单元300控制连接。
本发明提供的自助结算装置通过摄像单元100获取图像后，经过神经网络识别，能较好的得到商品的各项信息，并与商品上是防盗码配合，根据商品信息生成支付信息后，支付完成自主解码即可将商品带离，整个结算过程无需服务人员提供帮助即可完成，识别准确性高。商店运营成本低。
优选的，识别信息单元和识别判断单元220，用于按前述商品自动结算方法进行商品识别和判断。
优选的，商品自动结算装置还包括验证单元，用于判断支付成功商品的防盗码是否未解码；
如判断结果为是，则判断商品种类和数量是否有变化，如判断结果为无变化，则再次对商品的防盗码解码；
如判断结果为已全部解码成功，则在商品出门时检测到未解码防盗码时，执行报警指令。
优选的，验证单元还包括监控摄像头和核对单元，核对单元用于通过监控摄像头识别当前商品的种类和数量，并与支付成功的商品种类和数量进行对比，并显示未支付商品的种类和数量，监控摄像头与核对单元相连接，核对单元与显示单元300相连接。
可选地，摄像单元100包括两个普通网络摄像头，两个可调整任意角度的固定器，一台可运行图片上传的持续的计算机，一个高精度重量传感器。主要工作流程为：计算机上运行一个图像 抓取程序，该程序可以将同一时间的两个摄像头抓取的画面图像上传到远程服务器，远程服务器将识别结果返回。此方案成本极低，工作计算机也只需要最基础配置即可。
可选地，摄像单元100包括2-4个固定镜头高清摄像头，相应数量的可调节角度固定器，一个高精度重量传感器，一台带显存2G以上显卡的计算机。主要工作流程为，算机上运行一个图像抓取程序，该程序可以将同一时间的两个摄像头抓取的画面图像在本地识别。
可选地，商品自动结算装置可批量检测(低成本方案)，采用多个普通摄像头，从不同角度获得待检测商品的图像。
多个不同角度的摄像头可以解决商品在同一个2D图片中因为摆放角度和物品高度差异产生的遮挡问题。基本上3个摄像头可以实现无死角获取待识别所需信息，合适的摄像头机位情况下，2个摄像头也可以达到较理想效果。
优选的，可选地，摄像单元100包括第一摄像头和第二摄像头；
第一摄像头和第二摄像头分别从不同角度获取商品图像。
可选地，商品自动结算装置包括载物台，载物台含有重量传感器，用于测量载物台上商品总重量；
重量传感器与识别单元电连接，将载物台上商品的总重量输入识别单元。
在商品图像识别过程中，待结算商品常因堆叠或极端拍摄角度，导致物体被遮挡或大部分被遮挡，而无法得到足够的细节用准确识别商品。为了准确判断商品内有无堆叠情况，本发明结合重量传感器对图像识别结果进行校正，获取识别结果中的物品重量与识别装置内的重量传感器实际称重，如果不一致，则反馈商品处于堆叠状态。
本发明的又一方面还提供了一种自助收银台，自助收银台采用前述的商品自动结算方法进行商品识别。此处的自助既可以是无人值守状态，也可以为在防损员的监督下使用。仅需作到顾客进行结算操作即可。通过采用前述商品自动结算方法，顾客能高效、准确的完成计算过程，整个设备成本较低，无需使用电子标签。
本发明的又一方面还提供了一种自助收银台，自助收银台采用前述的商品自动结算装置。此处的自助既可以是无人值守状态，也可以为在防损员的监督下使用。仅需作到顾客进行结算操作即可。
图9示出了本发明通过的商品自动结算装置用于自助收银台的无人便利店的一种实施方式的时序示意图。也可以作为本发明提供的自助收银台的实施实例。如图9所示，使用了包含任一前述商品自动结算方法的商品自动结算装置，顾客在无人便利店中的购物步骤如下：
顾客选择完商品后，将所有商品放置于自助收银台(也是商品自动结算装置中的载物台)上；
载物台感应到重量>0，触发商品自动结算装置启动商品识别程序；
摄像头拍摄载物台上的商品，获得商品图片，并将商品图片Base64编码POST到图像识别服务器，进行图像识别；
图像识别的结果(包括所有商品品名、价格、总重量)的信息与载物台实际称量得到的总重量比对，得到差分数据；
当差分数据小于等于预设阈值时，判断为[实际称重与范围重量一致]，则向订单处理接口请求生成订单；
当差分数据大于预设阈值时，判断为[实际称重与范围重量不一致]，则在操作界面显示堆叠提示，提示顾客挪动商品，使摄像头可拍摄到堆叠在下层被遮挡住的商品；摄像头重新拍摄载物台上的商品，获得新的商品图片...直至差分数据小于等于预设阈值，向订单处理接口请求生成订单；
订单处理接口收到生成订单请求，发出支付二维码字符串，在操作界面生成支付二维码；
顾客扫描支付二维码；
支付成功后，消息SOCKET发送支付成功的消息，对载物台上的商品进行解码；
消息SOCKET向安全通道发送人脸识别消息；
顾客携带商品通过包括检测装置的安全通道，如未检测到未解码标签，大门开启，顾客走出无人购物便利店；如检测到未解码标签，则发出未支付警告，大门不开启。
支付成功后对商品的防盗码解码后，还包括对支付成功的商品防盗码是否解码进行验证的步骤：成功支付后是否有商品防盗码未解码。
如是，则判断商品种类和数量是否有变化，如无变化，则再次对商品防盗码解码。
如验证步骤的结果是已全部解码成功，则在商品出门时检测到未解码防盗码时，执行报警指令。
如验证步骤的结果是已全部解码成功，则在商品出门时检测到未解码防盗码时，通过监控摄像头识别当前商品的种类和数量，并与支付成功的商品种类和数量对比，将差异部分商品展示并显示未支付。
顾客携带商品通过包括检测装置的安全通道，如未检测到未解码标签，大门开启，顾客走出无人购物便利店；如检测到未解码标签，则发出未支付警告，大门不开启。
以上，仅是本发明的几个实施例，并非对本发明做任何形式的限制，虽然本发明以较佳实施例揭示如上，然而并非用以限制本发明，任何熟悉本专业的技术人员，在不脱离本发明技术方案的范围内，利用上述揭示的技术内容做出些许的变动或修饰均等同于等效实施案例，均属于技术方案范围内。

Claims (29)

  1. 一种商品自动结算方法，其特征在于，包括：
    获得含有商品的图像，所述商品设有防盗码；
    将所述含有商品的图像输入基于神经网络的识别系统，所述基于神经网络的识别系统输出商品信息；
    由输出的商品信息生成支付信息；
    支付成功后对所述商品的防盗码解码；
    所述获得含有商品的图像至少包括不同角度和/或不同景深的第一图像至第N图像；
    所述基于神经网络的识别系统包括基于区域卷积神经网络的第一神经网络；所述商品自动结算方法包括步骤：
    (a1)将所述第一图像输入所述第一神经网络，所述第一神经网络输出第一商品信息；将所述第N图像输入所述第一神经网络，所述第一神经网络输出第N商品信息；
    (b1)判断所述第N商品信息是否包含在所述第一商品信息中；
    如判断结果为是，则将所述第一商品信息作为所述商品信息输出；
    如判断结果为否，则输出反馈提示。
  2. 根据权利要求1所述的商品自动结算方法，其特征在于，所述步骤(a1)中还包括称量所述待检测商品重量的步骤，得到实际称量的商品总重量；
    所述步骤(b1)为：(b2)计算所述第一商品信息中商品总重量，与实际称量的商品总重量对比得到差分数据，判断所述差分数据是否小于等于预设阈值：
    如判断结果为是，则将所述第一商品信息作为所述商品信息输出；
    如判断结果为否，则输出所述反馈提示。
  3. 根据权利要求1所述的商品自动结算方法，其特征在于，所述基于神经网络的识别系统包括基于区域卷积神经网络的第一神经网络；所述商品自动结算方法包括步骤：
    (a3)将所述第一图像输入所述第一神经网络，所述第一神经网络输出第一商品信息；将所述第N图像输入所述第一神经网络，所述第一神经网络输出第N商品信息；
    (b3)判断所述第N商品信息是否包含在所述第一商品信息中；
    如判断结果为是，则将所述第一商品信息作为所述商品信息输出；
    如判断结果为否，则执行后续步骤；
    (c3)计算所述第一商品信息中商品总重量，与实际称量的商品总重量对比得到差分数据，判断所述差分数据是否小于等于预设阈值：
    如判断结果为是，则将所述第一商品信息作为所述商品信息输出；
    如判断结果为否，则输出反馈提示。
  4. 根据权利要求1或3所述的商品自动结算方法，其特征在于，所述步骤(b1)和所述步骤(b3)中判断所述第N商品信息是否包含在所述第一商品信息中的方法为，判断所述第N商品信息中的商品种类是否均存在于所述第一商品信息中。
  5. 根据权利要求1或3所述的商品自动结算方法，其特征在于，所述步骤(b1)和所述步骤(b3)中判断所述第N商品信息是否包含在所述第一商品信息中的方法为，判断所述第N商品信息中的商品数量是否小于等于所述第一商品信息中的商品数量。
  6. 根据权利要求1或3所述的商品自动结算方法，其特征在于，所述步骤(b1)和所述步骤(b3)中判断所述第N商品信息是否包含在所述第一商品信息中的方法为，判断所述第N商 品信息中的每种商品的数量是否小于等于所述第一商品信息中的商品数量。
  7. 根据权利要求1或3所述的商品自动结算方法，其特征在于，所述步骤(b1)和所述步骤(b3)为判断所述第N商品信息是否与所述第一商品信息一致；
    如判断结果为是，则将所述第一商品信息作为所述商品信息输出；
    如判断结果为否，则执行后续步骤。
  8. 根据权利要求7所述的商品自动结算方法，其特征在于，所述步骤(b1)和步骤(b3)中所述第N商品信息是否与所述第一商品信息一致，包括商品种类一致和每种商品的数量一致。
  9. 根据权利要求2或3所述的商品自动结算方法，其特征在于，所述步骤(b2)和步骤(c3)中预设阈值为0.1g至10kg中的至少一个数值。
  10. 根据权利要求2或3所述的商品自动结算方法，其特征在于，所述步骤(b2)和步骤(c3)中预设阈值为第一商品信息中重量最小的商品重量。
  11. 根据权利要求2或3所述的商品自动结算方法，其特征在于，所述步骤(b2)和步骤(c3)中预设阈值为第一商品信息中重量最小的商品重量的10％至80％中的至少一个数值。
  12. 根据权利要求1所述的商品自动结算方法，其特征在于，基于神经网络的识别系统包括基于区域卷积神经网络的第二神经网络，所述基于神经网络的识别系统由包括以下步骤的方法得到：
    获得每件所述商品的多角度图像的第一图像集；
    使用所述第一图像集训练所述第二神经网络，得到第一神经网络。
  13. 根据权利要求12所述的商品自动结算方法，其特征在于，所述训练所述第二神经网络的方法为监督学习方法。
  14. 根据权利要求12所述的商品自动结算方法，其特征在于，所述训练所述第二神经网络的方法为：
    采用监督学习，使用第一图像集训练所述第二神经网络，得到第三神经网络；
    获得所述商品图像的第二图像集；
    用第二图像集训练所述第三神经网络，得到第一神经网络。
  15. 根据权利要求14所述的商品自动结算方法，其特征在于，所述第二图像集包括经基于神经网络的识别系统输出商品信息的所述商品的图像。
  16. 根据权利要求14所述的商品自动结算方法，其特征在于，所述用第二图像集训练所述第三神经网络的过程未无监督学习。
  17. 根据权利要求1所述的商品自动结算方法，其特征在于，所述商品自动结算方法，包括步骤：
    (c1)所述步骤(b1)中判断结果为否时，识别所述第一商品信息与所述第N商品信息中差异商品；
    (d1)获取步骤(c1)中的所述差异商品的差异图像集，用所述差异图像集强化训练所述第一神经网络。
  18. 根据权利要求3所述的商品自动结算方法，其特征在于，所述商品自动结算方法中：
    (d3)为：所述步骤(c3)中判断结果为否时，识别所述第一商品信息与所述第N商品信息中差异商品；
    (e3)为：获取步骤(d3)中的所述差异商品的差异图像集，用所述差异图像集强化训练所述第一神经网络。
  19. 根据权利要求2所述的商品自动结算方法，其特征在于，所述商品自动结算方法，包括步骤：
    (c2)收集所述步骤(b2)中判断结果为否时，识别所述第一商品信息中的商品；
    (d2)获取步骤(c2)中的所述识别商品的收集图像集，用所述收集图像集强化训练所述第一神经网络。
  20. 根据权利要求3所述的商品自动结算方法，其特征在于，所述商品自动结算方法，包括步骤：
    (d3)收集所述步骤(c3)中判断结果为否时，识别所述第一商品信息中的商品；
    (e3)获取步骤(d3)中的所述识别商品的收集二图像集，用所述收集图像集强化训练所述第一神经网络。
  21. 根据权利要求1所述的商品自动结算方法，其特征在于，所述解码步骤后，还包括验证步骤：判断支付成功商品的防盗码是否未解码；
    如判断结果为是，则判断商品种类和数量是否有变化，如判断结果为无变化，则再次对所述商品的防盗码解码；
    如判断结果为已全部解码成功，则在商品出门时检测到未解码防盗码时，执行报警指令。
  22. 根据权利要求21所述的商品自动结算方法，其特征在于，所述检测到未解码防盗码步骤后，通过监控摄像头识别当前商品的种类和数量，并与支付成功的商品种类和数量进行对比，并显示未支付商品的种类和数量。
  23. 一种商品自动结算装置，其特征在于，包括：
    摄像单元，用于获取商品至少包括角度和/或景深不同的第一图像至第N图像，所述商品设有防盗码；
    识别信息单元，用于将所述第一图像输入所述第一神经网络，所述第一神经网络输出第一商品信息；将所述第N图像输入所述第一神经网络，所述第一神经网络输出第N商品信息；
    识别判断单元，用于判断所述第N商品信息是否包含在所述第一商品信息中；如判断结果为是，则将所述第一商品信息作为所述待检测商品信息输出；如判断结果为否，则输出反馈提示；
    显示单元，用于输出所述商品信息并生成显示支付信息，判断支付是否成功后发出解码指令，或显示所述反馈提示；
    解码单元，用于对所述防盗码进行解码；
    所述摄像单元与所述识别信息单元相连接，所述识别信息单元与所述识别判断单元相连接，所述识别判断单元与所述显示单元数据连接，所述解码单元与所述显示单元控制连接。
  24. 根据权利要求23所述的商品自动结算装置，其特征在于，所述识别信息单元和所述识别判断单元，用于按权利要求1～22中任一项所述商品自动结算方法进行商品识别和判断。
  25. 根据权利要求23所述的商品自动结算装置，其特征在于，所述商品自动结算装置还包 括验证单元，用于判断支付成功商品的防盗码是否未解码；
    如判断结果为是，则判断商品种类和数量是否有变化，如判断结果为无变化，则再次对所述商品的防盗码解码；
    如判断结果为已全部解码成功，则在商品出门时检测到未解码防盗码时，执行报警指令。
  26. 根据权利要求25所述的商品自动结算装置，其特征在于，所述验证单元还包括监控摄像头和核对单元，所述核对单元用于通过所述监控摄像头识别当前商品的种类和数量，并与支付成功的商品种类和数量进行对比，并显示未支付商品的种类和数量，所述监控摄像头与所述核对单元相连接，所述核对单元与所述显示单元相连接。
  27. 根据权利要求23所述的商品自动结算装置，其特征在于，还包括载物台，所述载物台含有重量传感器，用于测量所述载物台上商品的总重量；
    所述重量传感器与所述识别信息单元数据连接。
  28. 一种自助收银台，其特征在于，所述自助收银台采用权利要求1至22任一项所述的商品自动结算方法进行商品识别。
  29. 一种自助收银台，其特征在于，所述自助收银台采用权利要求23至27任一项所述的商品自动结算装置。
PCT/CN2018/077652 2017-09-27 2018-02-28 商品自动结算方法、装置、自助收银台 WO2019062018A1 (zh)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
CN201710889825.7 2017-09-27
CN201710889825 2017-09-27
CN201810155783.9 2018-02-23
CN201810155783.9A CN108269371B (zh) 2017-09-27 2018-02-23 商品自动结算方法、装置、自助收银台

Publications (1)

Publication Number Publication Date
WO2019062018A1 true WO2019062018A1 (zh) 2019-04-04

Family

ID=62774255

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/077652 WO2019062018A1 (zh) 2017-09-27 2018-02-28 商品自动结算方法、装置、自助收银台

Country Status (2)

Country Link
CN (1) CN108269371B (zh)
WO (1) WO2019062018A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11966900B2 (en) * 2019-07-19 2024-04-23 Walmart Apollo, Llc System and method for detecting unpaid items in retail store transactions

Families Citing this family (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110880223B (zh) * 2018-09-06 2022-03-04 阿里巴巴集团控股有限公司 商品结算方法、服务器、电子设备、介质和终端
JP7210221B2 (ja) 2018-10-19 2023-01-23 東芝テック株式会社 商品データ処理システム
JP7300699B2 (ja) * 2018-11-12 2023-06-30 株式会社イシダ 教師データ生成方法、教師データ生成プログラム、教師データ生成装置、及び商品識別装置
CN109508974B (zh) * 2018-11-29 2023-08-22 华南理工大学 一种基于特征融合的购物结账系统和方法
CN109871726A (zh) * 2018-12-28 2019-06-11 南京天创电子技术有限公司 一种基于qr码及图像识别的同类仪表示数预警方法
CN110032654B (zh) * 2019-04-19 2021-02-09 浪潮集团有限公司 一种基于人工智能的超市商品录入方法及系统
CN112735024A (zh) * 2019-10-14 2021-04-30 晋城三赢精密电子有限公司 物品防盜方法、装置、计算机装置及存储介质
WO2021081319A1 (en) * 2019-10-25 2021-04-29 7-Eleven, Inc. System and method for populating a virtual shopping cart based on video of a customer's shopping session at a physical store
CN111931882B (zh) * 2020-07-20 2023-07-21 五邑大学 货物自动结账方法、系统及存储介质
CN112801554A (zh) * 2021-04-01 2021-05-14 广州紫麦科技股份有限公司 多模式可选择的商品零售数据系统
CN114220232B (zh) * 2021-11-23 2023-06-27 浙江卡赢信息科技有限公司 用于漏单检测的金融支付系统及方法
CN114267139A (zh) * 2021-12-13 2022-04-01 湖南省金河计算机科技有限公司 一种基于物联网的智能pos机系统及其控制方法和电子设备

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104296852A (zh) * 2014-07-17 2015-01-21 赵肖肖 果蔬商品的自助称量方法及装置
KR20150059377A (ko) * 2013-11-22 2015-06-01 엔그램테크놀로지(주) 영상내 동작 및 객체인식 기술 기반의 자기 결제 시스템 및 방법
CN105989510A (zh) * 2015-02-06 2016-10-05 展讯通信（上海）有限公司 一种基于神经网络的在线商品推荐系统、装置及移动终端
US20170132496A1 (en) * 2015-11-05 2017-05-11 Microsoft Technology Licensing, Llc Hardware-efficient deep convolutional neural networks
CN106781121A (zh) * 2016-12-14 2017-05-31 朱明� 基于视觉分析的超市自助结账智能系统

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8571298B2 (en) * 2008-12-23 2013-10-29 Datalogic ADC, Inc. Method and apparatus for identifying and tallying objects
CN103093208B (zh) * 2013-01-23 2016-04-13 中国科学技术大学 一种果蔬识别的方法及系统
CN104077842B (zh) * 2014-07-02 2017-02-15 浙江大学 基于图像识别的自选餐厅自助付费装置及其使用方法
CN104077577A (zh) * 2014-07-03 2014-10-01 浙江大学 一种基于卷积神经网络的商标检测方法
CN104103135A (zh) * 2014-07-17 2014-10-15 张国铭 一种基于射频识别的物品自动结算纠错系统及方法
CN105719188B (zh) * 2016-01-22 2017-12-26 平安科技（深圳）有限公司 基于多张图片一致性实现保险理赔反欺诈的方法及服务器
CN106203533B (zh) * 2016-07-26 2019-09-20 厦门大学 基于混合训练的深度学习人脸验证方法
CN106384087A (zh) * 2016-09-05 2017-02-08 大连理工大学 一种基于多层网络人体特征的身份识别方法
CN106446937A (zh) * 2016-09-08 2017-02-22 天津大学 用于aer图像传感器的多层卷积识别系统
JP6165950B1 (ja) * 2016-09-20 2017-07-19 ヤフー株式会社 情報処理装置、情報処理方法および情報処理プログラム
CN106340137A (zh) * 2016-11-14 2017-01-18 贵州师范学院 一种基于深度学习的自动称重机及其程序控制方法
CN106529494A (zh) * 2016-11-24 2017-03-22 深圳市永达电子信息股份有限公司 一种基于多摄像头机型的人脸识别方法
CN106960214B (zh) * 2017-02-17 2020-11-20 北京一维弦科技有限责任公司 基于图像的物体识别方法
CN106886795B (zh) * 2017-02-17 2021-01-15 北京一维弦科技有限责任公司 基于图像中的显著物体的物体识别方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20150059377A (ko) * 2013-11-22 2015-06-01 엔그램테크놀로지(주) 영상내 동작 및 객체인식 기술 기반의 자기 결제 시스템 및 방법
CN104296852A (zh) * 2014-07-17 2015-01-21 赵肖肖 果蔬商品的自助称量方法及装置
CN105989510A (zh) * 2015-02-06 2016-10-05 展讯通信（上海）有限公司 一种基于神经网络的在线商品推荐系统、装置及移动终端
US20170132496A1 (en) * 2015-11-05 2017-05-11 Microsoft Technology Licensing, Llc Hardware-efficient deep convolutional neural networks
CN106781121A (zh) * 2016-12-14 2017-05-31 朱明� 基于视觉分析的超市自助结账智能系统

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
LI, JINGSONG ET AL.: "Supermarket Commodity Identification Using Convolutional Neural Networks", 2016 2ND INTERNATIONAL CONFERENCE ON CLOUD COMPUTING AND INTERNET OF THINGS (CCIOT, 23 October 2016 (2016-10-23), pages 115 - 119, XP033072542 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11966900B2 (en) * 2019-07-19 2024-04-23 Walmart Apollo, Llc System and method for detecting unpaid items in retail store transactions

Also Published As

Publication number Publication date
CN108269371A (zh) 2018-07-10
CN108269371B (zh) 2020-04-03

Similar Documents

Publication Publication Date Title
WO2019062018A1 (zh) 商品自动结算方法、装置、自助收银台
WO2019062017A1 (zh) 基于神经网络的商品识别方法、装置、自助收银台
US20200279310A1 (en) Notification system and methods for use in retail environments
WO2019062812A1 (zh) 用于自动结算的人机交互装置及其应用
US7646887B2 (en) Optical flow for object recognition
US9152828B2 (en) System and method for preventing cashier and customer fraud at retail checkout
JP6204491B2 (ja) セルフチェックアウト端末における不正行為の検証
US8267316B2 (en) Systems and methods for merchandise checkout
US8430311B2 (en) Systems and methods for merchandise automatic checkout
US10366379B2 (en) Remote weigh station with delayed fraud intervention
WO2020042966A1 (zh) 漏扫识别方法、装置、自助收银终端及系统
US20100114617A1 (en) Detecting potentially fraudulent transactions
US10372998B2 (en) Object recognition for bottom of basket detection
CN107134083A (zh) 终端附近事件发生的识别和成像
WO2015147333A1 (ja) 売上登録装置、プログラム及び売上登録方法
US20200019947A1 (en) Cashier Fraud Detecting System and Method and Product Image Selection Generation for Artificial Neural Network Learning Related Applications
US11188726B1 (en) Method of detecting a scan avoidance event when an item is passed through the field of view of the scanner
US20180365951A1 (en) Real time bypass detection in scanner
US20220414632A1 (en) Operation of a self-check out surface area of a retail store
JP2007080084A (ja) 異常報知装置および異常報知方法
US11657400B2 (en) Loss prevention using video analytics
US20190378389A1 (en) System and Method of Detecting a Potential Cashier Fraud
KR20210115033A (ko) 셀프 계산 소매점 환경에서 이상 검출을 위한 장치 및 방법
CN115546703A (zh) 自助收银的风险识别方法、装置、设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18862456

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205 DATED 19/08/2020)

122 Ep: pct application non-entry in european phase

Ref document number: 18862456

Country of ref document: EP

Kind code of ref document: A1