CN108320404B - Commodity identification method and device based on neural network and self-service cash register - Google Patents

Commodity identification method and device based on neural network and self-service cash register Download PDF

Info

Publication number
CN108320404B
CN108320404B CN201810135196.3A CN201810135196A CN108320404B CN 108320404 B CN108320404 B CN 108320404B CN 201810135196 A CN201810135196 A CN 201810135196A CN 108320404 B CN108320404 B CN 108320404B
Authority
CN
China
Prior art keywords
commodity
neural network
information
identification
commodity information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CN201810135196.3A
Other languages
Chinese (zh)
Other versions
CN108320404A (en
Inventor
陈子林
王良旗
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Binguo Kewei Beijing Technology Co ltd
Original Assignee
Binguo Kewei Beijing Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Binguo Kewei Beijing Technology Co ltd filed Critical Binguo Kewei Beijing Technology Co ltd
Priority to PCT/CN2018/077651 priority Critical patent/WO2019062017A1/en
Publication of CN108320404A publication Critical patent/CN108320404A/en
Application granted granted Critical
Publication of CN108320404B publication Critical patent/CN108320404B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07G—REGISTERING THE RECEIPT OF CASH, VALUABLES, OR TOKENS
    • G07G1/00—Cash registers
    • G07G1/0018—Constructional details, e.g. of drawer, printing means, input means
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07G—REGISTERING THE RECEIPT OF CASH, VALUABLES, OR TOKENS
    • G07G1/00—Cash registers
    • G07G1/0036—Checkout procedures
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07G—REGISTERING THE RECEIPT OF CASH, VALUABLES, OR TOKENS
    • G07G1/00—Cash registers
    • G07G1/12—Cash registers electronically operated
    • A—HUMAN NECESSITIES
    • A47—FURNITURE; DOMESTIC ARTICLES OR APPLIANCES; COFFEE MILLS; SPICE MILLS; SUCTION CLEANERS IN GENERAL
    • A47F—SPECIAL FURNITURE, FITTINGS, OR ACCESSORIES FOR SHOPS, STOREHOUSES, BARS, RESTAURANTS OR THE LIKE; PAYING COUNTERS
    • A47F9/00—Shop, bar, bank or like counters
    • A47F9/02—Paying counters
    • A47F9/04—Check-out counters, e.g. for self-service stores
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q20/00—Payment architectures, schemes or protocols
    • G06Q20/08—Payment architectures
    • G06Q20/20—Point-of-sale [POS] network systems
    • G06Q20/208—Input by product or record sensing, e.g. weighing or scanner processing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q20/00—Payment architectures, schemes or protocols
    • G06Q20/30—Payment architectures, schemes or protocols characterised by the use of specific devices or networks
    • G06Q20/32—Payment architectures, schemes or protocols characterised by the use of specific devices or networks using wireless devices
    • G06Q20/327—Short range or proximity payments by means of M-devices
    • G06Q20/3276—Short range or proximity payments by means of M-devices using a pictured code, e.g. barcode or QR-code, being read by the M-device
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07C—TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
    • G07C9/00—Individual registration on entry or exit
    • G07C9/30—Individual registration on entry or exit not involving the use of a pass
    • G07C9/32—Individual registration on entry or exit not involving the use of a pass in combination with an identity check
    • G07C9/37—Individual registration on entry or exit not involving the use of a pass in combination with an identity check using biometric data, e.g. fingerprints, iris scans or voice recognition
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07G—REGISTERING THE RECEIPT OF CASH, VALUABLES, OR TOKENS
    • G07G1/00—Cash registers
    • G07G1/0036—Checkout procedures
    • G07G1/0045—Checkout procedures with a code reader for reading of an identifying code of the article to be registered, e.g. barcode reader or radio-frequency identity [RFID] reader
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07G—REGISTERING THE RECEIPT OF CASH, VALUABLES, OR TOKENS
    • G07G1/00—Cash registers
    • G07G1/0036—Checkout procedures
    • G07G1/0045—Checkout procedures with a code reader for reading of an identifying code of the article to be registered, e.g. barcode reader or radio-frequency identity [RFID] reader
    • G07G1/0054—Checkout procedures with a code reader for reading of an identifying code of the article to be registered, e.g. barcode reader or radio-frequency identity [RFID] reader with control of supplementary check-parameters, e.g. weight or number of articles
    • G07G1/0072—Checkout procedures with a code reader for reading of an identifying code of the article to be registered, e.g. barcode reader or radio-frequency identity [RFID] reader with control of supplementary check-parameters, e.g. weight or number of articles with means for detecting the weight of the article of which the code is read, for the verification of the registration
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07G—REGISTERING THE RECEIPT OF CASH, VALUABLES, OR TOKENS
    • G07G1/00—Cash registers
    • G07G1/01—Details for indicating
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07G—REGISTERING THE RECEIPT OF CASH, VALUABLES, OR TOKENS
    • G07G3/00—Alarm indicators, e.g. bells
    • A—HUMAN NECESSITIES
    • A47—FURNITURE; DOMESTIC ARTICLES OR APPLIANCES; COFFEE MILLS; SPICE MILLS; SUCTION CLEANERS IN GENERAL
    • A47F—SPECIAL FURNITURE, FITTINGS, OR ACCESSORIES FOR SHOPS, STOREHOUSES, BARS, RESTAURANTS OR THE LIKE; PAYING COUNTERS
    • A47F9/00—Shop, bar, bank or like counters
    • A47F9/02—Paying counters
    • A47F9/04—Check-out counters, e.g. for self-service stores
    • A47F2009/041—Accessories for check-out counters, e.g. dividers
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/088—Non-supervised learning, e.g. competitive learning

Abstract

The invention discloses a commodity identification method, a commodity identification device and a self-service cash register based on a neural network, wherein the method comprises the following steps: obtaining an image containing a commodity to be detected; and inputting the image containing the to-be-detected commodity into a recognition system based on the neural network, and outputting the information of the to-be-detected commodity by the recognition system based on the neural network. According to the method, the commodity image is obtained through the common camera, the commodity information is obtained through the image recognition algorithm based on the neural network, a third-party identification is not needed, the user only needs to place the purchased commodity under the camera to obtain the image, the identification can be achieved, the use cost is low, and the identification accuracy is high.

Description

Commodity identification method and device based on neural network and self-service cash register
Technical Field
The invention relates to a commodity identification method and device based on a neural network and a self-service cash register, and belongs to the field of deep learning neural network and image identification.
Background
The machine recognition of commodities under the existing self-service settlement scene is mainly divided into two types:
the first is based on RFID electronic label (Radio Frequency Identification, also called Radio Frequency Identification) Identification settlement method, firstly, the corresponding relation of the goods is appointed for each electronic label with unique ID in the database, then the electronic label is pasted on all the goods sold, the unique ID of the electronic label is read out by a card reader during settlement, the information of the goods is inquired in the database according to the ID, thereby completing the Identification of the goods and settlement.
The second is that the user puts the bar code on the commodity under the code scanner by self, and the machine scans the code to realize the commodity 'identification'. The method needs the user to scan personally, various scanning misoperation is easy to occur, and the use cost is increased. For example, a product is repeatedly counted due to scanning codes for multiple times, and the product is difficult to correctly scan after the bar code of the product is deformed. Meanwhile, the method also has the problem of poor anti-theft effect.
Disclosure of Invention
In order to solve the technical problems, the invention provides a commodity identification method based on a neural network based on deep learning, the method obtains commodity images through a common camera, obtains commodity information by utilizing an image identification algorithm based on the neural network, does not need to help a third party mark, and can realize identification only by placing a purchased commodity under the camera to obtain the images by a user, so that the use cost is low, and the identification accuracy is high.
The commodity identification method based on the neural network comprises the following steps:
obtaining an image containing a commodity to be detected;
inputting an image containing a to-be-detected commodity into a recognition system based on a neural network, and outputting information of the to-be-detected commodity by the recognition system based on the neural network; obtaining an image containing a commodity to be detected, wherein the image is at least a two-dimensional image; obtaining images containing a commodity to be detected, wherein the images at least comprise first images to Nth images with different angles and/or depths of field; n is more than or equal to 2;
the neural network identification system comprises a first neural network based on a regional convolutional neural network; the commodity identification method based on the neural network comprises the following steps:
(a1) inputting the first image into a first neural network, and outputting first commodity information by the first neural network; inputting the Nth image into a first neural network, and outputting the Nth commodity information by the first neural network;
(b1) judging whether the Nth commodity information is contained in the first commodity information:
if the judgment result is yes, outputting the first commodity information as the to-be-detected commodity information;
and if the judgment result is negative, outputting a feedback prompt.
Optionally, obtaining first to nth images of which the images containing the to-be-detected commodity at least comprise different angles and/or different depths of field; n is 2-4; the first image is a front image of the commodity to be detected.
Optionally, the step (a1) further comprises the step of weighing the weight of the commodity to be detected, so as to obtain the actual weighed total weight of the commodity; and (b1) calculating the total weight of the commodities in the first commodity information (b2), comparing the total weight of the commodities with the actual weighed total weight of the commodities to obtain differential data, and judging whether the differential data is less than or equal to a preset threshold value:
if the judgment result is yes, outputting the first commodity information as the to-be-detected commodity information; and if the judgment result is negative, outputting a feedback prompt.
Optionally, the neural network identification system comprises a first neural network based on a regional convolutional neural network; the commodity identification method based on the neural network comprises the following steps:
the method comprises the following steps:
(a3) inputting the first image into a first neural network, and outputting first commodity information by the first neural network; inputting the Nth image into a first neural network, and outputting the Nth commodity information by the first neural network;
(b3) judging whether the Nth commodity information is contained in the first commodity information;
if the judgment result is yes, outputting the first commodity information as the to-be-detected commodity information;
if the judgment result is negative, executing the subsequent steps;
(c3) calculating the total weight of the commodities in the first commodity information, comparing the total weight of the commodities with the actual weighed commodity to obtain differential data, and judging whether the differential data is less than or equal to a preset threshold value:
if the judgment result is yes, outputting the first commodity information as the to-be-detected commodity information;
and if the judgment result is negative, outputting a feedback prompt.
Alternatively, the method of determining whether or not the nth product information is included in the first product information in the steps (b1) and (b3) is to determine whether or not the product types in the nth product information are both present in the first product information.
Alternatively, the determination as to whether or not the N-th item information is included in the first item information in steps (b1) and (b3) may be made as to whether or not the number of items in the N-th item information is equal to or less than the number of items in the first item information.
Alternatively, the determination as to whether or not the N-th item information is included in the first item information in steps (b1) and (b3) may be made as to whether or not the number of each item in the N-th item information is equal to or less than the number of items in the first item information.
Alternatively, the steps (b1) and (b3) are determining whether the nth item information coincides with the first item information;
if the judgment result is yes, outputting the first commodity information as the to-be-detected commodity information; if the judgment result is negative, the subsequent steps are executed.
Alternatively, whether the nth commodity information coincides with the first commodity information in the steps (b1) and (b3) includes commodity kind coincidence and the number of each commodity.
Optionally, the preset threshold value in the step (b2) and the step (c3) is at least one value of 0.1g to 10 kg.
Alternatively, the threshold value preset in the step (b2) and the step (c3) is the weight of the article having the smallest weight in the first article information.
Optionally, the threshold value preset in the step (b2) and the step (c3) is at least one value of 10% to 80% of the weight of the smallest weight commodity in the first commodity information.
Optionally, the feedback prompt in step (b2) and step (c3) includes at least one of a stack prompt and an error report.
Optionally, the number of the commodities to be detected in the image containing the commodities to be detected is more than or equal to 1.
Optionally, the number of the commodities to be detected in the image containing the commodities to be detected is 1-1000.
Optionally, the type of the commodity to be detected in the image containing the commodity to be detected is more than or equal to 1.
Optionally, the type of the commodity to be detected is 1-1000.
Optionally, the neural network-based recognition system comprises a second neural network based on a regional convolutional neural network, the neural network-based recognition system being obtained by a method comprising the steps of:
obtaining a first image set of multi-angle images of each commodity to be detected;
and training a second neural network by using the first image set to obtain a first neural network.
Optionally, the method of training the second neural network is a supervised learning method.
Optionally, the method of training the second neural network is:
adopting supervised learning, and training a second neural network by using the first image set to obtain a third neural network;
obtaining a second image set of the commodity image to be detected;
and training the third neural network by using the second image set to obtain the first neural network.
Optionally, the second set of images includes images of the item to be detected that output information of the item to be detected via a neural network-based recognition system.
Optionally, the identification accuracy of the second neural network to the commodity to be detected is more than 80%.
Optionally, the process of training the third neural network with the second set of images is unsupervised learning.
Optionally, the neural network based commodity identification method includes the steps of:
(a1) inputting the first image into a first neural network, and outputting first commodity information by the first neural network; inputting the Nth image into a first neural network, and outputting the Nth commodity information by the first neural network;
(b1) judging whether the Nth commodity information is contained in the first commodity information;
(c1) if the determination result in the step (b1) is negative, identifying a different commodity in the first commodity information and the nth commodity information;
(d1) acquiring the difference image set of the difference commodity in the step (c1), and strengthening and training the first neural network by using the difference image set.
Optionally, the neural network based commodity identification method includes the steps of:
(a3) inputting the first image into a first neural network, and outputting first commodity information by the first neural network; inputting the Nth image into a first neural network, and outputting the Nth commodity information by the first neural network;
(b3) judging whether the Nth commodity information is contained in the first commodity information;
if the judgment result is yes, outputting the first commodity information as the to-be-detected commodity information;
if the judgment result is negative, executing the subsequent steps;
(c3) calculating the total weight of the commodities in the first commodity information, comparing the total weight of the commodities with the actual weighed commodity to obtain differential data, and judging whether the differential data is less than or equal to a preset threshold value:
(d3) if the determination result in the step (c3) is negative, identifying a different commodity in the first commodity information and the nth commodity information;
(e3) and (d3) acquiring the difference image set of the difference commodity in the step (d3), and strengthening and training the first neural network by using the difference image set.
Optionally, the neural network based commodity identification method includes the steps of:
(a2) inputting the first image into a first neural network, and outputting first commodity information by the first neural network;
(b2) calculating the total weight of the commodities in the first commodity information, comparing the total weight of the commodities with the actual weighed commodity to obtain differential data, and judging whether the differential data is less than or equal to a preset threshold value:
(c2) identifying the commodity in the first commodity information when the judgment result in the collecting step (b2) is negative;
(d2) acquiring the collection image set for identifying the commodity in the step (c2), and strengthening and training the first neural network by using the collection image set.
Optionally, the neural network based commodity identification method includes the steps of:
(a3) inputting the first image into a first neural network, and outputting first commodity information by the first neural network; inputting the Nth image into a first neural network, and outputting the Nth commodity information by the first neural network;
(b3) judging whether the Nth commodity information is contained in the first commodity information;
if the judgment result is yes, outputting the first commodity information as the to-be-detected commodity information;
if the judgment result is negative, executing the subsequent steps;
(c3) calculating the total weight of the commodities in the first commodity information, comparing the total weight of the commodities with the actual weighed commodity to obtain differential data, and judging whether the differential data is less than or equal to a preset threshold value:
(d3) identifying the product in the first product information when the judgment result in the collecting step (c3) is negative;
(e3) acquiring the collection image set for identifying the commodity in the step (d3), and strengthening and training the first neural network by using the collection image set.
According to still another aspect of the present invention, there is provided a neural network-based product recognition apparatus, including:
the camera shooting unit is used for acquiring images containing the to-be-detected commodity, wherein the images at least comprise first images to Nth images with different angles and/or depths of field; n is more than or equal to 2;
the identification information unit is used for inputting the first image into a first neural network, and the first neural network outputs first commodity information; inputting the Nth image into a first neural network, and outputting the Nth commodity information by the first neural network;
an identification judgment unit configured to judge whether or not the nth commodity information is included in the first commodity information; if the judgment result is yes, outputting the first commodity information as the to-be-detected commodity information; if the judgment result is negative, outputting a feedback prompt;
the display unit is used for outputting information of the to-be-detected commodity and a feedback prompt;
the camera shooting unit is connected with the identification information unit, the identification information unit is connected with the identification judging unit, and the identification judging unit is connected with the display unit.
Optionally, the system comprises an identification information unit and an identification judgment unit, which are used for identifying and judging the commodity according to any one of the above commodity identification methods based on the neural network.
Optionally, the neural network based commodity identification device comprises an object stage, wherein the object stage comprises a weight sensor for measuring the total weight of the commodity on the object stage;
the weight sensor is electrically connected to the recognition unit and inputs the total weight of the commodity on the stage into the recognition unit.
According to another aspect of the invention, a self-service cash register is provided, and the self-service cash register adopts any one of the neural network-based commodity identification methods to identify commodities.
According to another aspect of the invention, a self-service cash register station is provided, wherein the self-service cash register station adopts any one of the neural network-based commodity identification devices.
The beneficial effects of the invention include but are not limited to:
(1) the commodity identification method based on the neural network provided by the invention fully utilizes the neural network to identify the commodity and judges the commodity information obtained by a plurality of images, thereby avoiding the identification error rate caused by the excessive dependence on image identification in the existing image identification field and improving the identification accuracy. The existing bar code or RFID electronic tag is not needed for identification, and the use cost is reduced.
(2) The commodity identification method based on the neural network provided by the invention continuously improves the identification accuracy of the method along with the increase of the use frequency through the sustainable learning of deep learning.
(3) According to the commodity identification method based on the neural network, the commodity picture is grabbed by the common camera, the batch commodity can be rapidly detected, and the cost and the speed of commodity identification are greatly reduced.
(4) The commodity identification method based on the neural network can realize low-cost and high-efficiency commodity identification and settlement under the self-service settlement scene.
(5) The commodity identification device based on the neural network realizes the correction of the identification result through the neural network identification and the multi-image comparison, and trains the neural network system by utilizing the image set obtained by the identification result, thereby continuously improving the identification accuracy.
(6) The self-service cash register station provided by the invention can realize the autonomous settlement of the user, and has high settlement efficiency and accurate settlement result.
Drawings
FIG. 1 is a schematic block diagram of a flow chart of a neural network-based product identification method in a first preferred embodiment of the invention;
FIG. 2 is a schematic block diagram of a flow chart of a neural network-based product identification method according to a second preferred embodiment of the present invention;
FIG. 3 is a schematic block diagram of the flow of a neural network-based product identification method in a third preferred embodiment of the present invention;
FIG. 4 is a schematic block diagram of the flow of a neural network-based product identification method in a fourth preferred embodiment of the present invention;
FIG. 5 is a schematic block diagram of a flow chart of a neural network-based product identification method according to a fifth preferred embodiment of the present invention;
FIG. 6 is a schematic block diagram of a neural network based article identification device according to a sixth preferred embodiment of the present invention;
FIG. 7 is a schematic block diagram of a flow chart of a neural network-based product identification method according to a seventh preferred embodiment of the present invention;
fig. 8 is a timing diagram illustrating an unmanned convenience store with a self-service checkout counter to which the neural network-based product recognition method of the present invention is applied.
Detailed Description
The technical solutions in the embodiments of the present invention are clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, shall fall within the scope of protection of the present invention.
Referring to fig. 1, the neural network-based product identification method provided by the present invention includes:
obtaining an image containing a commodity to be detected;
inputting an image containing a to-be-detected commodity into a recognition system based on a neural network, and outputting information of the to-be-detected commodity by the recognition system based on the neural network;
obtaining an image containing a commodity to be detected, wherein the image is at least a two-dimensional image;
obtaining images containing a commodity to be detected, wherein the images at least comprise first images to Nth images with different angles and/or depths of field; n is more than or equal to 2;
the neural network identification system comprises a first neural network based on a regional convolutional neural network; the commodity identification method based on the neural network comprises the following steps:
(a1) inputting the first image into a first neural network, and outputting first commodity information by the first neural network; inputting the Nth image into a first neural network, and outputting the Nth commodity information by the first neural network;
(b1) judging whether the Nth commodity information is contained in the first commodity information;
if the judgment result is yes, outputting the first commodity information as the to-be-detected commodity information; and if the judgment result is negative, outputting a feedback prompt.
The commodity identification method based on the neural network is mainly used for self-help shopping after self-help obtaining of settlement commodity information in an unattended environment. The method fully utilizes the neural network to identify the commodity and judges the commodity information obtained by the plurality of images, thereby avoiding the identification error rate caused by the excessive dependence on image identification in the prior image identification field and improving the identification accuracy. When the accurate commodity information cannot be acquired, the user can be prompted through feedback prompt, and whether the commodity to be settled cannot be accurately identified or not can be judged, so that the identification error can be corrected only by adjusting the commodity to be identified, and repeated code scanning or repeated attempts are not needed. The feedback prompt here includes at least one of a stack prompt and an error report. The method can be used for processing the commodities with unlimited types and quantities, for example, the quantity of the commodities to be detected in the image containing the commodities to be detected is more than or equal to 1. The number of the commodities to be detected in the image containing the commodities to be detected is 1-1000. The type of the commodity to be detected in the image containing the commodity to be detected is more than or equal to 1. The type of the commodity to be detected is 1-1000. The judged commodity information includes the commodity kind or the number of each commodity. And judging whether the commodity types and/or the commodity quantity are consistent. When the commodity identification method based on the neural network is used for settlement under an unattended environment, the commodity can be accurately identified only by using a common camera with a network networking function. And the RFID label is not needed to be used at all, so that the cost is reduced. Meanwhile, the problem that the settlement cannot be carried out due to misoperation and the like can be avoided.
Preferably, the first image is a front image of the commodity to be detected. The image is used as a main image for identification, and the identification accuracy can be improved.
Preferably, the first image to the Nth image which at least comprise different angles and/or different depths of field of the image containing the commodity to be detected are obtained; n is 2-4. By acquiring the multi-angle image, the identification accuracy of the neural network can be improved. The accuracy of subsequent recognition results is improved.
Referring to fig. 2, preferably, step (a1) further comprises the step of weighing the commodity to be detected, so as to obtain the actual weighed total weight of the commodity; and (b1) calculating the total weight of the commodities in the first commodity information (b2), comparing the total weight of the commodities with the actual weighed total weight of the commodities to obtain differential data, and judging whether the differential data is less than or equal to a preset threshold value: if the judgment result is yes, outputting the first commodity information as the to-be-detected commodity information; and if the judgment result is negative, outputting a feedback prompt. Meanwhile, for the obtained commodity information, the obtained result can be corrected by analyzing the weight of the commodity contained in the commodity information, so that the accuracy of the image recognition result is improved.
Referring to fig. 3, preferably, the neural network identification system includes a first neural network based on a regional convolutional neural network; the commodity identification method based on the neural network comprises the following steps: (a3) inputting the first image into a first neural network, and outputting first commodity information by the first neural network; inputting the Nth image into a first neural network, and outputting the Nth commodity information by the first neural network; (b3) judging whether the Nth commodity information is contained in the first commodity information; if the judgment result is yes, outputting the first commodity information as the to-be-detected commodity information; if the judgment result is negative, executing the subsequent steps; (c3) calculating the total weight of the commodities in the first commodity information, comparing the total weight of the commodities with the actual weighed commodity to obtain differential data, and judging whether the differential data is less than or equal to a preset threshold value: if the judgment result is yes, outputting the first commodity information as the to-be-detected commodity information; and if the judgment result is negative, outputting a feedback prompt.
By using the type and the commodity information as correction parameters, the obtained result can be better corrected, and the identification accuracy of the commodity is improved. The preset threshold value here may be at least one value of 0.1g to 10 kg. The preset threshold value may be a weight of the article with the smallest weight in the first article information. The preset threshold value may be at least one value of 10% to 80% of the weight of the article having the smallest weight in the first article information.
Preferably, the method of determining whether or not the nth product information is included in the first product information in the steps (b1) and (b3) is to determine whether or not both of the product types in the nth product information are present in the first product information.
Preferably, the determination as to whether or not the N-th item information is included in the first item information in the steps (b1) and (b3) is made as to whether or not the number of each item in the N-th item information is equal to or less than the number of items in the first item information.
Preferably, the steps (b1) and (b3) are determining whether the nth item information coincides with the first item information; if the judgment result is yes, outputting the first commodity information as the to-be-detected commodity information; if the judgment result is negative, the subsequent steps are executed.
Preferably, whether the nth commodity information coincides with the first commodity information in the steps (b1) and (b3) includes a commodity type coincidence and a quantity coincidence for each commodity.
Preferably, the neural network-based recognition system comprises a second neural network based on a regional convolutional neural network, the neural network-based recognition system being obtained by a method comprising the steps of: obtaining a first image set of multi-angle images of each commodity to be detected; and training a second neural network by using the first image set to obtain a first neural network. By using the second neural network, the obtained result can be used for training the first neural network, so that deep learning automatic system error correction is realized, and the identification accuracy of the neural identification system is automatically improved along with the improvement of the number of the identified commodities. The method is carried out according to the existing method. The multi-angle images of the commodities to be detected are used for training, and the recognition accuracy of the neural network recognition system when the commodities are shielded can be improved.
Preferably, the method of training the second neural network is a supervised learning method.
Preferably, the method for training the second neural network comprises the following steps: adopting supervised learning, and training a second neural network by using the first image set to obtain a third neural network; obtaining a second image set of the commodity image to be detected; and training the third neural network by using the second image set to obtain the first neural network.
Preferably, the second image set includes images of the item to be detected that output information of the item to be detected via a recognition system based on a neural network.
Preferably, the identification accuracy of the second neural network on the commodity to be detected is more than 80%. Preferably, the process of training the third neural network by the second image set is unsupervised learning. The method is carried out according to the existing method.
Referring to fig. 4, preferably, the neural network based commodity identification method includes the steps of:
(c1) or (d3) identifying a different product from the first product information and the nth product information when the judgment result in the step (b1) or (c3) is no;
(d1) or (e3) acquiring a difference image set of the difference commodity in the step (c1) or (d3), and strengthening and training the first neural network by using the difference image set.
And when the judgment result is negative, the difference commodity existing in the Nth commodity information is collected and an image set of the difference commodity is obtained, and the error correction capability of the system can be further improved by training the first neural network by using the difference image set. While this operation can also be used in the method as shown in figure 3.
Referring to fig. 5, preferably, the neural network based commodity identification method includes the steps of:
(c2) or (d3) identifying the commodity in the first commodity information when the judgment result in the collecting step (b2) or (c3) is negative;
(d2) or (e3) acquiring the collected image set for identifying the commodity in the step (c2) or (d3), and strengthening the training of the first neural network by using the collected image set.
This step can also be used in the method as shown in fig. 3, which is not described here in more detail. And when the detection result is negative, the first commodity information under the condition of multiple times of unrecognizable conditions is collected and used for training the first neural network, so that the recognition capability of the first neural network on the unrecognizable conditions is improved.
Referring to fig. 6, in the commodity identification method based on the neural network, when in use, a commodity to be detected is placed on an object stage, and N cameras are arranged around the commodity to be detected. Images of the commodity to be detected at all angles are acquired through the N cameras and are respectively recorded as P1 and P2. Among the N cameras, a camera located directly above the stage is a main camera and is denoted as a first camera, and an image acquired by the camera is a first image P1.
Uploading P1 and P2.. once PN to a local identification server or a cloud identification server, identifying each picture, respectively recording the identified commodity information as R1 and R2.. once RN, wherein the commodity information comprises the category information and the quantity information of commodities, wherein the identification result R1 of a main camera is first commodity information, and the identification results R2.. once RN of other cameras are second commodity information;
taking two cameras as an example, it is determined whether R2 (second article information) is included in R1 (first article information);
if the judgment result is yes, outputting R1 as the information of the commodity to be detected;
if the judgment result is no, calculating the total weight of the commodity in the R1, taking the absolute value of the result obtained by subtracting the total weight of the commodity actually weighed as difference data, and judging whether the difference data is less than or equal to a preset threshold value:
if the judgment result is yes, outputting R1 as the information of the commodity to be detected, and outputting a commodity information list of which the commodity information comprises the category, the quantity and the price of the commodity;
if the judgment result is negative, displaying a stacking prompt or error report information.
Referring to fig. 7, another aspect of the present invention also provides a neural network-based article recognition apparatus including:
the image pickup unit 100 is used for acquiring images containing the to-be-detected commodity, wherein the images at least comprise first images to Nth images with different angles and/or depths of field; n is more than or equal to 2;
an identification information unit 210, configured to input the first image into a first neural network, and the first neural network outputs first commodity information; inputting the Nth image into a first neural network, and outputting the Nth commodity information by the first neural network;
an identification judgment unit 220 for judging whether the nth commodity information is included in the first commodity information; if the judgment result is yes, outputting the first commodity information as the to-be-detected commodity information; if the judgment result is negative, outputting a feedback prompt;
the display unit 300 is used for outputting information of the to-be-detected commodity and feedback prompts;
the imaging unit 100 is connected to the identification information unit 210, the identification information unit 210 is connected to the identification determination unit 220, and the identification determination unit 220 is connected to the display unit 300. The above units can be implemented by setting corresponding programs on various existing devices.
Preferably, the identification information unit 210 and the identification judging unit 220 are used for identifying and judging the commodity according to the commodity identification method based on the neural network in any one of claims 1 to 21.
Optionally, the camera unit 100 includes two general webcams, two holders capable of adjusting any angle, a continuous computer capable of operating picture uploading, and a high-precision weight sensor. The main working flow is as follows: and an image capturing program runs on the computer, the program can upload picture images captured by the two cameras at the same time to a remote server, and the remote server returns the identification result. The scheme has extremely low cost, and the working computer only needs the most basic configuration.
Optionally, the camera unit 100 includes 2-4 lens-fixed high-definition cameras, a corresponding number of angle-adjustable holders, a high-precision weight sensor, and a computer with a video card with a video memory of 2G or more. The main work flow is that an image capturing program is operated on a computer, and the program can locally identify the picture images captured by two cameras at the same time.
Optionally, the neural network-based product recognition device may perform batch detection (low-cost scheme), and a plurality of common cameras are used to obtain images of the product to be detected from different angles.
The camera of a plurality of different angles can solve commodity because the angle of putting shelters from the problem that article difference in height produced in same 2D picture. Basically 3 cameras can realize that no dead angle acquires the required information of treating discernment, and under the suitable camera position condition, 2 cameras also can reach more ideal effect.
Preferably, optionally, the camera unit 100 comprises a first camera and a second camera;
the first camera and the second camera respectively obtain commodity images from different angles.
Optionally, the neural network based commodity identification device comprises an object stage, wherein the object stage comprises a weight sensor for measuring the total weight of the commodity on the object stage;
the weight sensor is electrically connected to the recognition unit and inputs the total weight of the commodity on the stage into the recognition unit.
In the process of commodity image identification, commodities to be settled are often stacked or extremely shot at an angle, so that objects are shielded or most of the objects are shielded, and sufficient details cannot be obtained for accurately identifying the commodities. In order to accurately judge whether the commodities are stacked or not, the invention combines the weight sensor to correct the image recognition result, obtains the weight of the articles in the recognition result and the actual weighing of the weight sensor in the recognition device, and feeds back that the commodities are stacked if the weights are not consistent.
The invention further provides a self-service cash register which adopts the commodity identification method based on the neural network to identify the commodities. The self-service can be in an unattended state or can be used under the supervision of a loss prevention person. Only the customer needs to settle the account. By adopting the commodity identification method based on the neural network, customers can efficiently and accurately complete the calculation process, the whole equipment is low in cost, and an electronic tag is not required.
The invention further provides a self-service cash register which adopts the commodity identification device based on the neural network. The self-service can be in an unattended state or can be used under the supervision of a loss prevention person. Only the customer needs to settle the account.
Fig. 8 is a timing diagram illustrating an embodiment of the neural network-based product recognition apparatus of the present invention used in an unmanned convenience store at a self-service checkout counter. Can also be used as an implementation example of the self-service cash register provided by the invention. As shown in fig. 8, in the case of using the neural network-based product recognition apparatus including any one of the neural network-based product recognition methods, the shopping procedure of the customer in the unmanned convenience store is as follows:
after the customer selects the commodities, all the commodities are placed on a self-service cash register (also a loading table in a commodity identification device based on a neural network);
when the object stage senses that the weight is greater than 0, triggering a commodity identification device based on a neural network to start a commodity identification program;
the camera shoots the commodity on the objective table to obtain a commodity picture, and codes POST (POST position) on the commodity picture Base64 to an image recognition server for image recognition;
comparing the information of the image identification result (including the names, prices and total weights of all commodities) with the total weight obtained by actual weighing of the objective table to obtain differential data;
when the difference data is less than or equal to a preset threshold value, judging that the actual weighing is consistent with the range weight, and requesting an order processing interface to generate an order;
when the difference data is larger than a preset threshold value, judging that the actual weighing is inconsistent with the range weight, displaying a stacking prompt on an operation interface, and prompting a customer to move the commodity so that the camera can shoot the commodity stacked on the lower layer and shielded; the camera shoots the commodities on the objective table again to obtain a new commodity picture, and an order is requested to be generated from the order processing interface until the difference data is less than or equal to a preset threshold value;
the order processing interface receives the order generation request, sends out a payment two-dimensional code character string and generates a payment two-dimensional code on the operation interface;
the customer scans the payment two-dimensional code;
after the successful payment, the message SOCKET sends a successful payment message to demagnetize the commodities on the objective table;
the message SOCKET sends a face recognition message to the secure channel;
the customer carries the commodity to pass through a safety channel comprising a detection device, if the label which is not demagnetized is not detected, the gate is opened, and the customer leaves an unmanned shopping convenience store; if the demagnetized label is detected, an unpaid warning is sent out, and the gate is not opened.
The above description is only for the purpose of illustrating the present invention and is not intended to limit the present invention in any way, and the present invention is not limited to the above description, but rather should be construed as being limited to the scope of the present invention.

Claims (23)

1. A commodity identification method based on a neural network is characterized by comprising the following steps:
obtaining an image containing a commodity to be detected;
inputting the image containing the to-be-detected commodity into a recognition system based on a neural network, and outputting information of the to-be-detected commodity by the recognition system based on the neural network;
the obtained image containing the commodity to be detected is at least a two-dimensional image;
the obtained images containing the to-be-detected commodity at least comprise first images to Nth images with different angles and/or depths of field; n is more than or equal to 2;
the neural network identification system comprises a first neural network based on a regional convolutional neural network; the commodity identification method based on the neural network comprises the following steps:
(a1) inputting the first image into the first neural network, and outputting first commodity information by the first neural network; inputting the Nth image into the first neural network, and outputting Nth commodity information by the first neural network;
(b1) judging whether the Nth commodity information is contained in the first commodity information;
if the judgment result is yes, outputting the first commodity information as the to-be-detected commodity information;
if the judgment result is negative, outputting a feedback prompt;
the commodity identification method based on the neural network comprises the following steps:
(c1) when the judgment result in the step (b1) is negative, identifying a different commodity in the first commodity information and the Nth commodity information;
(d1) acquiring the difference image set of the difference commodity in the step (c1), and strengthening and training the first neural network by using the difference image set.
2. The neural network-based commodity identification method according to claim 1, wherein the obtained images containing the commodity to be detected at least comprise first to Nth images at different angles and/or different depths of field; n is 2-4;
the first image is a front image of the commodity to be detected.
3. The neural network based commodity identification method according to claim 1, wherein said step (a1) further comprises a step of weighing the commodity to be detected, so as to obtain an actual weighed total weight of the commodity;
said step (b1) comprises step (b 2);
the step (b2) is as follows: calculating the total weight of the commodities in the first commodity information, comparing the total weight of the commodities with the actual weighed total weight of the commodities to obtain differential data, and judging whether the differential data is less than or equal to a preset threshold value: if the judgment result is yes, outputting the first commodity information as the to-be-detected commodity information;
and if the judgment result is negative, outputting a feedback prompt.
4. The neural network-based commodity identification method according to claim 1, wherein the neural network identification system comprises a first neural network based on a regional convolutional neural network; the commodity identification method based on the neural network comprises the following steps:
(a3) inputting the first image into the first neural network, and outputting first commodity information by the first neural network; inputting the Nth image into the first neural network, and outputting Nth commodity information by the first neural network;
(b3) judging whether the Nth commodity information is contained in the first commodity information;
if the judgment result is yes, outputting the first commodity information as the to-be-detected commodity information;
if the judgment result is negative, executing the subsequent steps;
(c3) calculating the total weight of the commodities in the first commodity information, comparing the total weight of the commodities with the actual weighed total weight of the commodities to obtain differential data, and judging whether the differential data is less than or equal to a preset threshold value:
if the judgment result is yes, outputting the first commodity information as the to-be-detected commodity information;
if the judgment result is negative, executing the step (d 3); the step (d3) is: identifying a difference commodity in the first commodity information and the Nth commodity information;
(e3) comprises the following steps: acquiring the difference image set of the difference commodity in the step (d3), and strengthening and training the first neural network by using the difference image set.
5. The neural-network-based product recognition method according to claim 1 or 4, wherein the step (b1) and the step (b3) of determining whether or not the N-th product information is included in the first product information are performed by determining whether or not both of the product types in the N-th product information are present in the first product information.
6. The neural network-based commodity identification method according to claim 1 or 4, wherein the step (b1) and the step (b3) of determining whether or not the N-th commodity information is included in the first commodity information are performed by determining whether or not the number of commodities in the N-th commodity information is equal to or less than the number of commodities in the first commodity information.
7. The neural network-based commodity identification method according to claim 1 or 4, wherein the step (b1) and the step (b3) of determining whether or not the N-th commodity information is included in the first commodity information are performed by determining whether or not the number of each commodity in the N-th commodity information is equal to or less than the number of commodities in the first commodity information.
8. The neural network-based commodity identification method according to claim 1 or 4, wherein the steps (b1) and (b3) are to determine whether the Nth commodity information coincides with the first commodity information;
if the judgment result is yes, outputting the first commodity information as the to-be-detected commodity information;
if the judgment result is negative, the subsequent steps are executed.
9. The neural network-based commodity identification method according to claim 8, wherein whether said nth commodity information coincides with said first commodity information in said steps (b1) and (b3) includes commodity kind coincidence and quantity coincidence for each commodity.
10. The neural network-based commodity identification method according to claim 3 or 4, wherein the preset threshold value in the steps (b2) and (c3) is at least one value from 0.1g to 10 kg.
11. The neural network-based commodity identification method according to claim 3 or 4, wherein the threshold value preset in the steps (b2) and (c3) is a commodity weight having a smallest weight among the first commodity information.
12. The neural network-based commodity identification method according to claim 3 or 4, wherein the threshold value preset in the steps (b2) and (c3) is at least one value of 10% to 80% of the weight of the commodity having the smallest weight in the first commodity information.
13. The neural network-based commodity identification method according to claim 1, wherein the neural network-based identification system comprises a second neural network based on a regional convolutional neural network, the neural network-based identification system being obtained by a method comprising:
obtaining a first image set of multi-angle images of each commodity to be detected;
and training the second neural network by using the first image set to obtain a first neural network.
14. The neural network-based commodity identification method according to claim 13, wherein the method for training the second neural network is a supervised learning method.
15. The method for commodity identification based on neural network as claimed in claim 13, wherein said method for training said second neural network is:
training the second neural network by using the first image set by adopting supervised learning to obtain a third neural network;
obtaining a second image set of the commodity image to be detected;
and training the third neural network by using the second image set to obtain the first neural network.
16. The neural network-based item identification method according to claim 15, wherein the second image set includes images of the item to be detected information output via the neural network-based identification system.
17. The neural network-based merchandise recognition method of claim 15, wherein the process of training the third neural network with the second set of images is unsupervised learning.
18. The neural network based commodity identification method according to claim 3, wherein the neural network based commodity identification method comprises the steps of:
(c2) collecting the commodities in the first commodity information when the judgment result in the step (b2) is negative;
(d2) acquiring the collection image set of the identified commodity in the step (c2), and strengthening and training the first neural network by using the collection image set.
19. The neural network based commodity identification method according to claim 4, wherein the neural network based commodity identification method comprises the steps of:
(d3) collecting the commodities in the first commodity information when the judgment result in the step (c3) is negative;
(e3) acquiring the collected image set of the identified commodity in the step (d3), and strengthening and training the first neural network by using the collected image set.
20. A neural network-based article recognition apparatus, comprising:
the camera shooting unit is used for acquiring images containing the to-be-detected commodity, wherein the images at least comprise first images to Nth images with different angles and/or depths of field; n is more than or equal to 2;
an identification information unit, configured to input the first image into the first neural network, where the first neural network outputs first commodity information; inputting the Nth image into the first neural network, and outputting Nth commodity information by the first neural network;
an identification determination unit configured to determine whether the nth commodity information is included in the first commodity information; if the judgment result is yes, outputting the first commodity information as the to-be-detected commodity information; if the judgment result is negative, outputting a feedback prompt;
the display unit is used for outputting the information of the to-be-detected commodity and the feedback prompt;
the camera shooting unit is connected with the identification information unit, the identification information unit is connected with the identification judging unit, and the identification judging unit is connected with the display unit;
the identification information unit and the identification judgment unit are used for identifying and judging the commodity according to the commodity identification method based on the neural network of any one of claims 1 to 19.
21. The neural network-based article identification device of claim 20, wherein the neural network-based article identification device comprises a stage, the stage comprising a weight sensor for measuring the total weight of the article on the stage;
the weight sensor is in data connection with the identification information unit.
22. A self-service checkout counter, characterized in that the self-service checkout counter performs merchandise identification using the neural network based merchandise identification method of any one of claims 1 to 19.
23. A self-service checkout counter wherein the neural network based article identification appliance of any one of claims 20 or 21 is employed.
CN201810135196.3A 2017-09-27 2018-02-09 Commodity identification method and device based on neural network and self-service cash register Expired - Fee Related CN108320404B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2018/077651 WO2019062017A1 (en) 2017-09-27 2018-02-28 Method, device, and self-service checkout counter for performing product recognition on the basis of neural network

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710891007 2017-09-27
CN2017108910070 2017-09-27

Publications (2)

Publication Number Publication Date
CN108320404A CN108320404A (en) 2018-07-24
CN108320404B true CN108320404B (en) 2020-04-03

Family

ID=62775868

Family Applications (6)

Application Number Title Priority Date Filing Date
CN201810050328.2A Pending CN108269369A (en) 2017-09-27 2018-01-18 Settle accounts case and its settlement method
CN201820085363.3U Expired - Fee Related CN209132890U (en) 2017-09-27 2018-01-18 Settle accounts case
CN201810135196.3A Expired - Fee Related CN108320404B (en) 2017-09-27 2018-02-09 Commodity identification method and device based on neural network and self-service cash register
CN201811134347.XA Active CN109559458B (en) 2017-09-27 2018-09-27 Neural network-based commodity identification cash registering method and self-service cash registering desk
CN201811133576.XA Active CN109559457B (en) 2017-09-27 2018-09-27 Neural network-based commodity identification cash registering method and self-service cash registering desk
CN201811133556.2A Active CN109559454B (en) 2017-09-27 2018-09-27 Neural network-based commodity identification cash registering method and self-service cash registering desk

Family Applications Before (2)

Application Number Title Priority Date Filing Date
CN201810050328.2A Pending CN108269369A (en) 2017-09-27 2018-01-18 Settle accounts case and its settlement method
CN201820085363.3U Expired - Fee Related CN209132890U (en) 2017-09-27 2018-01-18 Settle accounts case

Family Applications After (3)

Application Number Title Priority Date Filing Date
CN201811134347.XA Active CN109559458B (en) 2017-09-27 2018-09-27 Neural network-based commodity identification cash registering method and self-service cash registering desk
CN201811133576.XA Active CN109559457B (en) 2017-09-27 2018-09-27 Neural network-based commodity identification cash registering method and self-service cash registering desk
CN201811133556.2A Active CN109559454B (en) 2017-09-27 2018-09-27 Neural network-based commodity identification cash registering method and self-service cash registering desk

Country Status (2)

Country Link
CN (6) CN108269369A (en)
WO (1) WO2019062017A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20220309524A1 (en) * 2021-03-25 2022-09-29 Toshiba Tec Kabushiki Kaisha Information processing device, method, and behavior analysis system

Families Citing this family (28)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108986357A (en) * 2018-08-21 2018-12-11 深圳码隆科技有限公司 Merchandise news determines method, system and self-service system
CN109035630A (en) * 2018-08-21 2018-12-18 深圳码隆科技有限公司 Commodity information identification method and system
CN109472205B (en) * 2018-10-09 2021-07-30 深兰科技（上海）有限公司 Commodity identification method, commodity identification device, and storage medium
WO2020079651A1 (en) * 2018-10-17 2020-04-23 Supersmart Ltd. Imaging used to reconcile cart weight discrepancy
CN111220251A (en) * 2018-11-26 2020-06-02 重庆小雨点小额贷款有限公司 Method and device for determining number of cultured products, terminal and storage medium
CN111222388B (en) * 2018-12-11 2023-09-19 图灵通诺（北京）科技有限公司 Settlement method and system based on visual recognition
CN109684950A (en) * 2018-12-12 2019-04-26 联想(北京)有限公司 A kind of processing method and electronic equipment
CN109360358A (en) * 2018-12-25 2019-02-19 北京旷视科技有限公司 Payment devices, payment system and self-service accounts settling method
CN109711473A (en) * 2018-12-29 2019-05-03 北京沃东天骏信息技术有限公司 Item identification method, equipment and system
CN109741144B (en) * 2019-01-04 2021-08-10 南京旷云科技有限公司 Commodity verification method and device, host and equipment
CN109859418A (en) * 2019-01-23 2019-06-07 王强 A kind of shopping settlement method in unmanned market
CN109840503B (en) * 2019-01-31 2021-02-26 深兰科技（上海）有限公司 Method and device for determining category information
CN109977826B (en) * 2019-03-15 2021-11-02 百度在线网络技术（北京）有限公司 Object class identification method and device
CN109872168A (en) * 2019-03-15 2019-06-11 南京亿猫信息技术有限公司 A kind of anti-cheating system based on shopping tool
CN111368613B (en) * 2019-05-22 2023-04-25 深圳深知未来智能有限公司 Novel rapid loading method for intelligent unmanned sales counter
CN110956459A (en) * 2019-11-28 2020-04-03 浙江由由科技有限公司 Commodity processing method and system
CN111126384A (en) * 2019-12-12 2020-05-08 创新奇智(青岛)科技有限公司 Commodity classification system and method based on feature fusion
CN110942050A (en) * 2019-12-20 2020-03-31 华南理工大学 Automatic vending machine commodity identification system based on image processing
US11429800B2 (en) 2020-05-13 2022-08-30 Hong Fu Jin Precision Industry (Wuhan) Co., Ltd. Object recognition system and related device
CN111724546A (en) * 2020-05-20 2020-09-29 安徽谊品弘科技有限公司 Give birth to bright supermarket intelligence and receive silver-colored platform
CN111860629A (en) * 2020-06-30 2020-10-30 北京滴普科技有限公司 Jewelry classification system, method, device and storage medium
CN112132105A (en) * 2020-10-09 2020-12-25 深圳市智百威科技发展有限公司 Cash receiving method based on commodity visual intelligent identification
CN112466068B (en) * 2020-11-26 2021-07-09 融讯伟业（北京）科技有限公司 Intelligent weighing device and intelligent weighing method based on computer vision technology
CN113178032A (en) * 2021-03-03 2021-07-27 北京迈格威科技有限公司 Video processing method, system and storage medium
CN112700312A (en) * 2021-03-24 2021-04-23 浙江口碑网络技术有限公司 Method, server, client and system for settling account of object
CN113627393B (en) * 2021-09-09 2024-03-29 河北工业大学 Commodity identification method based on dual neural network and intelligent vending system
CN114267139A (en) * 2021-12-13 2022-04-01 湖南省金河计算机科技有限公司 Intelligent POS machine system based on Internet of things, control method thereof and electronic equipment
CN114399619A (en) * 2022-01-14 2022-04-26 南京苏胜天信息科技有限公司 Machine vision image recognition system and processing method thereof

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102326187A (en) * 2008-12-23 2012-01-18 数据逻辑扫描公司 Method and system for identifying and tallying objects
CN104077842A (en) * 2014-07-02 2014-10-01 浙江大学 Freestyle restaurant self-service payment device based on image identification and application method of device
CN104103135A (en) * 2014-07-17 2014-10-15 张国铭 Radio frequency identification based goods automatic clearing error correction system and method
CN106203533A (en) * 2016-07-26 2016-12-07 厦门大学 The degree of depth based on combined training study face verification method

Family Cites Families (34)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5965861A (en) * 1997-02-07 1999-10-12 Ncr Corporation Method and apparatus for enhancing security in a self-service checkout terminal
JP2009163331A (en) * 2007-12-28 2009-07-23 Toshiba Tec Corp Merchandise sales data processor and computer program
CN101477729A (en) * 2008-12-30 2009-07-08 于忠清 Self-help meal sale system and information processing method of the system
CN101552911B (en) * 2009-04-14 2012-09-26 重庆市海普软件产业有限公司 A remote outdoor monitoring apparatus and automatic monitoring control method
US8615476B2 (en) * 2009-04-15 2013-12-24 University Of Southern California Protecting military perimeters from approaching human and vehicle using biologically realistic neural network
CN102013019A (en) * 2010-12-03 2011-04-13 深圳市乐州光电技术有限公司 Information image recognition system and method
JP5554796B2 (en) * 2011-09-06 2014-07-23 東芝テック株式会社 Information processing apparatus and program
CN103093208B (en) * 2013-01-23 2016-04-13 中国科学技术大学 A kind of method and system of fruits and vegetables identification
CN104282094A (en) * 2013-07-02 2015-01-14 张�杰 Automatic cash register used in supermarket
CN103424404A (en) * 2013-08-01 2013-12-04 谢绍鹏 Material quality detection method and system
US10242036B2 (en) * 2013-08-14 2019-03-26 Ricoh Co., Ltd. Hybrid detection recognition system
CN103617681A (en) * 2013-11-11 2014-03-05 青岛中科英泰商用系统有限公司 Supermarket self-service settlement method and device with reminding function and multiple loss prevention measures
CN103632463A (en) * 2013-11-14 2014-03-12 成都博约创信科技有限责任公司 Settlement method based on image identification technology
JP6254895B2 (en) * 2014-04-18 2017-12-27 東芝テック株式会社 Reading apparatus and merchandise sales data processing apparatus
CN104077577A (en) * 2014-07-03 2014-10-01 浙江大学 Trademark detection method based on convolutional neural network
CN104240411A (en) * 2014-07-23 2014-12-24 南京工程学院 Intelligent shopping device for supermarket commodities in bulk
JP6451274B2 (en) * 2014-12-10 2019-01-16 カシオ計算機株式会社 Product processing system, product processing method and program
US10074041B2 (en) * 2015-04-17 2018-09-11 Nec Corporation Fine-grained image classification by exploring bipartite-graph labels
CN204706039U (en) * 2015-05-29 2015-10-14 杭州晟元芯片技术有限公司 A kind of bar code identifying device based on many camera lenses
CN106874296B (en) * 2015-12-14 2021-06-04 阿里巴巴集团控股有限公司 Method and device for identifying style of commodity
CN105719188B (en) * 2016-01-22 2017-12-26 平安科技（深圳）有限公司 The anti-method cheated of settlement of insurance claim and server are realized based on plurality of pictures uniformity
CN106169135A (en) * 2016-02-01 2016-11-30 唐超(北京)科技有限公司 Self-checkout loss prevention method of calibration and system
JP6283806B2 (en) * 2016-06-01 2018-02-28 サインポスト株式会社 Information processing system
CN106326852A (en) * 2016-08-18 2017-01-11 无锡天脉聚源传媒科技有限公司 Commodity identification method and device based on deep learning
CN106384087A (en) * 2016-09-05 2017-02-08 大连理工大学 Identity identification method based on multi-layer network human being features
CN106446937A (en) * 2016-09-08 2017-02-22 天津大学 Multi-convolution identifying system for AER image sensor
JP6165950B1 (en) * 2016-09-20 2017-07-19 ヤフー株式会社 Information processing apparatus, information processing method, and information processing program
CN106340137A (en) * 2016-11-14 2017-01-18 贵州师范学院 Automatic weighing machine based on deep learning and program control method thereof
CN106529494A (en) * 2016-11-24 2017-03-22 深圳市永达电子信息股份有限公司 Human face recognition method based on multi-camera model
CN106886795B (en) * 2017-02-17 2021-01-15 北京一维弦科技有限责任公司 Object identification method based on salient object in image
CN106960214B (en) * 2017-02-17 2020-11-20 北京一维弦科技有限责任公司 Object recognition method based on image
CN106959149A (en) * 2017-04-05 2017-07-18 西安电子科技大学 Fruits and vegetables are weighed and valuation intelligent electronic-scale automatically
CN107122730A (en) * 2017-04-24 2017-09-01 乐金伟 Free dining room automatic price method
CN107329925A (en) * 2017-06-29 2017-11-07 卜涛 A kind of intelligent electronic-scale, supermarket's pricing system and method using it

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102326187A (en) * 2008-12-23 2012-01-18 数据逻辑扫描公司 Method and system for identifying and tallying objects
CN104077842A (en) * 2014-07-02 2014-10-01 浙江大学 Freestyle restaurant self-service payment device based on image identification and application method of device
CN104103135A (en) * 2014-07-17 2014-10-15 张国铭 Radio frequency identification based goods automatic clearing error correction system and method
CN106203533A (en) * 2016-07-26 2016-12-07 厦门大学 The degree of depth based on combined training study face verification method

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
基于PCA的实时人脸识别系统的设计与实现;李姗姗;《中国优秀硕士学位论文全文数据库》;20131130;第I138-777页 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20220309524A1 (en) * 2021-03-25 2022-09-29 Toshiba Tec Kabushiki Kaisha Information processing device, method, and behavior analysis system

Also Published As

Publication number Publication date
CN109559454A (en) 2019-04-02
CN109559457A (en) 2019-04-02
WO2019062017A1 (en) 2019-04-04
CN109559458B (en) 2021-09-21
CN108320404A (en) 2018-07-24
CN109559458A (en) 2019-04-02
CN108269369A (en) 2018-07-10
CN109559454B (en) 2021-09-21
CN209132890U (en) 2019-07-19
CN109559457B (en) 2021-09-21

Similar Documents

Publication Publication Date Title
CN108320404B (en) Commodity identification method and device based on neural network and self-service cash register
CN108269371B (en) Automatic commodity settlement method and device and self-service cash register
US11151427B2 (en) Method and apparatus for checkout based on image identification technique of convolutional neural network
CN108335408B (en) Article identification method, device and system for vending machine and storage medium
US11501523B2 (en) Goods sensing system and method for goods sensing based on image monitoring
CN107464116B (en) Order settlement method and system
WO2019165892A1 (en) Automatic vending method and apparatus, and computer-readable storage medium
US7909248B1 (en) Self checkout with visual recognition
US11176597B2 (en) Associating shoppers together
WO2019062812A1 (en) Human-computer interaction device for automatic payment and use thereof
JP6549558B2 (en) Sales registration device, program and sales registration method
EP3901841A1 (en) Settlement method, apparatus, and system
CN111222870B (en) Settlement method, device and system
US11488126B2 (en) Cashier fraud detecting system and method and product image selection generation for artificial neural network learning related applications
CN112464697A (en) Vision and gravity sensing based commodity and customer matching method and device
CN111079478A (en) Unmanned goods selling shelf monitoring method and device, electronic equipment and system
CN111126990A (en) Automatic article identification method, settlement method, device, terminal and storage medium
CN111428743B (en) Commodity identification method, commodity processing device and electronic equipment
CN111222388B (en) Settlement method and system based on visual recognition
CN112507762A (en) Intelligent vending machine control method and system based on commodity image recognition
CN106600253A (en) Checkout Management System

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20200403

Termination date: 20210209