WO2019019291A1 - 基于卷积神经网络的图像识别技术的结算方法和装置 - Google Patents

基于卷积神经网络的图像识别技术的结算方法和装置 Download PDF

Info

Publication number
WO2019019291A1
WO2019019291A1 PCT/CN2017/100999 CN2017100999W WO2019019291A1 WO 2019019291 A1 WO2019019291 A1 WO 2019019291A1 CN 2017100999 W CN2017100999 W CN 2017100999W WO 2019019291 A1 WO2019019291 A1 WO 2019019291A1
Authority
WO
WIPO (PCT)
Prior art keywords
classification
settlement
classification result
model
trained
Prior art date
Application number
PCT/CN2017/100999
Other languages
English (en)
French (fr)
Inventor
吴一黎
Original Assignee
图灵通诺（北京）科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 图灵通诺（北京）科技有限公司 filed Critical 图灵通诺（北京）科技有限公司
Priority to JP2018567956A priority Critical patent/JP6709862B6/ja
Priority to US16/328,904 priority patent/US10853702B2/en
Publication of WO2019019291A1 publication Critical patent/WO2019019291A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q20/00—Payment architectures, schemes or protocols
    • G06Q20/08—Payment architectures
    • G06Q20/20—Point-of-sale [POS] network systems
    • G06Q20/208—Input by product or record sensing, e.g. weighing or scanner processing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/243—Classification techniques relating to the number of classes
    • G06F18/2431—Multiple classes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/254—Fusion techniques of classification results, e.g. of results related to same input data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G06N20/10—Machine learning using kernel methods, e.g. support vector machines [SVM]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/06—Buying, selling or leasing transactions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/24—Aligning, centring, orientation detection or correction of the image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G06V10/809—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of classification results, e.g. where the classifiers operate on the same input data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/64—Three-dimensional objects
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07G—REGISTERING THE RECEIPT OF CASH, VALUABLES, OR TOKENS
    • G07G1/00—Cash registers
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07G—REGISTERING THE RECEIPT OF CASH, VALUABLES, OR TOKENS
    • G07G1/00—Cash registers
    • G07G1/0036—Checkout procedures
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/90—Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/07—Target detection

Definitions

  • the invention belongs to the field of image recognition technology, and in particular relates to a settlement method and device for image recognition technology based on a convolutional neural network.
  • the first is a barcode-based settlement method, which is to identify an item by scanning a barcode on the item, and then settle the identified item, and the scanning operation is performed by the cashier.
  • a barcode-based settlement method which is to identify an item by scanning a barcode on the item, and then settle the identified item, and the scanning operation is performed by the cashier.
  • Complete or customer self-service completion The method has the following defects: scanning is troublesome, labor-intensive, and has certain requirements for operation, and generally only one item can be scanned at a time, and multiple items cannot be scanned at the same time, and the efficiency is low.
  • the second method is an RFID-based settlement method, in which a radio frequency small module that does not require a battery is attached to the commodity, and when the commodity passes through the settlement counter, the settlement station transmits a wireless signal to the commodity, and the radio frequency small module receives the The signal is then fed back to a settlement counter with the ID information of the item and then settled accordingly.
  • the method has the following defects: it is troublesome to attach a radio frequency small module to each product, and if the radio frequency small module is dropped from the commodity, whether it is naturally dropped or artificially torn off, the merchant will cause losses.
  • the product is a metal product
  • the RFID is attached thereto, and there may be a problem that the signal is shielded.
  • an aspect of the present invention provides a settlement method of an image recognition technology based on a convolutional neural network, comprising: an acquisition step of collecting a single commodity to be classified placed on a settlement counter. a plurality of pictures, the plurality of pictures are in one-to-one correspondence with the plurality of shooting angles; the target detecting step performs target detection on each of the pictures to obtain a rectangular area image, and the rectangular area image is a rectangular area containing the product Corresponding image; first classification step, root According to the plurality of rectangular area images and the pre-trained first-class classification model, a plurality of primary classification results are obtained correspondingly, the first-level classification model is an image recognition technology architecture based on a convolutional neural network and is trained by all commodities in the shopping place.
  • a model obtaining, according to a plurality of the primary classification results and a pre-trained first-order linear regression model, a plurality of primary classification results of the image; a confirmation step, using the primary classification result as a first classification result; Steps: settlement according to the first classification result.
  • the settlement method further comprises: a second classification step, if the primary classification result is a similar commodity, according to the plurality of The rectangular area image and the pre-trained secondary classification model respectively obtain a plurality of secondary classification results, and then acquire a plurality of the plurality of pictures according to the plurality of the secondary classification results and the pre-trained secondary linear regression model.
  • the secondary classification model is a model based on the image recognition technology architecture of the convolutional neural network in advance and trained by similar commodities in the shopping place, otherwise execution Confirm the steps.
  • the settlement method further comprises: pre-training according to the plurality of rectangular area images and the first classification result
  • the support vector machine model determines whether the products in the picture are consistent with the first classification result, and if they are consistent, the settlement step is performed, otherwise the customer is reminded that the goods are out of stock.
  • one camera is disposed directly above the article to be classified to take a picture from the top right to take a picture to collect a picture; and to arrange 4 around the item to be classified
  • the cameras capture pictures by taking pictures of the goods from obliquely above.
  • a settlement apparatus based on a convolutional neural network image recognition technology, including: an acquisition apparatus, configured to collect a plurality of pictures of a single item to be classified placed on a settlement counter, and a plurality of The image is in one-to-one correspondence with the plurality of shooting angles; the object detecting means is configured to perform object detection on each of the images to obtain a rectangular area image, the rectangular area image being an image corresponding to the rectangular area containing the commodity; a classifying device, configured to acquire a plurality of primary classification results according to the plurality of rectangular area images and a pre-trained first-level classification model, wherein the first-level classification model is a convolutional neural network-based image recognition technology architecture and is purchased a model for training all commodities in the site, obtaining a plurality of primary classification results of the images according to a plurality of the primary classification results and a pre-trained first-order linear regression model; and confirming means for using the primary classification results as a first classification result; and a settlement
  • the settlement device further includes: a second classification device, configured to: according to the plurality of rectangular region images and pre-trained if the primary classification result is a similar commodity a secondary classification model, corresponding to obtaining a plurality of secondary classification results, and then obtaining a plurality of secondary classification results of the pictures according to the plurality of the secondary classification results and the pre-trained secondary linear regression model, and
  • the classification result is the first classification result
  • the secondary classification model is a model based on the image recognition technology architecture of the convolutional neural network and trained by similar commodities in the shopping place; correspondingly, the settlement device is in the first
  • the settlement is performed according to the secondary classification result acquired by the second classification device, otherwise, the settlement is performed according to the primary classification result obtained by the first classification device.
  • the settlement device further includes: determination means for judging based on the plurality of rectangular area images and a pre-trained support vector machine model corresponding to the first classification result Whether the goods in the picture are consistent with the first classification result; and the selection device, if the settlement device is called in unison, otherwise the customer is reminded that the goods are out of stock.
  • the collection device is a camera, and one camera is disposed directly above the item to be classified to take a picture from the top right to take a picture to collect a picture; Four cameras are arranged around the classified products to take pictures of the goods from obliquely above to collect pictures.
  • a still further aspect of the present invention provides a settlement apparatus based on a convolutional neural network image recognition technology, comprising: a camera for collecting a plurality of pictures of a single item to be classified placed on a settlement counter, a plurality of said The picture corresponds to a plurality of shooting angles in one-to-one; a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to: perform target detection on each of the pictures to obtain a rectangular area image, The rectangular area image is an image corresponding to a rectangular area including an item; and correspondingly acquiring a plurality of primary classification results according to the plurality of the rectangular area images and the pre-trained first-level classification model, the first-level classification model is based on a volume a model of the image recognition technology of the neural network and trained by all the commodities in the shopping place, and obtaining a plurality of first-class classification results of the pictures according to the plurality of the primary classification results and the pre-trained first-order linear regression model;
  • the processor is further configured to: if the primary classification result is a similar commodity, according to the plurality of rectangular region images and the pre-trained secondary classification model, Corresponding to obtain multiple sub-category results, and then based on a plurality of said sub-category results and advance
  • the trained secondary linear regression model obtains the secondary classification results of the plurality of pictures, and uses the secondary classification result as the first classification result, and the secondary classification model is the image recognition based on the convolutional neural network in advance A model of technical architecture and training of similar commodities in a shopping place, otherwise the first-class classification result is used as the first classification result.
  • FIG. 1 is a schematic flowchart of a settlement method of an image recognition technology based on a convolutional neural network according to an embodiment of the present invention
  • FIG. 2 is a schematic flowchart of a settlement method of an image recognition technology based on a convolutional neural network according to another embodiment of the present invention
  • FIG. 3 is a schematic flowchart diagram of a settlement method of an image recognition technology based on a convolutional neural network according to another embodiment of the present invention.
  • FIG. 4 is a schematic structural diagram of a settlement apparatus of an image recognition technology based on a convolutional neural network according to an embodiment of the present invention.
  • an embodiment of the present invention provides a method for settlement of an image recognition technology based on a convolutional neural network, which includes:
  • the collecting step 101 a plurality of pictures of a single item to be classified placed on the settlement counter are collected, and the plurality of pictures are in one-to-one correspondence with the plurality of shooting angles.
  • the target detecting step 102 performs target detection on each picture to acquire a rectangular area image, which is an image corresponding to a rectangular area containing the product.
  • a first classification step 103 corresponding to acquiring a plurality of primary classification results according to the plurality of rectangular area images and the pre-trained first-level classification model, wherein the first-level classification model is an image recognition technology architecture based on a convolutional neural network and is included in the shopping place.
  • the model of commodity training acquires the primary classification results of multiple images based on multiple primary classification results and pre-trained first-order linear regression models.
  • the first classification result is used as the first classification result.
  • the settlement step 105 performs settlement based on the first classification result.
  • each image is processed by a first-level classification model, and corresponding plurality of classification results are obtained, and then data fusion is performed on the plurality of classification results to output a final result, that is, utilization
  • the linear regression model is processed to obtain which product the product is, thereby improving the accuracy of product identification.
  • another embodiment of the present invention provides a method for settlement of an image recognition technology based on a convolutional neural network, the method comprising the following steps:
  • Step 201 Collect a plurality of pictures of a single item to be classified placed on the settlement counter, and the plurality of pictures are in one-to-one correspondence with the plurality of shooting angles.
  • the product is photographed from a certain shooting angle, and a picture can be acquired. Since one shooting angle corresponds to one picture, different shooting angles are changed, so that pictures corresponding to different shooting angles can be acquired.
  • the number of shooting angles is multiple, multiple images can be collected, which ensures that key information for identifying the product is captured, which means that the appearance of the product plays an important role in product identification (or classification).
  • Information for example, when the product is mineral water, there are many types of mineral water. Different types of mineral water are mainly based on a pattern of plastic paper stuck on the mineral water bottle, which is the key information of mineral water.
  • a plurality of cameras such as five, four, and six, are arranged on the settlement counter in such a manner as to form a plurality of shooting angles, and the number of cameras is the same as the number of sheets of the picture.
  • the number of cameras is 5, and the shooting angle of the camera is described. It may be: one camera is arranged directly above the item to be classified, and the product is photographed from directly above and evenly arranged around the goods to be classified. The cameras are all photographed from obliquely above; or, five cameras are evenly arranged around the items to be classified, two of which take pictures from the product obliquely downward, and the other three cameras are diagonally upward. The product is photographed, and the embodiment does not limit the number and arrangement of the cameras.
  • the shooting angle in this article can refer to one factor of shooting direction, and can also refer to two factors: shooting direction and shooting distance. It can also refer to other factors or other quantity factors. The embodiment does not limit this.
  • the collection action can be triggered by a scale arranged on the settlement counter.
  • the scale is a scale with a pressure sensor, and the change of the weight sensed by the scale determines whether to trigger the shooting.
  • the camera starts to take pictures, which ensures that the picture that meets the requirements is taken, that is, the picture of the moment is taken after the customer puts the item on.
  • the triggering of the camera photographing action may adopt the technical means of graphic recognition and computer vision.
  • the camera firstly observes and photographs the area where the article is placed, for example, when the customer's hand is found, the user puts down a thing, and then the handle is held. Take it off, when capturing such an action from the video, go to the instruction to take a photo, that is, trigger the camera to take a photo.
  • Step 202 Perform target detection on each picture to obtain a rectangular area image, which is an image corresponding to a rectangular area containing an item.
  • a rectangular frame (or rectangular area) containing the product is drawn on each picture, and the image corresponding to the rectangular frame is an image for classifying the product. And output to step 203.
  • the number of pictures is five, five images corresponding to the rectangular area containing the product are obtained from the five pictures.
  • Step 203 Acquire a plurality of primary classification results according to the plurality of rectangular area images and the pre-trained first-level classification model, and the pre-trained first-level classification model is an image recognition technology architecture based on a convolutional neural network and is included in the shopping place. Model of commodity training.
  • the process of collecting data includes: 1) taking a photo of all the goods in the shopping place from various angles and in various postures to obtain a large number of photos. 2) Then mark these photos: mark the location, size, and category of the items in the photo.
  • the data included in the data set refers to the aforementioned photos and the annotations made on these photos.
  • the first-level classification model is a model based on the convolutional neural network image recognition technology architecture, and the first-level classification model is trained using the data of all commodities in the shopping place, and the training can be performed by gradient descent.
  • the trained first-level classification model classifies the products in each rectangular area image to obtain the primary classification result, which is an n-dimensional vector, where n represents the total quantity of goods in the shopping place, and each element in the vector
  • the meaning indicates that the primary classification model considers the probability that a single commodity to be classified belongs to each of the n commodities, and which component in the vector has the largest value, which means that the model considers that the commodity to be classified is the commodity corresponding to the commodity.
  • the rectangular area image is five sheets
  • the number of primary classification results is five n-dimensional vectors.
  • Step 204 Acquire a first-level classification result of multiple pictures according to a plurality of primary classification results and a pre-trained first-order linear regression model. If the primary classification result is a similar commodity, perform the following step 205; otherwise, the primary classification result is performed. As the first classification result.
  • step 203 when the first-level classification model is trained, the primary classification result output by the primary classification model is input as a primary linear regression model, and the correct classification of the commodities included in the image corresponding to the primary classification result is taken as The output of the first-order linear regression model is used to train the first-order linear regression model.
  • the trained first-order linear regression model performs data fusion on multiple primary classification results to obtain a first-level classification result, which indicates that the first-order linear regression model predicts which category of the goods in the shopping place in the image.
  • Step 205 Acquire multiple secondary classification results according to multiple rectangular area images and a pre-trained secondary classification model, and then acquire multiple images according to multiple secondary classification results and pre-trained secondary linear regression models.
  • the results of the classification are classified, and the results of the second classification are used as the first classification result.
  • the secondary classification model is a model based on the image recognition technology architecture of the convolutional neural network and trained by the commodity in the similar commodity group in the shopping place.
  • the secondary classification model is trained by using data of similar commodities in the data set established in step 203, and the training can be performed by gradient descent.
  • the difference between the secondary classification model and the primary classification model is that the data used in the training is different.
  • the data used by the primary classification model is the data of all the commodities in the shopping place, and the data used by the secondary classification model is the similar commodity data in the shopping place. .
  • the trained secondary classification model classifies the products in each rectangular area image to obtain the secondary classification result.
  • the secondary classification result is also an m-dimensional vector.
  • the meaning of each element in the vector indicates that the secondary classification model considers The probability that a single item to be classified belongs to each of the m similar items.
  • the rectangular area image is 5 sheets
  • the number of secondary classification results is 5 m-dimensional vectors, m is less than or equal to n, and represents the total number of similar items in the shopping place.
  • a secondary classification model can be trained for all groups of similar commodities.
  • a secondary classification model is trained for each group of similar commodities. At this time, if the primary classification result is a similar commodity, then the The secondary classification model corresponding to the primary classification result.
  • the secondary classification result output by the secondary classification model is used as the input of the secondary linear regression model, and the correct classification of the commodities included in the corresponding image of the secondary classification result is used as the secondary linear regression model.
  • Output to train a two-level linear regression model The trained secondary linear regression model performs data fusion on multiple sub-category results, and obtains a secondary classification result, which is used as the first classification result.
  • the secondary classification result represents the secondary linear regression model prediction image.
  • the item is which category of the item in the shopping place.
  • Step 206 Perform settlement according to the first classification result.
  • the customer pays the fee for the product placed on the settlement counter, and the product name can be displayed through the display on the settlement counter.
  • commodity prices, payment fees you can also promptly give the product name to the customer by voice.
  • the payment can be completed by scanning the two-dimensional code displayed on the display or by aligning the QR code of the account on the mobile terminal with the scanning code terminal on the settlement counter.
  • the method further includes:
  • Step 207 Determine, according to the plurality of rectangular area images and the pre-trained support vector machine model corresponding to the first classification result, whether the products in the picture are consistent with the first classification result, and if yes, perform step 206 above, otherwise Customer goods are not in stock.
  • a machine learning model (or a support vector machine model) of a support vector machine is constructed for each item in the shopping place, that is, each item has a support vector machine model corresponding thereto, and the data in the data set established by step 203 is used. The corresponding data is trained on the model.
  • the first-level classification model there is an intermediate calculation result, which is a vector with a length of 1024, which can be regarded as a feature of the picture, from which a vector is constructed to determine whether the product belongs to a certain Support vector machine model for each category of goods.
  • the support vector machine model corresponding to the first classification result is adopted.
  • the type judges the product contained in each rectangular area image to obtain a preliminary judgment result, and the preliminary judgment result indicates whether the item in the picture is consistent with the first classification result. If the number of rectangular area images is plural, the preliminary judgment results are plural. If the number of the plurality of preliminary judgment results is greater than or equal to the preset threshold, it is determined that the item in the picture is consistent with the first classification result; otherwise, the customer is reminded that the item has no inventory, that is, the item cannot be settled. For example, if the number of sheets in the picture is 5 and the preset threshold is 3, the preliminary judgment results are: consistent, consistent, inconsistent, inconsistent, and consistent, and the number of matches is 3.
  • the support vector machine model determines that the item in the picture is consistent with the first classification result, and considers that the foregoing classification process is correct, that is, identifying a correct item, at this time, the customer The item can be obtained through settlement. If the five preliminary judgment results are inconsistent, inconsistent, inconsistent, inconsistent, and consistent, the number of matches is 1. Since the consistent quantity is less than the preset threshold, the support vector machine model determines that the item in the picture is inconsistent with the first classification result, and the foregoing classification process is considered unsuccessful, that is, the recognition fails, and the voice and/or picture are passed at this time. Tip: 'There is no such item in Curry, the item cannot be identified', etc. to remind the customer that the item cannot be settled.
  • An embodiment of the present invention provides a settlement apparatus based on a convolutional neural network image recognition technology, which includes:
  • the collecting device 301 is configured to collect a plurality of pictures of a single item to be classified placed on the settlement counter, and the plurality of pictures are in one-to-one correspondence with the plurality of shooting angles.
  • the collecting device 301 is a camera, and one camera is arranged directly above the item to be classified to take pictures from the top right to collect pictures; four cameras are arranged around the items to be classified to Take a picture of the item diagonally above to capture the picture.
  • the target detecting means 302 is configured to perform target detection on each picture to acquire a rectangular area image, and the rectangular area image is an image corresponding to the rectangular area containing the product.
  • the first classifying device 303 is configured to obtain a plurality of primary classification results according to the plurality of rectangular region images and the pre-trained first-class classification model, and the first-level classification model is an image recognition technology architecture based on a convolutional neural network and is used in a shopping place.
  • the settlement device 305 is configured to perform settlement according to the first classification result.
  • the settlement device further includes: a second classification device, configured to obtain a plurality of secondary classification results according to the plurality of rectangular region images and the pre-trained secondary classification model if the primary classification result is a similar commodity, and then according to the plurality of secondary classification results
  • the secondary classification results and the pre-trained secondary linear regression model obtain the secondary classification results of multiple images, and the secondary classification results are used as the first classification result, and the secondary classification model is the image based on the convolutional neural network in advance. Identifying a technical architecture and training a similar product in a shopping mall; correspondingly, when the primary classification result is a similar commodity, the settlement device performs settlement according to the secondary classification result obtained by the second classification device, otherwise, according to the first classification device The first-level classification results are settled.
  • the settlement device further includes: determining means for determining whether the item in the picture is consistent with the first classification result according to the plurality of rectangular area images and the pre-trained support vector machine model corresponding to the first classification result; and selecting If the device calls the settlement device 305 in unison, the customer is reminded that the goods are out of stock.
  • the specific descriptions of the collection device 301 can be referred to the related contents of the steps 101 and 201 in the foregoing embodiment.
  • the target detection device 302 refer to the related content of steps 102 and 203 in the above embodiment.
  • a sorting device 303 refer to the related content of the steps 103 and 203 in the foregoing embodiment.
  • the confirming device 304 refer to the related content of the steps 104 and 204 in the foregoing embodiment.
  • the detailed description of the second classification device refer to the related content of the step 205 in the foregoing embodiment.
  • the determination device and the selection device refer to step 207 in the foregoing embodiment. The relevant content, will not be repeated here.
  • An embodiment of the present invention provides a settlement apparatus based on a convolutional neural network image recognition technology, including: a camera, a processor, and a memory.
  • the camera is used to collect a plurality of pictures of a single item to be classified placed on the settlement counter, and the plurality of pictures are in one-to-one correspondence with a plurality of shooting angles.
  • the memory is used to store instructions executable by the processor.
  • the processor is configured to: perform target detection on each picture to obtain a rectangular area image, and the rectangular area image is an image corresponding to the rectangular area containing the commodity; according to the multiple rectangular area image and the pre-trained first-class classification model, correspondingly acquiring A plurality of primary classification results, the first-level classification model is a model based on the image recognition technology architecture of the convolutional neural network and trained by all the commodities in the shopping place, and obtains multiple sheets according to the plurality of primary classification results and the pre-trained first-order linear regression model.
  • the first classification result is used as the first classification result; the settlement is performed according to the first classification result.
  • the processor is further configured to: if the primary classification result is a similar commodity, obtain a plurality of secondary classification results according to the plurality of rectangular region images and the pre-trained secondary classification model, and then according to the plurality of secondary classification results and The pre-trained two-level linear regression model obtains the secondary classification results of multiple images, and uses the secondary classification results as the first classification result.
  • the secondary classification model is based on the image recognition technology architecture based on convolutional neural network and is purchased. The model of similar commodity training in the place, otherwise the first classification result is used as the first classification result.
  • the customer placed a bottle of mineral water on the weighing platform.
  • the weighing scale of the weighing platform detected the weight change and the weight area was stable, data was collected from 5 cameras and 5 photos were collected.
  • the five photos are uniformly scaled to a preset size, and then the object is detected on the five photos by the target detection model.
  • the detection result is to find a rectangular area containing the object on the photo, and then each rectangular area is used.
  • all the probabilistic results are subjected to a first-order linear regression model to obtain the primary classification result of the object in all commodity categories.
  • the primary classification result belongs to a commodity category that needs to be further classified by the secondary classification model
  • the corresponding secondary classification model is retrieved, and each rectangular frame is classified to obtain a probability value of the object belonging to each category, and then the secondary linear regression model is used to obtain the classification result of the object in all commodity categories.
  • the support vector machine model or small model of this category, the rectangular frame in the five photos is judged to determine whether the product belongs to the category. If so, then the item category is returned, and if not, it is determined that the item does not belong to any of the item categories.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Multimedia (AREA)
  • Business, Economics & Management (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Accounting & Taxation (AREA)
  • Mathematical Physics (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Medical Informatics (AREA)
  • Finance (AREA)
  • Databases & Information Systems (AREA)
  • Strategic Management (AREA)
  • General Business, Economics & Management (AREA)
  • Signal Processing (AREA)
  • Development Economics (AREA)
  • Economics (AREA)
  • Marketing (AREA)
  • Image Analysis (AREA)
  • Cash Registers Or Receiving Machines (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

一种基于卷积神经网络的图像识别技术的结算方法和装置，该方法包括采集放置在结算台上的待分类的单个商品的多张图片，多张图片与多个拍摄角度一一对应（101）；对每张图片进行目标检测以获取矩形区域图像，矩形区域图像为与包含商品的矩形区域对应的图像（102）；根据多张矩形区域图像和预先训练的一级分类模型，对应获取多个初级分类结果，一级分类模型为基于卷积神经网络的图像识别技术架构且经购物场所内所有商品训练的模型，根据多个初级分类结果和预先训练的一级线性回归模型获取多张图片的一级分类结果（103）；以一级分类结果作为第一次分类结果（104）；根据第一次分类结果进行结算（105）。装置包括采集装置、目标检测装置、第一分类装置、确认装置和结算装置。该方法能准确识别商品，便于自助结算。

Description

基于卷积神经网络的图像识别技术的结算方法和装置 技术领域
本发明属于图像识别技术领域，特别涉及一种基于卷积神经网络的图像识别技术的结算方法和装置。
背景技术
顾客在超市、餐厅等购物场所看到自己喜欢或需要的商品时，需在结算台进行结算才能得到。
现有技术中，常用的结算方法有两种：第一种是基于条形码的结算方法，该方法是通过扫描商品上的条形码的方式识别商品，然后对识别的商品进行结算，扫描操作由收银员完成或者顾客自助完成。该方法具有如下缺陷：扫描比较麻烦、费人工、对操作有一定的要求，而且一般每次只能扫描一件商品，不能同时扫多件商品、效率低。第二种是基于RFID的结算方法，该方法是在商品上贴一个不需要电池的射频小模块，当该商品通过结算台时，结算台会向该商品发射无线信号，该射频小模块接收到该信号之后会回馈一个信号给结算台，该回馈信号中带有商品的ID信息，然后据此进行结算。该方法具有如下缺陷：需要在每件商品上贴射频小模块、比较麻烦，而且如果射频小模块从商品上掉落，无论自然掉落还是人为撕掉，都会给商家造成损失。此外，当商品为金属商品时，在其上贴附RFID，可能会存在信号被屏蔽问题。
发明内容
为了至少解决现有技术中存在的问题，本发明一方面提供了一种基于卷积神经网络的图像识别技术的结算方法，其包括：采集步骤，采集放置在结算台上的待分类的单个商品的多张图片，多张所述图片与多个拍摄角度一一对应；目标检测步骤，对每张所述图片进行目标检测以获取矩形区域图像，所述矩形区域图像为与包含商品的矩形区域对应的图像；第一分类步骤，根 据多张所述矩形区域图像和预先训练的一级分类模型，对应获取多个初级分类结果，所述一级分类模型为基于卷积神经网络的图像识别技术架构且经购物场所内所有商品训练的模型，根据多个所述初级分类结果和预先训练的一级线性回归模型获取多张所述图片的一级分类结果；确认步骤，以所述一级分类结果作为第一次分类结果；结算步骤，根据所述第一次分类结果进行结算。
在如上所述的结算方法中，优选地，在第一分类步骤之后，确认步骤之前，所述结算方法还包括：第二分类步骤，若所述一级分类结果为相似商品，则根据多张所述矩形区域图像和预先训练的二级分类模型，对应获得多个次级分类结果，再根据多个所述次级分类结果和预先训练的二级线性回归模型获取多张所述图片的二级分类结果，并以所述二级分类结果作为第一次分类结果，所述二级分类模型为预先基于卷积神经网络的图像识别技术架构且经购物场所内相似商品训练的模型，否则执行确认步骤。
在如上所述的结算方法中，优选地，在确认步骤之后，结算步骤之前，所述结算方法还包括：根据多张所述矩形区域图像和与所述第一次分类结果对应的预先训练的支持向量机模型，判断图片中的商品是否与所述第一次分类结果一致，若一致则执行结算步骤，否则提醒顾客商品无库存。
在如上所述的结算方法中，优选地，在待分类的商品的正上方布置1个摄像头，以从正上方向下对所述商品进行拍照来采集图片；在待分类的商品的四周布置4个摄像头，以从斜上方对所述商品进行拍照来采集图片。
本发明另一方面提供了一种基于卷积神经网络的图像识别技术的结算装置，其包括：采集装置，用于采集放置在结算台上的待分类的单个商品的多张图片，多张所述图片与多个拍摄角度一一对应；目标检测装置，用于对每张所述图片进行目标检测以获取矩形区域图像，所述矩形区域图像为与包含商品的矩形区域对应的图像；第一分类装置，用于根据多张所述矩形区域图像和预先训练的一级分类模型，对应获取多个初级分类结果，所述一级分类模型为基于卷积神经网络的图像识别技术架构且经购物场所内所有商品训练的模型，根据多个所述初级分类结果和预先训练的一级线性回归模型获取多张所述图片的一级分类结果；确认装置，用于以所述一级分类结果作为第一次分类结果；和结算装置，用于根据所述第一次分类结果进行结算。
在如上所述的结算装置中，优选地，所述结算装置还包括：第二分类装置，用于若所述一级分类结果为相似商品，则根据多张所述矩形区域图像和预先训练的二级分类模型，对应获得多个次级分类结果，再根据多个所述次级分类结果和预先训练的二级线性回归模型获取多张所述图片的二级分类结果，并以所述二级分类结果作为第一次分类结果，所述二级分类模型为预先基于卷积神经网络的图像识别技术架构且经购物场所内相似商品训练的模型；对应地，所述结算装置在所述一级分类结果为相似商品时，根据所述第二分类装置获取的二级分类结果进行结算，否则根据所述第一分类装置获取的一级分类结果进行结算。
在如上所述的结算装置中，优选地，所述结算装置还包括：判断装置，用于根据多张所述矩形区域图像和与第一次分类结果对应的预先训练的支持向量机模型，判断图片中的商品是否与第一次分类结果一致；和选择装置，若一致调用所述结算装置，否则提醒顾客商品无库存。
在如上所述的结算装置中，优选地，所述采集装置为摄像头，在待分类的商品的正上方布置1个摄像头，以从正上方向下对所述商品进行拍照来采集图片；在待分类的商品的四周布置4个摄像头，以从斜上方对所述商品进行拍照来采集图片。
本发明又一方面提供了一种基于卷积神经网络的图像识别技术的结算装置，其包括：摄像头，用于采集放置在结算台上的待分类的单个商品的多张图片，多张所述图片与多个拍摄角度一一对应；处理器；用于存储处理器可执行的指令的存储器；其中，所述处理器被配置为：对每张所述图片进行目标检测以获取矩形区域图像，所述矩形区域图像为与包含商品的矩形区域对应的图像；根据多张所述矩形区域图像和预先训练的一级分类模型，对应获取多个初级分类结果，所述一级分类模型为基于卷积神经网络的图像识别技术架构且经购物场所内所有商品训练的模型，根据多个所述初级分类结果和预先训练的一级线性回归模型获取多张所述图片的一级分类结果；以所述一级分类结果作为第一次分类结果；根据所述第一次分类结果进行结算。
在如上所述的结算装置中，优选地，所述处理器还被配置为：若所述一级分类结果为相似商品，则根据多张所述矩形区域图像和预先训练的二级分类模型，对应获得多个次级分类结果，再根据多个所述次级分类结果和预先 训练的二级线性回归模型获取多张所述图片的二级分类结果，并以所述二级分类结果作为第一次分类结果，所述二级分类模型为预先基于卷积神经网络的图像识别技术架构且经购物场所内相似商品训练的模型，否则以所述一级分类结果作为第一次分类结果。
本发明实施例通过上述技术方案带来的有益效果如下：
能准确识别商品，便于自助结算。
附图说明
图1为本发明一实施例提供的一种基于卷积神经网络的图像识别技术的结算方法的流程示意图；
图2为本发明另一实施例提供的一种基于卷积神经网络的图像识别技术的结算方法的流程示意图；
图3为本发明又一实施例提供的一种基于卷积神经网络的图像识别技术的结算方法的流程示意图；
图4为本发明实施例提供的一种基于卷积神经网络的图像识别技术的结算装置的结构示意图。
具体实施方式
为使本发明的目的、技术方案和优点更加清楚，下面将结合附图对本发明实施方式作进一步地详细描述。
参见图1，本发明一实施例提供一种基于卷积神经网络的图像识别技术的结算方法，其包括：
采集步骤101，采集放置在结算台上的待分类的单个商品的多张图片，多张图片与多个拍摄角度一一对应。
目标检测步骤102，对每张图片进行目标检测以获取矩形区域图像，矩形区域图像为与包含商品的矩形区域对应的图像。
第一分类步骤103，根据多张矩形区域图像和预先训练的一级分类模型，对应获取多个初级分类结果，一级分类模型为基于卷积神经网络的图像识别技术架构且经购物场所内所有商品训练的模型，根据多个初级分类结果和预先训练的一级线性回归模型获取多张图片的一级分类结果。
确认步骤104，以一级分类结果作为第一次分类结果。
结算步骤105，根据第一次分类结果进行结算。
综上，通过对单个商品采集多张图片，利用一级分类模型对每张图片进行处理，获取对应的多个分类结果，然后对多个分类结果进行数据融合以输出一个最终的结果，即利用线性回归模型进行处理，得到该商品为哪个商品，从而提高了商品识别的准确率。
参见图2，本发明另一实施例提供了一种基于卷积神经网络的图像识别技术的结算方法，该方法包括以下步骤：
步骤201，采集放置在结算台上的待分类的单个商品的多张图片，多张图片与多个拍摄角度一一对应。
在该步骤中，从某一拍摄角度对该商品进行拍照，可以采集一张图片。由于一个拍摄角度与一张图片相对应，变换不同的拍摄角度，从而可以采集与不同的拍摄角度对应的图片。当拍摄角度的数量为多个时，即可采集多张图片，如此能够确保捕捉到用于识别商品的关键信息，该关键信息是指商品外观上对商品识别(或称分类)起重要作用的信息，例如商品为矿泉水时，矿泉水的种类有很多，区分不同种类的矿泉水主要是依靠矿泉水瓶上贴的一圈塑料纸上的图案，该图案即为矿泉水的关键信息。
在结算台上按照形成多个拍摄角度的方式布置多个摄像头，如5个、4个、6个，摄像头的数量与图片的张数相同。以摄像头数量为5个对摄像头的拍摄角度进行说明，可以是：在待分类的商品的正上方布置1个摄像头，从正上方向下对商品进行拍照，在待分类的商品的四周均匀布置4个摄像头，都从斜上方对商品进行拍照；也可以是，在待分类的商品的四周均匀布置5个摄像头，其中两个摄像头从斜向下对商品进行拍照，另外3个摄像头从斜上方对商品进行拍照，本实施例不对摄像头的数量和布置方式进行限定。通常，摄像头的数量越多，则采集的图片数量越多，所有图片上含有的商品的信息也越多，如此有利于商品的分类，但这样会增大硬件的复杂度，增大运算量，所以可以根据实际情况来选择摄像头的数量。
需要说明的是，本文中的拍摄角度可以指拍摄方向一个因素，也可以指拍摄方向和拍摄距离两个因素，还可以指其他因素或其他数量的因素，本实 施例对此不进行限定。
采集动作(或称拍照动作)可以由结算台上布置的秤触发，如秤为具有压力传感器的秤，根据秤感应到的重量的变化来决定是否触发拍摄。当秤感受到重量发生了变化、并且该变化已经稳定下来时，摄像头去开始拍照，这样能够保证拍到符合要求的图片，就是顾客把东西放上去以后，拍到这个时刻的图片。在其他的实施例中，摄像头拍照动作的触发可以采用图形识别、计算机视觉的技术手段，摄像头首先对放置物品的区域持续观察和拍摄，比如当发现顾客的手伸进来、放下一个东西、再把手拿开，当从视频中捕捉到这样一个动作时，去下拍照的指令，即触发摄像头进行拍照。
步骤202，对每张图片进行目标检测以获取矩形区域图像，该矩形区域图像为与包含商品的矩形区域对应的图像。
具体地，在对每张图片进行目标检测时，会在每张图片上拉出一个包含商品的矩形框(或称矩形区域)，该矩形框所对应的图像是用于对商品进行分类的图像，输出至步骤203。当图片数量为5张时，则从5张图片中会获取5张与包含商品的矩形区域对应的图像。
步骤203，根据多张矩形区域图像和预先训练的一级分类模型，对应获取多个初级分类结果，预先训练的一级分类模型为基于卷积神经网络的图像识别技术架构且经购物场所内所有商品训练的模型。
具体地，采集数据建立数据集，采集数据的过程包括：1)对购物场所内所有商品从各个角度以及在各个姿态下拍照来获取大量的照片。2)然后对这些照片进行标注：对照片中商品的位置、大小以及类别进行标注。数据集包括的数据是指前述这些照片以及这些照片上进行的标注。一级分类模型为基于卷积神经网络的图像识别技术架构的模型，并使用购物场所内所有商品的数据对一级分类模型进行了训练，训练时可以通过梯度下降的方式进行。
训练好的一级分类模型对每张矩形区域图像中的商品进行分类，得到初级分类结果，该初级分类结果为一个n维向量，n表示购物场所内商品的总数量，向量中每个元素的含义表示一级分类模型认为待分类的单个商品属于n个商品中每个商品的概率，向量中哪个元素的值最大，那意味着模型认为待分类的商品为该元素对应的商品。当矩形区域图像为5张时，初级分类结果的数量为5个n维向量。
步骤204，根据多个初级分类结果和预先训练的一级线性回归模型获取多张图片的一级分类结果，若一级分类结果为相似商品，则执行下述步骤205，否则以一级分类结果作为第一次分类结果。
具体地，将在步骤203中，训练一级分类模型时，一级分类模型输出的初级分类结果作为一级线性回归模型的输入，该初级分类结果对应的图片中所包含的商品的正确分类作为一级线性回归模型的输出，以此来训练一级线性回归模型。训练好的一级线性回归模型对多个初级分类结果进行数据融合，得到一个一级分类结果，该一级分类结果表示一级线性回归模型预测图片中商品为购物场所内商品中哪个类别。
购物场所内的商品有多种，在该多种商品中会存在一些外观相近及通过视觉易混淆的商品，将这些商品称为相似商品，如黄元帅苹果和黄色的雪花梨。若待分类的单个商品为相似商品时，一级分类模型难以准确地对该商品进行分类，如把黄元帅的苹果与黄色的雪花梨弄混，将黄元帅的苹果分类为黄色的雪花梨，因此需要执行下述步骤205，否则直接将一级分类结果作为第一次分类结果，用于结算。
步骤205，根据多张矩形区域图像和预先训练的二级分类模型，对应获得多个次级分类结果，再根据多个次级分类结果和预先训练的二级线性回归模型获取多张图片的二级分类结果，并以二级分类结果作为第一次分类结果，二级分类模型为预先基于卷积神经网络的图像识别技术架构且经购物场所内相似商品组中商品训练的模型。
具体地，利用在步骤203中建立的数据集中的相似商品的数据对二级分类模型进行训练，训练时可以通过梯度下降的方式进行。二级分类模型和一级分类模型的区别在于训练时所使用的数据不同，一级分类模型使用的数据为购物场所内所有商品的数据，二级分类模型使用的数据为购物场所内相似商品数据。
训练好的二级分类模型对每张矩形区域图像中的商品进行分类，得到次级分类结果，该次级分类结果也为一个m维向量，向量中每个元素的含义表示二级分类模型认为待分类的单个商品属于m个相似商品中每个商品的概率。当矩形区域图像为5张时，次级分类结果的数量为5个m维向量，m小于等于n，且表示购物场所内相似商品的总数量。
实际中，购物场所内的相似商品有多组，如一组相似商品中包括黄元帅苹果和黄色的雪花梨，另一组相似商品中包括散装的盐和散装的白糖；再一组相似商品中包括碱面和面粉。可以针对所有组相似商品训练一个二级分类模型，为了进一步提高对商品分类的准确率，针对每组相似商品训练一个二级分类模型，此时，若一级分类结果为相似商品，则调用该一级分类结果对应的二级分类模型。
将训练二级分类模型时，二级分类模型输出的次级分类结果作为二级线性回归模型的输入，该次级分类结果对应的图片中所包含的商品的正确分类作为二级线性回归模型的输出，以此来训练二级线性回归模型。训练好的二级线性回归模型对多个次级分类结果进行数据融合，得到一个二级分类结果，并以其作为第一次分类结果，该二级分类结果表示二级线性回归模型预测图片中商品为购物场所内商品中哪个类别。
步骤206，根据第一次分类结果进行结算。
第一次分类结果获取后，再获取与第一次分类结果对应的商品价格，则顾客为放置在结算台上的商品所需支付的费用就确定了，可以通过结算台上的显示器显示商品名称、商品价格、支付费用，还可以通过语音将商品名称提示给顾客。顾客支付费用时，可以通过扫描显示器显示的二维码或将移动终端上自己账户的二维码对准结算台上的扫码终端完成支付。
为了避免出现分类错误，提高结算的准确率，参见图3，在步骤206之前还包括：
步骤207，根据多张矩形区域图像和与第一次分类结果对应的预先训练的支持向量机模型，判断图片中的商品是否与第一次分类结果一致，若一致则执行上述步骤206，否则提醒顾客商品无库存。
具体地，为购物场所内的每个商品构建支持向量机的机器学习模型(或称支持向量机模型)，即每一个商品都有与其对应的支持向量机模型，并用步骤203建立的数据集中商品对应的数据对该模型进行训练。在构建一级分类模型时，存在一个中间计算结果，其为一个长度是1024的向量，可以将该向量看成是图片的一个特征，据此构建了一个从该向量到判断该商品是否属于某个类别商品的支持向量机模型。
第一次分类结果获取后，采用与该第一次分类结果对应的支持向量机模 型对每张矩形区域图像中所包含的商品进行判断，得到初步判断结果，该初步判断结果表示该张图片中的商品是否与第一次分类结果一致。矩形区域图像的数量为多张，则初步判断结果共有多个。若多个初步判断结果中一致的数量大于等于预设的阈值，则判断该张图片中的商品与第一次分类结果一致，否则，提醒顾客该商品无库存，即无法结算。如图片的张数为5张，预设的阈值为3，5个初步判断结果依次为：一致、一致、不一致、不一致、一致，则一致的数量为3。由于一致的数量等于预设的阈值，所以支持向量机模型判断该图片中的商品与第一次分类结果一致，则认为前述分类过程是正确的，即识别到一种正确的商品，此时顾客可以通过结算获得该商品。若5个初步判断结果依次为不一致、不一致、不一致、不一致、一致，则一致的数量为1。由于一致的数量小于预设的阈值，所以支持向量机模型判断该图片中的商品与第一次分类结果不一致，则认为前述分类过程是不成功的，即识别失败，此时通过语音和/画面提示：‘库里没有该商品，无法识别该商品’等来提醒顾客该商品无法结算。
本发明一实施例提供了一种基于卷积神经网络的图像识别技术的结算装置，其包括：
采集装置301，用于采集放置在结算台上的待分类的单个商品的多张图片，多张图片与多个拍摄角度一一对应。
优选地，采集装置301为摄像头，在待分类的商品的正上方布置1个摄像头，以从正上方向下对商品进行拍照来采集图片；在待分类的商品的四周布置4个摄像头，以从斜上方对商品进行拍照来采集图片。
目标检测装置302，用于对每张图片进行目标检测以获取矩形区域图像，矩形区域图像为与包含商品的矩形区域对应的图像。
第一分类装置303，用于根据多张矩形区域图像和预先训练的一级分类模型，对应获取多个初级分类结果，一级分类模型为基于卷积神经网络的图像识别技术架构且经购物场所内所有商品训练的模型，根据多个初级分类结果和预先训练的一级线性回归模型获取多张图片的一级分类结果；
确认装置304，用于以一级分类结果作为第一次分类结果；和
结算装置305，用于根据第一次分类结果进行结算。
该结算装置还包括：第二分类装置，用于若一级分类结果为相似商品，则根据多张矩形区域图像和预先训练的二级分类模型，对应获得多个次级分类结果，再根据多个次级分类结果和预先训练的二级线性回归模型获取多张图片的二级分类结果，并以二级分类结果作为第一次分类结果，二级分类模型为预先基于卷积神经网络的图像识别技术架构且经购物场所内相似商品训练的模型；对应地，结算装置在一级分类结果为相似商品时，根据第二分类装置获取的二级分类结果进行结算，否则根据第一分类装置获取的一级分类结果进行结算。
该结算装置还包括：判断装置，用于根据多张矩形区域图像和与第一次分类结果对应的预先训练的支持向量机模型，判断图片中的商品是否与第一次分类结果一致；和选择装置，若一致调用结算装置305，否则提醒顾客商品无库存。
需要说明的是，关于采集装置301的具体描述可参见上述实施例中步骤101和201的相关内容，关于目标检测装置302的具体描述可参见上述实施例中步骤102和203的相关内容，关于第一分类装置303的具体描述可参见上述实施例中步骤103和203的相关内容，关于确认装置304的具体描述可参见上述实施例中步骤104和204的相关内容，关于结算装置305的具体描述可参见上述实施例中步骤105和206的相关内容，关于第二分类装置的具体描述可参见上述实施例中步骤205的相关内容，关于判断装置和选择装置的具体描述可参见上述实施例中步骤207的相关内容，此处不再一一赘述。
本发明一实施例提供了一种基于卷积神经网络的图像识别技术的结算装置，其包括：摄像头、处理器和存储器。
摄像头用于采集放置在结算台上的待分类的单个商品的多张图片，多张图片与多个拍摄角度一一对应。存储器用于存储处理器可执行的指令。处理器被配置为：对每张图片进行目标检测以获取矩形区域图像，矩形区域图像为与包含商品的矩形区域对应的图像；根据多张矩形区域图像和预先训练的一级分类模型，对应获取多个初级分类结果，一级分类模型为基于卷积神经网络的图像识别技术架构且经购物场所内所有商品训练的模型，根据多个初级分类结果和预先训练的一级线性回归模型获取多张图片的一级分类结果； 以一级分类结果作为第一次分类结果；根据第一次分类结果进行结算。
处理器还被配置为：若一级分类结果为相似商品，则根据多张矩形区域图像和预先训练的二级分类模型，对应获得多个次级分类结果，再根据多个次级分类结果和预先训练的二级线性回归模型获取多张图片的二级分类结果，并以二级分类结果作为第一次分类结果，二级分类模型为预先基于卷积神经网络的图像识别技术架构且经购物场所内相似商品训练的模型，否则以一级分类结果作为第一次分类结果。
下面以购买一瓶矿泉水为例对本方法的具体过程进行说明：
顾客在结账过程中，向称重台上放置了一瓶矿泉水，当称重台的电子秤检测到重量变化，并且重量区域稳定时，开始从5个摄像头采集数据，采集到了5张照片。首先将这5张照片统一缩放到某个预设的尺寸，然后用目标检测模型在这5张照片上检测物体，检测结果为在照片上找出一个包含物体的矩形区域，之后将各个矩形区域统一缩放到某个预设的尺寸，然后用一级分类模型对各个矩形区域进行分类，得到该物体属于各个类别商品的概率。之后将所有的概率结果经过一个一级线性回归模型，得到该物体的在所有商品类别中的一级分类结果，若该一级分类结果属于需要用二级分类模型进行进一步分类的商品类别，则调取相应的二级分类模型，对各个矩形框进行分类，得到该物体属于各个类别的一个概率值，然后再用二级线性回归模型得到该物体在所有商品类别中的分类结果。最后再用该类别的支持向量机模型(或称小模型)对5张照片中的矩形框进行判断，判别该商品是否属于该类别。如果是，那么就返回该商品类别，如果不是，就认定该物品不属于任何一个商品类别。
综上，本发明实施例带来的有益效果如下：
能准确识别商品，便于自助结算。
由技术常识可知，本发明可以通过其它的不脱离其精神实质或必要特征的实施方案来实现。因此，上述公开的实施方案，就各方面而言，都只是举例说明，并不是仅有的。所有在本发明范围内或在等同于本发明的范围内的改变均被本发明包含。

Claims (10)

  1. 一种基于卷积神经网络的图像识别技术的结算方法，其特征在于，所述结算方法包括：
    采集步骤，采集放置在结算台上的待分类的单个商品的多张图片，多张所述图片与多个拍摄角度一一对应；
    目标检测步骤，对每张所述图片进行目标检测以获取矩形区域图像，所述矩形区域图像为与包含商品的矩形区域对应的图像；
    第一分类步骤，根据多张所述矩形区域图像和预先训练的一级分类模型，对应获取多个初级分类结果，所述一级分类模型为基于卷积神经网络的图像识别技术架构且经购物场所内所有商品训练的模型，根据多个所述初级分类结果和预先训练的一级线性回归模型获取多张所述图片的一级分类结果；
    确认步骤，以所述一级分类结果作为第一次分类结果；
    结算步骤，根据所述第一次分类结果进行结算。
  2. 根据权利要求1所述的结算方法，其特征在于，在第一分类步骤之后，确认步骤之前，所述结算方法还包括：
    第二分类步骤，若所述一级分类结果为相似商品，则根据多张所述矩形区域图像和预先训练的二级分类模型，对应获得多个次级分类结果，再根据多个所述次级分类结果和预先训练的二级线性回归模型获取多张所述图片的二级分类结果，并以所述二级分类结果作为第一次分类结果，所述二级分类模型为预先基于卷积神经网络的图像识别技术架构且经购物场所内相似商品训练的模型，否则执行确认步骤。
  3. 根据权利要求1或2所述的结算方法，其特征在于，在确认步骤之后，结算步骤之前，所述结算方法还包括：
    根据多张所述矩形区域图像和与所述第一次分类结果对应的预先训练的支持向量机模型，判断图片中的商品是否与所述第一次分类结果一致，若一致则执行结算步骤，否则提醒顾客商品无库存。
  4. 根据权利要求1所述的结算方法，其特征在于，在待分类的商品的正 上方布置1个摄像头，以从正上方向下对所述商品进行拍照来采集图片；
    在待分类的商品的四周布置4个摄像头，以从斜上方对所述商品进行拍照来采集图片。
  5. 一种基于卷积神经网络的图像识别技术的结算装置，其特征在于，所述结算装置包括：
    采集装置，用于采集放置在结算台上的待分类的单个商品的多张图片，多张所述图片与多个拍摄角度一一对应；
    目标检测装置，用于对每张所述图片进行目标检测以获取矩形区域图像，所述矩形区域图像为与包含商品的矩形区域对应的图像；
    第一分类装置，用于根据多张所述矩形区域图像和预先训练的一级分类模型，对应获取多个初级分类结果，所述一级分类模型为基于卷积神经网络的图像识别技术架构且经购物场所内所有商品训练的模型，根据多个所述初级分类结果和预先训练的一级线性回归模型获取多张所述图片的一级分类结果；
    确认装置，用于以所述一级分类结果作为第一次分类结果；和
    结算装置，用于根据所述第一次分类结果进行结算。
  6. 根据权利要求5所述的结算装置，其特征在于，所述结算装置还包括：
    第二分类装置，用于若所述一级分类结果为相似商品，则根据多张所述矩形区域图像和预先训练的二级分类模型，对应获得多个次级分类结果，再根据多个所述次级分类结果和预先训练的二级线性回归模型获取多张所述图片的二级分类结果，并以所述二级分类结果作为第一次分类结果，所述二级分类模型为预先基于卷积神经网络的图像识别技术架构且经购物场所内相似商品训练的模型；
    对应地，所述结算装置在所述一级分类结果为相似商品时，根据所述第二分类装置获取的二级分类结果进行结算，否则根据所述第一分类装置获取的一级分类结果进行结算。
  7. 根据权利要求5所述的结算装置，其特征在于，所述结算装置还包括：
    判断装置，用于根据多张所述矩形区域图像和与第一次分类结果对应的预先训练的支持向量机模型，判断图片中的商品是否与第一次分类结果一致；和
    选择装置，若一致调用所述结算装置，否则提醒顾客商品无库存。
  8. 根据权利要求5所述的结算装置，其特征在于，所述采集装置为摄像头，在待分类的商品的正上方布置1个摄像头，以从正上方向下对所述商品进行拍照来采集图片；
    在待分类的商品的四周布置4个摄像头，以从斜上方对所述商品进行拍照来采集图片。
  9. 一种基于卷积神经网络的图像识别技术的结算装置，其特征在于，所述结算装置包括：
    摄像头，用于采集放置在结算台上的待分类的单个商品的多张图片，多张所述图片与多个拍摄角度一一对应；
    处理器；
    用于存储处理器可执行的指令的存储器；
    其中，所述处理器被配置为：
    对每张所述图片进行目标检测以获取矩形区域图像，所述矩形区域图像为与包含商品的矩形区域对应的图像；根据多张所述矩形区域图像和预先训练的一级分类模型，对应获取多个初级分类结果，所述一级分类模型为基于卷积神经网络的图像识别技术架构且经购物场所内所有商品训练的模型，根据多个所述初级分类结果和预先训练的一级线性回归模型获取多张所述图片的一级分类结果；以所述一级分类结果作为第一次分类结果；根据所述第一次分类结果进行结算。
  10. 根据权利要求9所述的结算装置，其特征在于，所述处理器还被配置为：
    若所述一级分类结果为相似商品，则根据多张所述矩形区域图像和预先训练的二级分类模型，对应获得多个次级分类结果，再根据多个所述次级分 类结果和预先训练的二级线性回归模型获取多张所述图片的二级分类结果，并以所述二级分类结果作为第一次分类结果，所述二级分类模型为预先基于卷积神经网络的图像识别技术架构且经购物场所内相似商品训练的模型，否则以所述一级分类结果作为第一次分类结果。
PCT/CN2017/100999 2017-07-24 2017-09-08 基于卷积神经网络的图像识别技术的结算方法和装置 WO2019019291A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2018567956A JP6709862B6 (ja) 2017-07-24 2017-09-08 畳み込みニューラルネットワーク画像認識技術による会計方法及び設備
US16/328,904 US10853702B2 (en) 2017-07-24 2017-09-08 Method and apparatus for checkout based on image identification technique of convolutional neural network

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710607759.X 2017-07-24
CN201710607759 2017-07-24

Publications (1)

Publication Number Publication Date
WO2019019291A1 true WO2019019291A1 (zh) 2019-01-31

Family

ID=63804471

Family Applications (2)

Application Number Title Priority Date Filing Date
PCT/CN2017/100999 WO2019019291A1 (zh) 2017-07-24 2017-09-08 基于卷积神经网络的图像识别技术的结算方法和装置
PCT/CN2017/115443 WO2019019525A1 (zh) 2017-07-24 2017-12-11 基于卷积神经网络的图像识别技术的结算方法和装置

Family Applications After (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/115443 WO2019019525A1 (zh) 2017-07-24 2017-12-11 基于卷积神经网络的图像识别技术的结算方法和装置

Country Status (4)

Country Link
US (2) US10853702B2 (zh)
JP (2) JP6709862B6 (zh)
CN (4) CN109299715B (zh)
WO (2) WO2019019291A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2020134507A (ja) * 2019-02-12 2020-08-31 株式会社イシダ 計量機
US11128737B1 (en) * 2019-08-28 2021-09-21 Massachusetts Mutual Life Insurance Company Data model monitoring system
CN117077083A (zh) * 2023-10-10 2023-11-17 上海英内物联网科技股份有限公司 装箱物品自动识别统计方法

Families Citing this family (38)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190034897A1 (en) 2017-07-26 2019-01-31 Sbot Technologies Inc. Self-Checkout Anti-Theft Vehicle Systems and Methods
JP6821009B2 (ja) * 2017-12-25 2021-01-27 ▲図▼▲霊▼通▲諾▼（北京）科技有限公司Ｙｉ Ｔｕｎｎｅｌ （Ｂｅｉｊｉｎｇ） Ｔｅｃｈｎｏｌｏｇｙ Ｃｏ．，Ｌｔｄ． 会計方法、装置及びシステム
US20200151692A1 (en) * 2018-04-18 2020-05-14 Sbot Technologies, Inc. d/b/a Caper Inc. Systems and methods for training data generation for object identification and self-checkout anti-theft
JP7218984B2 (ja) * 2018-04-27 2023-02-07 日本電気株式会社 商品登録装置、商品登録方法及びプログラム
CN111222388B (zh) * 2018-12-11 2023-09-19 图灵通诺（北京）科技有限公司 基于视觉识别的结算方法和系统
JP7261022B2 (ja) * 2019-01-30 2023-04-19 キヤノン株式会社 情報処理システム、端末装置及びその制御方法、プログラム、記憶媒体
CN109816045A (zh) * 2019-02-11 2019-05-28 青岛海信智能商用系统股份有限公司 一种商品识别方法及装置
CN109919086A (zh) * 2019-03-05 2019-06-21 北京美餐巧达科技有限公司 一种基于图像识别的智能餐台系统及结算方法
US11170186B2 (en) * 2019-03-29 2021-11-09 Ncr Corporation Scanner bar
CN110032654B (zh) * 2019-04-19 2021-02-09 浪潮集团有限公司 一种基于人工智能的超市商品录入方法及系统
CN110414559B (zh) * 2019-06-26 2021-09-03 武汉大学 智能零售柜商品目标检测统一框架的构建方法及商品识别方法
CN110718016A (zh) * 2019-09-02 2020-01-21 上海理工大学 一种生鲜自助售卖系统
CN110706075B (zh) * 2019-09-27 2022-04-29 支付宝(杭州)信息技术有限公司 一种基于ai模型的物料铺设的方法和设备
CN110781823B (zh) * 2019-10-25 2022-07-26 北京字节跳动网络技术有限公司 录屏检测方法、装置、可读介质及电子设备
US11205094B2 (en) * 2019-10-29 2021-12-21 Accel Robotics Corporation Multi-angle rapid onboarding system for visual item classification
US11743418B2 (en) 2019-10-29 2023-08-29 Accel Robotics Corporation Multi-lighting conditions rapid onboarding system for visual item classification
CN110987140A (zh) * 2019-11-28 2020-04-10 浙江由由科技有限公司 一种商品称重辅助设备及称重设备
CN111062786B (zh) * 2019-12-25 2023-05-23 创新奇智(青岛)科技有限公司 一种基于建立商品外观特征映射表的模型更新方法
CN111104988B (zh) * 2019-12-28 2023-09-29 Oppo广东移动通信有限公司 图像识别方法及相关装置
CN111401499A (zh) * 2020-02-11 2020-07-10 王娅雯 一种基于ai条形码防伪的方法
US11809999B2 (en) * 2020-02-24 2023-11-07 Zebra Technologies Corporation Object recognition scanning systems and methods for implementing artificial based item determination
CN111368900A (zh) * 2020-02-28 2020-07-03 桂林电子科技大学 一种图像目标物识别方法
WO2021179137A1 (zh) * 2020-03-09 2021-09-16 图灵通诺（北京）科技有限公司 结算方法、装置和系统
CN111401422B (zh) * 2020-03-09 2024-03-08 南京览笛信息科技有限公司 一种基于多角度深度推理的深度网络图像分类方法
CN111613225A (zh) * 2020-04-27 2020-09-01 深圳壹账通智能科技有限公司 基于语音及图像处理的道路违规自动上报方法及系统
CN111428698A (zh) * 2020-06-10 2020-07-17 北京云迹科技有限公司 一种模型样本采集方法、更新方法、电子设备及存储介质
CN111860219B (zh) * 2020-06-30 2024-01-05 杭州科度科技有限公司 一种高速占道判断方法、装置及电子设备
CN112001419A (zh) * 2020-07-22 2020-11-27 李峰 一种防伪识别方法和装置
US20220277313A1 (en) * 2021-02-26 2022-09-01 Ncr Corporation Image-based produce recognition and verification
CN113095228B (zh) * 2021-04-13 2024-04-30 地平线(上海)人工智能技术有限公司 图像中的目标检测方法、装置及计算机可读存储介质
CN113344012B (zh) * 2021-07-14 2022-08-23 马上消费金融股份有限公司 物品识别方法、装置及设备
CN113688893A (zh) * 2021-08-19 2021-11-23 西交利物浦大学 一种图片分类的方法、装置、电子设备及存储介质
CN113743481B (zh) * 2021-08-20 2024-04-16 北京电信规划设计院有限公司 类人化图像识别方法和系统
US11681997B2 (en) * 2021-09-30 2023-06-20 Toshiba Global Commerce Solutions Holdings Corporation Computer vision grouping recognition system
JP2023077805A (ja) * 2021-11-25 2023-06-06 東芝テック株式会社 決済者監視装置及びそのプログラム、決済者監視方法
CN114299395A (zh) * 2021-12-30 2022-04-08 百富计算机技术（深圳）有限公司 商品结算处理方法、装置、终端设备及存储介质
CN114660112B (zh) * 2022-05-25 2022-09-02 河南源宏高分子新材料有限公司 一种再生聚酯材料温度阻隔性能的检测方法
CN114813635B (zh) * 2022-06-28 2022-10-04 华谱智能科技(天津)有限公司 煤炉燃烧参数的优化方法及电子设备

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130058541A1 (en) * 2011-09-06 2013-03-07 Toshiba Tec Kabushiki Kaisha Image processing apparatus and method
CN103034839A (zh) * 2012-12-04 2013-04-10 南京航空航天大学 基于图像识别技术的食堂支付系统及方法
CN103729777A (zh) * 2013-12-12 2014-04-16 福建伊时代信息科技股份有限公司 基于图像识别技术的网络购物方法、购物装置及购物系统
CN104077842A (zh) * 2014-07-02 2014-10-01 浙江大学 基于图像识别的自选餐厅自助付费装置及其使用方法
CN106096932A (zh) * 2016-06-06 2016-11-09 杭州汇萃智能科技有限公司 基于餐具形状的菜品自动识别系统的计价方法

Family Cites Families (41)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5729623A (en) * 1993-10-18 1998-03-17 Glory Kogyo Kabushiki Kaisha Pattern recognition apparatus and method of optimizing mask for pattern recognition according to genetic algorithm
US5497314A (en) * 1994-03-07 1996-03-05 Novak; Jeffrey M. Automated apparatus and method for object recognition at checkout counters
AUPQ212499A0 (en) * 1999-08-10 1999-09-02 Ajax Cooke Pty Ltd Item recognition method and apparatus
JP3412592B2 (ja) * 2000-02-08 2003-06-03 松下電器産業株式会社 個人情報認証方法
JP2007293575A (ja) * 2006-04-25 2007-11-08 Shigeru Furuno 電子商取引システム
CN102119387B (zh) * 2008-08-08 2014-12-03 株式会社尼康 检索辅助系统、检索辅助方法以及检索辅助程序
CN102063616A (zh) * 2010-12-30 2011-05-18 上海电机学院 一种基于图像特征匹配的商品自动识别系统及方法
JP2012247968A (ja) * 2011-05-27 2012-12-13 Toshiba Tec Corp 情報処理装置、情報処理方法及び制御プログラム
WO2013188442A2 (en) * 2012-06-11 2013-12-19 Stylinity, Inc. Photographic stage
US9269022B2 (en) * 2013-04-11 2016-02-23 Digimarc Corporation Methods for object recognition and related arrangements
US10296814B1 (en) * 2013-06-27 2019-05-21 Amazon Technologies, Inc. Automated and periodic updating of item images data store
CN103412938B (zh) * 2013-08-22 2016-06-29 成都数之联科技有限公司 一种基于图片交互式多目标提取的商品比价方法
CN103632460A (zh) * 2013-11-08 2014-03-12 青岛中科英泰商用系统有限公司 同步向顾客提供交易过程视频的超市自助结算用防损方法
JP6145416B2 (ja) * 2014-02-28 2017-06-14 富士フイルム株式会社 商品検索装置及び方法、商品検索システム
US9275293B2 (en) * 2014-02-28 2016-03-01 Thrift Recycling Management, Inc. Automated object identification and processing based on digital imaging and physical attributes
US20150310601A1 (en) * 2014-03-07 2015-10-29 Digimarc Corporation Methods and arrangements for identifying objects
US10129507B2 (en) * 2014-07-15 2018-11-13 Toshiba Global Commerce Solutions Holdings Corporation System and method for self-checkout using product images
JP6336351B2 (ja) * 2014-08-08 2018-06-06 東芝テック株式会社 情報処理装置及びプログラム
JP6428062B2 (ja) * 2014-09-04 2018-11-28 富士ゼロックス株式会社 情報処理装置及び情報処理プログラム
CN105718833B (zh) * 2014-12-23 2018-09-14 东芝泰格有限公司 图像识别装置及商品信息处理装置
CN105989527A (zh) * 2015-02-10 2016-10-05 黄金富知识产权咨询(深圳)有限公司 一种基于图像辨识的商品销售系统和相应方法
US10282722B2 (en) * 2015-05-04 2019-05-07 Yi Sun Huang Machine learning system, method, and program product for point of sale systems
US20160350336A1 (en) * 2015-05-31 2016-12-01 Allyke, Inc. Automated image searching, exploration and discovery
CN106355188B (zh) * 2015-07-13 2020-01-21 阿里巴巴集团控股有限公司 图像检测方法及装置
CN106023220B (zh) * 2016-05-26 2018-10-19 史方 一种基于深度学习的车辆外观部件图像分割方法
US9928448B1 (en) * 2016-09-23 2018-03-27 International Business Machines Corporation Image classification utilizing semantic relationships in a classification hierarchy
WO2018073510A1 (fr) * 2016-10-20 2018-04-26 Jes Labs Systeme d'identification ou d'aide a l'identification d'un produit ou ensemble de produits
CN106548145A (zh) * 2016-10-31 2017-03-29 北京小米移动软件有限公司 图像识别方法及装置
CN206209991U (zh) * 2016-11-18 2017-05-31 吴观众 一种基于图像识别的餐费自助结算装置
US20180197218A1 (en) * 2017-01-12 2018-07-12 Verizon Patent And Licensing Inc. System and method for object detection in retail environment
US10600043B2 (en) * 2017-01-31 2020-03-24 Focal Systems, Inc. Automated checkout system through mobile shopping units
CN109409175B (zh) * 2017-08-16 2024-02-27 图灵通诺（北京）科技有限公司 结算方法、装置和系统
JP6821009B2 (ja) * 2017-12-25 2021-01-27 ▲図▼▲霊▼通▲諾▼（北京）科技有限公司Ｙｉ Ｔｕｎｎｅｌ （Ｂｅｉｊｉｎｇ） Ｔｅｃｈｎｏｌｏｇｙ Ｃｏ．，Ｌｔｄ． 会計方法、装置及びシステム
MX2020008117A (es) * 2018-01-31 2021-08-24 Walmart Apollo Llc Sistemas y metodos para verificar etiquetas legibles por maquina asociadas con mercancias.
US20200151692A1 (en) * 2018-04-18 2020-05-14 Sbot Technologies, Inc. d/b/a Caper Inc. Systems and methods for training data generation for object identification and self-checkout anti-theft
US10867186B2 (en) * 2018-05-15 2020-12-15 Genetec Inc. Transaction monitoring
CN112585667A (zh) * 2018-05-16 2021-03-30 康耐克斯数字有限责任公司 智能平台柜台展示系统和方法
WO2020056041A1 (en) * 2018-09-11 2020-03-19 Pointivo, Inc. Improvements in data acquistion, processing, and output generation for use in analysis of one or a collection of physical assets of interest
US11080559B2 (en) * 2018-09-18 2021-08-03 Focal Systems, Inc. Product onboarding machine
US11423648B2 (en) * 2018-09-27 2022-08-23 Ncr Corporation Item recognition processing over time
US10783403B2 (en) * 2018-10-29 2020-09-22 EMC IP Holding Company LLC Automated design testing through deep learning

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130058541A1 (en) * 2011-09-06 2013-03-07 Toshiba Tec Kabushiki Kaisha Image processing apparatus and method
CN103034839A (zh) * 2012-12-04 2013-04-10 南京航空航天大学 基于图像识别技术的食堂支付系统及方法
CN103729777A (zh) * 2013-12-12 2014-04-16 福建伊时代信息科技股份有限公司 基于图像识别技术的网络购物方法、购物装置及购物系统
CN104077842A (zh) * 2014-07-02 2014-10-01 浙江大学 基于图像识别的自选餐厅自助付费装置及其使用方法
CN106096932A (zh) * 2016-06-06 2016-11-09 杭州汇萃智能科技有限公司 基于餐具形状的菜品自动识别系统的计价方法

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2020134507A (ja) * 2019-02-12 2020-08-31 株式会社イシダ 計量機
JP7398779B2 (ja) 2019-02-12 2023-12-15 株式会社イシダ 計量機
US11128737B1 (en) * 2019-08-28 2021-09-21 Massachusetts Mutual Life Insurance Company Data model monitoring system
CN117077083A (zh) * 2023-10-10 2023-11-17 上海英内物联网科技股份有限公司 装箱物品自动识别统计方法
CN117077083B (zh) * 2023-10-10 2024-01-05 上海英内物联网科技股份有限公司 装箱物品自动识别统计方法

Also Published As

Publication number Publication date
JP2019526848A (ja) 2019-09-19
US11151427B2 (en) 2021-10-19
CN109300263B (zh) 2024-06-21
CN209028742U (zh) 2019-06-25
JP6709320B2 (ja) 2020-06-10
CN109300263A (zh) 2019-02-01
JP2019536121A (ja) 2019-12-12
WO2019019525A1 (zh) 2019-01-31
JP6709862B2 (ja) 2020-06-17
US20190220692A1 (en) 2019-07-18
CN109299715A (zh) 2019-02-01
CN109299715B (zh) 2021-07-13
JP6709320B6 (ja) 2020-08-05
JP6709862B6 (ja) 2020-07-22
CN207993164U (zh) 2018-10-19
US10853702B2 (en) 2020-12-01
US20190244055A1 (en) 2019-08-08

Similar Documents

Publication Publication Date Title
WO2019019291A1 (zh) 基于卷积神经网络的图像识别技术的结算方法和装置
CN109559457B (zh) 基于神经网络识别商品的收银方法以及自助收银台
US11501523B2 (en) Goods sensing system and method for goods sensing based on image monitoring
WO2022111059A1 (zh) 基于计算机视觉技术的智能称重装置与智能称重方法
CN109508974B (zh) 一种基于特征融合的购物结账系统和方法
KR101960899B1 (ko) 제품 식별 방법
US10503961B2 (en) Object recognition for bottom of basket detection using neural network
CN111222389B (zh) 商超货架上商品的分析方法和系统
WO2019062812A1 (zh) 用于自动结算的人机交互装置及其应用
CN110555356A (zh) 自助结帐系统、方法与装置
WO2019127618A1 (zh) 结算方法、装置和系统
US10372998B2 (en) Object recognition for bottom of basket detection
WO2021179137A1 (zh) 结算方法、装置和系统
WO2021179138A1 (zh) 商超货架上商品的分析方法和系统
US20230037427A1 (en) Identifying barcode-to-product mismatches using point of sale devices and overhead cameras
CN110909698A (zh) 电子秤识别结果输出方法、系统、装置以及可读存储介质
CN111222388B (zh) 基于视觉识别的结算方法和系统
CN111428743B (zh) 商品识别方法、商品处理方法、装置及电子设备
CN115862043A (zh) 用于商品辨识的系统和方法
CN111126110A (zh) 一种商品信息识别方法、结算方法、装置及无人零售系统
CN109741119A (zh) 结账方法、装置、系统及计算机可读存储介质
CN115631585A (zh) 一种用于商品结算的智能自助称重系统

Legal Events

Date Code Title Description
ENP Entry into the national phase

Ref document number: 2018567956

Country of ref document: JP

Kind code of ref document: A

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17918749

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17918749

Country of ref document: EP

Kind code of ref document: A1