WO2021175601A1 - Création et mise à jour d'une base de données de produits - Google Patents

Création et mise à jour d'une base de données de produits Download PDF

Info

Publication number
WO2021175601A1
WO2021175601A1 PCT/EP2021/053947 EP2021053947W WO2021175601A1 WO 2021175601 A1 WO2021175601 A1 WO 2021175601A1 EP 2021053947 W EP2021053947 W EP 2021053947W WO 2021175601 A1 WO2021175601 A1 WO 2021175601A1
Authority
WO
WIPO (PCT)
Prior art keywords
goods
scanning
product information
neural network
optical
Prior art date
Application number
PCT/EP2021/053947
Other languages
German (de)
English (en)
Inventor
Philipp Kleinlein
Philip Koene
Frank Schaefer
Original Assignee
BSH Hausgeräte GmbH
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by BSH Hausgeräte GmbH filed Critical BSH Hausgeräte GmbH
Priority to EP21706897.2A priority Critical patent/EP4115370A1/fr
Publication of WO2021175601A1 publication Critical patent/WO2021175601A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q20/00—Payment architectures, schemes or protocols
    • G06Q20/08—Payment architectures
    • G06Q20/20—Point-of-sale [POS] network systems
    • G06Q20/203—Inventory monitoring
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/08—Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
    • G06Q10/087—Inventory or stock management, e.g. order filling, procurement or balancing against orders
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07G—REGISTERING THE RECEIPT OF CASH, VALUABLES, OR TOKENS
    • G07G1/00—Cash registers
    • G07G1/0036—Checkout procedures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Definitions

  • the invention relates to the provision of a product database, in particular for recognizing a household item.
  • an inventory of existing consumables can be kept automatically. For example, a supply of vacuum cleaner bags can be replenished in good time without having to keep large stocks and without the risk of not having any bags available when needed.
  • a best-before date can be tracked for perishable goods, especially food. If the goods threaten to spoil, a notice on how to use the goods can be given in good time. Used or expired goods can be placed on a shopping list or ordered automatically.
  • refrigerators with a camera located therein are known from the prior art, which can detect the objects located in the refrigerator by means of a neural network.
  • the probability of correct object recognition depends heavily on the quality of the training of the neural network.
  • An object on which the present invention is based is to provide an improved technique for creating or maintaining a product database with goods that can be procured especially for a household.
  • the invention solves this task by means of the subjects of the independent claims. Subclaims reflect preferred embodiments.
  • an apparatus comprises an optical scanning device for providing optical scanning of goods, a cash register system with preferably a scanning device for providing product information about the goods, and a processing device for assigning product information to optical scanning.
  • the scanning device can for example comprise an optical camera.
  • labeled training data can be provided for an artificial neural network, the training data each comprising a scanning and product information.
  • the device is preferably set up to transmit the training data to an external point.
  • the device can be used, for example, in a supermarket or another retailer as part of a billing or payment process, which can also be called “checkout”.
  • all goods collected by a customer in the shop are recorded (e.g. scanned) by means of the cash register system, whereby the quantities and prices of the goods are also determined.
  • the information can in particular include a designation or a unique code such as an EAN code for the goods.
  • EAN code a unique code
  • a large number of optical scans can be created and clearly assigned to the goods in a normal operation of the cash register system.
  • information from several cash register systems can be evaluated. A relationship between an optically detectable property of the goods and the goods themselves can be determined automatically and with a high degree of reliability.
  • the processing device is set up to provide training data for an artificial neural network, the training data each comprising a scan and product information.
  • the training data are also called “labeled”, whereby the product information can represent the “label”, that is, an indication of the identity of the item presented.
  • the training data can be used for supervised learning of the neural network. Especially when trai- When data is collected from many of the devices described herein, a necessary number of labeled training data for learning to recognize a product can be generated efficiently and in a timely manner.
  • the scanning device can be attached, for example, in the area of a scanner, a scale or a manual detection station of the cash register system.
  • the optical scanning can be installed in the area of a shop counter or a conveyor belt for goods to be purchased. It is preferred that the goods do not have a predetermined preferred orientation during scanning, so that a statically attached scanning device can scan the goods from a random perspective. A neural network trained with a large number of such scans can then recognize the goods from all practically occurring perspectives.
  • the scanning device can comprise an optical camera. It is preferred that the camera provides a colored, high-resolution and high-contrast image. Active lighting can be provided for this purpose, optionally in a predetermined wavelength range.
  • the camera can support a short exposure time in order to reduce motion blur in the scan.
  • the scanning device can be set up for three-dimensional scanning of the goods.
  • a device for three-dimensional scanning of the goods can be used in addition or as an alternative to an optical camera.
  • a plurality of optical cameras are used which are mounted at locations that are slightly offset from one another and whose scanning fields overlap one another.
  • Such a camera can also be called a stereo camera.
  • a ToF camera can be used in which a distance of a point on the surface of the goods is determined on the basis of a travel time of light from a light source in the region of the camera to the goods and from there to the camera.
  • a camera can also be referred to as a depth camera.
  • a system comprises a device described herein for scanning the goods; and an external location configured to receive scans and associated product information from a plurality of devices.
  • the external location is preferably set up to receive information from a large number of the devices described.
  • a dedicated processing device is provided on the part of the external point, for example in order to process received information, to store it in a data memory or to process stored information.
  • the external point can be set up to train an artificial neural network to recognize goods on the basis of the received scans and the associated product information.
  • the external location comprises a data memory for recording a multiplicity of scans of the goods and associated product information in each case.
  • the information collected can be compared or merged with one another. Deviating provisions can be excluded. Particularly when the goods are mass-produced, information that is determined at a large number of sales points can be quickly and reliably updated, for example a change in a presentation, a packaging or container size or another aspect of the goods.
  • the processing device of the external location is set up to train an artificial neural network for recognizing the goods on the basis of an optical scan of the goods.
  • the provided neural network can be used, for example, in a household in order to recognize goods on the basis of optical scanning, as will be explained in more detail below.
  • a method comprises the following steps: providing an optical scanning of a product; Acquiring product information of the goods; and assigning the product information to the optical scanning.
  • the optical scanning can in particular take place with the aid of an optical camera.
  • the product information can in particular be provided by a cash register system, for example as part of an invoice or payment process for the goods.
  • the cash register system can use a scanning device.
  • the method can be used to provide data sets that are used to train a neural network can be used.
  • the training data can each include a sample and product information.
  • the training data can be transferred to an external location.
  • the method can preferably be carried out in the area of a cash register system, in particular a supermarket or retailer. Further processing of the collected data is preferably carried out at an external point.
  • the external point can be provided centrally for a large number of points at which the method can be carried out.
  • the method can be carried out in whole or in part by means of a processing device which is comprised by a device described herein.
  • the processing device can comprise a programmable microcomputer or microcontroller and the method can be in the form of a computer program product with program code means.
  • the computer program product can also be stored on a computer-readable data carrier. Additional features or advantages of the method can be transferred to the device or the system or vice versa.
  • an apparatus comprises a processing device, a communication device and an optical scanning device.
  • the communication device is set up to receive an artificial neural network from the external location
  • the processing device is set up to recognize goods scanned by the scanning device by means of the neural network.
  • the neural network usually comprises a computerized data structure which is relatively compact and can usually be recorded in a few 100 kBytes.
  • the neural network can be implemented on the processing device, or the processing device can implement the neural network with the data structure.
  • the following scenario can be realized: Many users buy goods in many supermarkets. These goods are scanned on standard cash register systems in order to enable them to be paid for. During this scanning process, a barcode attached to the goods is usually scanned, which uniquely identifies the goods. The goods can be photographed with the help of a camera located near the cash register system. The photos can be labeled using the barcode scanned by the cash register system. The data obtained in this way can be transferred to an external location, e.g. an Internet server or cloud Service, where they are used to train a neural network. The neural network trained in this way can be downloaded by the users to their refrigeration devices, where it is used to identify the goods in the refrigeration device.
  • an external location e.g. an Internet server or cloud Service
  • the labeled training data can be used location-based to train the neural network. For example, it is conceivable that a neural network is trained only with labeled training data from Bavaria (or another limited area) and that this neural network is only downloaded to cooling devices in Bavaria (or the other limited area). In this way it can be ensured that the training level of the neural network is optimally matched to the locally offered goods, whereby the error rate in object recognition can be significantly reduced.
  • FIG. 1 shows a system with a first device
  • FIG. 2 shows a flow chart of a method
  • Figure 3 shows a further system with a second device.
  • FIG. 1 shows a system 100 with a first device 105 and a second device 110.
  • the devices 105 and 110 can be designed to be integrated with one another.
  • devices 105 and 110 are physically separate from one another, but communicatively connected to one another, as detailed below.
  • the second device 110 can be designed as a central component which is set up to process information from a plurality of first devices 105.
  • the second device 110 can then also be referred to as an external location.
  • the first device 105 is preferably attached in the area of a cash register system 115, which can be used in particular in retail trade in order to determine a purchase price for a number of goods 120 selected by a customer.
  • the goods 120 can be intended in particular for use in a household, in particular a private household, and include, for example, food, drugstore items, animal feed, printed matter or household appliances.
  • Goods 120 shown as an example include sen a poultry, a detergent and a vegetable. Other goods 120 are also possible and in principle the present invention can also be used in the intermediate or wholesale trade.
  • the checkout system 115 usually comprises a processing device 120, which can be connected to a first scanning device 125, for example a scanning device, an input device 130 and / or a scale 135.
  • Information about a recorded product 120 or a single or total price can be provided by means of an output device 140.
  • the output device 140 can comprise a visual display and / or a printer.
  • Product information about a product 120 can be provided via an interface 142, in particular an identification of the product 120.
  • the identification can include an output that can be made available on the output device 140.
  • the identification is preferably directed to a human user and includes a naturally linguistic designation.
  • the goods 120 are usually detected by scanning a label attached to it, for example a barcode or barcode, by means of the first scanning device 125. Loose goods 120 can be identified and counted by a person or weighed using the scale 135. A certain quantity of the goods 120 can be entered by means of the input device 130.
  • the first device 105 includes at least one of a second scanner 145 and a third scanner 150; as well as a processing device 155 and an optional communication device 160 for communication with the second device 110 if this is set up separately from the first device 105.
  • the second scanning device 145 preferably comprises an optical camera for providing a still or moving image of a product 120 that is processed in the checkout system 115.
  • the scanning device 145 can comprise one or more lighting devices that can emit light of predetermined wavelength ranges. Both visible and invisible wavelength ranges can be used for this.
  • the third scanning device 150 is preferably set up to determine geometric dimensions of a product 120 and preferably also works optically.
  • the third scanning device 150 can comprise a LiDAR sensor, a depth camera, a stereo camera or a ToF camera.
  • both scanning devices 145, 150 are designed to be integrated with one another.
  • Each of the scanning devices 145, 150 can also be provided multiple times in the area of the cash register system 115, preferably with a variation of a perspective of a product 120.
  • the scanning devices 145, 150 are set up less to detect an identification of the goods 120 - for example a barcode - than for an optically recognizable property, for example a color, a design, a size, an aspect ratio or a Labeling.
  • the purpose of the scanning is preferably to determine reproducibly detectable optical features of the goods 120.
  • Goods 120 with great optical variation, such as loose goods, fruit, vegetables or fresh meat, can have individually different optical features, which can nevertheless be used to identify them.
  • the scanning devices 145, 150 are preferably mounted in the area of the checkout system 115 in such a way that their scans can be assigned as clearly as possible to other scans, inputs or determinations of the cash register system 115, so that it is ensured that the scans of the devices 145, 150 relate to the same Wa re 120 like a product information provided by the cash register system 115.
  • the scanning devices 125, 145, 150 can be spatially close to one another.
  • the scanning devices 145, 150 are positioned in such a way that they not yet align the goods 120 feel, or the goods 120 after a reorientation that is as random as possible, for example at the end of an inclined plane.
  • the first device 105 and the second device 110 each include a communication device 165 for mutual data transmission.
  • the second device 110 further comprises a processing device 165 and preferably a data memory 170. It is proposed to assign a scan of a product 120 by means of one of the scan devices 145, 150 to product information provided by the checkout system 115.
  • a data record can preferably be created in which the product information includes a label of an optical scan.
  • Generated data sets can be used to train an artificial neural network to create a to enable identification of the goods 120 by means of the network.
  • a trained neural network is trained and / or provided by the second device 110.
  • FIG. 2 shows a flow chart of a method 200 which can be carried out in particular by means of one or both devices 105, 110 of a system 100. It should be noted that the method 200 can comprise a plurality of sub-methods, one of which can also be carried out by means of another system, as is described in more detail below with reference to FIG. 3.
  • a step 205 goods 120 that are received in the area of a cash register system 115, in particular for billing purposes, can be optically scanned.
  • product information of the goods 120 provided by the checkout system 115 can be recorded. It is preferably ensured that steps 205 and 210 are based on the same goods 120.
  • a data record can be created in which the product information is assigned to the scanning that has taken place.
  • the product information can be specified in the form of a label.
  • a boundary for example as a “bounding box” of the depicted goods 120, can be inserted on the scanning. The limitation can easily be determined if the scanning device 145, 150 is attached statically, so that an image part that does not belong to the goods 120 is known.
  • a repository of specific data records can be created.
  • the repository can include a large number of specific data records that can be sorted according to certain criteria. Exemplary ordering criteria include an acquisition time, a determination time, a destination or a determination confidence. Outdated or incorrectly recognized data records can be removed from the repository. Data records from the repository can be made available for training a neural network.
  • an artificial neural network can be trained using previously determined data sets.
  • the trained network can be set up to recognize a commodity 120 on the basis of its optical scanning.
  • information associated with the goods 120 can be provided, for example its designation.
  • further information is assigned to the goods 120, for example an ingredient, a place of purchase, a purchase price, a quantity or an estimated or specified expiry date. This information can then also be provided.
  • the neural network can be used in a different context than one of the devices 105, 110.
  • the network can be used in a household to identify goods 120 that have been procured.
  • the goods 120 can be optically scanned in a step 230, with graphical features and / or dimensions being able to be recorded.
  • the goods 120 can be recognized.
  • Associated information can be provided.
  • An inventory of goods 120 present in the household can be updated to include goods 120.
  • a used or withdrawn item 120 can be identified in the same way and an inventory can be updated accordingly.
  • FIG. 3 shows a system 300 with a third device 305, which is attached to a cooling device 310 by way of example.
  • the refrigerator 310 may include, for example, a refrigerator, a freezer, a combination of both, a wine cooler, or an uncooled pantry.
  • a product 120 is arranged in the cooling device 310 by way of example.
  • the third device 305 comprises a processing device 315 and an optical scanning device 320 preferably directed onto an interior of the cooling device 310.
  • a communication device 325 is optionally provided.
  • a data structure can be obtained from an external point, for example the second device 110, which represents an artificial neural network.
  • the neural network can be implemented by executing the data structure by the processing device 315.
  • the scanning device 320 may include one or more cameras.
  • the scanning device 320 is optionally set up to determine a geometric dimension of a product 120.
  • a light source for illuminating the interior of the cooling device 310 is also optionally provided.
  • the scanning device 320 is adjustably attached to the cooling device 310 in order to determine optical scans from different perspectives.
  • the third device 305 can recognize which goods 120 are in the refrigerator 310 by performing an optical scan of the interior using the scanning device 320 and identifying one or more goods 120 recognizable on the scan by means of the neural network.
  • the device 305 can then provide an inventory of the cooling device 310. For example, it can be determined in an automated manner whether or not certain quantities of predetermined goods 120 are present in front of them. In one embodiment, for example, it can be determined which ingredients are available for the preparation of a predetermined recipe, or which quantities of which ingredients have to be procured in order to convert the recipe into a meal of a predetermined quantity.

Landscapes

  • Business, Economics & Management (AREA)
  • General Physics & Mathematics (AREA)
  • Accounting & Taxation (AREA)
  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Business, Economics & Management (AREA)
  • Strategic Management (AREA)
  • Finance (AREA)
  • Economics (AREA)
  • Development Economics (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Human Resources & Organizations (AREA)
  • Marketing (AREA)
  • Operations Research (AREA)
  • Quality & Reliability (AREA)
  • Tourism & Hospitality (AREA)
  • Cash Registers Or Receiving Machines (AREA)

Abstract

Un dispositif comprend une unité de balayage optique pour fournir un balayage optique d'un article ; une unité de traitement et une interface pour un système de caisse enregistreuse. Le système de caisse enregistreuse est conçu pour fournir des informations de produit par rapport à l'article, et l'unité de traitement est conçue pour associer les informations de produit à un balayage optique.
PCT/EP2021/053947 2020-03-02 2021-02-18 Création et mise à jour d'une base de données de produits WO2021175601A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP21706897.2A EP4115370A1 (fr) 2020-03-02 2021-02-18 Création et mise à jour d'une base de données de produits

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
DE102020202657.2 2020-03-02
DE102020202657.2A DE102020202657A1 (de) 2020-03-02 2020-03-02 Erstellung und Aktualisierung einer Produktdatenbank

Publications (1)

Publication Number Publication Date
WO2021175601A1 true WO2021175601A1 (fr) 2021-09-10

Family

ID=74672316

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2021/053947 WO2021175601A1 (fr) 2020-03-02 2021-02-18 Création et mise à jour d'une base de données de produits

Country Status (3)

Country Link
EP (1) EP4115370A1 (fr)
DE (1) DE102020202657A1 (fr)
WO (1) WO2021175601A1 (fr)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
AT526299A1 (de) * 2022-07-06 2024-01-15 Manuel Loew Beer Mba Computerimplementiertes Verfahren zur Identifikation eines Artikels, Computerprogrammprodukt, sowie System

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018154411A2 (fr) * 2017-02-22 2018-08-30 Savis Retail Private Limited Distributeurs automatiques et procédés de distribution de produits
US20190236363A1 (en) * 2018-01-31 2019-08-01 Walmart Apollo, Llc Systems and methods for verifying machine-readable label associated with merchandise

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018144650A1 (fr) 2017-01-31 2018-08-09 Focal Systems, Inc. Système de caisse automatisée par l'intermédiaire d'unités d'achat mobiles
US20190034897A1 (en) 2017-07-26 2019-01-31 Sbot Technologies Inc. Self-Checkout Anti-Theft Vehicle Systems and Methods

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018154411A2 (fr) * 2017-02-22 2018-08-30 Savis Retail Private Limited Distributeurs automatiques et procédés de distribution de produits
US20190236363A1 (en) * 2018-01-31 2019-08-01 Walmart Apollo, Llc Systems and methods for verifying machine-readable label associated with merchandise

Also Published As

Publication number Publication date
EP4115370A1 (fr) 2023-01-11
DE102020202657A1 (de) 2021-09-02

Similar Documents

Publication Publication Date Title
CN107045641B (zh) 一种基于图像识别技术的货架识别方法
CN108846621B (zh) 一种基于策略模块的库存管理系统
CN101268478B (zh) 采用视频分析检测可疑活动的方法及装置
DE4217045C2 (de) System zum Verkauf von Waren
CN101809601B (zh) 基于图像处理的货架图提取
DE3913162C2 (de) Selbstabfertigungs-Vorrichtung zur unbeaufsichtigten Abfertigung von zum Kauf ausgewählten Artikeln
DE60022245T2 (de) Verfahren und Vorrichtung zur Detektion und Dekodierung eines zusätzlichen Strichkodes
US5154260A (en) Method and system for automated processing of articles
DE2113068A1 (de) Vorrichtung zur Handhabung von Waren
EP2105845A1 (fr) Procédé d'identification
WO2021175601A1 (fr) Création et mise à jour d'une base de données de produits
DE19921748A1 (de) System und Verfahren zur Steuerung automatisierter und personalloser Logistik-, Verleih- und Verkaufsysteme
DE102008044795A1 (de) Warenerfassung an Selbstbedienungskassensystemen
EP2230650B1 (fr) Dispositif et procédé de détection de marchandises
EP3896629A1 (fr) Suivi des produits végétaux et/ou animaux
US20170091703A1 (en) Tracking merchandise using watermarked bags
DE202008018010U1 (de) Obst und Gemüse Identifikations- und Preiskalkulations-System für Kassen
CN109767236B (zh) 基于位置大数据的商品竞争态势分析方法、设备、介质
DE102021100085A1 (de) Verfahren zur synchronisierung einer strichcodedekodiererung mit einer videokamera zur genauigkeitsverbesserung einer verlustprävention an einzelhandels-pos
EP3866128A1 (fr) Système de détection et procédé de détection des objets pour ou dans un distributeur automatique d'acceptation
AT526299A1 (de) Computerimplementiertes Verfahren zur Identifikation eines Artikels, Computerprogrammprodukt, sowie System
US11720922B2 (en) Sensor technology for identifying and assessing store inventory and merchandising
DE102006018836A1 (de) System und Verfahren zur Schätzung von Objektbeständen
DE102019106057A1 (de) Verfahren, Vorrichtung zur Datenverarbeitung und Haltevorrichtung zur Klassierung von Lebensmitteln
DE4035305A1 (de) Verfahren und vorrichtung zur warenerfassung und abrechnung fuer selbstbedienungslaeden

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21706897

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2021706897

Country of ref document: EP

Effective date: 20221004