WO2020037762A1 - Procédé et système d'identification d'informations de produit - Google Patents

Procédé et système d'identification d'informations de produit Download PDF

Info

Publication number
WO2020037762A1
WO2020037762A1 PCT/CN2018/107333 CN2018107333W WO2020037762A1 WO 2020037762 A1 WO2020037762 A1 WO 2020037762A1 CN 2018107333 W CN2018107333 W CN 2018107333W WO 2020037762 A1 WO2020037762 A1 WO 2020037762A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
product
information
commodity
text
Prior art date
Application number
PCT/CN2018/107333
Other languages
English (en)
Chinese (zh)
Inventor
黄鼎隆
斯科特·马修·罗伯特
傅恺
郭胜
Original Assignee
深圳码隆科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳码隆科技有限公司 filed Critical 深圳码隆科技有限公司
Publication of WO2020037762A1 publication Critical patent/WO2020037762A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07G—REGISTERING THE RECEIPT OF CASH, VALUABLES, OR TOKENS
    • G07G1/00—Cash registers
    • G07G1/0036—Checkout procedures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images

Definitions

  • the present application relates to the technical field of unmanned sales, and in particular, to a method and system for identifying commodity information.
  • Unmanned sales is a type of commercial automation. It is not limited by time and place, it can save manpower and facilitate transactions. It is a new form of commercial retail. It is also called a 24-hour unmanned smart supermarket.
  • the unmanned sales model it is particularly important to identify the information of the sales merchandise that is automated, because it involves the subsequent settlement of the sales merchandise.
  • the existing unmanned supermarkets have a cumbersome method for identifying the merchandise information, and the recognition results have large errors and are not accurate.
  • the purpose of this application includes providing a method and system for identifying product information, identifying product information from multi-angle product images, and improving the accuracy of the recognition result.
  • an embodiment of the present application provides a method for identifying commodity information, including:
  • the identifying the image and obtaining the product information and image characteristics of the product include:
  • An image feature of the commodity in the image is determined according to a pre-trained commodity image recognition model.
  • the identifying the text contained in the image block includes:
  • the characters corresponding to the found feature information are obtained.
  • the generating commodity information according to the text includes:
  • the preset template includes multiple types of templates belonging to different types of products, and the corresponding text is extracted from the text contained in the image block and filled into the preset template according to the preset template.
  • Generating the commodity information includes:
  • the corresponding text in the text contained in the image block is extracted to fill in the found template to generate the product information.
  • the image of the obtained commodity includes:
  • the images carrying the same product identifier are integrated to obtain an image of the product.
  • the image block that recognizes that the image contains text includes:
  • the image block where the text is located is identified and separated from the image.
  • the method further includes:
  • the method further includes:
  • the commodity image recognition model is trained based on the image of the commodity.
  • training the product image recognition model based on the image of the product includes:
  • Collect a sample set of product images the image set includes a plurality of image samples, and each of the image samples includes a feature set and a label;
  • the image sample set is imported into the deep neural network model to train the image sample set according to the feature sets and labels of the plurality of image samples to obtain the commodity image recognition model.
  • the method further includes:
  • the product information of the product is converted into voice information of the corresponding product, and the voice information is output.
  • an embodiment of the present application further provides a product information identification system, including:
  • An image acquisition module configured to acquire an image of a commodity
  • An image recognition module configured to recognize the image and obtain product information and image characteristics of the product
  • a synthesis module is configured to synthesize the commodity information and the image features to obtain commodity identification information.
  • the image acquisition module includes a cavity, a tray, a motor, and a camera device disposed on a cavity wall of the cavity;
  • the imaging device is configured to photograph the commodity placed on the tray and rotated by the motor.
  • the image acquisition module includes a cavity and a plurality of camera devices arranged circumferentially on a cavity wall of the cavity;
  • the imaging device is configured to photograph the merchandise at various angles.
  • the embodiments of the present application provide a method and a system for identifying product information.
  • identifying the product information and image features in a product image and combining the two types of information, namely, the product information and image features, to obtain product identification information.
  • the product information identification solution provided in this embodiment can identify product information from multi-angle product images, and combine image features to obtain product identification information, which improves the The accuracy of the recognition results.
  • FIG. 1 is a block diagram of a product information identification system according to an embodiment of the present application.
  • FIG. 2 is a flowchart of a method for identifying commodity information according to an embodiment of the present application
  • step S120 in FIG. 2 is a flowchart of a sub-step of step S120 in FIG. 2;
  • step S220 in FIG. 3 is a flowchart of a sub-step of step S220 in FIG. 3;
  • FIG. 5 is a flowchart of a method for generating commodity information according to an embodiment of the present application.
  • FIG. 6 is a flowchart of sub-steps of step S150 in FIG. 2.
  • a method and system for identifying commodity information provided in the embodiments of the present application, identify commodity information from multi-angle commodity images, and combine image features to obtain commodity identification information, which can improve the accuracy of the recognition result.
  • an embodiment of the present application provides a product information identification system.
  • the product information identification system includes a processing device and an image acquisition module in communication with the processing device.
  • the processing device includes a processor and a memory.
  • the memory stores a software function module stored in the memory in a form of software or firmware.
  • the processor runs a software program stored in the memory and Modules, such as the image recognition module and the synthesis module in the embodiment of the present invention, thereby executing various functional applications and data processing.
  • the method for identifying commodity information in the embodiment of the present invention is implemented together with the image acquisition module.
  • FIG. 2 is a flowchart of a method for identifying commodity information according to an embodiment of the present application.
  • the method for identifying product information includes the following steps:
  • Step S110 obtaining an image of a commodity
  • Step S120 identify the image, obtain the product information and image characteristics of the product
  • Step S130 Combining the commodity information and the image features to obtain commodity identification information.
  • the image acquisition module captures a commodity and acquires an image of the commodity, and sends the image to a processing device.
  • the processor recognizes the product information and image characteristics in the product image, and combines the two types of information, that is, the product information and image characteristics, to obtain the product identification information, which is similar to the existing product identification information simply extracted from the product image This can improve the reliability of recognition results.
  • step S120 provided in the foregoing embodiment includes:
  • Step S210 identifying an image block containing text in the image
  • Step S220 identifying characters contained in the image block, and generating commodity information according to the characters;
  • Step S230 Determine the image features of the commodities in the image according to a pre-trained commodity image recognition model.
  • the processor recognizes the image block containing the text from the product image, and then recognizes the text from the image block to generate the text from the text. At the same time, the processor determines the Image features.
  • the step of identifying text contained in an image block may be implemented by the following sub-steps:
  • Step S221 performing a binarization process on the image block, and dividing the binarized image block into a plurality of text blocks;
  • Step S222 extracting text features of each of the text blocks
  • Step S223, searching a pre-stored feature library to obtain feature information matching the text feature;
  • Step S224 Obtain a character corresponding to the found feature information according to a preset correspondence between the feature information and the character.
  • a feature library is stored in the memory in advance, and the feature database stores a plurality of characters and feature information associated with each character.
  • the image block is binarized. In this way, subsequent consideration of multi-level values of pixels will not be involved in the processing of the image, and the amount of data processing and compression will become smaller.
  • the image block is converted into an image including a plurality of points with a gray value of 0 or 255.
  • a point with a gray value of 0 generally indicates a background area. In this way, the image block after the binarization process can be conveniently divided into a plurality of text blocks.
  • feature extraction can be performed on each text block to obtain text features.
  • the text corresponding to the found feature information is obtained according to the pre-stored relationship between the text and the feature information. In this way, the text contained in the image block can be obtained.
  • generating the product information according to the text in step S220 may also be implemented by the following steps, including:
  • Step S310 Extract the corresponding text from the text contained in the image block according to the preset template and fill it into the preset template to generate product information.
  • the preset template includes items such as a product name, a product type, a production date, a place of origin, and an ingredient list, and the recognized text is filled into corresponding items of the template according to the items in the preset template to generate product information.
  • the preset template includes multiple types of templates belonging to different types of products, and there may be differences in product information that is not required for similar products.
  • product information For example, food products need to obtain information such as production date and shelf life .
  • clothing products it is not necessary to obtain the above information, but to obtain information such as clothing size and clothing materials. Therefore, in order to obtain corresponding product information for different types of products to improve the comprehensiveness and accuracy of the obtained product information, please refer to FIG. 5.
  • Step S310 in this embodiment includes the following sub-steps:
  • Step S311 extract a product name to which the product belongs from the text contained in the image block;
  • step S312 a template belonging to the same category as the product name is found from a plurality of types of templates belonging to different types of goods included in the preset template;
  • step S313 the corresponding text in the text contained in the image block is extracted to be filled into the found template to generate the product information.
  • a preset template is stored in the memory in advance, wherein the preset template includes multiple types of templates belonging to different types of products, for example, different types of books, food products, and clothing products, respectively.
  • Product template includes multiple types of templates belonging to different types of products, for example, different types of books, food products, and clothing products, respectively.
  • the text obtained by the controller from the image block includes the name of the product and other information of the product.
  • a template that belongs to the same category as the product name can be found from a plurality of types of templates that belong to different types of products included in the preset template. For example, when the name of a product included in the text obtained from the image block is "... bread", a template belonging to a food product may be obtained from multiple types of templates.
  • Corresponding text in the text contained in the extracted image block is filled into the found template to generate product information.
  • corresponding templates can be obtained to fill in the information to ensure the accuracy and completeness of the information of different types of products.
  • the information of the merchandise for sale can also be conveniently obtained to help the user to purchase the merchandise.
  • the controller may convert the commodity information into corresponding voice information and output the voice information.
  • a common manner in the prior art may be adopted, which is not described in this embodiment.
  • step S110 may also be implemented by the following steps, including:
  • Step S410 Acquire multiple images of a 360-degree product, in which multiple images of the same type of product carry the same product identifier;
  • step S420 the images carrying the same product identifier are integrated to obtain an image of the product.
  • the image acquisition module can obtain multiple images of the product in all directions, combine the multiple images of various directions to obtain the product image, and then identify the image features from the combined product image And product information;
  • step S210 provided in the above embodiment may also be implemented by the following steps:
  • step S510 the text in the image is separated from the image.
  • step S520 the image block where the text is located is identified and separated from the image.
  • the text and the image are separated first, and then the image block containing the text is separated;
  • the above method further includes:
  • step S140 the product identification information is updated to a plurality of product identification centers and a plurality of product information distribution centers, so that the product identification center and the product information distribution center identify the products according to the product identification information.
  • the method further includes:
  • step S150 a commodity image recognition model is trained based on the commodity image.
  • a commodity image recognition model is trained based on multiple omnidirectionally merged commodity images as samples.
  • step S150 may include the following sub-steps:
  • Step S151 Collect a sample set of product images, where the image set includes a plurality of image samples, and each of the image samples includes a feature set and a label;
  • Step S152 constructing a deep neural network model
  • Step S153 Import the image sample set into the deep neural network model to train the image sample set according to the feature sets and labels of the multiple image samples to obtain the commodity image recognition model.
  • the processor may construct a deep neural network in advance, and collect a large number of image samples to form a commodity image sample set, where each of the image samples in the image sample set includes a feature set and a label.
  • the label is used to identify a category of the image sample.
  • the obtained image sample set is imported into the constructed deep neural network model for training to obtain a commodity image recognition model.
  • the image sample set may include a training subset and a test subset.
  • the commodity image recognition model is obtained by training the training subset.
  • the test image subset may also be used to verify the product image recognition model to obtain the recognition accuracy rate of the product image recognition model. According to the obtained recognition accuracy, the training parameters of the constructed deep neural network model are adjusted to continuously optimize the model.
  • the image acquisition module is configured to acquire an image of a commodity
  • the image recognition module is configured to recognize an image and obtain commodity information and image characteristics of the commodity
  • the combination module is configured to combine product information and image features to obtain product identification information.
  • the image acquisition module includes a cavity, a tray, a motor, and a camera device disposed on a cavity wall of the cavity;
  • the imaging device is configured to photograph a product placed on a tray and rotated by a motor.
  • the image acquisition module includes a cavity and a plurality of camera devices arranged circumferentially on the cavity wall of the cavity; the camera device is configured to take pictures of goods from various angles, without Rotate can also get a full range of images of the product from all angles;
  • the product information identification system provided in the embodiment of the present application has the same technical features as the method for identifying the product information provided in the foregoing embodiment, so it can also solve the same technical problems and achieve the same technical effect.
  • the product information identification method and system computer program product provided in the embodiments of the present application include a computer-readable storage medium storing program code, and the program code includes instructions that can be used to execute the method described in the foregoing method embodiment.
  • program code includes instructions that can be used to execute the method described in the foregoing method embodiment.
  • the terms “installation”, “connected”, and “connected” should be understood in a broad sense unless otherwise specified and limited, for example, they may be fixed connections or removable connections , Or integrally connected; it can be mechanical or electrical; it can be directly connected, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two elements.
  • the specific meanings of the above terms in this application can be understood in specific situations.
  • the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium.
  • the technical solution of this application is essentially a part that contributes to the existing technology or a part of the technical solution can be embodied in the form of a software product.
  • the computer software product is stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in the embodiments of the present application.
  • the foregoing storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disks or optical disks and other media that can store program codes .
  • An embodiment of the present application further provides an electronic device including a memory, a processor, and a computer program stored on the memory and executable on the processor.
  • the processor executes the computer program, the steps of the method for identifying the commodity information provided in the foregoing embodiment are implemented. .
  • An embodiment of the present application further provides a computer-readable storage medium.
  • a computer program is stored on the computer-readable storage medium.
  • the steps of the method for identifying product information in the foregoing embodiment are performed.
  • the commodity information recognition scheme recognizes commodity information and image features in a commodity image, and combines the commodity information and image features to obtain commodity identification information, which improves the accuracy of the recognition result.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

La présente invention se rapporte au domaine technique de la vente en libre-service et concerne un procédé et un système d'identification d'informations de produit, le procédé consistant à : acquérir une image d'un produit ; identifier l'image et obtenir des informations du produit et une caractéristique d'image ; et faire la synthèse des informations de produit et de la caractéristique d'image, puis obtenir des informations d'identification de produit. Les informations de produit peuvent être identifiées à partir d'images du produit sous de multiples angles, ce qui permet d'améliorer la précision d'identification.
PCT/CN2018/107333 2018-08-21 2018-09-25 Procédé et système d'identification d'informations de produit WO2020037762A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810957289.4 2018-08-21
CN201810957289.4A CN109035630A (zh) 2018-08-21 2018-08-21 商品信息识别方法和系统

Publications (1)

Publication Number Publication Date
WO2020037762A1 true WO2020037762A1 (fr) 2020-02-27

Family

ID=64627561

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/107333 WO2020037762A1 (fr) 2018-08-21 2018-09-25 Procédé et système d'identification d'informations de produit

Country Status (2)

Country Link
CN (1) CN109035630A (fr)
WO (1) WO2020037762A1 (fr)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112307967A (zh) * 2020-10-30 2021-02-02 浪潮金融信息技术有限公司 一种商品识别模型训练优化方法
CN112329774A (zh) * 2020-11-10 2021-02-05 杭州微洱网络科技有限公司 基于图像的商品尺码表自动生成方法
CN112613950A (zh) * 2020-12-28 2021-04-06 上海寻梦信息技术有限公司 品类识别模型训练方法、品类识别方法及相关设备

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111080399A (zh) * 2019-11-22 2020-04-28 汉口北进出口服务有限公司 一种商品信息处理方法和装置
CN112163600B (zh) * 2020-09-09 2021-06-08 成都理工大学 一种基于机器视觉的商品识别方法

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103699523A (zh) * 2013-12-16 2014-04-02 深圳先进技术研究院 产品分类方法和装置
CN104462380A (zh) * 2014-12-11 2015-03-25 北京中细软移动互联科技有限公司 商标检索方法
CN104504104A (zh) * 2014-12-30 2015-04-08 百度在线网络技术（北京）有限公司 用于搜索引擎的图片物料处理方法、装置和搜索引擎
US9881223B2 (en) * 2014-04-22 2018-01-30 Lenovo (Singapore) Pte. Ltd. Forming scanned composite document with optical character recognition function

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5244864B2 (ja) * 2010-06-29 2013-07-24 ヤフー株式会社 商品識別装置、方法及びプログラム
CN102799848B (zh) * 2012-06-21 2016-09-21 深圳市江波龙电子有限公司 商品信息识别方法和商品信息识别装置
CN106909946A (zh) * 2017-03-02 2017-06-30 深圳明创自控技术有限公司 一种多模态融合的商品分类系统
CN107292248B (zh) * 2017-06-05 2023-04-07 广州诚予国际市场信息研究有限公司 一种基于图像识别技术的商品管理方法及系统
CN107705066B (zh) * 2017-09-15 2022-01-07 唯品会(海南)电子商务有限公司 一种商品入库时信息录入方法及电子设备
CN108269369A (zh) * 2017-09-27 2018-07-10 中山市宾哥网络科技有限公司 结算箱及其结算方法
CN107748973A (zh) * 2017-09-29 2018-03-02 时瑞科技（深圳）有限公司 商品识别系统及自动识别售卖或入库方法
CN107958252A (zh) * 2017-11-23 2018-04-24 深圳码隆科技有限公司 一种商品识别方法和设备
CN108345912A (zh) * 2018-04-25 2018-07-31 电子科技大学中山学院 一种基于rgbd信息与深度学习的商品快速结算系统

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103699523A (zh) * 2013-12-16 2014-04-02 深圳先进技术研究院 产品分类方法和装置
US9881223B2 (en) * 2014-04-22 2018-01-30 Lenovo (Singapore) Pte. Ltd. Forming scanned composite document with optical character recognition function
CN104462380A (zh) * 2014-12-11 2015-03-25 北京中细软移动互联科技有限公司 商标检索方法
CN104504104A (zh) * 2014-12-30 2015-04-08 百度在线网络技术（北京）有限公司 用于搜索引擎的图片物料处理方法、装置和搜索引擎

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112307967A (zh) * 2020-10-30 2021-02-02 浪潮金融信息技术有限公司 一种商品识别模型训练优化方法
CN112329774A (zh) * 2020-11-10 2021-02-05 杭州微洱网络科技有限公司 基于图像的商品尺码表自动生成方法
CN112329774B (zh) * 2020-11-10 2023-07-28 广州探域科技有限公司 基于图像的商品尺码表自动生成方法
CN112613950A (zh) * 2020-12-28 2021-04-06 上海寻梦信息技术有限公司 品类识别模型训练方法、品类识别方法及相关设备
CN112613950B (zh) * 2020-12-28 2024-03-29 上海寻梦信息技术有限公司 品类识别模型训练方法、品类识别方法及相关设备

Also Published As

Publication number Publication date
CN109035630A (zh) 2018-12-18

Similar Documents

Publication Publication Date Title
WO2020037762A1 (fr) Procédé et système d'identification d'informations de produit
US10116861B1 (en) Guided image capture user interface
WO2018218860A1 (fr) Procédé et dispositif de recommandation de produit de base
WO2020010827A1 (fr) Procédé et dispositif de règlement d'achat basé sur un distributeur automatique de type ouvert
US20180253674A1 (en) System and method for identifying retail products and determining retail product arrangements
CN111311316B (zh) 商户画像的刻画方法、装置、电子设备、验证方法及系统
US20200027148A1 (en) Smart shopping cart, server, smart shopping system and method
CN104112216A (zh) 存货管理与行销的图像辨识方法
WO2019080674A1 (fr) Dispositif, procédé, appareil, support et dispositif électronique de caisse en libre-service
CN110750697B (zh) 商户分类方法、装置、设备及存储介质
WO2020037728A1 (fr) Procédé, système, support d'informations et système de vente automatisée destinés à déterminer des informations de produit
CN113329261B (zh) 视频处理方法及装置
CN114169952A (zh) 一种商品推荐方法、服务器、购物车和购物系统
WO2021179138A1 (fr) Procédé et système permettant d'analyser des produits sur une étagère de supermarché
CN113935774A (zh) 图像处理方法、装置、电子设备及计算机存储介质
EP3779842A1 (fr) Procédé et système de demande d'informations de marchandise
CN110826481A (zh) 数据处理方法、商品识别方法、服务器和存储介质
JP2012014220A (ja) 商品識別装置、方法及びプログラム
CN113177151A (zh) 一种潜在客户筛选方法
CN116739836A (zh) 一种基于知识图谱的餐饮数据分析方法及系统
CN112883719A (zh) 一种品类词识别方法、模型训练方法、装置及系统
CN116308562A (zh) 线下零售用户智能筛选方法
US20190355039A1 (en) Methods of managing and querying product inventory data and inventory management server
CN113255748A (zh) 一种商品识别模型的特征底库更新方法及装置
CN113111800A (zh) 无人货柜商品上架判断方法、设备和存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18930607

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18930607

Country of ref document: EP

Kind code of ref document: A1