WO2020237480A1 - 基于图像识别的控制方法与装置 - Google Patents

基于图像识别的控制方法与装置 Download PDF

Info

Publication number
WO2020237480A1
WO2020237480A1 PCT/CN2019/088640 CN2019088640W WO2020237480A1 WO 2020237480 A1 WO2020237480 A1 WO 2020237480A1 CN 2019088640 W CN2019088640 W CN 2019088640W WO 2020237480 A1 WO2020237480 A1 WO 2020237480A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
information
template
database
recognition result
Prior art date
Application number
PCT/CN2019/088640
Other languages
English (en)
French (fr)
Inventor
唐立三
胡飞凰
周冠兴
Original Assignee
西门子股份公司
西门子(中国)有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 西门子股份公司, 西门子(中国)有限公司 filed Critical 西门子股份公司
Priority to PCT/CN2019/088640 priority Critical patent/WO2020237480A1/zh
Priority to CN201980096186.6A priority patent/CN113841156B/zh
Publication of WO2020237480A1 publication Critical patent/WO2020237480A1/zh

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition

Definitions

  • the present invention relates to the field of data processing, in particular to a control method and device based on image recognition.
  • a bill of materials (Bill of Materials, BOM) is used to describe the composition of a company's products, which can indicate the structural relationship between the final assembly, sub-assembly, components, parts, parts, and raw materials, and the required quantity.
  • the manufacturing process of the product is implemented based on the bill of materials.
  • the material information is attached.
  • Material information is usually expressed in the form of delivery notes and packaging labels. These labels are often printed by the material supplier based on a specified format.
  • the present invention proposes a control method and device based on image recognition.
  • a control method based on image recognition including: performing feature extraction on an image to obtain at least one feature image associated with the image; and performing feature extraction on the image based on the at least one feature image. Dividing to obtain at least one image sub-block and position information of the at least one image sub-block; and performing index extraction on the at least one image sub-block to obtain at least one index associated with the at least one image sub-block Information, and obtain the description information associated with the at least one image sub-block based on the at least one index information, and then obtain the recognition result of the image.
  • the image recognition method can realize semantic recognition after the image is divided, and then determine the information contained in the image.
  • the image is divided based on at least a division template associated with the template mark; if the at least one feature image does not include The template tag divides the image based on the type of the at least one characteristic image.
  • the image can be flexibly divided. When there is a division template, the image can be divided and recognized faster; when there is no division template, the image can also be divided according to the type of the characteristic image, thereby extending the application scope of the present invention.
  • the image is divided based on the division template; if the division template matches the image If the degree is less than the first threshold, the first part of the image is divided based on the division template, and the second part is divided based on the characteristic image type in the second part of the image, wherein, The first part is a part of the image that matches the division template, and the remaining part of the image includes the second part.
  • a preset form of information text represents the recognition result.
  • the identified content can be presented in a specified form, while reducing the complexity of database maintenance and facilitating integration with other systems.
  • the verification information from the database is obtained, and based on the verification information, it is verified whether the recognition result is abnormal. If the recognition result is abnormal, then based on the type and content of the abnormality, a check is generated.
  • the first update operation instruction of the database is to update the part associated with the recognition result in the database. When the database is updated, the efficiency and accuracy of image recognition can be improved.
  • the quality inspection information of the object to be tested associated with the image is obtained; if the quality inspection information is abnormal, the database is generated The second update operation instruction to update the part associated with the quality inspection information in the database.
  • This embodiment provides for determining whether to maintain the database according to the quality of the object to be tested (for example, the received material).
  • the image before the feature extraction is performed on the image, the image is preprocessed so that the difference between the pixels in the specified area of the image and other pixel areas is greater than or equal to the second threshold .
  • the performance of text recognition can be improved.
  • Another aspect of the present invention also provides an image recognition-based device, including: an extraction module configured to perform feature extraction on an image to obtain at least one feature image associated with the image; and a division module configured to perform feature extraction based on the at least A feature image is used to divide the image to obtain at least one image sub-block and position information of the at least one image sub-block; the identification module is configured to perform index extraction on the at least one image sub-block to obtain and At least one index information associated with the at least one image sub-block, and the description information associated with the at least one image sub-block is obtained based on the at least one index information, so as to obtain a recognition result of the image.
  • an extraction module configured to perform feature extraction on an image to obtain at least one feature image associated with the image
  • a division module configured to perform feature extraction based on the at least A feature image is used to divide the image to obtain at least one image sub-block and position information of the at least one image sub-block
  • the identification module is configured to perform index extraction on the at least one image sub-block to
  • a computer storage medium on which computer executable instructions are stored, and when the executable instructions are executed, the aforementioned method is executed.
  • a computer device which includes a memory and a processor, and computer executable instructions are stored on the memory, and when the executable instructions are executed, the processor is caused to execute the aforementioned method.
  • Another aspect of the present invention also proposes a computer program product, which is tangibly stored on a computer-readable medium and includes computer-executable instructions that, when executed, cause at least one processing The device executes the aforementioned method.
  • the material receiving and inspection system can be better integrated with other production systems, the entire factory can be automated, a large number of manual operations and human errors can be reduced, and the cost can be reduced.
  • Fig. 1 is a flowchart of an image recognition method according to an embodiment of the present invention
  • FIG. 2 is a structural diagram of an image recognition device according to an embodiment of the present invention.
  • Figure 3 is a flow chart of material inspection according to an embodiment of the present invention.
  • Fig. 4 is a schematic diagram of a network architecture of a process control system according to an embodiment of the present invention.
  • Semantics refers to the meaning of the field, and multiple different fields can express the same semantics.
  • the semantics of the image refers to the meaning of the characteristic images, characters, etc. in the image.
  • the present invention proposes a data processing method and system based on image recognition to realize the automatic process of material receiving and detection.
  • the following describes the data processing method and system for image recognition proposed by the present invention with reference to the accompanying drawings.
  • FIG. 1 is a flowchart of an image recognition method according to an embodiment of the present invention.
  • Step S101 preprocessing the obtained image.
  • the obtained image type, definition, color or other characteristics may be different. Therefore, in this step, the obtained image will be preprocessed so that the The image is easier to recognize.
  • the preprocessing in this embodiment can be inverting colors, adjusting contrast, removing gray areas, removing bubbles, etc., so as to make the difference between pixels in a specified area (for example, characters, feature codes) and surrounding pixels in the image Greater than or equal to the specified threshold.
  • Step S102 Perform feature image extraction on the preprocessed image.
  • the classification of the characteristic image in the image can be determined, for example, a characteristic code, a table, and a predetermined mark. It is understandable that an image may include one or more types of different characteristic images. For example, when the characteristic image includes a predetermined mark (for example, a supplier’s logo, a characteristic code, etc.), the corresponding supplier can be identified by identifying the logo in the image, and then the supplier can be found in the database. The corresponding partition template.
  • a predetermined mark for example, a supplier’s logo, a characteristic code, etc.
  • the above-mentioned characteristic image is stored in a database in the form of key information, and is continuously improved during the life cycle.
  • the key information can be stored in the database in the form of key-value pairs.
  • the keyword K can be the category of the feature image, the formatted logo of the related company, the name of the related company, etc.
  • the corresponding value V can be the information set of these keywords, for example, company information, delivery note templates, and material packaging labels, etc. Wait.
  • the image can be divided based on the characteristic image to determine at least one image sub-block and the position information of the sub-block.
  • Step S103 is performed to determine whether the feature image points to an existing division template. Specifically, if the feature image includes a template mark, it is further determined whether the degree of matching between the divided template associated with the template mark and the image is greater than or equal to a specified threshold (step S104). It is understandable that the template mark here can be the supplier's logo or other designated marks.
  • the matching degree between the division template and the image is less than the specified threshold, the first part of the image is divided based on the division template, and the second part is divided based on the characteristic image type in the second part of the image, where the first part is the image
  • the remaining part of the image includes the second part (step S105); if the matching degree between the partition template and the image is greater than or equal to the specified threshold, the image is divided based on the partition template (step 109).
  • the matching degree here can be the similarity between the division template and the layout of the image sub-blocks in the image, or other parameters that measure the degree of matching between the division template and the image.
  • the partition template A corresponding to the supplier can be found in the database through the feature image
  • the partition template A can be used to partition the image. It is understandable that in the image provided by the supplier A, at least a part of the image is regularly distributed according to the existing division template A. Therefore, for this part of the image with regularity, the image can be divided based on the division template A to obtain image sub-blocks and corresponding position information; for the remaining part of the image, it can be based at least on the characteristics of the feature image included in this part. According to the type, at least one image sub-block is obtained.
  • the remaining part of the image may also include a third part, which is divided according to a specified division rule, for example, divided by factors such as size, position, and shape.
  • the image sub-block corresponding to the feature code can be determined based on a preset rule.
  • the image sub-block corresponds to an area formed by at least one edge of the barcode extending a specified length in a direction perpendicular to the edge for subsequent identification.
  • the image includes a table or other extraction marks (for example, blank lines, black spaces, bold characters)
  • the area corresponding to the table or the area corresponding to the extraction mark can also be set according to the specified rules Divide from the image.
  • the upper and lower edges of the table can be used as the basis to extend the specified length upwards and downwards respectively to determine the image sub-blocks corresponding to the table.
  • the position and size relationship between the image sub-blocks and the characteristic image here can be set based on specific applications, and there is no need to list them all here.
  • the existing division rules corresponding to supplier A can also be updated, that is, the division template can be adjusted based on the correctness of the final recognition result.
  • the image partition rule can be determined based on the type of the extracted characteristic image, and then the image is partitioned based on the partition rule (step S110).
  • Step S106 Recognize the divided image sub-blocks.
  • index extraction is performed on the image sub-blocks (for example, to identify characters and feature codes in the image) to obtain the index information contained in the image sub-blocks.
  • the characteristic image is a barcode
  • the corresponding image sub-block may include a character/two-dimensional code located above the barcode, and index information can be obtained by recognizing the character/two-dimensional code.
  • the number strings represented by the multiple barcodes may overlap.
  • the description information associated with the multiple barcodes that is, the product category and related information, can be found in the database through the index information, the number string and the position of the barcode corresponding to the barcode.
  • Step S107 Based on the description information, obtain a recognition result with a specified representation form.
  • the term "phone number” may be expressed in different languages or abbreviations (for example, Tel, phone, telephone).
  • "Tel, phone, phone” are all mapped to the "phone number” in the database. Therefore, through learning models and/or matching rules, one entry corresponding to multiple fields of different representations can be obtained in the database, that is, many-to-one conversion can be realized.
  • both the learning model and the matching rule are used for semantic matching, so as to convert the recognition content of the image sub-blocks into entries in a preset form of information text (for example, standardized information text).
  • the recognized content in a specified representation form (for example, standardized information text), it is convenient for users to use and integrate between multiple systems.
  • a specified representation form for example, standardized information text
  • phone number expressed in different languages or abbreviations can be expressed in designated terms (for example, Tel No.), and other systems can directly call the identification content without re-identification or conversion. .
  • Step S108 verify the recognition result.
  • the verification information from the database is obtained, and based on the verification information, whether there is an abnormality in the identification result is verified, that is, the identification content can be verified by the information that has been stored in the database. If the recognition result is abnormal, based on the type and content of the abnormality, a first update operation instruction for the database is generated. It is understandable that when the database is operated according to the first update operation instruction, the part related to the recognition result (for example, the template involved in image recognition, the division rule, etc.) can be updated. As an example, at least one of the following items can be updated: key information, division rules, division templates, learning models, order information.
  • the identification result may be verified by calling order information in the database, and the verified and corrected identification content may be provided to the database. It is understandable that the recognition result with the specified representation can be easily verified with the information in the database.
  • the template can be updated based on the verification result. If the verification result is accurate, the division template is generated or updated based on the above division rule for subsequent use. For example, when there is no division template, the division template may be generated based on at least the index information and the position of the image sub-block.
  • FIG. 2 is a structural diagram of an image recognition device according to an embodiment of the invention.
  • the image processing device 200 includes a preprocessing module 201, an extraction module 202, a division module 203, an identification module 204, and a verification module 205.
  • the preprocessing module 201 is configured to preprocess the obtained image so that the difference between the pixel of the text in the image and other pixels reaches a specified threshold.
  • the extraction module 202 receives the preprocessed image, and then performs feature image extraction on the image to obtain at least one feature image associated with the image, such as feature codes, tables, and/or other extraction marks (for example, blank lines, black Spaces, bold letters, etc.).
  • the dividing module 203 divides the image based on the obtained characteristic image to obtain image sub-blocks respectively corresponding to the characteristic image and positions of the image sub-blocks, wherein the image sub-blocks include the characteristic image.
  • the recognition module 204 recognizes the image sub-block to determine the index information of the image sub-block, and then determines the data record corresponding to the index information.
  • the recognition module 204 also converts the recognized content into an item of information text in a preset form based on the specified matching rule.
  • the verification module 205 is configured to obtain verification information from the verification database, and based on the verification information, verify whether the aforementioned identification content is abnormal, and if there is an abnormality, generate a corresponding database update operation instruction based on the type and content of the abnormality.
  • Fig. 3 is a flow chart of material inspection according to an embodiment of the present invention.
  • step S301 is performed to obtain the recognition result of the image representing the material, and then step S302 is performed to determine whether the recognition result is abnormal.
  • step S303 is executed to obtain the material inspection result; otherwise, step S306 is executed to update the relevant content in the database. For example, a first update operation instruction to the database is generated to update the part of the database associated with the image recognition result.
  • Step S304 is executed to determine whether the inspection of the material has passed, that is, whether the inspection result of the material is abnormal. If the accurate and accurate batch reaches a certain value, the inspection plan can be adjusted periodically (S305), for example, to reduce the material If it is inaccurate, execute step S306 to generate a second update operation instruction.
  • the inspection plan of the object to be tested can be updated in the database based on the type of the abnormality and/or the stage of the abnormality.
  • Fig. 4 is a schematic diagram of a network architecture of a process control system according to an embodiment of the present invention.
  • the data processing system DPS receives image data from the image acquisition device IMD (for example, multiple images of the delivery note and/or material package label captured by the field device), and determines the received image data by recognizing the image. Of materials.
  • the aforementioned image acquisition device may include, but is not limited to, a 2D color or grayscale camera, a 3D camera, and the like.
  • the 2D camera can be used to check whether the material package is damaged, and the 3D camera can be used to check whether the shape of the material is consistent with the pre-recorded shape in the database.
  • the data processing system DPS communicates with the ERP system to query material orders and supplier information from the ERP system, and can also submit material inspection results to the ERP system to perform corresponding operations on the material order, such as completing the purchase Order or return non-conforming materials.
  • the data processing system DPS also communicates with the failure data system EDS to obtain information on the use of materials and quality inspections in the production process, such as information on the use of materials, information on the quality consistency of the materials in the production process, and then update The contents of the database involved in the image recognition and material inspection of the data processing system, such as models and rules.
  • the data processing system DPS is also in communication connection with a remote server (for example, a cloud server) to implement material usage data analysis, update models and rules, manage suppliers, etc. on the cloud server.
  • a remote server for example, a cloud server
  • the processes of the methods recorded in FIGS. 1 and 3 also represent computer-readable instructions, and the computer-readable instructions include a program executed by a processor.
  • the program can be embodied in a tangible computer readable medium, such as CD-ROM, floppy disk, hard disk, digital versatile disk (DVD), Blu-ray disk, or other forms of memory.
  • some or all of the steps in the example methods in Figures 1 and 3 can use application specific integrated circuits (ASIC), programmable logic devices (PLD), field programmable logic devices (EPLD), discrete logic, hardware, and firmware. Any combination of etc. is realized.
  • Information can be stored on a readable medium for any time. It is understandable that the computer-readable instructions can also be stored in a network server or on a cloud platform for the convenience of users.
  • the present invention also provides a computer device, which includes a processor and a memory.
  • the memory is used to store instructions, and when the instructions are executed, the processor executes the methods described in FIGS. 1 and 3.
  • Another aspect of the present invention also provides a computer program product, which is tangibly stored on a computer-readable medium and includes computer-executable instructions.
  • the computer-executable instructions When executed, at least one processor Perform the method described in Figures 1 and 3.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • Image Analysis (AREA)

Abstract

本发明公开了基于图像识别的控制方法与装置。该方法包括:对图像进行特征提取,以获得与图像相关联的至少一个特征图像;基于至少一个特征图像来对图像进行划分,以获得至少一个图像子块以及至少一个图像子块的位置信息;以及对至少一个图像子块进行索引提取,以获得与至少一个图像子块相关联的至少一个索引信息,并且基于至少一个索引信息获得与至少一个图像子块相关联的描述信息,进而获得图像的识别结果。通过采用本发明的技术方案,可以提升图像识别的效率,还可以使得具备图像识别的物料接收和检验的系统可以更好地与其他生产系统集成。

Description

基于图像识别的控制方法与装置 技术领域
本发明涉及数据处理领域,特别涉及一种基于图像识别的控制方法与装置。
背景技术
物料清单(Bill of Materials,BOM)用来描述企业产品组成,其可以表明产品的总装件、分装件、组件、部件、零件、直到原材料之间的结构关系,以及所需的数量。
在产品的制造过程基于物料清单来实施。当材料从材料供应商交付到产品制造工厂时,会附上材料信息。材料信息通常以交货单和包装标签的形式来表示,这些标签往往由材料供应商基于指定的格式进行打印。
发明内容
针对传统方法的物料计划和质量策略很难及时更新的,本发明提出一种基于图像识别的控制方法与装置。
本发明一方面提出了一种基于图像识别的控制方法,包括:对图像进行特征提取,以获得与所述图像相关联的至少一个特征图像;基于所述至少一个特征图像来对所述图像进行划分,以获得至少一个图像子块以及所述至少一个图像子块的位置信息;以及对所述至少一个图像子块进行索引提取,以获得与所述至少一个图像子块相关联的至少一个索引信息,并且基于所述至少一个索引信息获得与所述至少一个图像子块相关联的描述信息,进而获得所述图像的识别结果。该图像识别方法可以实现对图像的划分后进行语义识别,进而确定图像中所包含的信息。
在一种实施方式中,如果所述至少一个特征图像中包括模板标记,则至少基于与所述模板标记相关联的划分模板来对所述图像进行划分;如果所述 至少一个特征图像中未包括所述模板标记,则基于所述至少一个特征图像的类型来对所述图像进行划分。通过此实施方式,可以灵活地对图像进行划分处理。当存在划分模板时,可以较快地划分、识别图像;当不存在划分模板时,也可以根据特征图像的类型来划分图像,从而延展了本发明的应用范围。
在一种实施方式中,如果所述划分模板与所述图像的匹配度大于等于第一阈值,则基于所述划分模板来对所述图像进行划分;如果所述划分模板与所述图像的匹配度小于所述第一阈值,则基于所述划分模板对所述图像的第一部分进行划分,并且基于所述图像的第二部分中的特征图像类型来对所述第二部分进行划分,其中,所述第一部分为所述图像中与所述划分模板相匹配的部分,所述图像的剩余部分包括所述第二部分。该实施方式更加详细地阐述了如何利用划分模板来划分图像,并且实现了划分模板与特征图像结合使用的特点。
在一种实施方式中,基于所述描述信息,以预设形式的信息文本表示所述识别结果。通过执行该实施方式,可以将所识别的内容以指定的形式呈现,同时降低了数据库维护的复杂性,便于与其它系统集成。
在一种实施方式中,获得来自数据库的验证信息,基于所述验证信息来验证所述识别结果是否存在异常,若所述识别结果存在异常,则基于所述异常的类型以及内容,生成对所述数据库的第一更新操作指示,以更新所述数据库中与所述识别结果相关联的部分。当数据库更新后,可以提升图像识别的效率以及准确度。
在一种实施方式中,若所述图像的识别结果不存在异常,则获得与所述图像相关联的待测物的质量检验信息;如果所述质量检验信息存在异常,则生成对所述数据库的第二更新操作指示,以更新所述数据库中与所述质量检验信息相关联的部分。该实施方式提供了根据待测物(譬如,所接收的物料)的质量来确定是否对数据库进行维护。
在一种实施方式中,在对所述图像进行特征提取之前,对所述图像进行预处理,以使得所述图像中的指定区域的像素与其他像素区域之间的差值大于等于第二阈值。通过执行该实施方式,可以提高文本识别的性能。
本发明另一方面还提出了基于图像识别的装置,包括:提取模块,配置为对图像进行特征提取,以获得与所述图像相关联的至少一个特征图像;划分模块,配置为基于所述至少一个特征图像来对所述图像进行划分,以获得 至少一个图像子块以及所述至少一个图像子块的位置信息;识别模块,配置为对所述至少一个图像子块进行索引提取,以获得与所述至少一个图像子块相关联的至少一个索引信息,并且基于所述至少一个索引信息获得与所述至少一个图像子块相关联的描述信息,进而获得所述图像的识别结果。
本发明另一方面还提出了计算机存储介质,该介质上其上存储有计算机可执行指令,当所述可执行指令被执行时,执行前述的方法。
本发明另一方面还提出了计算机装置,其包括存储器与处理器,所述存储器上存储有计算机可执行指令,当所述可执行指令被执行时,使得所述处理器执行前述的方法。
本发明另一方面还提出了计算机程序产品,所述计算机程序产品被有形地存储在计算机可读介质上,并且包括计算机可执行指令,所述计算机可执行指令在被执行时,使至少一个处理器执行前述的方法。
通过采用本发明的技术方案,可以使得物料接收和检查系统可以更好地与其他生产系统集成,能够实现整个工厂的自动化,减少了大量的人工操作以及人为的错误,降低了成本。
附图说明
参考附图示出并阐明实施例。这些附图用于阐明基本原理,从而仅仅示出了对于理解基本原理必要的方面。这些附图不是按比例的。在附图中,相同的附图标记表示相似的特征。
图1为依据本发明实施例的图像识别方法的流程图;
图2为依据本发明实施例的图像识别装置架构图;
图3为依据本发明实施例的物料检验流程图;
图4为依据本发明实施例的过程控制系统的网络架构示意图。
具体实施方式
在以下优选的实施例的具体描述中,将参考构成本发明一部分的所附的附图。所附的附图通过示例的方式示出了能够实现本发明的特定的实施例。示例的实施例并不旨在穷尽根据本发明的所有实施例。可以理解,在不偏离本发明的范围的前提下,可以利用其他实施例,也可以进行结构性或者逻辑性的修改。因此,以下的具体描述并非限制性的,且本发明的范围由所附的 权利要求所限定。
对于相关领域普通技术人员已知的技术、方法和设备可能不作详细讨论,但在适当情况下,所述技术、方法和设备应当被视为说明书的一部分。
首先,对本发明所涉及到的术语进行阐述。语义是指字段所要表达的意思,多个不同的字段可以表示同一个语义。图像的语义则是指图像中的特征图像、字符等等所要表达的意思。
通过大量的实践,发明人发现对于来自不同供应商的物料清单,材料信息往往会采用不同的语言,甚至是以手写的方式来呈现。为了获得准确的物料清单,材料接收和检查高度依赖于工厂的人工操作,这导致了较低的生产率,还会导致异常和产品质量问题,例如,相关人员必须比较一些无规则的数字来检查材料类型,当错误类型材料被当成正确的批次时,会导致最终的产品不合格。另外,由于需要大量的人工操作和非实时反馈,基于传统方法的物料计划和质量策略很难及时更新。
基于上述问题,本发明提出了一种基于图像识别的数据处理方法和系统,来实现物料接收和检测的自动化过程。下面结合附图来阐述本发明所提出的用于图像识别的数据处理方法和系统。
首先,请参阅图1,其为依据本发明实施例的图像识别方法的流程图。
步骤S101:对所获得的图像进行预处理。
由上文可知,对于不同的图像来源,所获得的图像类型、清晰度、色彩或是其他特征可能不同,因此,在该步骤中,将对所获得的图像进行预处理,以使得经预处理的图像更加便于识别。本实施例中的预处理可以是反色、调整对比度、去除灰色区域、去除气泡等方式,以使得图像中的指定的区域(譬如,字符、特征码)的像素与周边的像素之间的差大于等于指定的阈值。
步骤S102:对经预处理的图像进行特征图像提取。
通过提取到的至少一个特征图像,可以确定图像中与特征图像的分类,譬如,特征码、表格、预定的标记等。可以理解的,一幅图像中可以包括一类或多类不同的特征图像。例如,当特征图像包括预定的标记(譬如,供应商的徽标、特征码等)时,可以通过识别图像中的徽标来确定与之相对应的供应商,进而在数据库中找到与该供应商相对应的划分模板。
在一种实施方式中,上述的特征图像以关键信息的形式存储在数据库中,并且在生命周期内持续改进。譬如,该关键信息可以按照键值对的形式 存储在数据库中。关键字K可以是特征图像的类别、相关公司的格式化标识、相关公司的名称等,相应的值V可以是这些关键字的信息集,例如,公司信息、交货单模板和材料包装标签等等。
然后,可以基于特征图像来对图像进行划分,以确定至少一个图像子块以及该子块的位置信息。
执行步骤S103以判断特征图像是否指向现有的划分模板。具体地,如果特征图像中包括模板标记,则进一步判断与模板标记相关联的划分模板与图像的匹配度是否大于等于指定阈值(步骤S104)。可以理解的,这里的模板标记可以是供应商的徽标也可以是其他指定的标记。如果划分模板与图像的匹配度小于指定阈值,则基于划分模板对图像的第一部分进行划分,并且基于图像的第二部分中的特征图像类型来对第二部分进行划分,其中,第一部分为图像中与划分模板相匹配的部分,图像的剩余部分包括第二部分(步骤S105);如果该划分模板与图像的匹配度大于等于指定阈值,则基于划分模板来对图像进行划分(步骤109)。这里的匹配度可以是划分模板与图像中图像子块布局的相似度,也可以是其它衡量划分模板与图像之间匹配程度的参数。
例如,若通过该特征图像能够在数据库中找到与供应商相对应的划分模板A,则可以利用该划分模板A对图像进行划分。可以理解的,在供应商A所提供的图像中,该图像中的至少一部分是依照现有的划分模板A规律性地分布。因此,针对具有规律的这一部分图像,可以基于划分模板A来对图像进行划分,以获得图像子块以及相应的位置信息;针对图像的剩余部分,则可以至少基于该部分所包括的特征图像的类型来进行划分,从而获得至少一个图像子块。换而言之,图像的剩余部分还可以包括第三部分,该第三部分以指定的划分规则进行划分,譬如,以尺寸、位置、形状等因素来划分。
以特征码(例如,条形码、二维码等)为例,可以基于预设的规则来确定与该特征码对应的图像子块。具体而言,当该特征码为条形码时,该图像子块对应于该条形码的至少一条边缘在与该边缘垂直方向上延伸指定的长度后形成的区域,以供后续识别。可以理解的,当图像中包括表格或是其他提取标记(例如,空行、黑色空格、粗体字)时,也可以将与该表格对应的区域或与提取标记对应的区域,以指定的规则从图像中划分出。譬如,可以将表格的上下边缘作为基础,分别向上、向下延伸指定的长度,进而确定与 该表格相对应的图像子块。这里的图像子块与特征图像之间的位置尺寸关系可以基于具体的应用来设定,在此无需穷尽列举。在一种实施方式中,通过上述步骤,还可以对现有的与供应商A相对应的划分规则进行更新,即基于最终的识别结果的正确性,来调整划分模板。
若数据库中不存在与该供应商相对应的划分模板,则可以基于所提取的特征图像的类型来确定图像的划分规则,进而基于该划分规则来对该图像进行划分(步骤S110)。
步骤S106:对划分后的图像子块进行识别。
在该步骤中,对图像子块进行索引提取(譬如,识别图像中的字符、特征码等等),以获得图像子块的所包含的索引信息。譬如,当特征图像是条形码时,与其相对应的图像子块中可以包括位于条形码上方的字符/二维码,通过识别该字符/二维码,可以获得索引信息。
当图像中包括多个条形码时,该多个条形码识别后所表示的数字串可能存在重合。可以通过与该条形码对应的索引信息、数字串以及条形码的位置来在数据库中找到该多个条形码相关联的描述信息,即,产品类别以及相关信息。
步骤S107:基于描述信息,获得具有指定表示形式的识别结果。
在实际应用中,对于相同的语义关键词,不同的供应商可能会给出不同的类型。例如,对于不同的供应商,词条“电话号码”可能采用不同语言或缩写的方式来表示(例如,Tel、phone、电话)。在该步骤中,基于预设的学习模型和/或匹配规则,将“Tel、phone、电话”均映射到数据库中的“电话号码”。因此,通过学习模型和/或匹配规则,可以在数据库中获得与多个不同表示形式的字段相对应的一个词条,即实现多对一的转换。换而言之,学习模型、匹配规则均用于语义匹配,以将图像子块的识别内容转换成以预设形式的信息文本(例如,标准化信息文本)中的条目。
通过将所识别的内容以指定的表示形式(譬如,标准化信息文本)来表示,可以便于用户的使用以及多个系统间的集成。例如,采用不同语言或缩写的方式来表示的术语“电话号码”能够以指定的术语(譬如,Tel No.)来表示,其它系统可以直接调用该识别内容时,而不需要再次进行识别或转换。
步骤S108:对识别结果进行验证。
在该步骤中,获得来自数据库的验证信息,基于验证信息来验证识别结 果是否存在异常,即可以通过数据库中已经预存的信息对该识别内容进行验证。若识别结果存在异常,则基于异常的类型以及内容,生成对数据库的第一更新操作指示。可以理解的,当根据第一更新操作指示来操作数据库时,可以对与识别结果相关的部分(譬如,图像识别所涉及到的模板、划分规则等等)进行更新。作为示例,可以对以下项中的至少一个进行更新:关键信息、划分规则、划分模板、学习模型、订单信息。在一种实施方式中,可以通过调取数据库中的订单信息来对该识别结果进行验证,并将验证且修正后的识别内容提供到数据库。可以理解的,具有指定的表示形式的识别结果可以便于和数据库中的信息进行验证。
在一种实施方式中,对于前述的两种情形:(1)不存在划分模板(2)存在划分模板但图像还有一部分无法与划分模板匹配,可以基于验证结果来更新模板。若验证结果准确,则基于上述的划分规则来生成或更新划分模板,以供后续使用。例如,当不存在划分模板时,可以至少基于图像子块的索引信息以及位置来生成划分模板。
图2为依据本发明实施例的图像识别装置架构图。
图像处理装置200包括预处理模块201、提取模块202、划分模块203、识别模块204以及验证模块205。具体地,预处理模块201配置为对获得的图像进行预处理,以使得图像中的文字的像素与其他像素之间的差值达到指定的阈值。提取模块202接收经预处理的图像,然后对该图像进行特征图像提取,以获得与图像相关联的至少一个特征图像,譬如,特征码、表格和/或其他提取标记(例如,空行、黑色空格、粗体字等)。
划分模块203基于所获得的特征图像来对图像进行划分,以获得分别与该特征图像相对应的图像子块以及该图像子块的位置,其中,图像子块包括该特征图像。识别模块204对图像子块进行识别,以确定图像子块的索引信息,进而确定与该索引信息相对应的数据记录。识别模块204还基于指定的匹配规则,将识别内容转换为预设形式的信息文本的条目。
验证模块205配置为获得来自验证数据库的验证信息,基于所述验证信息来验证上述的识别内容是否存在异常,若存在异常,则基于异常的类型以及内容,生成相应的数据库更新操作指示。
图3为依据本发明实施例的物料检验流程图。
首先,执行步骤S301,以获得用于表征物料的图像的识别结果,进而 执行步骤S302以判断该识别结果是否产生异常。当该识别结果准确时,执行步骤S303,以获得物料检验结果;反之,则执行步骤S306来以更新数据库中的相关内容。譬如,生成对数据库的第一更新操作指示,以更新据库中与图像识别结果相关联的部分。
执行步骤S304,判断物料的检验是否通过,即判断物料的检验结果是否产生异常,若准确且准确的批次达到一定的数值,可以周期性地调整检验计划(S305),譬如,减少对该物料的检验项目;若不准确,则执行步骤S306来生成第二更新操作指示。当根据第二更新操作指示来操作数据库时,可以对数据库中与质量检验相关的部分进行更新。可以理解的,可以基于异常的类型和/或异常产生的阶段,来在数据库中更新待测物的检验计划。
图4为依据本发明实施例的过程控制系统的网络架构示意图。
如图所示,数据处理系统DPS接收来自图像获得设备IMD的图像数据(譬如,由现场设备捕获的交货单和/或材料包标签的多个图像),通过对图像识别来确定所收到的物料。
上述图像获得设备可以包括但不限于2D彩色或灰度型摄像头、3D摄像头等。2D摄像头可以用于检查材料包是否被损坏,3D摄像头可以用于检查物料的形状与数据库中预先记录的形状是否一致。
数据处理系统DPS与ERP系统通信连接,以从ERP系统中查询材料订单和供应商信息,并且也可以向ERP系统提交物料的检验结果,以针对该物料的订单执行相应的操作,譬如,完成采购订单或退回不合格物料。
另外,数据处理系统DPS还与故障数据系统EDS通信连接,以获得物料在生产过程中的使用和质量检验信息,譬如,物料的使用信息、物料在生产过程中的质量一致性的信息,进而更新数据处理系统在图像识别、物料检验中的所涉及的数据库中的内容,譬如,模型和规则。
在一个实施方式中,数据处理系统DPS还与远程服务器(譬如,云服务器)通信连接,以在云服务器上实现物料使用数据分析、更新模型和规则、管理供应商等等。
图1、3中所记载的方法的流程还代表计算机可读指令,该计算机可读指令包括由处理器执行的程序。该程序可被实体化在被存储于有形计算机可读介质中,该有形计算机可读介质如CD-ROM、软盘、硬盘、数字通用光盘(DVD)、蓝光光盘或其它形式的存储器。替代的,图1、3中的示例方法中 的一些步骤或所有步骤可利用专用集成电路(ASIC)、可编程逻辑器件(PLD)、现场可编程逻辑器件(EPLD)、离散逻辑、硬件、固件等的任意组合被实现。在可读介质上信息可以存储任意时间。可以理解的,该计算机可读指令还可以存储在网络服务器中、云端平台上,以便于用户使用。
本发明还提出了一种计算机装置,该计算机装置包括处理器以及存储器。该存储器用于存储指令,当指令在执行时使得处理器执行图1、3中所记载的方法。
本发明另一方面还提出了计算机程序产品,该计算机程序产品被有形地存储在计算机可读介质上,并且包括计算机可执行指令,当该计算机可执行指令在被执行时,使至少一个处理器执行图1、3中所记载的方法。
上文通过附图和优选实施例对本发明进行了详细展示和说明,然而本发明不限于这些已揭示的实施例,本领域技术人员从中推导出来的其他方案也在本发明的保护范围之内。

Claims (17)

  1. 基于图像识别的控制方法,其特征在于,包括:
    对图像进行特征提取,以获得与所述图像相关联的至少一个特征图像;
    基于所述至少一个特征图像来对所述图像进行划分,以获得至少一个图像子块以及所述至少一个图像子块的位置信息;以及
    对所述至少一个图像子块进行索引提取,以获得与所述至少一个图像子块相关联的至少一个索引信息,并且基于所述至少一个索引信息获得与所述至少一个图像子块相关联的描述信息,进而获得所述图像的识别结果。
  2. 如权利要求1所述的方法,其特征在于,还包括:
    如果所述至少一个特征图像中包括模板标记,则至少基于与所述模板标记相关联的划分模板来对所述图像进行划分;
    如果所述至少一个特征图像中未包括所述模板标记,则基于所述至少一个特征图像的类型来对所述图像进行划分。
  3. 如权利要求2所述的方法,其特征在于,还包括:
    如果所述划分模板与所述图像的匹配度大于等于第一阈值,则基于所述划分模板来对所述图像进行划分;
    如果所述划分模板与所述图像的匹配度小于所述第一阈值,则基于所述划分模板对所述图像的第一部分进行划分,并且基于所述图像的第二部分中的特征图像类型来对所述第二部分进行划分,
    其中,所述第一部分为所述图像中与所述划分模板相匹配的部分,所述图像的剩余部分包括所述第二部分。
  4. 如权利要求1所述的方法,其特征在于,还包括:
    基于所述描述信息,以预设形式的信息文本表示所述识别结果。
  5. 如权利要求1所述的方法,其特征在于,还包括:
    获得来自数据库的验证信息,基于所述验证信息来验证所述识别结果是否存在异常,若所述识别结果存在异常,则基于所述异常的类型以及内容, 生成对所述数据库的第一更新操作指示,以更新所述数据库中与所述识别结果相关联的部分。
  6. 如权利要求5所述的方法,其特征在于,包括:
    若所述图像的识别结果不存在异常,则获得与所述图像相关联的待测物的质量检验信息;
    如果所述质量检验信息存在异常,则生成对所述数据库的第二更新操作指示,以更新所述数据库中与所述质量检验信息相关联的部分。
  7. 如权利要求1所述的方法,其特征在于,包括:
    在对所述图像进行特征提取之前,对所述图像进行预处理,以使得所述图像中的指定区域的像素与其他像素区域之间的差值大于等于第二阈值。
  8. 基于图像识别的装置,其特征在于,包括:
    提取模块,配置为对图像进行特征提取,以获得与所述图像相关联的至少一个特征图像;
    划分模块,配置为基于所述至少一个特征图像来对所述图像进行划分,以获得至少一个图像子块以及所述至少一个图像子块的位置信息;
    识别模块,配置为对所述至少一个图像子块进行索引提取,以获得与所述至少一个图像子块相关联的至少一个索引信息,并且基于所述至少一个索引信息获得与所述至少一个图像子块相关联的描述信息,进而获得所述图像的识别结果。
  9. 如权利要求8所述的装置,其特征在于,所述划分模块还配置为:
    如果所述至少一个特征图像中包括模板标记,则至少基于与所述模板标记相关联的划分模板来对所述图像进行划分;
    如果所述至少一个特征图像中未包括所述模板标记,则基于所述至少一个特征图像的类型来对所述图像进行划分。
  10. 如权利要求9所述的装置,其特征在于,所述划分模块还配置为:
    如果所述划分模板与所述图像的匹配度大于等于第一阈值,则基于所述 划分模板来对所述图像进行划分;
    如果所述划分模板与所述图像的匹配度小于所述第一阈值,则基于所述划分模板对所述图像的第一部分进行划分,并且基于所述图像的第二部分中的特征图像类型来对所述第二部分进行划分,
    其中,所述第一部分为所述图像中与所述划分模板相匹配的部分,所述图像的剩余部分包括所述第二部分。
  11. 如权利要求8所述的装置,其特征在于,所述识别模块还配置为:
    基于所述描述信息,以预设形式的信息文本表示所述识别结果。
  12. 如权利要求8所述的装置,还包括:
    验证模块,配置为获得来自数据库的验证信息,基于所述验证信息来验证所述识别结果是否存在异常,若所述识别结果存在异常,则基于所述异常的类型以及内容,生成对所述数据库的第一更新操作指示,以更新所述数据库中与所述识别结果相关联的部分。
  13. 如权利要求12所述的装置,其特征在于,所述验证模块还配置为:
    若所述图像的识别结果不存在异常,则获得与所述图像相关联的待测物的质量检验信息,
    如果所述质量检验信息表示异常,则生成对所述数据库的第二更新操作指示,以更新所述数据库中与所述质量检验信息相关联的部分。
  14. 如权利要求8所述的装置,其特征在于,还包括:
    预处理模块,配置为在对所述图像进行特征图像提取之前,对所述图像进行预处理,以使得所述图像中的指定区域的像素与其他区域像素之间的差值大于等于第二阈值。
  15. 计算机存储介质,其上存储有计算机可执行指令,当所述可执行指令被执行时,执行如权利要求1至7中任一项所述的方法。
  16. 计算机设备,包括存储器与处理器,所述存储器上存储有计算机可 执行指令,当所述可执行指令被执行时,使得所述处理器执行如权利要求1至7中任一项所述的方法
  17. 计算机程序产品,所述计算机程序产品被有形地存储在计算机可读介质上,并且包括计算机可执行指令,所述计算机可执行指令在被执行时,使至少一个处理器执行如权利要求1至7中任一项所述的方法。
PCT/CN2019/088640 2019-05-27 2019-05-27 基于图像识别的控制方法与装置 WO2020237480A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2019/088640 WO2020237480A1 (zh) 2019-05-27 2019-05-27 基于图像识别的控制方法与装置
CN201980096186.6A CN113841156B (zh) 2019-05-27 2019-05-27 基于图像识别的控制方法与装置

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2019/088640 WO2020237480A1 (zh) 2019-05-27 2019-05-27 基于图像识别的控制方法与装置

Publications (1)

Publication Number Publication Date
WO2020237480A1 true WO2020237480A1 (zh) 2020-12-03

Family

ID=73552194

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/088640 WO2020237480A1 (zh) 2019-05-27 2019-05-27 基于图像识别的控制方法与装置

Country Status (2)

Country Link
CN (1) CN113841156B (zh)
WO (1) WO2020237480A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113807651A (zh) * 2021-08-09 2021-12-17 中建二局安装工程有限公司 基于大数据的构件全过程管理系统、终端设备及管理方法
CN116740581A (zh) * 2023-08-16 2023-09-12 深圳市欢创科技有限公司 确定材质识别模型的方法、回到基站的方法及电子设备
CN117170314A (zh) * 2023-11-02 2023-12-05 天津诺瑞信精密电子有限公司 一种切割钻孔数控机床控制方法及系统

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160217344A1 (en) * 2015-01-22 2016-07-28 Microsoft Technology Licensing, Llc. Optimizing multi-class image classification using patch features
CN105824886A (zh) * 2016-03-10 2016-08-03 西安电子科技大学 基于马尔科夫随机场的快速食物识别方法
CN106708904A (zh) * 2015-11-17 2017-05-24 北京奇虎科技有限公司 对图像进行搜索的方法及装置

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3607228B2 (ja) * 1998-12-17 2005-01-05 松下電器産業株式会社 映像検索データ生成装置および映像検索データ生成方法並びに映像検索装置および映像検索方法
CN102339289B (zh) * 2010-07-21 2014-04-23 阿里巴巴集团控股有限公司 文字信息与图像信息的匹配识别方法及装置
DE102012005325A1 (de) * 2012-03-19 2013-09-19 Ernst Pechtl Maschinelles Bilderkennungsverfahren basierend auf einem Kl-System
CN102831694B (zh) * 2012-08-09 2015-01-14 广州广电运通金融电子股份有限公司 一种图像识别系统及图像存储控制方法
CN105809088B (zh) * 2014-12-30 2019-07-19 清华大学 车辆识别方法和系统
CN106096028A (zh) * 2016-06-24 2016-11-09 北京旷视科技有限公司 基于图像识别的文物索引方法及装置
CN106294535B (zh) * 2016-07-19 2019-06-25 百度在线网络技术(北京)有限公司 网站的识别方法和装置
CN107833280B (zh) * 2017-11-09 2021-05-11 交通运输部天津水运工程科学研究所 一种基于地理网格与图像识别相结合的户外移动增强现实方法
CN108427959A (zh) * 2018-02-07 2018-08-21 北京工业大数据创新中心有限公司 基于图像识别的机台状态采集方法及系统
CN108319964B (zh) * 2018-02-07 2021-10-22 嘉兴学院 一种基于混合特征和流形学习的火灾图像识别方法
CN108446717A (zh) * 2018-02-07 2018-08-24 苏州工业大数据创新中心有限公司 一种基于图像识别的机台状态采集方法及系统

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160217344A1 (en) * 2015-01-22 2016-07-28 Microsoft Technology Licensing, Llc. Optimizing multi-class image classification using patch features
CN106708904A (zh) * 2015-11-17 2017-05-24 北京奇虎科技有限公司 对图像进行搜索的方法及装置
CN105824886A (zh) * 2016-03-10 2016-08-03 西安电子科技大学 基于马尔科夫随机场的快速食物识别方法

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113807651A (zh) * 2021-08-09 2021-12-17 中建二局安装工程有限公司 基于大数据的构件全过程管理系统、终端设备及管理方法
CN113807651B (zh) * 2021-08-09 2024-02-23 中建二局安装工程有限公司 基于大数据的构件全过程管理系统、终端设备及管理方法
CN116740581A (zh) * 2023-08-16 2023-09-12 深圳市欢创科技有限公司 确定材质识别模型的方法、回到基站的方法及电子设备
CN116740581B (zh) * 2023-08-16 2023-10-27 深圳市欢创科技有限公司 确定材质识别模型的方法、回到基站的方法及电子设备
CN117170314A (zh) * 2023-11-02 2023-12-05 天津诺瑞信精密电子有限公司 一种切割钻孔数控机床控制方法及系统
CN117170314B (zh) * 2023-11-02 2024-01-26 天津诺瑞信精密电子有限公司 一种切割钻孔数控机床控制方法及系统

Also Published As

Publication number Publication date
CN113841156A (zh) 2021-12-24
CN113841156B (zh) 2024-06-28

Similar Documents

Publication Publication Date Title
KR102177550B1 (ko) 이미지화된 pid 도면에서 설계 정보를 자동으로 인식하여 분류하는 방법
US10936820B2 (en) Post-filtering of named entities with machine learning
US11756323B2 (en) Method of automatically recognizing and classifying design information in imaged PID drawing and method of automatically creating intelligent PID drawing using design information stored in database
WO2020237480A1 (zh) 基于图像识别的控制方法与装置
CN113344857B (zh) 缺陷检测网络的训练方法、缺陷检测方法和存储介质
US20120106787A1 (en) Apparatus and methods for analysing goods packages
CN111695439A (zh) 图像结构化数据提取方法、电子装置及存储介质
CN110705952A (zh) 一种合同审核方法及装置
US20220292861A1 (en) Docket Analysis Methods and Systems
CN112418813B (zh) 基于智能解析识别的aeo资质智能评级管理系统、方法及存储介质
Caldeira et al. Industrial optical character recognition system in printing quality control of hot-rolled coils identification
CN114758341A (zh) 一种智能化合同图像识别与合同要素抽取方法及装置
WO2022111247A1 (zh) 一种报表分析方法及装置
US11605219B2 (en) Image-processing device, image-processing method, and storage medium on which program is stored
KR102282025B1 (ko) 컴퓨터를 이용한 문서 분류 및 문자 추출 방법
CN111414889B (zh) 基于文字识别的财务报表识别方法及装置
CN112613367A (zh) 票据信息文本框获取方法、系统、设备及存储介质
CN112418204A (zh) 基于纸质文档的文本识别方法、系统及计算机介质
CN116664066A (zh) 一种管理企业计划收入与实际收入的方法及系统
CN110688445B (zh) 一种数字化档案建设方法
CN111159997A (zh) 一种企业投标文档的智能校验方法
CN110889365A (zh) 一种标签信息获取方法及装置
CN111047261A (zh) 一种仓储物流委托单识别方法及系统
JP2020009466A (ja) 陳列状況判定システム
CN110991411A (zh) 适用于物流行业的智能单证结构化提取方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19930739

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19930739

Country of ref document: EP

Kind code of ref document: A1