WO2023184644A1

WO2023184644A1 - Product information processing method and apparatus based on rpa and ai, and device and medium

Info

Publication number: WO2023184644A1
Application number: PCT/CN2022/091293
Authority: WO
Inventors: 陈愫恺
Original assignee: 来也科技(北京)有限公司
Priority date: 2022-03-31
Filing date: 2022-05-06
Publication date: 2023-10-05
Also published as: CN114495127A

Abstract

The present disclosure provides a product information processing method and apparatus based on RPA and AI, and a device and a medium, which relate to the field of AI and RPA. The method comprises: obtaining, by means of an RPA robot, a product package appearance corresponding to a target product; recognizing text content in the product package appearance on the basis of an OCR technology; obtaining document content in a reference document; comparing the text content with the document content to determine a first difference part in the text content that is different from the document content; and marking anomalies for the first difference part in the text content, and/or marking anomalies for the area in the product package appearance where the first difference part is located.

Description

Product information processing methods, devices, equipment and media based on RPA and AI

Cross-references to related applications

This application is filed based on a Chinese patent application with application number 202210332711.3 and a filing date of March 31, 2022, and claims the priority of the Chinese patent application. The entire content of the Chinese patent application is hereby incorporated by reference into this application.

Technical field

The present disclosure relates to the fields of Artificial Intelligence (AI for short) and Robotic Process Automation (RPA for short), and in particular to a product information processing method, device, equipment and medium based on RPA and AI.

Background technique

RPA uses specific "robot software" to simulate human operations on computers and automatically execute process tasks according to rules.

AI is a technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence.

Intelligent Document Processing (IDP) is based on Optical Character Recognition (OCR), Computer Vision (CV), Natural Language Processing (NLP), Knowledge Graph ( Artificial intelligence technologies such as Knowledge Graph (KG) are used to identify, classify, extract, and verify various types of documents, and are a new generation of automation technology that help enterprises realize the intelligence and automation of document processing.

For products, different styles of packaging may be designed in different periods. For example, for different holidays, product packaging will be designed to match the atmosphere of each holiday. For example, when the product is linked with different celebrities or games, packaging will also be designed. New product packaging and so on. Among them, product packaging generally includes nutritional information, ingredient information, manufacturer, address, place of origin and other information. If the above information is incorrect, it may cause certain legal issues. Therefore, it is very important to check the product information on the product packaging.

In related technologies, when designing new product packaging, employees from multiple departments conduct multiple checks on the product packaging.

However, the above-mentioned manual multiple verification methods are not only inefficient, but also the accuracy of the verification results cannot be guaranteed. In addition, when there are multiple manufacturers, addresses and origins, manual verification is difficult and laborious, and it is easy to miss.

Contents of the invention

The present disclosure aims to solve one of the technical problems in the related art, at least to a certain extent.

To this end, the present disclosure proposes a product information processing method, device, equipment and medium based on RPA and AI to realize automatic verification of product information on product packaging diagrams through RPA robots. On the one hand, it can reduce the amount of manual participation. Free up human resources and reduce labor costs. On the other hand, it can improve the efficiency of product information verification, avoid error-prone manual verification, and improve the accuracy of product information verification results.

The first embodiment of the present disclosure proposes a product information processing method based on RPA and AI. The method is executed by an RPA robot and includes:

Obtain the product packaging image corresponding to the target product, and identify the text content in the product packaging image based on optical character recognition OCR technology;

Obtain a reference document and obtain document content in the reference document, where the document content includes product information corresponding to the target product;

Compare the text content and the document content to determine a first difference part in the text content that is different from the document content;

The first difference part is marked abnormally in the text content, and/or the area where the first difference part is located is marked abnormally in the product packaging diagram.

The second embodiment of the present disclosure proposes a product information processing device based on RPA and AI, applied to RPA robots, including:

The first acquisition module is used to obtain the product packaging diagram corresponding to the target product;

A recognition module, used to identify the text content in the product packaging image based on optical character recognition OCR technology;

The second acquisition module is used to obtain a reference document and obtain the document content in the reference document, where the document content includes product information corresponding to the target product;

A comparison module, configured to compare the text content and the document content to determine the first difference part in the text content that is different from the document content;

A marking module is configured to mark the first difference part abnormally in the text content, and/or mark the area where the first difference part is located in the product packaging diagram abnormally.

The third embodiment of the present disclosure provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the present disclosure is implemented. The method described in the above embodiment of the first aspect.

The fourth embodiment of the present disclosure provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the method described in the first embodiment of the disclosure is implemented.

The fifth aspect embodiment of the present disclosure proposes a computer program product, which includes a computer program. When executed by a processor, the computer program implements the method described in the above first aspect embodiment of the present disclosure.

Obtain the product packaging diagram corresponding to the target product through the RPA robot, and identify the text content in the product packaging diagram based on OCR technology; obtain the reference document, and obtain the document content in the reference document, where the document content includes the product corresponding to the target product information; compare the text content and the document content to determine the first difference part in the text content that is different from the document content; mark the first difference part in the text content as abnormal, and/or, mark the first difference part in the product packaging diagram The area where the first difference part is located is marked as abnormal. As a result, the RPA robot can be used to automatically check the product information on the product packaging map. On the one hand, it can reduce the amount of manual participation, release human resources, and reduce labor costs. On the other hand, it can improve the efficiency of checking product information, and also It can avoid the error-prone situation of manual verification and improve the accuracy of product information verification results.

Additional aspects and advantages of the disclosure will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the disclosure.

Description of drawings

The above and/or additional aspects and advantages of the present disclosure will become apparent and readily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

Figure 1 is a schematic flowchart of a product information processing method based on RPA and AI provided by an embodiment of the present disclosure.

Figure 2 is a schematic flowchart of a product information processing method based on RPA and AI provided by an embodiment of the present disclosure.

Figure 3 is a schematic diagram of each sub-image obtained after segmenting the product packaging image in an embodiment of the present disclosure.

Figure 4 is a partial schematic diagram of a product packaging diagram in an embodiment of the present disclosure.

Figure 5 is a schematic flowchart of a product information processing method based on RPA and AI provided by an embodiment of the present disclosure.

Figure 6 is a schematic diagram of a verification report in an embodiment of the present disclosure.

Figure 7 is a schematic flowchart of a product information processing method based on RPA and AI provided by an embodiment of the present disclosure.

Figure 8 is a schematic flowchart of a product information processing method based on RPA and AI provided by an embodiment of the present disclosure.

Figure 9 is a schematic diagram of the first nutritional component information in an embodiment of the present disclosure.

Figure 10 is a schematic flowchart of a product information processing method based on RPA and AI provided by an embodiment of the present disclosure.

Figure 11 is a schematic diagram of the implementation principle of an embodiment of the present disclosure.

Figure 12 is a partial schematic diagram of a product packaging diagram in an embodiment of the present disclosure.

Figure 13 is a schematic diagram of OCR recognition results in an embodiment of the present disclosure.

Figure 14 is a schematic diagram of the ingredient information extraction results in the embodiment of the present disclosure.

Figure 15 is a schematic diagram of OCR recognition results in an embodiment of the present disclosure.

Figure 16 is a schematic diagram of the extraction results of the factory name, factory address and production license number in the embodiment of the present disclosure.

Figure 17 is a schematic diagram of the third attribute field in an embodiment of the present disclosure.

Figure 18 is a schematic diagram of a configuration template in an embodiment of the present disclosure.

Figure 19 is a schematic diagram of extraction rules or extraction rules for ingredients in an embodiment of the present disclosure.

Figure 20 is a schematic diagram of OCR recognition results in an embodiment of the present disclosure.

Figure 21 is a schematic diagram of OCR recognition results in an embodiment of the present disclosure.

Figure 22 is a schematic structural diagram of a product information processing device based on RPA and AI provided by an embodiment of the present disclosure.

23 illustrates a block diagram of an exemplary electronic device suitable for implementing embodiments of the present disclosure.

Detailed ways

Embodiments of the present disclosure are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present disclosure and are not to be construed as limitations of the present disclosure.

This disclosure proposes a product information processing method, device, equipment and medium based on RPA and AI.

The RPA- and AI-based product information processing methods, devices, equipment, and media of embodiments of the present disclosure are described below with reference to the accompanying drawings. Before describing the embodiments of the present disclosure in detail, in order to facilitate understanding, common technical terms are first introduced:

"RPA", the abbreviation of Robotic Process Automation, provides professional and comprehensive process automation solutions for enterprises and individuals. RPA uses specific "robot software" to simulate human operations on computers and automatically execute process tasks according to rules. That is, the RPA robot can quickly and accurately collect data from the user operation interface by simulating the user's mouse and keyboard operations, process the data based on clear logical rules, and then quickly and accurately input it into another system or interface. As a result, labor cost investment can be significantly reduced, existing office efficiency can be effectively improved, and work can be completed accurately, stably, and quickly.

"AI" is the abbreviation of Artificial Intelligence. It is a technical science that researches and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. AI is the study of using computers to simulate certain human thinking processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.). It has both hardware-level technology and software-level technology. AI hardware technology generally includes technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; AI software technology mainly includes computer vision technology, speech recognition technology, and Natural Language Processing (NLP). technology and machine learning/deep learning, big data processing technology, knowledge graph technology and other major directions.

"Commodities" are the fruits of labor produced for sale and the products of labor used for exchange. For example, products can include food, daily necessities, health products, etc.

"Target product" can be any product. For example, the target product can be a certain food, a certain daily necessities, etc.

"Product packaging drawing", also known as product packaging design drawing, refers to an image including the packaging design of the target product.

"Product information" refers to information related to the target product. For example, the product information may include nutritional information, ingredient information (or composition information), manufacturer, address, origin and other information of the target product.

"Reference document", or document to be compared, refers to a document that includes product information corresponding to the target product. For example, the reference document can be a structured document, such as an Excel document, or the reference document can also be unstructured. Documents, such as Word documents, etc. It should be understood that when the reference document is an unstructured document, in order to facilitate the RPA robot to perform information comparison, the unstructured reference document can be converted into a structured document.

"Optical Character Recognition (OCR)" refers to the process in which electronic equipment examines characters printed on paper, determines their shape by detecting dark and light patterns, and then uses character recognition methods to translate the shape into computer text; That is, for printed characters, it uses optical methods to convert the text in the paper document into a black and white dot matrix image file, and uses recognition software to convert the text in the image into a text format for further editing and processing by word processing software. .

"First attribute field" refers to the attribute field included in the text content corresponding to the product packaging diagram. For example, the first attribute field may include: production license (or production license number, production number), address, Manufacturer, ingredients, storage conditions, shelf life, production date, net content, product type, etc.

"First attribute value" refers to the attribute value corresponding to the first attribute field in the text content. For example, taking the target product as food as an example, the attribute values corresponding to ingredients can be: drinking water, cheese powder, citric acid, etc.

The "second attribute field" refers to the attribute field included in the document content in the reference document. Correspondingly, the second attribute value refers to the attribute value corresponding to the second attribute field in the document content. It should be noted that the second attribute field is the standard attribute field corresponding to the target product, and the second attribute value is the standard attribute value corresponding to the target product.

It should be understood that the first attribute field and/or the first attribute value may have errors in the design process, but the second attribute field and the second attribute value are related to the target product, and the correctly written attribute field and attribute value .

"Set vocabulary list" refers to a preset vocabulary list, which can also be called a custom vocabulary list. The setting vocabulary includes various attribute fields related to the product information of the target product, which are recorded as third attribute fields in this disclosure. For example, the third attribute field may include: production license, address, manufacturer, ingredients, storage conditions, shelf life, production date, net content, product type, etc.

It should be noted that, considering the accuracy of OCR recognition results, for some attribute fields, such as ingredients, the OCR recognition result may be "material", resulting in the word "mixing" not being recognized, or the recognition result may be "ingredient", resulting in There are extra spaces in the recognition results. In response to the above situation, for each attribute field related to product information, you can also count the multiple explanations or multiple possible explanations of the attribute field, and compare the attribute field and the multiple explanations or multiple possible explanations corresponding to the attribute field. Arguments are used as the third attribute field and are set in the setting vocabulary.

For example, for the attribute field "ingredients", the set word list may include: "mixing", "materials", "ingredients", "ingredients", etc.

"Third attribute value" refers to the attribute value corresponding to the third attribute field in the text content corresponding to the product packaging diagram of the target product. For example, taking the target product as food as an example, the attribute value corresponding to the ingredients can be: Drinking water, cheese powder, citric acid, etc.

"Target document" refers to a document containing the product packaging diagram of the target product. For example, the target document can be a PDF (Portable Document Format) document, or it can also be a PSD (PSD is Adobe's graphic design Design documents in formats such as the special format of the software Photoshop), Adobe Illustrator (specifically the file extension of Adobe Illustrator, which is a vector graphics file format).

"First nutritional ingredient information" is the nutritional ingredient information related to the target product included in the text content. For example, taking the target product as food as an example, the first nutritional ingredient information can include: energy, protein, fat, carbohydrate, etc. Ingredient information.

"Second nutritional information" refers to the nutritional information related to the target product included in the document content. It should be understood that the first nutritional information may be wrong in the design process, but the second nutritional information is related to the target product and the correct nutritional information is written.

"Regular expressions", also known as regular expressions, are used to retrieve or replace text that matches a certain pattern (or rule).

"Any text fragment" refers to any text fragment in the first nutritional ingredient information, wherein the same text fragment contains adjacent characters, and/or contains an interval of the first set number (such as 1 or 2, etc.) each character of the space.

"Adjacent text fragment" refers to the text fragment adjacent to the position of "any text fragment" in the "first nutritional ingredient information". For example, the "adjacent text fragment" can be: located on the left side of "any text fragment", Text snippets for the right, top, and bottom sides.

As an example, taking the target product as food, the first nutritional ingredient information can be as shown in Table 1:

Table 1

Assuming that "any text fragment" is "carbohydrate" in Table 1, the "adjacent text fragment" can be "5.8g", "fat", and "sodium".

"Target detection algorithm" belongs to the field of computer vision in the field of AI. It can be based on the target detection algorithm in deep learning technology to detect whether the required content is included in the image.

The product information processing method based on RPA and AI provided by the embodiments of the present disclosure can be applied to RPA robots, which can run in any electronic device with computing capabilities. The electronic device may be a personal computer, a mobile terminal, etc. The mobile terminal is, for example, a mobile phone, a tablet computer, a personal digital assistant and other hardware devices with various operating systems.

As shown in Figure 1, the product information processing method based on RPA and AI can include the following steps:

Step 101: Obtain the product packaging image corresponding to the target product, and identify the text content in the product packaging image based on OCR technology.

In the embodiment of the present disclosure, the product packaging image may be an image in image formats such as JPG (or JPEG (Joint Photographic Experts Group, Joint Photographic Experts Group)), PNG (Portable Network Graphics, Portable Network Graphics), etc.

In a possible implementation of the embodiment of the present disclosure, the RPA robot can directly obtain the product packaging diagram corresponding to the target product.

As an example, you can manually upload or send product packaging pictures to the device where the RPA robot is located. For example, business personnel can take photos of the target product through an image collection device (such as a camera, mobile terminal, etc.) to obtain the product in image file format. Packaging diagram, or the business personnel can scan the paper document containing the product packaging diagram to obtain a document in PDF format, and take a screenshot of the product packaging diagram in the above document to obtain the product packaging diagram in image file format. After obtaining the product packaging diagram, the business personnel can upload or send the product packaging diagram to the device where the RPA robot is located.

In another possible implementation of the embodiment of the present disclosure, the RPA robot can also indirectly obtain the product packaging diagram corresponding to the target product.

As an example, the RPA robot can obtain the target document containing the product packaging diagram. For example, the target document can be manually uploaded or sent to the device where the RPA robot is located, so that after the RPA robot obtains the target document, it can extract it from the target document. Product packaging picture. For example, RPA robots can identify and intercept product packaging images from target documents based on target detection algorithms.

In the embodiment of the present disclosure, after the RPA robot obtains the product packaging image, it can perform character recognition on the product packaging image based on OCR technology to obtain the text content of the product packaging image.

Step 102: Obtain the reference document and obtain the document content in the reference document, where the document content includes product information corresponding to the target product.

In the embodiment of the present disclosure, the RPA robot can obtain the reference document, for example, by manually uploading or sending the reference document to the device where the RPA robot is located. After the RPA robot obtains the reference document, it can read the document content in the reference document.

Step 103: Compare the text content and the document content to determine the first difference part in the text content that is different from the document content.

In the embodiment of the present disclosure, the RPA robot can compare the text content and the document content to determine the first difference part in the text content that is different from the document content.

Step 104: Make an abnormality mark on the first difference part in the text content, and/or make an exception mark on the area where the first difference part is located in the product packaging diagram.

In a possible implementation of the embodiment of the present disclosure, the RPA robot can annotate the above-mentioned first difference part abnormally in the text content. For example, the RPA robot can adjust the font and/or font size of the first difference part in the text content (such as increasing the font size, italicizing and/or bolding the font, etc.), and color the adjusted first difference part. Mark; alternatively, the RPA robot can also directly color mark the first difference part in the text content. For example, the first difference part can be colored in a striking color (such as red, blue, etc.). This disclosure is for There are no restrictions.

In another possible implementation of the embodiment of the present disclosure, the RPA robot can determine the area where the first difference part is located in the product packaging diagram, and mark the above-mentioned area as abnormal in the product packaging diagram. For example, a label box can be added to the edge of the above area; or underlines, wavy lines, etc. can be added under the characters in the above area, and this disclosure does not limit this.

In another possible implementation of the embodiment of the present disclosure, the RPA robot can also simultaneously mark the first difference part in the text content as abnormal, and mark the area where the first difference part is located in the product packaging diagram. Exception annotation.

In one embodiment, after the RPA robot annotates the text content abnormally, it can also display the annotated text content, and/or after the RPA robot annotates the product packaging diagram abnormally, it can also display the annotated product packaging. diagram so that relevant personnel can be informed of the comparison results in a timely manner.

The product information processing method based on RPA and AI in the embodiment of the present disclosure uses the RPA robot to obtain the product packaging diagram corresponding to the target product, and based on OCR technology, identifies the text content in the product packaging diagram; obtains the reference document, and obtains the reference document The document content, wherein the document content includes product information corresponding to the target product; compare the text content and the document content to determine the first difference part in the text content that is different from the document content; compare the first difference in the text content Make an abnormal mark on the part, and/or make an abnormal mark on the area where the first difference part is located in the product packaging diagram. As a result, the RPA robot can be used to automatically check the product information on the product packaging map. On the one hand, it can reduce the amount of manual participation, release human resources, and reduce labor costs. On the other hand, it can improve the efficiency of checking product information, and also It can avoid the error-prone situation of manual verification and improve the accuracy of product information verification results.

In order to clearly illustrate how the RPA robot compares text content and document content in any embodiment of the disclosure, the disclosure also proposes a product information processing method based on RPA and AI.

As shown in Figure 2, the product information processing method based on RPA and AI can include the following steps:

Step 201: Obtain the product packaging image corresponding to the target product, and identify the text content in the product packaging image based on OCR technology.

In any embodiment of the present disclosure, the product packaging image can be divided into at least one sub-image by manual selection, so that character recognition can be performed on at least one sub-image based on OCR technology to obtain the text content.

That is, in this disclosure, the RPA robot can respond to the interception operation triggered by the relevant personnel, divide the product packaging image into at least one sub-image, and perform character recognition on at least one sub-image based on OCR technology to obtain the text content.

As an example, taking the target product as food, the relevant personnel can divide the product packaging diagram into six sub-areas as shown in Figure 3 through circle selection.

In any embodiment of the present disclosure, the RPA robot can identify and extract at least one target area from the product packaging image based on the target detection algorithm in deep learning technology, where the target area includes character information. The RPA robot can perform character recognition on at least one target area based on OCR technology to obtain text content.

Step 202: Obtain the reference document and obtain the document content in the reference document, where the document content includes product information corresponding to the target product.

The execution process of steps 201 to 202 can refer to the execution process of any embodiment of the present disclosure, and will not be described again here.

Step 203: Extract each first attribute field from the text content, and extract a first attribute value matching each first attribute field from the text content.

In the embodiment of the present disclosure, the first attribute field can be extracted from the text content, and the first attribute value matching each first attribute field can be extracted from the text content.

As an example, taking the target product as food, the part of the product packaging diagram can be shown in Figure 4. The text fragment before ":" can be used as the first attribute field, and the text fragment after ":" can be used as the third attribute field. The first attribute value corresponding to an attribute field.

As another example, an attribute table can be set in advance, and the attribute table includes each attribute field related to the target product. Therefore, in the present disclosure, the text content matching each attribute field in the attribute table can be extracted. After extracting each first attribute field, the first attribute value corresponding to each first attribute field can be extracted from the text content based on the set extraction rule or extraction rule.

For example, the attribute value between two adjacent first attribute fields can be extracted from the text content and used as the first attribute value corresponding to the previous one of the two adjacent first attribute fields. The character content after the last first attribute field can be the first attribute value corresponding to the last first attribute field.

It should be noted that in actual application, the inventor analyzed a large number of packaging design drawings and found that the characters after the last attribute field not only include attribute values, but may also include other characters, such as "Keep the environment clean, please Don’t throw away empty bottles” etc.

In response to the above situation, in this disclosure, a large number of packaging design drawings can be analyzed and counted, the statement located after the last attribute field in each packaging design drawing is determined, and the ending identifier is set based on the above statement, such as the ending identifier It can be "keep environment", etc., so that when the RPA robot recognizes that the text content contains the end identifier, it can intercept the character content between the last first attribute field and the end identifier, and use it as the third attribute field corresponding to the last first attribute field. An attribute value.

Step 204: Compare each first attribute field and the first attribute value corresponding to each first attribute field with each second attribute field and the second attribute value corresponding to each second attribute field in the document content.

In the embodiment of the present disclosure, each first attribute field in the text content and the first attribute value corresponding to each first attribute field can be combined with each second attribute field and each second attribute field in the document content corresponding to the first attribute value. Compare the two attribute values.

Step 205: When there is a mismatch between the first target attribute field and the second attribute field in each first attribute field, use the first target attribute field and/or the first attribute value corresponding to the first target attribute field as the third attribute field. A difference part.

In the embodiment of the present disclosure, when it is determined that at least one attribute field (denoted as the first target attribute field in this disclosure) does not match the second attribute field among the first attribute fields, the first target attribute field may be and/or the first attribute value corresponding to the first target attribute field, as the first difference part.

Step 206: In each first attribute field, there is a situation where the second target attribute field matches the second attribute field, but the first attribute value corresponding to the second target attribute field does not match the second attribute value corresponding to the second attribute field. Next, the first attribute value corresponding to the second target attribute field is used as the first difference part.

In the embodiment of the present disclosure, it is determined that among the first attribute fields, there is one attribute field (denoted as the second target attribute field in this disclosure) that matches the second attribute field, but the first attribute corresponding to the second target attribute field If the value does not match the second attribute value corresponding to the second attribute field, the first attribute value corresponding to the second target attribute field may be used as the first difference part.

Step 207: Make an abnormality mark on the first difference part in the text content, and/or make an exception mark on the area where the first difference part is located in the product packaging diagram.

The execution process of step 207 can be referred to the execution process of any embodiment of the present disclosure, and will not be described again.

The product information processing method based on RPA and AI in the embodiment of the present disclosure can avoid important content in product information by comparing each attribute field and attribute value in the text content with the attribute fields and attribute values in the document content respectively. omission detection, thereby improving the accuracy of product information verification results.

It should be noted that, considering the accuracy of OCR recognition results, for some attribute fields, such as ingredients, the OCR recognition result may be "material", resulting in the word "mixing" not being recognized, or the recognition result may be "ingredient", resulting in There are extra spaces in the recognition results. The above situation will cause the RPA robot to be unable to recognize the attribute field "ingredients" and thus be unable to extract the attribute value corresponding to "ingredients". This will result in the RPA robot being unable to compare the ingredient information in the product packaging diagram.

In response to the above problem, in this disclosure, for each attribute field related to the target product, multiple statements or possible statements of the attribute field can also be counted, and the attribute field and the multiple statements or statements corresponding to the attribute field can be counted. Various possible expressions are used as the third attribute field and are set in the setting vocabulary. Therefore, in the present disclosure, the third attribute value corresponding to each third attribute field in the set vocabulary table can be extracted from the text content based on the set vocabulary table, so that the third attribute value can be compared with each third attribute field in the document content. Compare the two attribute values. The above process will be described in detail below with reference to Figure 5.

As shown in Figure 5, based on the embodiment shown in Figure 2, the product information processing method based on RPA and AI can also include the following steps:

Step 301: Obtain a setting vocabulary, where the setting vocabulary includes at least one third attribute field.

In the embodiment of the present disclosure, the set word list is preset. In the present disclosure, the RPA robot can obtain the preset set word list.

Step 302: Extract third attribute values matching each third attribute field in the set vocabulary from the text content.

In this embodiment of the present disclosure, the RPA robot can extract the third attribute value from the text content that matches each third attribute field in the set vocabulary. The specific implementation process is similar to step 203 and will not be described again here.

Step 303: Compare the third attribute value corresponding to each third attribute field with the second attribute value corresponding to each second attribute field in the document content.

Step 304: If there is a mismatch between the target attribute value and the second attribute value among the third attribute values, the target attribute value is used as the first difference part.

In this embodiment of the present disclosure, the third attribute value corresponding to each third attribute field can be compared with the second attribute value corresponding to each second attribute field in the document content. If there is If at least one attribute value (referred to as the target attribute value in this disclosure) does not match the second attribute value, the target attribute value may be used as the first difference part. If each third attribute value matches the second attribute value, no processing is required.

It should be noted that this disclosure does not limit the execution sequence of steps 301 to 304. For example, steps 301 to 304 can be executed after step 206, or steps 301 to 304 can also be executed in parallel with steps 203 to 206, or steps 301 to 304 may also be executed before step 203, and so on. In other words, steps 301 to 304 only need to be executed before step 207.

It should be noted that, when there is a first difference part in the text content, in order to enable relevant personnel to check and/or modify the product packaging diagram in a timely manner, in any embodiment of the present disclosure, the RPA robot also Prompt information can be sent, where the prompt information is used to prompt to check and/or modify the first difference part in the product packaging diagram.

For example, the RPA robot can send prompt information to a designated account (such as an email account); for another example, the device where the RPA robot is located can be logged in with instant messaging software, and the RPA robot can send prompt information to the instant messaging account of the relevant person.

In any embodiment of the present disclosure, the RPA robot can determine the target product according to the corresponding relationship between the first attribute field and the first attribute value, the corresponding relationship between the third attribute field and the third attribute value in the text content. Generate and display a verification report for at least one of the first nutritional ingredient information, so that relevant personnel can verify the product packaging diagram based on the above verification report. For example, the reconciliation report can be as shown in Figure 6.

In any embodiment of the present disclosure, the RPA robot can not only send prompt information, but also generate verification reports.

In a possible implementation of the embodiment of the present disclosure, the RPA robot can also compare the document content and the text content to determine the second difference part in the document content that is different from the text content. The comparison method is the same as in the above embodiment. The method of comparing text content and document content is similar and will not be described in detail here. In this disclosure, when the RPA robot determines that there is a second difference part in the document content, it can annotate the second difference part in the document content and display the annotated document content. The marking method of the second difference part is similar to the marking method of the first difference part, and will not be described again here.

The product information processing method based on RPA and AI in the embodiment of the present disclosure further extracts each attribute value in the text content according to the set vocabulary, and compares each extracted attribute value with each attribute value in the document content. It can avoid the situation where attribute values are missed due to low accuracy of OCR recognition results, thereby improving the accuracy of product information verification results.

As shown in Figure 7, the product information processing method based on RPA and AI can include the following steps:

Step 401: Obtain the product packaging image corresponding to the target product, and identify the text content in the product packaging image based on OCR technology.

Step 402: Obtain the reference document and obtain the document content in the reference document, where the document content includes product information corresponding to the target product.

The execution process of steps 401 to 402 can refer to the execution process of any embodiment of the present disclosure, and will not be described again here.

Step 403: Extract the first nutritional component information of the target product from the text content, and extract the second nutritional component information from the document content.

In the embodiment of the present disclosure, the first nutritional ingredient information of the target product can be extracted from the text content. For example, by dividing the product packaging image into multiple sub-images for illustration, the first nutritional ingredient information can be included in a certain sub-region, which is recorded as a target sub-image in this disclosure. For example, the target sub-image can be as shown in Figure 3 As shown in sub-image 1, character recognition can be performed on the target sub-area based on OCR technology to obtain the first nutritional component information.

That is to say, the text content is composed of OCR recognition results corresponding to multiple sub-images. The target sub-image containing the first nutritional component information can be determined from the multiple sub-images, and the OCR corresponding to the target sub-image can be determined from the text content. Recognition results.

In the embodiment of the present disclosure, the RPA robot can also extract the second nutritional component information from the document content.

Step 404: Compare each component information in the first nutritional component information with the corresponding component information in the second nutritional component information.

Step 405: If there is a mismatch between the target component information in the first nutritional component information and the corresponding component information in the second nutritional component information, use the target component information as the first difference part.

In the embodiment of the present disclosure, each component information (such as energy, protein, fat and other component information) in the first nutritional component information can be matched with the corresponding component information in the second nutritional component information. In the first nutritional component information When there is a mismatch between at least one component information (denoted as target component information in this disclosure) and the corresponding component information in the second nutritional component information, the target component information can be used as the first difference part.

Step 406: Make an abnormality mark on the first difference part in the text content, and/or make an exception mark on the area where the first difference part is located in the product packaging diagram.

The execution process of step 406 can be referred to the execution process of any embodiment of the present disclosure, and will not be described again here.

The product information processing method based on RPA and AI in the embodiment of the present disclosure can realize the verification of the table content in the product packaging diagram by comparing the nutritional information in the text content with the nutritional information in the document content to avoid Omissions of product information are checked, thereby improving the reliability of the verification results.

It should be noted that because most of the nutritional ingredients tables are tables without borders, it is difficult for the current general table recognition algorithm to identify the nutritional labeling. For example, the general table recognition algorithm cannot clearly identify the left, middle, and right sides of the table without borders. , line break, etc. For example, if the carbohydrates in sub-image 1 in Figure 3 are changed to another line, they may become as shown in Table 2 or Table 3:

Table 2

碳水化carbs	5.8g5.8g	2％2%
合物compound

table 3

It is understandable that Tables 2 and 3 are easier to understand for people, but it is difficult for machines to judge that carbohydrate is a complete word, so the current general table recognition algorithm is more difficult to recognize. To address the above problems, in the present disclosure, after extracting the first nutritional ingredient information from the text content, regular replacement can be performed on incorrectly identified ingredient information in the first nutritional ingredient information. The above process will be described in detail below with reference to Figure 8 .

As shown in Figure 8, the product information processing method based on RPA and AI can include the following steps:

Step 501: Obtain the product packaging image corresponding to the target product, and identify the text content in the product packaging image based on OCR technology.

Step 502: Obtain the reference document and obtain the document content in the reference document, where the document content includes product information corresponding to the target product.

Step 503: Extract the first nutritional component information of the target product from the text content, and extract the second nutritional component information from the document content.

The execution process of steps 501 to 503 can refer to the execution process of any embodiment of the present disclosure, and will not be described again here.

Step 504: For any component information in the first nutritional component information, obtain a regular expression matching any component information.

In the embodiment of the present disclosure, the regular expression corresponding to each ingredient information can be preset, so that in the present disclosure, the RPA robot can obtain the regular expression corresponding to each ingredient information in the first nutritional ingredient information.

Step 505: Match the regular expression with any component information.

Step 506: If there is no match, any component information is replaced based on the regular expression.

In the embodiment of the present disclosure, for any component information in the first nutritional component information, if the any component information does not match the corresponding regular expression, then the corresponding regular expression can be based on the any component information. Any component information is replaced. And if any component information matches the corresponding regular expression, there is no need to perform replacement processing on any component information.

For example, the unit corresponding to "carbohydrate" is "g". If the unit corresponding to "carbohydrate" in the first nutritional ingredient information is "9", you can use the regular expression corresponding to the "carbohydrate" to "9" is automatically replaced with "g".

For another example, as shown in Figure 1, the last item "NRV" corresponding to each nutrient ingredient is the percentage of the nutrient required throughout the day. If the unit corresponding to "NRV" in each ingredient information in the first nutrient ingredient information is not "%", and other symbols, you can use the regular expression corresponding to each component information to automatically replace other symbols with "%".

For another example, assume that the first nutritional component information in the OCR recognition result is shown in Figure 9. According to the regular expression corresponding to "carbohydrates", it can be determined that the first item in the component information corresponding to "carbohydrates" lacks the word "物" , the second item contains the word "thing", you can use the regular expression corresponding to the "carbohydrate" to replace "carbohydrate" in the first item with "carbohydrate", and automatically replace "thing 5.8g" is "5.8g".

It should be noted that the above-mentioned examples of this disclosure only use regular expressions to replace any component information. In actual application, any component information can also be replaced by writing logical judgments at the code level. . For example, the code logic can be: determine whether the last number in each ingredient information contains a unit. If it does not contain a unit, the last digit can be automatically replaced with a unit that matches the ingredient information, such as replacing "9" with " g".

Step 507: Compare each component information in the replaced first nutritional component information with the corresponding component information in the second nutritional component information.

Step 508: If there is a mismatch between the target component information in the replaced first nutritional component information and the corresponding component information in the second nutritional component information, use the target component information as the first difference part.

Step 509: Make an abnormality mark on the first difference part in the text content, and/or make an exception mark on the area where the first difference part is located in the product packaging diagram.

The execution process of step 509 can be referred to the execution process of any embodiment of the present disclosure, and will not be described again here.

The product information processing method based on RPA and AI in the embodiment of the present disclosure obtains a regular expression that matches any component information by targeting any component information in the first nutritional component information; the regular expression is matched with any component information Match; if not, any component information will be replaced based on the regular expression. As a result, the OCR recognition results can be supplemented, corrected and optimized, thereby further improving the accuracy and reliability of the product information comparison results.

This disclosure also proposes an RPA and AI product information processing method.

As shown in Figure 10, the product information processing method based on RPA and AI can include the following steps:

Step 601: Obtain the product packaging image corresponding to the target product, and identify the text content in the product packaging image based on OCR technology.

Step 602: Obtain the reference document and obtain the document content in the reference document, where the document content includes product information corresponding to the target product.

Step 603: Extract the first nutritional component information of the target product from the text content, and extract the second nutritional component information from the document content.

The execution process of steps 601 to 603 can refer to the execution process of any embodiment of the present disclosure, and will not be described again here.

Step 604: For any text segment in the first nutritional ingredient information, determine whether the semantics of any text segment is complete.

In the embodiment of the present disclosure, for any text fragment in the first nutritional ingredient information, it can be determined whether the semantics of any text fragment is complete.

As an example, it can be determined whether the semantics of any text fragment is complete based on a semantic analysis algorithm.

It can be understood that, under normal circumstances, the reasons for the identification errors of the first nutritional ingredient information include unit identification errors on the one hand, and item (such as protein, carbohydrates, trans fatty acids, vitamin D, etc.) identification errors on the other hand, among which , the reason for project identification errors is generally: the project name is long, which causes OCR to classify some characters in the project name into the content (such as the column for each 100mL in Figure 9).

Therefore, in response to the above problems, in the present disclosure, as another example, statistical analysis can be performed on the packaging design drawings of a large number of commodities, each item included in the nutritional composition table corresponding to different commodities can be determined, and the above items can be written into the item table , so in the present disclosure, the text fragment where each item in the first nutritional component information is located can be matched with the name of each item in the item table. If the text fragment where a certain item in the first nutritional component information is located matches the item name in the item table If the names of each item do not match, it is determined that the semantics of the text fragment in which the item is located is incomplete.

Step 605: If the semantics of any text fragment is incomplete, obtain adjacent text fragments adjacent to any text fragment from the nutritional composition information.

In the embodiment of the present disclosure, if the semantics of any of the above text fragments is incomplete, the adjacent text fragments adjacent to any of the text fragments can be obtained from the nutritional component information.

Step 606: If the semantics of the adjacent text segments are incomplete, determine semantically complete sub-segments from the adjacent text segments.

Step 607: Extract other characters excluding sub-segments from the adjacent text fragments, and classify the other characters into any text fragment.

In the embodiment of the present disclosure, it can be determined whether the semantics of the adjacent text fragments are complete. If the semantics of the adjacent text fragments are incomplete, the semantically complete sub-segments can be determined from the adjacent text fragments, and the sub-segments in the adjacent text fragments can be extracted. characters, so that other characters can be included in any text fragment.

When the semantics of the adjacent text fragments are complete, the next adjacent text fragment adjacent to any of the above text fragments can be obtained, and whether the semantics of the next adjacent text fragment is complete is determined. If the semantics of the next adjacent text fragment is not Complete, the semantically complete sub-segment can be determined from the next adjacent text segment, and other characters in the next adjacent text segment except the sub-segment can be extracted, so that other characters can be classified into any text segment.

Step 608: Remove other characters from adjacent text segments.

In the embodiment of the present disclosure, the RPA robot can also remove other characters from adjacent text segments to ensure the accuracy of the first nutritional ingredient information recognition result.

Step 609: Compare each component information in the updated first nutritional component information with the corresponding component information in the second nutritional component information.

Step 610: If there is a mismatch between the target component information in the updated first nutritional component information and the corresponding component information in the second nutritional component information, use the target component information as the first difference part.

Step 611: Make an abnormality mark on the first difference part in the text content, and/or make an exception mark on the area where the first difference part is located in the product packaging diagram.

The execution process of steps 609 to 611 can refer to the execution process of any embodiment of the present disclosure, and will not be described again here.

As an example, the RPA robot can be installed on the verification platform side, so that in the present disclosure, automatic verification of product information can be completed on the verification platform side. For example, the implementation principle of the embodiment of the present disclosure can be shown in Figure 11, specifically: Includes the following parts:

The first part is to upload the product packaging pictures to the verification platform. Among them, the format of the product packaging diagram can be JPG, PNG and other picture formats (or image formats), or you can upload design documents in PDF documents, PSD and other formats, and the product packaging diagram can be extracted from the above documents.

For example, relevant personnel can upload images or documents to the verification platform through the web page.

The second part is to cut the product packaging diagram. The product packaging image can be cut into multiple sub-images. For example, the relevant personnel can manually select the areas in the product packaging image that need to be OCR recognized, and cut the above areas to obtain each sub-area.

As an example, since the uploaded product packaging image is large, in order to accurately identify the text information in the product packaging image, you can manually select the part to be identified from the product packaging image. For example, a human can circle the area where the nutritional label as shown in Figure 12 is located. For another example, humans can circle each area as shown in Figure 3.

Among them, nutritional ingredient tables, because most of them are tables without borders, it is difficult for the current general table recognition algorithm to identify the nutrition table. For example, the general table recognition algorithm cannot clearly identify the left, middle, right, line breaks, etc. in tables without borders. . For example, if the carbohydrates in Figure 3 are changed to another line, they may become as shown in Table 2 or Table 3. It is understandable that Tables 2 and 3 are easier to understand for people, but it is difficult for machines to judge that carbohydrate is a complete word, so the current general table recognition algorithm is more difficult to recognize. In response to the above problems, the verification platform will supplement and optimize the OCR recognition results by writing logical judgments at the code level for specific words in the nutrition facts table.

For example, the OCR recognition results of the nutritional ingredients table in Figure 3 can be shown in Figure 9. The OCR recognition results can be supplemented and optimized, and the optimized OCR recognition results can be shown in Table 1.

To extract ingredients, as shown in Figure 13, the newline symbols in the OCR recognition results can be removed to obtain a long text, which can then be checked against the configuration template on the platform (the configuration template includes information for extracting the corresponding attributes of each attribute field). Attribute value extraction rules or extraction rules), extract ingredient information from the OCR recognition results. For example, the verification platform extracts the ingredient information from the OCR recognition results in Figure 13, and the extraction results can be shown in Figure 14.

The extraction of the manufacturer (hereinafter referred to as the factory name), place of production and address (hereinafter referred to as the factory address), and production license (or production license number) is similar to the ingredients. For example, OCR recognition is performed on the image area where the factory name and address are located, and the recognition result can be shown in Figure 15. The line breaks in the OCR recognition results can be removed to obtain a long text, and then the template can be configured to extract the text from the OCR recognition results. Extract the factory name and address. For example, the verification platform extracts the factory name, factory address and food production license number from the OCR recognition results in Figure 15, and the extraction results can be shown in Figure 16.

That is to say, in this disclosure, the attribute fields to be extracted can be defined on the verification platform side, such as extracting the manufacturer (hereinafter referred to as factory name), place of origin and address (hereinafter referred to as factory address), etc. As an example, the defined attribute fields can be as shown in Figure 17, so that attribute values matching each attribute field can be extracted from the OCR recognition results, and then each extracted attribute field and attribute value can be subsequently compared with the attribute value in the document content. Each attribute field is compared with the attribute value.

Furthermore, a custom vocabulary list (referred to as a set vocabulary list in this disclosure) can also be set on the verification platform side, and the set vocabulary list is used to cooperate with the extraction. For example, the ingredient information must appear after the word "ingredients" or "ingredients:". However, considering the accuracy of the OCR recognition results, it is possible that the word "ingredients" will be recognized, but the word "matching" will not be recognized, or the word "ingredients" will be recognized. If there are spaces in the middle of the "ingredients", these can be configured in the vocabulary as enumerations.

When configuring the above configuration template, you can use the custom vocabulary corresponding to each attribute field. For example, the configuration template can be as shown in Figure 18.

Figure 19 shows the extraction rules for ingredients. It can identify whether the text content includes words in the custom word list corresponding to ingredients. If it does, any 0 to 500 characters in the text content after the word can be output to In the ingredient field, it is the attribute value corresponding to the ingredient field. If the character content located after the word in the text content includes words in the segmented vocabulary of the custom vocabulary list (recorded as the ending identifier in this disclosure), there is no need to extract the character information after the ending identifier, that is, between the word and the ending identifier The character content between them is used as the attribute value corresponding to the ingredient.

The third part is to upload the reference documents to the verification platform. In order to make the comparison or verification results more accurate and reduce the verification error rate, the format of the reference document can be a standard structured document, such as an Excel document. If you cannot use a structured document, you can use a document with a fixed template structure, such as a Word document.

For example, relevant personnel can upload reference documents to the verification platform through the web page.

The fourth part is to perform OCR recognition on the product packaging image to obtain the text content. In order to improve the accuracy of the recognition results, it is necessary to ensure that the product packaging image is clear enough. According to tests on different images, the size of the cut image is above 8MB, which can ensure a high recognition accuracy.

The fifth part is document extraction and understanding. Can convert unstructured document content into structured data. In one embodiment, business personnel can compose the reference document according to a set format, so that there is no need to perform structured conversion of the document content of the reference document. For example, the intelligent document understanding capability in the IDP system can be used to intelligently extract key information from the document content and convert unstructured document content into structured data.

The sixth part is information comparison to determine the first difference part in the text content that is different from the document content, and/or to determine the second difference part in the document content that is different from the text content. You can use OCR technology and the document information extraction function to compare the text content extracted in the fourth part and the document content extracted in the fifth part. The comparison logic is: classify the text content into categories, such as attribute fields and attribute values, first nutritional ingredient information, etc.; compare the classified text content with the corresponding content in the document content in sequence, if the tags are inconsistent or have multiple out of the text part. In addition, the document content can also be checked against the text content in sequence (or called back-checking) to ensure that all content in the text content participates in the verification, so as to avoid certain content not participating in the comparison and reducing the accuracy of the verification results. situation occurs.

In addition, the text content can also be logically corrected to improve the accuracy of the OCR recognition results, thereby improving the accuracy of the verification results. For example, for different ingredient information in the nutrition table, you can perform regular replacement in the code logic. For example, the unit of protein is g. If the unit of protein in the OCR recognition result is 9, you can replace 9 with g to improve the performance. The accuracy of OCR recognition results.

Part 7, results display. The comparison results can be displayed on the web page. For example, the first difference part can be marked in the text content, and the second difference part can be marked in the document content. In addition, the location of the first difference part can also be marked on the product packaging diagram.

It should be noted that the added amount of lactic acid bacteria in the product packaging picture in Figure 13 is: 1.0×10 ⁷ CFU/100g, but the OCR recognition result is: 1.0×107CFU/100g, that is, the power is not distinguished in the OCR recognition result. In view of the above situation, the RPA robot can recognize that the two attribute values are different, that is, 1.0×10 ⁷ CFU/100g is different from 1.0×107CFU/100g. The attribute value of 1.0×107CFU/100g can be marked in the text content. , let humans check whether there are errors here.

It should be noted that taking into account the special circumstances of the product packaging design of some products, for example, under normal circumstances, the text in the product packaging image is arranged from left to right or top to bottom. However, the product packaging image of some products The text in may be displayed wrapped around, displayed in the form of wavy lines, etc. In this case, the OCR recognition result will be different from the document content.

As an example, OCR recognition is performed on sub-image 1 in Figure 3, and the recognition result can be as shown in Figure 20. However, for the image in Figure 21, the OCR recognition results may be wrong. In response to the above situation, the RPA robot can mark the differences in the text content, and/or mark the location of the differences in the product packaging diagram, and manually check whether there are errors.

In one embodiment, the RPA robot can also generate verification results, which can be downloaded by the user.

Finally, the annotated text content, annotated document content, annotated product packaging diagram, and verification report can be reviewed manually. The verification of product information by the verification platform or RPA robot can be completed in a shorter time. Generally, it only takes 1-3 minutes to complete the verification, which not only improves the verification efficiency, but also improves the accuracy of the verification results. Only the differences are manually reviewed, which can reduce the workload of relevant personnel and improve work efficiency.

Corresponding to the product information processing method based on RPA and AI provided by the above embodiments of FIG. 1 to FIG. 10 , the present disclosure also provides a product information processing device based on RPA and AI. The product information processing device corresponds to the product information processing method based on RPA and AI provided by the above embodiments of Figures 1 to 10. Therefore, the implementation of the product information processing method based on RPA and AI is also applicable to the product information processing method provided by the embodiment of the present disclosure. The product information processing device based on RPA and AI will not be described in detail in the embodiment of this disclosure.

As shown in Figure 22, the product information processing device 2200 based on RPA and AI is applied to RPA robots and may include: a first acquisition module 2210, an identification module 2220, a second acquisition module 2230, a comparison module 2240 and an annotation module 2250.

Among them, the first acquisition module 2210 is used to acquire the product packaging diagram corresponding to the target product.

The recognition module 2220 is used to recognize text content in product packaging images based on optical character recognition OCR technology.

The second acquisition module 2230 is used to acquire the reference document and acquire the document content in the reference document, where the document content includes product information corresponding to the target product.

The comparison module 2240 is used to compare the text content and the document content to determine the first difference part in the text content that is different from the document content.

The marking module 2250 is configured to mark the first difference part abnormally in the text content, and/or mark the area where the first difference part is located in the product packaging diagram abnormally.

In a possible implementation of the embodiment of the present disclosure, the comparison module 2240 is configured to: extract each first attribute field from the text content, and extract the first attribute matching each first attribute field from the text content. value; compare each first attribute field and the first attribute value corresponding to each first attribute field with each second attribute field and the second attribute value corresponding to each second attribute field in the document content; When there is a mismatch between the first target attribute field and the second attribute field in an attribute field, the first target attribute field and/or the first attribute value corresponding to the first target attribute field is used as the first difference part; in each If there is a second target attribute field in the first attribute field that matches the second attribute field, but the first attribute value corresponding to the second target attribute field does not match the second attribute value corresponding to the second attribute field, the second attribute field will be The first attribute value corresponding to the target attribute field is used as the first difference part.

In a possible implementation of the embodiment of the present disclosure, the comparison module 2240 is also used to: obtain a setting vocabulary list, where the setting vocabulary list includes at least a third attribute field; extract and extract from the text content Set the third attribute value matching each third attribute field in the vocabulary; compare the third attribute value corresponding to each third attribute field with the second attribute value corresponding to each second attribute field in the document content; When there is a mismatch between the target attribute value and the second attribute value among the third attribute values, the target attribute value is used as the first difference part.

In a possible implementation of the embodiment of the present disclosure, the comparison module 2240 is configured to: extract the first nutritional component information of the target product from the text content, and extract the second nutritional component information from the document content; Each component information in the first nutritional component information is compared with the corresponding component information in the second nutritional component information; when there is a mismatch between the target component information in the first nutritional component information and the corresponding component information in the second nutritional component information, The target component information is used as the first difference part.

In a possible implementation of the embodiment of the present disclosure, the text content includes the first nutritional ingredient information of the target product. The product information processing device 2200 based on RPA and AI may also include:

The first processing module is used to extract the first nutritional component information from the text content; for any component information in the first nutritional component information, obtain a regular expression that matches any component information; combine the regular expression with any component information The information is matched; if there is no match, any component information is replaced based on the regular expression.

The second processing module is used to extract the first nutritional component information from the text content; for any text fragment in the first nutritional component information, determine whether the semantics of any text fragment is complete; if the semantics of any text fragment is incomplete , then obtain the adjacent text segments adjacent to any text segment from the nutritional composition information; if the semantics of the adjacent text segments are incomplete, determine the semantically complete sub-segments from the adjacent text segments; extract the sub-segments from the adjacent text segments other characters, group other characters into any one text segment, and exclude other characters from adjacent text segments.

In a possible implementation of the embodiment of the present disclosure, the first acquisition module 2210 is configured to: acquire a target document containing a product packaging diagram; and extract the product packaging diagram from the target document.

In a possible implementation of the embodiment of the present disclosure, the recognition module 2220 is configured to: respond to the interception operation, segment the product packaging image into at least one sub-image; and perform character recognition on the at least one sub-image based on OCR technology , to get the text content.

In a possible implementation of the embodiment of the present disclosure, the identification module 2220 is configured to: identify and extract at least one target area from the product packaging image based on a target detection algorithm, where the target area includes character information; based on OCR Technology that performs character recognition on at least one target area to obtain text content.

In a possible implementation of the embodiment of the present disclosure, the annotation module 2250 is used to adjust the font and/or font size of the first difference part in the text content; and color-mark the adjusted first difference part. .

In a possible implementation of the embodiment of the present disclosure, the comparison module 2240 is also used to compare the document content and the text content to determine the second difference part in the document content that is different from the text content.

The annotation module 2250 is also used to annotate the second difference part abnormally in the document content.

The product information processing device 2200 based on RPA and AI may also include:

Display module, used to display the annotated document content.

In a possible implementation of the embodiment of the present disclosure, the product information processing device 2200 based on RPA and AI may also include:

The sending module is used to send prompt information, where the prompt information is used to prompt to check and/or modify the first difference part in the product packaging diagram.

and / or,

A generation module for generating and displaying a verification report, wherein the verification report includes a correspondence between the first attribute field and the first attribute value in the text content, a correspondence between the third attribute field and the third attribute value, and At least one item of the first nutritional ingredient information of the target product.

The product information processing device based on RPA and AI in the embodiment of the present disclosure obtains the product packaging diagram corresponding to the target product through the RPA robot, and based on OCR technology, identifies the text content in the product packaging diagram; obtains the reference document, and obtains the reference document The document content, wherein the document content includes product information corresponding to the target product; compare the text content and the document content to determine the first difference part in the text content that is different from the document content; compare the first difference in the text content Make an abnormal mark on the part, and/or make an abnormal mark on the area where the first difference part is located in the product packaging diagram. As a result, the RPA robot can be used to automatically check the product information on the product packaging map. On the one hand, it can reduce the amount of manual participation, release human resources, and reduce labor costs. On the other hand, it can improve the efficiency of checking product information, and also It can avoid the error-prone situation of manual verification and improve the accuracy of product information verification results.

An embodiment of the present disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, any one of the foregoing methods is implemented. The product information processing method based on RPA and AI described in the example.

Embodiments of the present disclosure also provide a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the product information based on RPA and AI as described in any of the foregoing method embodiments is implemented. Approach.

An embodiment of the present disclosure also provides a computer program product. When the instruction processor in the computer program product is executed, the product information processing method based on RPA and AI as described in any of the foregoing method embodiments is implemented.

23 illustrates a block diagram of an exemplary electronic device suitable for implementing embodiments of the present disclosure. The electronic device 12 shown in FIG. 23 is only an example and should not bring any limitations to the functions and scope of use of the embodiments of the present disclosure.

As shown in Figure 23, electronic device 12 is embodied in the form of a general computing device. Components of the electronic device 12 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and a bus 18 connecting various system components (including memory 28 and processing unit 16).

Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics accelerated port, a processor, or a local bus using any of a variety of bus structures. For example, these architectures include but are not limited to Industry Standard Architecture (hereinafter referred to as: ISA) bus, Micro Channel Architecture (Micro Channel Architecture; hereafter referred to as: MAC) bus, enhanced ISA bus, video electronics Standards Association (Video Electronics Standards Association; hereinafter referred to as: VESA) local bus and Peripheral Component Interconnection (hereinafter referred to as: PCI) bus.

Electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by electronic device 12, including volatile and nonvolatile media, removable and non-removable media.

The memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (Random Access Memory; hereinafter referred to as: RAM) 30 and/or cache memory 32. Electronic device 12 may further include other removable/non-removable, volatile/non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write to non-removable, non-volatile magnetic media (not shown in Figure 23, commonly referred to as a "hard drive"). Although not shown in FIG. 23, a disk drive for reading and writing a removable non-volatile disk (e.g., a "floppy disk") and a removable non-volatile optical disk (e.g., a compact disk read-only memory) may be provided. Disc Read Only Memory (hereinafter referred to as: CD-ROM), Digital Video Disc Read Only Memory (hereinafter referred to as: DVD-ROM) or other optical media) read and write optical disc drives. In these cases, each drive may be connected to bus 18 through one or more data media interfaces. Memory 28 may include at least one program product having a set (eg, at least one) of program modules configured to perform the functions of embodiments of the present disclosure.

A program/utility 40 having a set of (at least one) program modules 42, including but not limited to an operating system, one or more application programs, other program modules, and program data, may be stored, for example, in memory 28 , each of these examples or some combination may include the implementation of a network environment. Program modules 42 generally perform functions and/or methods in the embodiments described in this disclosure.

Electronic device 12 may also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), may also communicate with one or more devices that enable a user to interact with electronic device 12, and/or with Any device (eg, network card, modem, etc.) that enables the electronic device 12 to communicate with one or more other computing devices. This communication may occur through input/output (I/O) interface 22. Moreover, the electronic device 12 can also communicate with one or more networks (such as a local area network (Local Area Network; hereinafter referred to as: LAN), a wide area network (Wide Area Network; hereinafter referred to as: WAN)) and/or a public network, such as the Internet, through the network adapter 20 ) communication. As shown, network adapter 20 communicates with other modules of electronic device 12 via bus 18 . It should be understood that, although not shown in the figures, other hardware and/or software modules may be used in conjunction with electronic device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives And data backup storage system, etc.

The processing unit 16 executes programs stored in the memory 28 to perform various functional applications and data processing, such as implementing the methods mentioned in the previous embodiments.

In the description of this specification, reference to the terms "one embodiment," "some embodiments," "an example," "specific examples," or "some examples" or the like means that specific features are described in connection with the embodiment or example. , structures, materials, or features are included in at least one embodiment or example of the present disclosure. In this specification, the schematic expressions of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, those skilled in the art may combine and combine different embodiments or examples and features of different embodiments or examples described in this specification unless they are inconsistent with each other.

In addition, the terms “first” and “second” are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the quantity of indicated technical features. Therefore, features defined as "first" and "second" may explicitly or implicitly include at least one of these features. In the description of the present disclosure, "plurality" means at least two, such as two, three, etc., unless otherwise expressly and specifically limited.

Any process or method descriptions in flowcharts or otherwise described herein may be understood to represent modules, segments, or portions of code that include one or more executable instructions for implementing customized logical functions or steps of the process. , and the scope of the preferred embodiments of the present disclosure includes additional implementations in which functions may be performed out of the order shown or discussed, including in a substantially simultaneous manner or in the reverse order, depending on the functionality involved, which shall It should be understood by those skilled in the art to which embodiments of the present disclosure belong.

The logic and/or steps represented in the flowcharts or otherwise described herein, for example, may be considered a sequenced list of executable instructions for implementing the logical functions, and may be embodied in any computer-readable medium, For use by, or in combination with, instruction execution systems, devices or devices (such as computer-based systems, systems including processors or other systems that can fetch instructions from and execute instructions from the instruction execution system, device or device) or equipment. For the purposes of this specification, a "computer-readable medium" may be any device that can contain, store, communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. More specific examples (non-exhaustive list) of computer readable media include the following: electrical connections with one or more wires (electronic device), portable computer disk cartridges (magnetic device), random access memory (RAM), Read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium may even be paper or other suitable medium on which the program may be printed, as the paper or other medium may be optically scanned, for example, and subsequently edited, interpreted, or otherwise suitable as necessary. process to obtain the program electronically and then store it in computer memory.

It should be understood that various parts of the present disclosure may be implemented in hardware, software, firmware, or combinations thereof. In the above embodiments, various steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if it is implemented in hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: discrete logic gate circuits with logic functions for implementing data signals; Logic circuits, application specific integrated circuits with suitable combinational logic gates, programmable gate arrays (PGA), field programmable gate arrays (FPGA), etc.

Those of ordinary skill in the art can understand that all or part of the steps involved in implementing the methods of the above embodiments can be completed by instructing relevant hardware through a program. The program can be stored in a computer-readable storage medium. The program can be stored in a computer-readable storage medium. When executed, one of the steps of the method embodiment or a combination thereof is included.

In addition, each functional unit in various embodiments of the present disclosure may be integrated into one processing module, each unit may exist physically alone, or two or more units may be integrated into one module. The above integrated modules can be implemented in the form of hardware or software function modules. If the integrated module is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

The storage media mentioned above can be read-only memory, magnetic disks or optical disks, etc. Although the embodiments of the present disclosure have been shown and described above, it can be understood that the above-mentioned embodiments are illustrative and should not be construed as limitations of the present disclosure. Those of ordinary skill in the art can make modifications to the above-mentioned embodiments within the scope of the present disclosure. The embodiments are subject to changes, modifications, substitutions and variations.

Claims

A product information processing method based on robotic process automation RPA and artificial intelligence AI, wherein the method is executed by an RPA robot and includes:

Obtain the product packaging image corresponding to the target product, and identify the text content in the product packaging image based on optical character recognition OCR technology;

Obtain a reference document and obtain document content in the reference document, where the document content includes product information corresponding to the target product;

Compare the text content and the document content to determine a first difference part in the text content that is different from the document content;

The first difference part is marked abnormally in the text content, and/or the area where the first difference part is located is marked abnormally in the product packaging diagram.
The method according to claim 1, wherein the comparing the text content and the document content to determine a first difference part in the text content that is different from the document content includes:

Extract each first attribute field from the text content, and extract a first attribute value matching each of the first attribute fields from the text content;

Compare each first attribute field and the first attribute value corresponding to each first attribute field with each second attribute field and the second attribute value corresponding to each second attribute field in the document content. Comparison;

If there is a mismatch between the first target attribute field and the second attribute field in each of the first attribute fields, the first target attribute field and/or the first target attribute field corresponding to the first target attribute field are Attribute value, as the first difference part;

In each of the first attribute fields, there is a second target attribute field that matches the second attribute field, but the first attribute value corresponding to the second target attribute field does not match the second attribute corresponding to the second attribute field. If the values do not match, the first attribute value corresponding to the second target attribute field is used as the first difference part.
The method of claim 2, further comprising:

Obtain a setting vocabulary, wherein the setting vocabulary includes at least one third attribute field;

Extract third attribute values matching each of the third attribute fields in the set vocabulary from the text content;

Compare the third attribute value corresponding to each of the third attribute fields with the second attribute value corresponding to each of the second attribute fields in the document content;

If there is a mismatch between the target attribute value and the second attribute value in each of the third attribute values, the target attribute value is used as the first difference part.
The method according to claim 1, wherein the comparing the text content and the document content to determine a first difference part in the text content that is different from the document content includes:

Extract the first nutritional component information of the target commodity from the text content, and extract the second nutritional component information from the document content;

Compare each component information in the first nutritional component information with the corresponding component information in the second nutritional component information;

When there is a mismatch between the target component information in the first nutritional component information and the corresponding component information in the second nutritional component information, the target component information is used as the first difference part.
The method according to any one of claims 1 to 4, wherein the text content includes the first nutritional ingredient information of the target commodity, and the optical character recognition (OCR) technology is used to identify the information in the packaging image of the commodity. After the text content, the method also includes:

Extract the first nutritional ingredient information from the text content;

For any component information in the first nutritional component information, obtain a regular expression matching the any component information;

Match the regular expression with any of the component information;

If there is no match, any component information is replaced based on the regular expression.
The method according to any one of claims 1 to 5, wherein the text content includes the first nutritional ingredient information of the target commodity, and the optical character recognition (OCR) technology is used to identify the information in the packaging image of the commodity. After the text content, the method also includes:

Extract the first nutritional ingredient information from the text content;

For any text fragment in the first nutritional ingredient information, determine whether the semantics of any text fragment is complete;

If the semantics of any text fragment is incomplete, then obtain adjacent text fragments adjacent to any text fragment from the nutritional composition information;

If the adjacent text segment is semantically incomplete, determining a semantically complete sub-segment from the adjacent text segment;

Extract other characters in the adjacent text segment except the sub-segment, classify the other characters into any text segment, and eliminate the other characters from the adjacent text segment.
The method according to any one of claims 1-6, wherein said obtaining the product packaging diagram corresponding to the target product includes:

Obtain the target document containing the product packaging diagram;

Extract the product packaging image from the target document.
The method according to any one of claims 1 to 7, wherein the identifying the text content in the product packaging image based on optical character recognition (OCR) technology includes:

In response to the interception operation, segment the product packaging image into at least one sub-image;

Based on the OCR technology, character recognition is performed on the at least one sub-image to obtain the text content.
The method according to any one of claims 1 to 7, wherein the identifying the text content in the product packaging image based on optical character recognition (OCR) technology includes:

Based on a target detection algorithm, identify and extract at least one target area from the product packaging image, wherein the target area includes character information;

Based on the OCR technology, character recognition is performed on the at least one target area to obtain the text content.
The method according to any one of claims 1-9, wherein the abnormal annotation of the first difference part in the text content includes:

In the text content, adjust the font and/or font size of the first difference part;

Color-mark the adjusted first difference part.
The method according to any one of claims 1-10, wherein the method further includes:

Compare the document content and the text content to determine a second difference part in the document content that is different from the text content;

Mark the second difference part abnormally in the document content;

Display the annotated document content.
The method according to any one of claims 1-11, wherein the method further includes:

Send prompt information, wherein the prompt information is used to prompt to check and/or modify the first difference part in the product packaging diagram;

and / or,

Generate and display a verification report, wherein the verification report includes the corresponding relationship between the first attribute field and the first attribute value in the text content, the corresponding relationship between the third attribute field and the third attribute value, and the corresponding relationship between the third attribute field and the third attribute value. At least one item of the first nutritional ingredient information of the target product.
A product information processing device based on robotic process automation RPA and artificial intelligence AI, which is applied to RPA robots and includes:

The first acquisition module is used to obtain the product packaging diagram corresponding to the target product;

A recognition module, used to identify the text content in the product packaging image based on optical character recognition OCR technology;

The second acquisition module is used to obtain a reference document and obtain the document content in the reference document, where the document content includes product information corresponding to the target product;

A comparison module, configured to compare the text content and the document content to determine the first difference part in the text content that is different from the document content;

A marking module is configured to mark the first difference part abnormally in the text content, and/or mark the area where the first difference part is located in the product packaging diagram abnormally.
The device according to claim 13, wherein the comparison module is used for:

Extract each first attribute field from the text content, and extract a first attribute value matching each of the first attribute fields from the text content;

Compare each first attribute field and the first attribute value corresponding to each first attribute field with each second attribute field and the second attribute value corresponding to each second attribute field in the document content. Comparison;

If there is a mismatch between the first target attribute field and the second attribute field in each of the first attribute fields, the first target attribute field and/or the first target attribute field corresponding to the first target attribute field are Attribute value, as the first difference part;

In each of the first attribute fields, there is a second target attribute field that matches the second attribute field, but the first attribute value corresponding to the second target attribute field does not match the second attribute corresponding to the second attribute field. If the values do not match, the first attribute value corresponding to the second target attribute field is used as the first difference part.
The device according to claim 14, wherein the comparison module is also used for:

Obtain a setting vocabulary, wherein the setting vocabulary includes at least one third attribute field;

Extract third attribute values matching each of the third attribute fields in the set vocabulary from the text content;

Compare the third attribute value corresponding to each of the third attribute fields with the second attribute value corresponding to each of the second attribute fields in the document content;

If there is a mismatch between the target attribute value and the second attribute value in each of the third attribute values, the target attribute value is used as the first difference part.
The device according to claim 13, wherein the comparison module is used for:

Extract the first nutritional component information of the target commodity from the text content, and extract the second nutritional component information from the document content;

Compare each component information in the first nutritional component information with the corresponding component information in the second nutritional component information;

When there is a mismatch between the target component information in the first nutritional component information and the corresponding component information in the second nutritional component information, the target component information is used as the first difference part.
The device according to any one of claims 13 to 16, wherein the text content includes the first nutritional ingredient information of the target commodity, and the device further includes:

A first processing module, configured to extract the first nutritional component information from the text content; for any component information in the first nutritional component information, obtain a regular expression matching the any component information; The regular expression is matched with the any component information; if there is no match, the any component information is replaced based on the regular expression.
An electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the implementation as described in any one of claims 1-12 is achieved. Methods.
A non-transitory computer-readable storage medium on which a computer program is stored, wherein when the computer program is executed by a processor, the method according to any one of claims 1-12 is implemented.
A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-12.