CN115578729B - AI intelligent process arrangement method for digital staff - Google Patents

Abstract

The invention discloses a digital employee AI intelligent process layout method, comprising: collecting the original image of a paper document with process information, performing difference grayscale on the original image, and obtaining several difference grayscale images; Rotate at a preset angle to obtain several rotated grayscale images; expand the rotated grayscale images, use Hough transform to detect the character line formed by each line of characters after expansion, and obtain a character trend map; according to the character trend map, expand Perform perspective transformation on the previously rotated grayscale image to obtain the rectified image; extract the arrow logo in the rectified image, use the arrow logo as auxiliary information to perform affine transformation on the rectified image, rotate to obtain the restored image, and input the restored image after binarization Go to the character recognition module for recognition, and sequentially extract process information to complete the arrangement. The invention can obtain accurate character direction, avoid recognition errors caused by special angles and the like, and is beneficial to improve processing speed and accuracy.

Description

AI intelligent process orchestration method for digital employees

technical field

The invention relates to the technical field of data processing, in particular to a digital employee AI intelligent process arrangement method.

Background technique

At present, if the process information recorded in paper documents is manually entered into the computer, the efficiency is low, so image recognition is a common solution. Among them, OCR (Optical Character Recognition, Optical Character Recognition) refers to the process in which electronic equipment checks characters printed on paper, determines their shapes by detecting dark and bright patterns, and then uses character recognition methods to translate the shapes into computer data. For printed characters, the text in the paper document is optically converted into a black and white dot matrix image file, and the text in the image is converted into a text format by the recognition software for further editing and processing by the word processing software. The main indicators to measure the performance of an OCR system are: rejection rate, false recognition rate, recognition speed, product stability, ease of use, etc. RPA digital workers deeply integrate traditional character recognition with machine learning, can parse data from non-standard documents, and help convert handwritten text characters into machine-readable formats. In most cases, OCR is mainly used to simplify paper business and convert it into digital business, such as: PDF, scanned documents, paper invoices, fax and handwritten documents, etc.

But for the identification of paper documents, because the documents may not be neatly placed or the angles are not aligned, the direction of the characters actually photographed may change. At the same time, some documents contain tables or flow charts, and the conventional character direction judgment cannot be used. Therefore, the prior art aims at this problem, especially when the angle difference is large, it may be difficult to identify accurately.

Contents of the invention

Aiming at the problem that it is difficult to judge the angle or direction of characters in the paper document recognition process in the prior art, the present invention provides a digital employee AI intelligent process layout method, which is mainly aimed at the pre-processing of character recognition, and automatically corrects the angle and direction of characters, avoiding the Recognition errors or recognition failures are caused by special forms or flow charts, which is conducive to improving the speed and accuracy of processing, and the obtained results are accurate and clear for subsequent recognition.

The following is the technical scheme of the present invention.

The digital employee AI intelligent process orchestration method includes the following steps:

S1: Collect the original image of the paper document with process information, perform differential grayscale conversion on the original image, and obtain several differential grayscale images;

S2: Rotate the difference grayscaled images at a preset angle to obtain several rotated grayscaled images;

S3: Inflate the rotated grayscale image, use the Hough transform to detect the character line formed by each line of characters after the expansion, and obtain the character direction map;

S4: Perform perspective transformation on the rotated grayscale image before expansion according to the character direction diagram to obtain a correction diagram;

S5: Extract the arrow logo in the rectified image, use the arrow logo as auxiliary information to perform affine transformation on the rectified image, rotate to obtain the restored image, binarize the restored image and input it to the character recognition module for recognition, and sequentially extract process information to complete the arrangement .

The present invention can prevent the problem of unclear images that may occur in single grayscale through differential grayscale, and through the rotation of the preset angle, it can ensure that at least one image with a smaller angle than the straightening angle appears, so as to reduce errors in the subsequent transformation process Finally, through a series of transformations and using arrows to indicate the identification process information, it can avoid identification errors or identification failures caused by special forms or flow charts, which is conducive to improving the speed and accuracy of processing.

As preferably, said performing differential grayscale on the original image includes:

The original image is grayscaled with the average value of the RGB value as the grayscale value to obtain an average grayscaled image;

The original image is grayscaled with the maximum value in the RGB value as the grayscale value to obtain the maximum grayscaled image;

The weighted average grayscale is performed on the original image with the RGB value combined with the preset weight to obtain a weighted average grayscale image.

As a preference, the acquisition process of the preset weight includes:

Calculate the ratio of the pixels whose R value is greater than the critical value to the total pixels in the original image to obtain the first ratio, calculate the ratio of the pixels whose G value is greater than the critical value to the total pixels to obtain the second ratio, and calculate the pixels whose B value is greater than the critical value and the ratio of the total pixels to obtain the third ratio;

According to the magnitude of the first ratio, the second ratio, and the third ratio, the preset weight of each value of RGB is determined in equal proportion.

In this solution, take the ratio of pixels whose R value is greater than the critical value to the total pixels as an example. The larger the first ratio, the larger the proportion of the color of the R value in the image as a whole, and the greater the impact on the image. Therefore, etc. When the ratio determines the preset weight of each value of RGB, the greater the weight obtained by the R value, the smaller the ratio, the smaller the weight obtained; this method can strengthen the difference brought about by the color characteristics of the image, especially suitable for text For recognition image processing tasks, compared with general images, in text-based images, there is usually an obvious gap between the color parameters of the text and the background. This method can amplify the difference caused by the parameter gap. The threshold value is generally set to about 128, which can be adjusted according to actual needs.

As a preference, said rotating the difference gray-scaled image at a preset angle to obtain several rotated gray-scaled images, including:

Set the preset angles to -90 degrees, 90 degrees, and 180 degrees, select a preset angle for each difference grayscale image in turn, and rotate to obtain several rotated grayscale images. Generally speaking, it is easiest to recognize when the angle between an image with an uncertain angle and the expected straightening angle is less than 45 degrees, but in fact the image may be placed flat or upside down, which will seriously increase the difficulty of recognition. Therefore, through the above By rotating, at least one image with an included angle of less than 45 degrees with the straight angle can be obtained, and the probability of accurate recognition increases, which is beneficial to character recognition.

Preferably, the perspective transformation is performed on the rotated grayscale image before expansion according to the character direction diagram to obtain a rectification diagram, including:

Taking any character line in the character direction graph as the reference line, the pixels of the rotated grayscale image before expansion are partially stretched or compressed, so that the rest of the character lines are parallel to the reference line, and a correction map is obtained.

Preferably, said extracting the arrow mark in the rectified image, using the arrow mark as auxiliary information to perform affine transformation on the rectified image, and rotating to obtain the restored image, includes:

Judging the direction marked by the arrow in the same correction diagram, obtaining several unit vectors, calculating the total vector of several unit vectors, and judging the direction (x, y) of the total vector;

Rotate the correction map until the character line in the correction map is in the horizontal position, and y is less than or equal to 0 in the pointing direction (x, y) of the total vector, and the candidate map is obtained;

According to the actual rotation angle of the candidate image relative to the original image, at least one qualified candidate image is retained as the restored image.

This solution optimizes the recognition of flow charts with arrows. Generally, the flow charts are from top to bottom as a whole. However, because the direction of the arrows of partial forks is not consistent, the judgment is based on the direction of the total vector. When If y is less than or equal to 0 after rotation, it means that the total vector has a downward component, regardless of whether it is biased to the left or to the right, it meets the conditions. This step filters out rotated and inverted images.

Preferably, the screening is carried out according to the actual rotation angle of the candidate image relative to the original image, and at least one qualified candidate image is retained as the restored image, including:

Judging the actual rotation angles of different candidate images obtained by processing the same original image relative to the original image, calculating the numerical distribution of each actual rotation angle, retaining the actual rotation angles whose numerical difference is within 10%, and deleting the remaining actual rotation angles candidate graphs, and the remaining candidate graphs are used as restored graphs. Although the straight line of the character is in a horizontal position, it is not ruled out that the image is processed as an inversion. This possibility can be reduced on the basis of introducing the arrow judgment, and this possibility can be basically eliminated by further screening.

As a preference, the calculation process of the actual rotation angle includes:

Record the preset angle p rotated by each rotated grayscale image;

Record the rotation angle q when the correction image is rotated to obtain the candidate image;

The actual rotation angle C=q+p, where clockwise rotation is recorded as positive, and counterclockwise rotation is recorded as negative.

The substantive effects of the present invention include: using digital employees to perform AI text recognition on the image to be recognized, and obtaining a number of grayscale images highlighting different color characteristics through differential grayscale, which is convenient to obtain the most clear results; through the rotation of the preset angle At least one image with an angle of less than 45 degrees to the straight angle can be obtained, and the probability of accurate recognition increases; through the overall judgment of the direction of the arrow, the correction process is assisted; through the progressive steps of the above steps, they interact with each other , can gradually reduce the error of angle and direction, increase the success rate of correction, and finally accurately judge the direction of the text, and there will be no inversion of the text, which is suitable for the initial recognition of the flow chart.

Description of drawings

Fig. 1 is a flowchart of an embodiment of the present invention.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions will be clearly and completely described below in conjunction with the embodiments. Apparently, the described embodiments are only some of the embodiments of the present invention, not all of them. the embodiment. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

It should be understood that in various embodiments of the present invention, the sequence numbers of the processes do not mean the order of execution, and the execution order of the processes should be determined by their functions and internal logic, rather than by the implementation order of the embodiments of the present invention. The implementation process constitutes no limitation.

It should be understood that in the present invention, "comprising" and "having" and any variations thereof are intended to cover a non-exclusive inclusion, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to Those steps or elements are not explicitly listed, but may include other steps or elements not explicitly listed or inherent to the process, method, product or apparatus.

It should be understood that in the present invention, "plurality" means two or more. "And/or" is just an association relationship describing associated objects, which means that there can be three kinds of relationships, for example, and/or B, which can mean: A exists alone, A and B exist at the same time, and B exists alone. . The character "/" generally indicates that the contextual objects are an "or" relationship. "Includes A, B and C", "Includes A, B, C" means that A, B, and C are all included, "includes A, B, or C" means includes one of A, B, and C, "Containing A, B and/or C" means containing any 1 or any 2 or 3 of A, B and C.

The technical solution of the present invention will be described in detail below with specific examples. The embodiments may be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

Example:

The digital employee AI intelligent process orchestration method, as shown in Figure 1, includes the following steps:

S1: Collect the original image of the paper document with process information, perform differential grayscale conversion on the original image, and obtain several differential grayscale images. include:

The original image is grayscaled with the average value of the RGB values as the grayscale value to obtain an average grayscaled image;

The original image is grayscaled with the maximum value in the RGB value as the grayscale value to obtain the maximum grayscaled image;

The weighted average grayscale is performed on the original image with the RGB value combined with the preset weight to obtain a weighted average grayscale image.

Among them, the acquisition process of the preset weight includes:

Calculate the ratio of the pixels whose R value is greater than the critical value to the total pixels in the original image to obtain the first ratio, calculate the ratio of the pixels whose G value is greater than the critical value to the total pixels to obtain the second ratio, and calculate the pixels whose B value is greater than the critical value and the ratio of the total pixels to obtain the third ratio;

According to the magnitude of the first ratio, the second ratio, and the third ratio, the preset weight of each value of RGB is determined in equal proportion.

In this solution, take the ratio of pixels whose R value is greater than the critical value to the total pixels as an example. The larger the first ratio, the larger the proportion of the color of the R value in the image as a whole, and the greater the impact on the image. Therefore, etc. When the ratio determines the preset weight of each value of RGB, the greater the weight obtained by the R value, the smaller the ratio, the smaller the weight obtained; this method can strengthen the difference brought about by the color characteristics of the image, especially suitable for text For recognition image processing tasks, compared with general images, in text-based images, there is usually an obvious gap between the color parameters of the text and the background. This method can amplify the difference caused by the parameter gap.

In most cases, the conventional grayscale method can get the expected effect, but sometimes it can't work. For example, in a captured image, the background and the text color are very similar, and the background color is greenish , its RGB value is (180, 250, 100), which accounts for 70% of the entire image, while the color of the text part is bluish, and its RGB value is (100, 180, 250), accounting for 30% of the entire image. If it is grayscaled by the average value, the gray value of the background and text is the same, which is obviously not conducive to subsequent recognition; if it is grayed by the maximum value, the gray value of the background and text is still the same, which is obviously not good. for subsequent identification. Although in most cases, the grayscale values obtained by these processing methods will not be equal, but at this time, other methods must be relied on for grayscale conversion.

If the weighted average grayscale of this embodiment is adopted, when the critical value is 128, the pixels with an R value greater than 128 account for 70%, so the first ratio is 0.7; the G values of all pixels are greater than 128, so the second The ratio is 1; similarly, the third ratio is 0.3. Determine the default weight of each value of RGB in equal proportions, then the default weight of R is 0.35, the default weight of G is 0.5, and the default weight of B is 0.15. Therefore, the gray value of the background is 203, and the gray value of the text is 0.35. The gray value is 162.5. Since the original image itself is very similar in color, the difference seen by the naked eye after grayscale is not obvious, but compared with the conventional method, there is already a clear distinction, and by adjusting the threshold, a clearer image can also be obtained.

S2: Rotating the difference grayscaled images at a preset angle to obtain several rotated grayscaled images. include:

Set the preset angles to -90 degrees, 90 degrees, and 180 degrees, select a preset angle for each difference grayscale image in turn, and rotate to obtain several rotated grayscale images. Generally speaking, it is easiest to recognize when the angle between an image with an uncertain angle and the desired straightening angle is less than 45 degrees, but in fact, the image may be placed flat or upside down, which will seriously increase the difficulty of recognition. Therefore, through the above By rotating, at least one image with an included angle of less than 45 degrees to the square angle can be obtained, and the probability of accurate recognition increases, which is beneficial to character recognition.

S3: Dilate the rotated grayscale image, use Hough transform to detect the character line formed by each line of characters after dilation, and obtain the character direction map.

The most commonly used method of skew correction is Hough transform, whose principle is to expand the image and connect intermittent text into a straight line, which is convenient for straight line detection.

S4: Perspective transformation is performed on the rotated grayscale image before expansion according to the character direction map to obtain a correction map. include:

Taking any character line in the character direction graph as the reference line, the pixels of the rotated grayscale image before expansion are partially stretched or compressed, so that the rest of the character lines are parallel to the reference line, and a correction map is obtained. This process is similar to the process of keystone correction, which can correct the angle caused by the shooting position.

S5: Extract the arrow logo in the rectified image, use the arrow logo as auxiliary information to perform affine transformation on the rectified image, rotate to obtain the restored image, binarize the restored image and input it to the character recognition module for recognition, and sequentially extract process information to complete the arrangement . Used to correct a skewed image to a horizontal position. include:

Judging the direction marked by the arrow in the same correction diagram, obtaining several unit vectors, calculating the total vector of several unit vectors, and judging the direction (x, y) of the total vector;

Rotate the correction map until the character line in the correction map is in the horizontal position, and y is less than or equal to 0 in the pointing direction (x, y) of the total vector, and the candidate map is obtained;

According to the actual rotation angle of the candidate image relative to the original image, at least one qualified candidate image is retained as the restored image.

This solution optimizes the recognition of flow charts with arrows. Generally, the flow charts are from top to bottom as a whole. However, because the direction of the arrows of partial forks is not consistent, the judgment is based on the direction of the total vector. When If y is less than or equal to 0 after rotation, it means that the total vector has a downward component, regardless of whether it is biased to the left or to the right, it meets the conditions. This step filters out rotated and inverted images.

In addition, the extraction process of the restoration map includes:

Judging the actual rotation angles of different candidate images obtained by processing the same original image relative to the original image, calculating the numerical distribution of each actual rotation angle, retaining the actual rotation angles whose numerical difference is within 10%, and deleting the remaining actual rotation angles candidate graphs, and the remaining candidate graphs are used as restored graphs.

The calculation process of the actual rotation angle includes:

Record the preset angle p rotated by each rotated grayscale image;

Record the rotation angle q when the correction image is rotated to obtain the candidate image;

The actual rotation angle C=q+p, where clockwise rotation is recorded as positive, and counterclockwise rotation is recorded as negative.

Although the straight line of the character is in a horizontal position, it is not ruled out that the image is processed as an inversion. This possibility can be reduced on the basis of introducing the arrow judgment, and this possibility can be basically eliminated by further screening.

What needs to be explained is that, through the step-by-step progression of the above steps and their interaction with each other, the error of angle and direction can be gradually reduced, the success rate of correction can be increased, and finally the direction of the text can be accurately judged without inversion of the text. Applicable to the initial identification of flow charts. The above steps achieve the result that 1+1 is greater than 2. If any step is missing, other steps will lose the optimal effect, resulting in inaccurate results.

In this embodiment, the problem of unclear images that may occur in single grayscale can be prevented through differential grayscale, and through the rotation of the preset angle, at least one image with a smaller angle to the alignment can be guaranteed to reduce the subsequent transformation process The probability of error, finally, through a series of transformations and the use of arrows to indicate the identification process information, there will be no text inversion, and it can avoid identification errors or identification failures due to special forms or flow charts, which is conducive to improving processing speed and accuracy.

In the embodiments provided in this application, it should be understood that the disclosed structures and methods may be implemented in other ways, or some features may be omitted or not implemented.

In addition, the embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

If an integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the software product is stored in a storage medium Among them, several instructions are included to make a device (which may be a single-chip microcomputer, a chip, etc.) or a processor (processor) execute all or part of the steps of the methods in various embodiments of the present application. The above-mentioned storage medium includes: U disk, mobile hard disk, read only memory (read only memory, ROM), random access memory (random access memory, RAM), magnetic disk or optical disk, and other various media that can store program codes.

The above content is only the specific implementation of the application, but the scope of protection of the application is not limited thereto. Anyone familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the application, and should covered within the scope of protection of this application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims (4)

1. The digital employee AI intelligent process arrangement method is characterized in that it comprises the following steps: S1: Collect the original image of the paper document with process information, perform differential grayscale conversion on the original image, and obtain several differential grayscale images; S2: Rotate the difference grayscaled images at a preset angle to obtain several rotated grayscaled images; S3: Inflate the rotated grayscale image, use the Hough transform to detect the character line formed by each line of characters after the expansion, and obtain the character direction map; S4: Perform perspective transformation on the rotated grayscale image before expansion according to the character direction diagram to obtain a correction diagram; S5: Extract the arrow logo in the rectified image, use the arrow logo as auxiliary information to perform affine transformation on the rectified image, rotate to obtain the restored image, binarize the restored image and input it to the character recognition module for recognition, and sequentially extract process information to complete the arrangement ; The said original image is differentially grayscaled, including: The original image is grayscaled with the average value of the RGB value as the grayscale value to obtain an average grayscaled image; The original image is grayscaled with the maximum value in the RGB value as the grayscale value to obtain the maximum grayscaled image; The weighted average grayscale is performed on the original image with the RGB value combined with the preset weight to obtain a weighted average grayscale image; The extraction of the arrow mark in the rectified image, using the arrow mark as auxiliary information to perform affine transformation on the rectified image, and rotating to obtain the restored image includes: Judging the direction marked by the arrow in the same correction diagram, obtaining several unit vectors, calculating the total vector of several unit vectors, and judging the direction (x, y) of the total vector; Rotate the correction map until the character line in the correction map is in the horizontal position, and y is less than or equal to 0 in the pointing direction (x, y) of the total vector, and the candidate map is obtained; Screening according to the actual rotation angle of the candidate image relative to the original image, retaining at least one qualified candidate image as a restoration image; said screening according to the actual rotation angle of the candidate image relative to the original image, retaining at least one qualified candidate image As a restoration map, including: Judging the actual rotation angles of different candidate images obtained by processing the same original image relative to the original image, calculating the numerical distribution of each actual rotation angle, retaining the actual rotation angles whose numerical difference is within 10%, and deleting the remaining actual rotation angles Candidate graphs, and the remaining candidate graphs are used as restored graphs; The calculation process of the actual rotation angle includes: Record the preset angle p rotated by each rotated grayscale image; Record the rotation angle q when the correction image is rotated to obtain the candidate image; The actual rotation angle C=q+p, where clockwise rotation is recorded as positive, and counterclockwise rotation is recorded as negative. 2. The digital employee AI intelligent process arrangement method according to claim 1, wherein the acquisition process of the preset weight comprises: Calculate the ratio of the pixels whose R value is greater than the critical value to the total pixels in the original image to obtain the first ratio, calculate the ratio of the pixels whose G value is greater than the critical value to the total pixels to obtain the second ratio, and calculate the pixels whose B value is greater than the critical value and the ratio of the total pixels to obtain the third ratio; According to the magnitude of the first ratio, the second ratio, and the third ratio, the preset weight of each value of RGB is determined in equal proportion. 3. The digital employee AI intelligent process arrangement method according to claim 1, characterized in that, the said difference grayscale images are rotated at a preset angle to obtain several rotated grayscale images, including: Set the preset angles to -90 degrees, 90 degrees, and 180 degrees, select a preset angle for each difference grayscale image in turn, and rotate to obtain several rotated grayscale images. 4. The digital employee AI intelligent process layout method according to claim 1, characterized in that, the perspective transformation of the rotated grayscale image before expansion is performed according to the character trend diagram to obtain the correction diagram, including: Taking any character line in the character direction graph as the reference line, the pixels of the rotated grayscale image before expansion are partially stretched or compressed, so that the rest of the character lines are parallel to the reference line, and a correction map is obtained.

Images