CN114419632A - A method, device and system for generating OCR training samples - Google Patents

Abstract

The invention discloses an OCR training sample generation method, device and system, and relates to the field of computer vision. The method comprises the following steps: a character outline extraction step, namely extracting all character outlines based on the original image, determining an erasing area mask by combining the erasing area coordinates, and obtaining a repairing area mask; an image repairing and filling step, namely performing image repairing and filling according to a mask of a repairing area and pixel information around the repairing area to obtain a background template after characters are erased; and a random text generation step, namely generating a random text in each generation area, thereby obtaining a new sample picture and a corresponding marking information file. The method combines the technologies of character contour extraction algorithm, image restoration and the like, fully utilizes the background information of the original picture to generate a high-quality training picture, and simultaneously generates a labeling file (containing character content and position information) corresponding to the picture, so that the method avoids the tedious labeling work and can be directly used for OCR model training.

Description

A method, device and system for generating OCR training samples

technical field

The present invention relates to the field of computer vision, in particular to a method, device and system for generating OCR training samples.

Background technique

OCR (Optical Character Recognition, Optical Character Recognition) tasks widely exist in life and business scenarios. The current best method for this task is to use deep learning techniques for text localization and recognition. However, in real scenarios, there is often a shortage of training samples in a certain format, and deep learning relies on huge training samples. Therefore, the method of automatically generating OCR training samples emerges as the times require.

Current methods for generating OCR samples are roughly divided into two categories:

The first category is template-based methods. That is, input a clean template image, and then write random text in the specified position of the template to generate an OCR sample of the template style. This method has two main defects: one is that we cannot obtain template images in most cases; the other is that template images are often clean and undeformed ideal images, which are far from the real images mixed with various noises. The sample authenticity is poor.

The second category is image editing techniques based on deep learning. This type of method has four main defects: the first defect is the poor generalization ability of the model. Since the OCR image features and text features of different layouts are very different, for the image layout or text style that has not appeared in the model training process, The model often performs poorly; the second defect is that it is difficult to obtain training data. Due to the limited generalization ability of the model, if you want to train an image editing model for a certain layout, you first need a large amount of training data similar to the layout, and the current scene is originally It is because of the lack of data that image editing technology is needed; the third defect is that the samples for training the image editing model need pixel-level annotation, which is very expensive and time-consuming; the fourth defect is that the model inference speed is relatively slow, and it often runs. A deep learning inference framework is required, which greatly increases the engineering complexity.

SUMMARY OF THE INVENTION

The invention relates to a method for generating OCR training samples. In view of the lack of samples often encountered in the OCR training process, the applicant innovatively combined text contour extraction algorithms and image restoration technologies to make full use of the background information of the original images to generate high-quality training images and generate annotation files corresponding to the images. (including text content and location information), eliminating the tedious and labor-intensive labeling work, and can be directly used for OCR model training. Therefore, based on the traditional computer vision method, the present invention directly processes the real original OCR picture, and overcomes the above drawbacks.

According to a first aspect of the present invention, a method for generating an OCR training sample is provided, wherein the input information is an original image and coordinates of an erased area, and the method includes:

The text outline extraction step, extracts all text outlines based on the original image, determines the erasure area mask (mask) in combination with the erasure area coordinates, and obtains the repair area mask;

In the image repairing and filling step, image repairing and filling is performed according to the repaired area mask and the pixel information around the repaired area to obtain the background template after erasing the text;

In the random text generation step, random text is generated in each generation area, thereby obtaining a new sample image and the corresponding annotation information file.

Further, the text outline extraction step specifically includes:

Convert the input original image to a single-channel grayscale image, and then adaptively binarize it to obtain all text outline masks, the text area value is 1, and the background area value is 0;

According to the coordinates of the erasing area, the erasing area mask is obtained, the value of the erasing area is 1, and the value of other areas is 0;

Multiply all the text outline masks with the corresponding position pixels of the erased area mask to obtain the erased area text outline mask;

Morphological expansion is performed on the erased region text outline mask, thereby obtaining the repaired region mask.

Further, the swelling core of the morphological swelling is 2 or 3.

Further, the image repairing and filling step specifically includes:

Determine the area to be repaired in the original image according to the repair area mask;

Poll each pixel of the area to be repaired in order from outside to inside, and calculate the pixel value that should be filled in the repaired point according to the information of the known pixels around a certain pixel to become a known pixel;

Calculate the pixel value of the next pixel inward;

Iterative step by step, the area to be repaired gradually shrinks and becomes smaller, until the area to be repaired is repaired, and the repaired background template after erasing the text is obtained.

Further, the sorting algorithm from outside to inside is the Fast Marching Method.

Further, according to the information of the known pixels around a certain pixel, the method for calculating the pixel value that should be filled in the repaired point includes the weighted average of the known pixel values in the neighborhood, the INPAINT_NS or the INPAINT_TELEA method.

Here, INPAINT_NS (Navier-Stokes-based inpainting method) and INPAINT_TELEA (image gradient-based fast matching method, also known as Telea method) are two common image inpainting algorithms.

Further, the generation area is a location area where sample characters need to be generated in the background template after erasing the characters.

Further, the random text generation step specifically includes:

For a certain generation area, determine the expected length w of the random text for the generation area, set the font size to s, and estimate the number of characters in the random text n=int(w/s);

Generate redundant random text with length of n*k characters, k is a redundant multiple, and the value is a positive integer;

Determine the final generated random text and its actual length L according to the relationship between the length of the redundant random text and the expected length w of the random text;

Randomly determine the position of the generated text in the generation area, write the final generated random text and determine its label information;

Poll each generated area to obtain a new sample image and the corresponding annotation information file.

Further, the determining the expected length w of the random text for the generation area specifically includes:

Take the maximum value of the length and width of the generated area as the maximum length of the generated text, the minimum value of the length and width as the minimum length of the generated text, and randomly select an integer between the minimum and maximum values as the expected length of the random text , denoted as w.

Further, in the described step of generating redundant random texts of n*k character lengths, if the corpus type is specified, then the redundant random texts of n*k character lengths are generated with this corpus type; if the corpus type is not specified, then Randomly generate redundant random text of length n*k characters.

Further, the generation method of the specified corpus includes:

Regular expression generation: used to generate corpus with clear rules, compile the required corpus rule features into regular expressions, and then randomly generate strings according to the regular expressions; or

Database random acquisition: used to generate corpus with unclear rules or a certain fixed type, obtain all the content of the corpus through public information, and store the data in the database or data file by entry, and randomly obtain one item from it when it is randomly generated. Can.

Further, the value of k is 3.

Further, according to the relationship between the length of the redundant random text and the expected length w of the random text, it is determined that the final generated random text and its actual length L specifically include:

Starting from the first character of the redundant random text, count the actual length of each character and accumulate them in turn, until the total length of the characters just satisfies "one more character will exceed w", and take it as the final generated random text, record is "text", and records the actual length L of the final generated random text.

Further, randomly determine the position of the generated text in the generation area, write the final generated random text and determine that its label information specifically includes:

Let the coordinates of the upper left corner of the generated area be (x1, x2) and the coordinates of the lower right corner be (x2, y2), then the x-axis coordinate range of the starting point of the upper left corner of the final generated random text is [x1, (x2-w)] , The y-axis coordinate range is [y1, (y2-s)], and an integer point (x, y) is randomly selected within this range as the starting position of the final generated random text;

On the background template, with (x,y) as the starting position of the upper left corner of the text, write the final generated random text, and its label information is: the coordinates are [(x,y),(x+L,y) ,(x+L,y+s),(x,y+s)]; the text content is "text".

Further, after the step of randomly determining the position of the generated text in the generation area, writing the final generated random text and determining its label information, it also includes adjusting the font size and color of the final generated random text. step.

Further, the adjusting the font size of the finally generated random text specifically includes: determining the width h of the erasing area, and setting the size of the finally generated random text to h by default.

Further, the adjusting the font color of the finally generated random text specifically includes:

For a certain generated area, select the corresponding area from the repair area mask (mask), and extract the text outline of the area to form the text backbone area;

Extract the area corresponding to the generated area from the original image, and for the three RGB channels of this area, average all pixel values in the text backbone area of each channel, which is the color value of each channel in the text backbone area;

The color value is used as the color of the generated text, so that the font color of the final generated random text is approximately the same as the original text color.

Here, commonly used backbone extraction algorithms include K3M algorithm, Zhang-Suen algorithm, and the like.

According to a second aspect of the present invention, an apparatus for generating OCR training samples is provided. The apparatus operates based on the method provided in any of the preceding aspects, and the apparatus includes:

The text outline extraction module is used to extract all text outlines based on the original image, determine the erasure area mask in combination with the erasure area coordinates, and obtain the repair area mask;

Image inpainting and filling module, which is used for image inpainting and filling according to the repaired area mask and the pixel information around the repaired area to obtain the background template after erasing the text;

The random text generation module is used to generate random text in each generation area, thereby obtaining a new sample image and the corresponding annotation information file.

According to a third aspect of the present invention, there is provided a system for generating OCR training samples, the system comprising: a processor and a memory for storing executable instructions; wherein the processor is configured to execute the executable instructions, to perform the OCR training sample generation method as described in any of the above aspects.

According to a fourth aspect of the present invention, a computer-readable storage medium is provided, characterized in that a computer program is stored thereon, and when the computer program is executed by a processor, the OCR training sample generation according to any one of the above aspects is realized. method.

Beneficial effects of the present invention:

1. Invented a method for generating OCR training samples, which can be used to solve the problem of lack of samples when using deep learning to train OCR models;

2. Innovatively use the expanded text outline as the "image to be repaired area", and use the image repair algorithm to erase the text. It can not only ensure that the text traces are wiped clean, but also maximize the preservation of surrounding pixel information for the text area during image restoration, so as to obtain the best restoration effect;

3. Innovatively proposes an optional adaptive text size module, which can adaptively restore different text sizes in different positions in the real sample according to needs, so that the generated image is closer to the real sample in terms of text size;

4. Innovatively propose an optional adaptive text color module, which can adaptively restore different text colors in different positions in the real sample according to needs, so that the generated image is closer to the real sample in terms of text color;

5. Annotation files can be generated at the same time, including text position coordinates and text content information required for OCR training. Eliminate the tedious and labor-intensive answer labeling work.

Description of drawings

In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present invention, and for those of ordinary skill in the art, other drawings can also be obtained according to the structures shown in these drawings without creative efforts.

FIG. 1 shows an example of an original image according to an embodiment of the present invention.

FIG. 2 shows an example of an extracted text outline according to an embodiment of the present invention.

FIG. 3 shows an example of a generated background template according to an embodiment of the present invention.

FIG. 4 shows an example of a new sample and a new text box generated according to an embodiment of the present invention.

FIG. 5 shows an example of the result of adding an adaptive text size module according to an embodiment of the present invention.

FIG. 6 shows an example of the result of adding an adaptive text color module according to an embodiment of the present invention.

FIG. 7 shows an example of the result of adding a corpus configuration module according to an embodiment of the present invention.

FIG. 8 shows a block flow diagram according to an embodiment of the present invention.

FIG. 9 shows a flowchart of an algorithm of a text outline extraction module according to an embodiment of the present invention.

FIG. 10 shows a flowchart of an image repair module algorithm according to an embodiment of the present invention.

FIG. 11 shows a flowchart of a random generation module algorithm according to an embodiment of the present invention.

FIG. 12 shows a flowchart of an adaptive text size module algorithm according to an embodiment of the present invention.

FIG. 13 shows a flowchart of an adaptive text color module algorithm according to an embodiment of the present invention.

FIG. 14 shows a schematic diagram of a text outline of a certain generation area according to an embodiment of the present invention.

FIG. 15 shows a schematic diagram of a character outline backbone according to an embodiment of the present invention.

FIG. 16 shows an original image of the generation area according to an embodiment of the present invention.

FIG. 17 shows a schematic diagram of the original FIG. 1 according to an embodiment of the present invention.

FIG. 18 shows a schematic diagram of generating FIG. 1 according to an embodiment of the present invention.

FIG. 19 shows a schematic diagram of the original FIG. 2 according to an embodiment of the present invention.

FIG. 20 shows a schematic diagram of generating FIG. 2 according to an embodiment of the present invention.

FIG. 21 shows a schematic diagram of the original FIG. 3 according to an embodiment of the present invention.

FIG. 22 shows a schematic diagram of generating FIG. 3 according to an embodiment of the present invention.

The realization, functional characteristics and advantages of the present invention will be further described with reference to the accompanying drawings in conjunction with the embodiments.

Detailed ways

Exemplary embodiments will be described in detail herein, examples of which are illustrated in the accompanying drawings. Where the following description refers to the drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the illustrative examples below are not intended to represent all implementations consistent with this disclosure. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present disclosure as recited in the appended claims.

The terms "first", "second", etc. in the description and claims of the present disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It is to be understood that the data so used are interchangeable under appropriate circumstances such that the embodiments of the disclosure described herein can, for example, be practiced in sequences other than those illustrated or described herein.

Furthermore, the terms "comprising" and "having" and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those expressly listed Rather, those steps or units may include other steps or units not expressly listed or inherent to these processes, methods, products or devices.

Multiple, including two or more.

And/or, it should be understood that the term "and/or" used in the present disclosure is merely an association relationship for describing associated objects, indicating that three kinds of relationships may exist. For example, A and/or B can mean that A exists alone, A and B exist at the same time, and B exists alone.

The present invention provides a method for generating an OCR training sample, which mainly relates to an image text outline extraction technology, an image backbone extraction technology, and an image repairing and filling technology:

Text outline extraction technology:

There are traditional computer vision-based methods and deep learning-based methods for text contour extraction. Traditional methods are mainly various binarization methods, including fixed threshold binarization, OTSU binarization, and adaptive binarization. The advantages of such traditional methods are that the algorithm is simple and fast, and the interpretability is strong.

The method based on deep learning is mainly an image segmentation algorithm, which has the advantages of strong anti-noise ability, not easily affected by brightness, and strong robustness. The disadvantage is that the labeling of text outline training samples is very difficult, and deep learning requires model training and the inference speed is slow. Therefore, text contour extraction based on deep learning methods is extremely rare in industrial scenarios.

Image backbone extraction technology:

Image backbone extraction techniques are used to reduce objects within a binarized image to a 1-pixel wide representation. It works by continuously iterating over the image, removing pixels on object boundaries without changing their connectivity, until no more pixels can be removed. The present invention uses this technology to extract the backbone of the text in the image.

Image inpainting and filling techniques:

Image inpainting and filling techniques can be divided into three categories: image filling, image inpainting, and texture synthesis. Image filling is generally used when the area to be filled is large. The general idea is to set the size of the filling size for each iteration, find an area that best matches the pixels around the area to be filled, copy its pixels directly to the area to be filled, and iterate until all the areas to be filled are filled. The image inpainting technology is generally used in the case where the area to be filled is small. The general idea is to poll each pixel in the area to be filled from the outside to the inside, and calculate the value of the pixel to be filled by some algorithm according to the known pixel information around the pixel. , until all pixels are filled. Texture synthesis is generally used in images with obvious texture features, such as cells and brick walls, and the content to be filled is the growth of texture in a known area. The OCR sample images do not meet the characteristics of texture synthesis. The present invention conducts detailed experiments on the two methods of image filling and image inpainting in the experimental stage, and finds that the method using image inpainting has the best effect in this scene.

Example

The invention relates to a method for generating OCR training samples. This method can adaptively generate a large number of high-quality OCR samples to solve the problem of lack of OCR training samples.

If you want to use Figure 1 as the original image, generate a large number of samples of this layout. For the convenience of presentation, the text box added later is not only the "erasing area" where we want to erase the text, but also the "generating area" where we want to generate the text (the erasing area and the generating area do not have to overlap).

The first step is to enter the text outline extraction module, and extract all text outlines in the "erased area" on the entire image, as shown in Figure 2.

The second step, enter the image repair module, take the text outline (the white part in Figure 2) as the damaged part of the original image, repair and fill the image according to the pixel information around the damaged part, and obtain the background template after erasing the text, such as shown in Figure 3.

The third step is to enter the random generation module, generate random text of the specified style in the specified generation area, and record the coordinates and text content of the generated text as the annotation information, as shown in Figure 4.

The overall module flow chart is shown in Figure 8 below, and the algorithm flow chart inside each module is shown in the description section of each module.

Text outline extraction module

As shown in Figure 9 above, the input of the text outline extraction module is the original image and the coordinates of the "erased area". For the input image, we first convert it into a single-channel grayscale image, and then adaptively binarize it to obtain a mask with a pixel value of 0 and 1, that is, a text outline mask, denoted as mask1, where the value The area with a value of 1 is the text area, and the area with a value of 0 is the background area. For all the input "erased area" coordinates, we also construct a mask, so that the value of the erased area is 1, and the value of other areas is 0, denoted as mask2. Multiply the corresponding position pixel of mask1 and mask2. For the part with mask2 value of 0, the multiplication result is 0; for the part with mask2 value of 1, the multiplication result is the original value of mask1. In this way, the mask of "Erase Area Text Outline" can be obtained.

Since the edges of the text contours extracted by binarization may be incomplete, the traces of the text edges cannot be completely erased after repairing. We perform morphological expansion on the "erased area text contour" mask obtained in the previous step (the expansion kernel is 2 or 3). , to ensure that the text outline can completely cover the text in the original image, so as to obtain the repair area mask, as shown in Figure 2.

If we don't want to erase any text on the picture, just enter an empty "Erase Area Coordinates". At this time, the constructed mask2 value is all 0, and the result of multiplication with mask1 is also all 0, and it is still all 0 after morphological expansion, so the mask value of the "repair area" obtained by this module is all 0, that is, no repair is required. any area.

If we want to erase all the text on the picture, in addition to entering the coordinates of all the text as the method of erasing the area, we can also ignore mask2 directly, use mask1 directly as the "erasing area text mask", and then shape it Learn to dilate to get the repair area mask.

Image inpainting module

The function of the image restoration module is to erase the text that needs to be erased on the picture and generate a background template. The repair area mask obtained from the previous module is used to tell the module the specific area that needs to be repaired. Now we poll each pixel of the area to be repaired in order from outside to inside, (the sorting algorithm from outside to inside can be freely selected, such as FastMarching Method), according to the known pixel information around the pixel , calculates the pixel value that the repair point should be filled with. The reason why we choose the order from outside to inside, that is, fill the pixels around the area to be repaired first and then fill the inner pixels, is because there are more known pixels in the neighborhood of the pixels around the contour, and there are more abundant known pixels. information, so that the fill value calculated based on the surrounding pixels is more accurate and smooth. When a pixel to be repaired is filled, the pixel is considered to be a known pixel. When calculating the surrounding pixels to be filled, it can be used as known pixel information and provide a reference value. The specific method for calculating the pixel value based on the pixel information around a pixel can be freely selected, such as the weighted average of the known pixel values in the neighborhood, [INPAINT_NS], [INPAINT_TELEA] and other methods that take into account both speed and effect.

Through the step-by-step iteration of the above method, the area to be repaired gradually shrinks and becomes smaller until the entire area is repaired, and a repaired background template image is obtained, as shown in Figure 3.

Randomly generated modules

The random generation module is used to generate random text and annotation files at specified or random positions of the background template. By setting the generation number, this module is used to quickly generate a specified number of new samples. This module requires the user to specify the font size and color of the generated text. If the user does not specify it, the set default value will be used.

For each "generating area", we take the maximum length and width as the maximum length of the generated text, take the minimum length and width as the minimum length of the generated text, and then randomly select one between the minimum and maximum values. The integer is used as the actual length of the text generated this time, which is denoted as w. This not only realizes the randomness of the length of the generated text, but also ensures that the length of the text does not exceed the generated area. We record the set font size as s, and estimate the number of characters in the text n=int(w/s). If the corpus type is specified, we use the corpus type to generate redundant texts with a length of n*k characters. If the corpus type is not specified, we randomly generate redundant texts with a length of n*k characters. Among them, k is the redundancy multiple. Since the above estimated number of characters is the case where the width of each character is exactly equal to the height of the character, and the aspect ratio of different characters is not necessarily strictly 1:1, we enlarge n by k times to ensure that the number of redundant text characters generated is sufficient, usually k is 3. Then start from the first character of the redundant text, count the actual length of each character and accumulate them in turn, until the total length of the characters just satisfies "one more character will exceed w", and it is used as the final generated text, recorded as text, and record its real length at this time as L.

Next, we implement the random position function of text generation in the generated area. In order to ensure that the generated text does not exceed the generation area, set the coordinates of the upper left corner of the generated area to be (x1, x2) and the coordinates of the lower right corner to be (x2, y2), then the x-axis coordinate range of the starting point of the upper left corner of the text should be [x1,( x2-w)], the y-axis coordinate range should be [y1,(y2-s)]. In this range, an integer point is randomly selected as the starting position of the generated text to realize the function of random position.

Then we write the "text" string of the specified color and size on the image, starting at (x, y). The annotation information of the text line can also be obtained: the coordinates are [(x,y),(x+L,y),(x+L,y+s),(x,y+s)], and the text content is " text". After polling each "generating area" in this process, a new image and the corresponding answer annotation file can be obtained.

So far, the OCR training sample generation has been completed, and the effect is shown in Figure 4.

Adaptive text size module

The adaptive text size module is an optional module, used to adaptively adjust the text size of each generated text line, so that the size of the text generated in each area is approximately the same as the size of the original text at that location.

In order to retain more background information, the designated erasing area should be close to the text. At this time, the width of the erasing area is the size of the characters in it, denoted as h. We use this feature to set the font size of the generated text to h by default, and the size of the text generated in this area is approximately the same as the size of the original text in this position.

There are three advantages of this module:

1. The size of the text generated in each area can be kept approximately the same as the size of the text in the original position, and the different font features in different positions of the original image can be restored to the maximum extent;

2. Eliminate the tedious steps of manually setting a fixed font size for each generated text area;

3. Solve the problem that the font size is difficult to control by manually setting the font.

The sample generation effect after adding this module is shown in Figure 5.

Adaptive text color module

The adaptive text color module is an optional module used to adaptively adjust the font color of each generated text line, so that the color of the generated text in each area is approximately the same as the original text color.

For each generated region, we select the corresponding generated region from the "Text Outline Extraction Module", as shown in Figure 14. Backbone extraction is performed on the text outline of this area, as shown in Figure 15. The purpose of extracting the backbone is to filter out some background pixels that may be included in the border of the text outline. The color of these background pixels is often very different from the color of the text, which will introduce errors. Then, for the three RGB channels of the original image of the generated area (as shown in Figure 16), we take the R channel as an example, and average all the pixel values in the R channel text backbone area, which is the R channel color of the text backbone in this area. value. In this way, the average value of each channel is obtained, which is the RGB three-channel color average value of the text in the current area. The color value is used as the color of the generated text, so that the color of the generated text in each area is approximately the same as the original text color.

There are three advantages of this module:

1. The color of the text generated in each area can be kept approximately the same as the color of the text in the original position, and the color characteristics of different fonts in different positions of the original image can be restored to the maximum extent;

2. Eliminate the tedious steps of manually setting a fixed font color for each generated text area;

3. Solve the problem that the font color is manually set, and the color pixel value is difficult to control.

The sample generation effect after adding this module is shown in Figure 6.

Corpus Configuration Module

The corpus configuration module is an optional module, which is used to configure the corpus type of each generated text line, so that the corpus meaning of the generated text in each area is consistent with the original text. This module is embedded in the "Random Generation Module", as shown in Figure 11.

This module uses two methods to generate the specified corpus: 1. Regular expression generation; 2. Random acquisition from database.

The method of regular expression generation is mainly used to generate corpus with clear rules, such as mobile phone numbers, ID numbers, etc. This kind of corpus has fixed character length and encoding rules. We encode the required corpus rule features into regular expressions, and then randomly generate strings according to the regular expressions.

The method of random database acquisition is mainly used to generate unclear rules or certain fixed corpus, such as company name, bank name, country name, province, city, language, etc. For this type of corpus, we must first obtain all the content of the corpus through public information, and store the data in the database or data file by entry, and randomly obtain one from it when it is randomly generated.

The sample generation effect after adding this module is shown in Figure 7. Figures 17-22 are the effect diagrams of using this method to generate virtual samples for OCR samples of different styles.

It should be noted that, herein, the terms "comprising", "comprising" or any other variation thereof are intended to encompass non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements, It also includes other elements not expressly listed or inherent to such a process, method, article or apparatus. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in a process, method, article or apparatus that includes the element.

The above-mentioned serial numbers of the embodiments of the present invention are only for description, and do not represent the advantages or disadvantages of the embodiments.

From the description of the above implementations, those skilled in the art can clearly understand that the above implementation method can be implemented by means of software plus a necessary general hardware platform, and of course hardware can also be used, but in many cases the former is better implementation. Based on this understanding, the technical solutions of the present invention can be embodied in the form of software products in essence or the parts that make contributions to the prior art, and the computer software products are stored in a storage medium (such as ROM/RAM, magnetic disk, CD), including several instructions to make a terminal (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) execute the methods described in the various embodiments of the present invention.

The embodiments of the present invention have been described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific embodiments, which are merely illustrative rather than restrictive. Under the inspiration of the present invention, without departing from the scope of protection of the present invention and the claims, many forms can be made, which all belong to the protection of the present invention.

Claims (17)

1. an OCR training sample generation method, is characterized in that, input information is original image and erasure area coordinate, and described method comprises: In the text outline extraction step, all text outlines are extracted based on the original image, and the erased area mask is determined in combination with the erased area coordinates, and the repaired area mask is obtained; In the image repairing and filling step, image repairing and filling is performed according to the repaired area mask and the pixel information around the repaired area to obtain the background template after erasing the text; In the random text generation step, random text is generated in each generation area, thereby obtaining a new sample image and the corresponding annotation information file. 2. OCR training sample generation method according to claim 1, is characterized in that, described character outline extraction step specifically comprises: Convert the input original image to a single-channel grayscale image, and then adaptively binarize it to obtain all text outline masks, the text area value is 1, and the background area value is 0; According to the coordinates of the erasing area, the erasing area mask is obtained, the value of the erasing area is 1, and the value of other areas is 0; Multiply all the text outline masks with the corresponding position pixels of the erased area mask to obtain the erased area text outline mask; Morphological expansion is performed on the erased region text outline mask, thereby obtaining the repaired region mask. 3. OCR training sample generation method according to claim 1, is characterized in that, described image repair filling step specifically comprises: Determine the area to be repaired in the original image according to the repair area mask; Poll each pixel of the area to be repaired in order from outside to inside, and calculate the pixel value that should be filled in the repaired point according to the information of the known pixels around a certain pixel to become a known pixel; Calculate the pixel value of the next pixel inward; Iterative step by step, the area to be repaired gradually shrinks and becomes smaller, until the area to be repaired is repaired, and the repaired background template after erasing the text is obtained. 4. The method for generating OCR training samples according to claim 3, wherein the sorting algorithm from outside to inside is a fast marching algorithm. 5. The OCR training sample generation method according to claim 3, wherein the method for calculating the pixel value that should be filled in the repair point according to the information of the known pixels around a certain pixel point comprises: the neighborhood is known Pixel value weighted average, INPAINT_NS or INPAINT_TELEA method. 6. OCR training sample generation method according to claim 1, is characterized in that, described random text generation step specifically comprises: For a certain generation area, determine the expected length w of the random text for the generation area, set the font size to s, and estimate the number of characters in the random text n=int(w/s); Generate redundant random text with length of n*k characters, k is a redundant multiple, and the value is a positive integer; Determine the final generated random text and its actual length L according to the relationship between the length of the redundant random text and the expected length w of the random text; Randomly determine the position of the generated text in the generation area, write the final generated random text and determine its label information; Poll each generated area to obtain a new sample image and the corresponding annotation information file. 7. OCR training sample generation method according to claim 6, is characterized in that, described determining the random text expected length w for this generation area specifically comprises: Take the maximum value of the length and width of the generated area as the maximum length of the generated text, the minimum value of the length and width as the minimum length of the generated text, and randomly select an integer between the minimum and maximum values as the expected length of the random text , denoted as w. 8. OCR training sample generation method according to claim 6, is characterized in that, in described generating the redundant random text step of n*k character length, as specified corpus type, then generates n* with this specified corpus type Redundant random text with a length of k characters; if the corpus type is not specified, a redundant random text with a length of n*k characters is randomly generated. 9. OCR training sample generation method according to claim 8, is characterized in that, described generating the redundant random text of n*k character length with this specified corpus type specifically comprises: Regular expression generation: used to generate corpus with clear rules, compile the required corpus rule features into a regular expression, and then randomly generate redundant random text with a length of n*k characters according to the regular expression; or Database random acquisition: used to generate unclear rules or a certain fixed corpus, obtain all the content of the corpus through public information, store the data in the database or data file by entry, and randomly obtain a n *k characters long redundant random text is sufficient. 10. OCR training sample generation method according to claim 6, is characterized in that, described according to the relationship between redundant random text length and random text expected length w, determine the random text finally generated and its actual length L specifically include: Starting from the first character of the redundant random text, count the actual length of each character and accumulate them in turn, until the total length of the characters just satisfies "one more character will exceed w", and take it as the final generated random text, record is "text", and records the actual length L of the final generated random text. 11. OCR training sample generation method according to claim 6, is characterized in that, randomly determines the position of this generated text in the generation area, writes the random text of this final generation and determines that its label information specifically comprises: Let the coordinates of the upper left corner of the generated area be (x1, x2) and the coordinates of the lower right corner be (x2, y2), then the x-axis coordinate range of the starting point of the upper left corner of the final generated random text is [x1, (x2-w)] , The y-axis coordinate range is [y1, (y2-s)], and an integer point (x, y) is randomly selected within this range as the starting position of the final generated random text; On the background template, with (x,y) as the starting position of the upper left corner of the text, write the final generated random text, and its label information is: the coordinates are [(x,y),(x+L,y) ,(x+L,y+s),(x,y+s)]; the text content is "text". 12. The OCR training sample generation method according to claim 6, characterized in that, after the step of randomly determining the generated text in the generating area, writing the final generated random text and determining its labeling information, further Including the step of adjusting the font size and color of the finally generated random text. 13. The method for generating OCR training samples according to claim 12, wherein the adjusting the font size of the finally generated random text specifically comprises: determining the width h of the erasing region, and adjusting the font size of the finally generated random text. The size is set to h by default. 14. The OCR training sample generation method according to claim 12, wherein the adjusting the font color of the finally generated random text specifically comprises: For a certain generated area, select the corresponding area from the repair area mask, and extract the text outline of the area to form the text backbone area; Extract the area corresponding to the generated area from the original image, and for the three RGB channels of this area, average all pixel values in the text backbone area of each channel, which is the color value of each channel in the text backbone area; The color value is used as the color of the generated text, so that the font color of the final generated random text is approximately the same as the original text color. 15. A device for generating OCR training samples, the device operating based on the method according to any one of claims 1 to 14, the device comprising: The text outline extraction module is used to extract all text outlines based on the original image, determine the erasure area mask in combination with the erasure area coordinates, and obtain the repair area mask; Image inpainting and filling module, which is used for image inpainting and filling according to the repaired area mask and the pixel information around the repaired area to obtain the background template after erasing the text; The random text generation module is used to generate random text in each generation area, thereby obtaining a new sample image and the corresponding annotation information file. 16. An OCR training sample generation system, the system comprising: a processor and a memory for storing executable instructions; wherein the processor is configured to execute the executable instructions to execute the steps according to claims 1 to 10 . The OCR training sample generation method described in any one of 14. 17. A computer-readable storage medium, wherein a computer program is stored thereon, and when the computer program is executed by a processor, the method for generating an OCR training sample according to any one of claims 1 to 14 is implemented.

Images