RU2806012C1 - Method for neural network control of text data on document images - Google Patents

Abstract

FIELD: neural networks.

SUBSTANCE: method for neural network control of text data on document images. In the method, input image f of a text field is supplied to the input, and for image f it is known that in the original document the text information on it has property A, width W_f of image f of the text field is also known, and the input field of image f is in the RGB colour space containing the text field of the document is processed by a neural network detector for monitoring text data according to the following algorithm: the image of field f is converted into a single-channel image, after which it is fed to the input of a trained full-convolutional neural network; at the output, the neural network assigns values for each vertical line corresponding to the middle of the receptive field and w_A - confidence estimates for 2 possible classes: class in which the property is missing in the text field; class A, in which the property is present in the text field; calculate the amounts and S_A values of confidence estimates for the 2 possible classes along all vertical lines of the image ƒ text field: check for the presence of an anomaly in f, and if an anomaly is not found in the text field in question, then the image ƒ the text field in question has property A if if an anomaly is found in the text field under consideration, then image f of the text field is considered to have property A in presence of an anomaly, if the condition is met, otherwise image f of the text field does not have property A.

EFFECT: ensuring control of text data on document images.

1 cl, 10 dwg, 1 tbl

Description

The claimed technical solution relates to the field of verification specifically designed to determine the identity or authenticity of paper money or similar securities, or to separate foreign banknotes or others different from those being checked, in particular to a method of neural network control of text data on document images.

Various neural network font detectors are known from the prior art that are capable of monitoring the type of font presented in a text field image. In addition to font type control detectors, there are also neural network approaches that monitor specific font properties, for example, monowidth. Thus, in the source of information (see Chirvonaya, Anastasiya N., Alexander V. Sheshkus, and Vladimir L. Arlazarov. “Monospaced font detection using character segmentation and Fourier transform.” Twelfth International Conference on Machine Vision (ICMV 2019). Vol. 11433. SPIE, 2020) discloses a method that allows you to detect monowidth fonts based on a segmentation neural network and Fourier transform.

However, the known approaches have the following disadvantages:

- the impossibility of applying detectors to images of text fields of arbitrary size due to the selected neural network architectures;

- inapplicability to cases of partial violation of the homogeneity of the properties of text fields, which does not make it possible to use it to identify counterfeit and defective documents;

- lack of information about where exactly in the image an anomaly is present, if it exists;

- lack of a generalized approach to the data application control method, capable of determining the properties of text fields of various types.

There are also methods for identifying fonts based on typographical features of documents (see Zramdini, Abdelwahab, and Rolf Ingold. “Optical font recognition using typographical features.” IEEE Transactions on pattern analysis and machine intelligence 20.8 (1998): 877-882), texture analysis (see Zhu, Yong, Tieniu Tan, and Yunhong Wang. “Font recognition based on global texture analysis.” IEEE Transactions on pattern analysis and machine intelligence 23.10 (2001): 1192-1200), etc. Such methods have a significant drawback in the form of a lack of resistance to imperfect images.

As a result, when working with various features of text data application, for example, in documents, there is a need for a generalized approach to a method for monitoring data application based on a neural network detector for monitoring document text data.

The objective of the claimed invention is to eliminate the disadvantages of the prior art. The technical result consists in providing a method for neural network control of text data on document images, which makes it possible to apply it to the features of text data of various types and control them, the ability to apply it to images of any size, as well as the ability to detect the presence and location of partial violations of the homogeneity of the properties in question if available.

The stated problem is solved, and the stated technical result is achieved through the claimed method of neural network control of text data on document images.

The figures show:

Fig. 1: a) Image of a German ID card document; b) Text field made using laser engraving with a tactile effect; c) Text fields made using conventional printing techniques. Figure 2: a) Example of the font used in a British driving license; b) An example of the font used in German ID cards; c) Example of font used in Philippine ID cards

Fig. 3: Example of a laser engraved document text field

Fig. 4: Example of a document text field made with laser perforation

Fig. 5: Flowchart of the algorithm

Fig. 6: a) An example of a text field made by laser engraving; b) Example of anomaly detection

Fig. 7: Example of text fields correctly recognized as laser engraved

Fig. 8: Example of correctly recognized text fields made using conventional printing techniques

Fig. 9: Example of correctly recognized text fields made using the OCR-A font

Fig. 10: Example of correctly recognized text fields made using Latin fonts (not OCR-A)

When producing identity documents, various methods of applying text information are often used. Depending on the application method, the corresponding text field has different properties that can be visually observed and verified in document images.

In fig. Figure 1 shows an example of an identification document (German identification card), in which different fields are applied in different ways, as a result of which they have different properties: laser engraving of text (see Fig. 1b), conventional printing technology (see Fig. 1c).

The technique of applying text fields using a printer is used on passports, birth certificates, etc. This application method is common due to the simplicity of the equipment. However, on the other hand, such fields are the most unprotected. To check such text fields, font analysis is used: analysis of type, point, slope, boldness, character spacing and also ink color. In fig. Figure 2 shows examples of different fonts used in documents from different countries.

The laser engraving technique is a method of applying text fields to documents, which allows you to engrave document surface elements with black material (see Fig. 3). This ensures high reliability of personalization, which greatly complicates forgery and various manipulations with the document. Attempting to change the engraved information will result in visually obvious damage to the document.

Laser perforation technique is a method of applying text data to documents, which allows you to burn holes in the paper using a thin laser beam according to a specific pattern.

The diameter of the laser-perforated holes gradually decreases from the first perforated page of the document to the last. Burn marks left by the laser beam are visible along the edges of the holes. The holes can vary in shape: round, triangular, square, star-shaped, etc. (see Fig. 4).

Implementation of the invention.

Let the input image ƒ of a text field (for example, Fig. 1b) of a document be supplied as input. Let it be known for the image ƒ that in the original document the text information on it has property A. The width W _f of the image ƒ of the text field is also known.

It is necessary to check the presence of property A in the image in question ƒ text field.

Description of the operating algorithm of the neural network detector

The algorithm parameters are as follows:

• W _s - minimum width for anomaly detection;

• T - minimum threshold for detection class when searching for an anomaly with a width greater than W _s ;

We consider an image ƒ in the RGB color space, containing a text field (Fig. 1b) of the document.

The output expects a response from the neural network detector: does the text field in question have a feature (this case is assigned a class

A) or does not have (class ).

The input field ƒ is processed by a neural network text data monitoring detector using the following algorithm:

1. The field image ƒ is converted into a single-channel image, after which it is fed to the input of a trained fully convolutional neural network (the architecture is presented in Table 1).

2. At the output, the neural network assigns values for each vertical line corresponding to the middle of the receptive field and w _A - confidence estimates for 2 possible classes:

• Class in which the property is missing in the text field;

• class A, in which the property is present in the text field.

3. The amounts are calculated and S _A values of confidence ratings for 2 possible classes along all vertical lines of the image ƒ text field:

4. A check is made for the presence of an anomaly in ƒ.

Let us call an anomaly a situation in which the homogeneity of property A of the text field was partially violated in the text field ƒ. For example, this situation is considered in the example (Fig. 6b), where laser engraving is considered as property A.

Anomaly detection occurs according to the following principle:

1) A search is performed for the maximum possible length L of the anomaly in the image ƒ of the text field, provided

2) An anomaly was found if L≥W _s .

5. If no anomaly is found in the text field under consideration, then the image ƒ of the text field under consideration has property A if

If an anomaly is found in the text field under consideration, then the image ƒ of the text field is considered to have property A in the presence of an anomaly if the condition is met

In other cases, the image ƒ of the text field does not have property A.

Implementation of a neural network detector

The detector is trained on the basis of a fully convolutional neural network with the architecture presented in Table 1. When training the network, single-channel images of 77x27 pixels were supplied as input, and at the output of the network, pseudo-probabilities of belonging to two possible classes are expected: text data has property A or does not have this property.

Visualization of the algorithm's operation

For ease of visual perception of the detector’s operation, each field in the figures (see Fig. 6, 7, 8, 9, 10) corresponds to the graph below, where for clarity, the horizontal red bar is responsible for the value of the confidence estimate (pseudo-probability) equal to 0.5, and along the vertical axis The values of confidence estimates for the value w _A are located from 0 to 1. Since the network architecture used in the work (Table 1) is fully convolutional, an image of any width can be fed to the input of the trained network. Thus, when detecting anomalies (Fig. 6b), the fully convolutional algorithm is able not only to determine their presence, but also to indicate in which parts of the considered text string image these anomalies are located.

Thus, the claimed method makes it possible to control the features and properties of images of text fields on a document, as well as: This method will allow:

- strengthen quality control and identify defective documents from images;

- confirm the authenticity of a document using its image.

Example 1.

As property A, one of the methods of applying text data was considered - the laser engraving technique (Fig. 1b). The neural network detector presented in the work was trained on images of text fields that have this property, for example, images of text fields of German identification cards, as well as text fields that do not have this property. In Fig. 7 and Fig. Figure 8 shows a visualization of the results of the trained neural network detector for monitoring laser-engraved text data.

Example 2.

The OCR-A font was considered as property A of text data (Fig. 2c). The neural network detector presented in the work was trained on synthetic data consisting of images of text data made using the OCR-A font and other Latin fonts. In Fig. 9 and Fig. Figure 10 shows a visualization of the result of the work of a trained neural network detector for monitoring text data made using the OCR-A font.

Example 3 (anomaly detection)

Let us consider the operation of the algorithm in the case of example 1, when the homogeneity of the property was partially violated in the field ƒ, i.e. Not all information has been replaced, but only part of it. In the example shown (Fig. 6b), the text field around the edges remains the same, i.e. made with laser engraving, and the central part was replaced with regular flat numerals without laser engraving.

Let the threshold T=0.85. In this case, the resulting anomaly length L≥W _s => the anomaly is found, and the sum of confidence estimates satisfies the condition which means the text field ƒ has properties A in the presence of an anomaly.

Claims (7)

A method for neural network control of text data on document images, including feeding an input image f of a text field to the input, and for image f it is known that in the original document the text information on it has property A, the width W _{f of} the image f of the text field is also known, characterized in that that the input image field f in the RGB color space containing the text field of the document is processed by a neural network text data monitoring detector according to the following algorithm: - the image of the field f is converted into a single-channel image, after which it is fed to the input of the trained fully convolutional neural network; - at the output, the neural network assigns values for each vertical line corresponding to the middle of the receptive field and w _A - confidence estimates for 2 possible classes: - Class in which the property is missing in the text field; class A, in which the property is present in the text field; - calculate amounts and S _A values of confidence ratings for 2 possible classes along all vertical lines of the image ƒ text field: - check for the presence of an anomaly in f, and if an anomaly is not found in the text field under consideration, then the image ƒ of the text field under consideration has property A, if if an anomaly is found in the text field under consideration, then the image f of the text field is considered to have property A in the presence of an anomaly if the condition is met in other cases, the image f of the text field does not have property A.