RU2249251C2 - Automatic recognition of symbols on structural background by combination of models of symbols and background - Google Patents

Abstract

FIELD: automated recognition of symbols.

SUBSTANCE: method includes following stages: tuning, forming symbols models, recognition, recording background model together with background of read image, separating model of registered background from elementary image of background, combining for each position of symbol of model of letters and/or digits with elementary displaying of appropriate background, forming of combined models, comparison of unknown symbols to combined models, recognition of each unknown symbol as appropriate symbol, combined model of which is combined with it best in accordance to "template comparison" technology.

EFFECT: higher efficiency.

10 cl, 10 dwg

Description

State of the art

The present invention relates to a method for automatically recognizing characters printed on any material suitable for this, even when the background has very contrasting structures, which therefore interfere strongly with the structure of the printed characters.

The vast majority of known character recognition systems come up with a solution to this problem, trying to separate the characters from the background through the use of a threshold system, often very sophisticated and complex.

Unfortunately, this technology does not lead to success in cases where the contrast of the background structures is very significant, especially if the position of the characters is variable with respect to the mentioned structures. As a result, symbol images in some cases contain some background characters (those that exceed the level of the corresponding threshold value) or sometimes these images are not complete, because part of the symbol structure does not exceed the corresponding threshold value.

This, in particular, refers to the case of control of bank tickets, in which the printing of series numbers is carried out during a phase separate (usually subsequent) from the printing phase of the rest of the image, and usually using other printing equipment. Thus, the registration cannot be completely perfect and, therefore, the series numbers “move” in relation to the background.

This means that if these numbers are printed on the structured area of a given bank ticket, that is, on the area containing any pattern, then they “move” or can be shifted arbitrarily in relation to the structure (pattern) of the background. In addition to the above, in the above cases, even the search and segmentation of characters may not lead to success due to the presence of specific background structures.

Indeed, even taking into account the huge number of variations, the process of distinguishing and recognizing characters almost always goes through the following steps, in which

- reading images of a document and, especially, an object containing symbols to be recognized. The reading of these images is provided through an electronic camera, after which these images are usually processed in order to increase contrast and reduce interference;

- search on the image (now electronic) of the position of the characters to be recognized. This search is often based on an analysis of sharp changes in illumination (such as the transition from white to black) and, in particular, the spatial distribution of these transitions;

- segmentation of the identified zone into sections, each of which contains only one symbol. This segmentation is carried out, for example, by analyzing the projection of the black density onto a segment parallel to the base of the symbol line: the minimum values of this density can be correlated with the white space between the symbols;

- each symbol isolated in this way is compared with the prototypes (models) of all letters and / or all numbers, either in terms of the degree of their coincidence (technologies known under the English name "template matching"), or in terms of the sequence of characteristic structures, such for example, like vertical lines, horizontal lines, oblique lines, etc. (technologies known as "features extraction" or structural analysis).

In any case, it is obvious that if the part of the image segmented as a symbol contains structures that are extraneous to the shape of this symbol itself (for example, lines belonging to the background structure), the risk of insolvency or failure of comparison with the mentioned prototypes is very high. This danger can also be caused by the loss of the distinctive parts of the structure of this symbol, followed by a sharp threshold transition in the phase of separation of this symbol from the background.

That is why the previous approaches to solving the problem of automatic recognition of characters printed on highly structured backgrounds with high contrast are not quite satisfactory.

SUMMARY OF THE INVENTION

In accordance with the invention, objects on which the characters to be recognized are printed are subjected to optical analysis using well-known optoelectronic means, such as, for example, a CCD-type camera (linear or matrix, black and white or color) with the desired resolution to form electronic images of characters to be recognized.

In the following presentation, the term "image" will be used in the sense of an electronic image, in particular, a discrete system of values of light saturation, usually organized in the form of a rectangular matrix. Each element of this matrix, or the so-called pixel, is a measure of the intensity of light reflected by the corresponding part of the object. For color images, their general description consists of three matrices corresponding to the components of red, green, and blue for each image element or pixel. To simplify the presentation, the following description relates to the case of a black and white image. In this case, the extension for the color image is carried out by repeating the same operations for the three matrices.

The basis of the present invention is the task of implementing automatic recognition on electronic images of characters printed on a highly structured background, the contrast of which can even be comparable with the contrast of the structures of these symbols themselves (as shown in the example illustrated in Fig. I / 4c).

The first step of the method in accordance with the invention consists in forming a background model, which can be obtained by reading images of one or more samples, which shows only the background image without any symbols (as shown in the example illustrated in Fig. I / 4b) .

As such a model, it is possible to use, in particular, the arithmetic mean of the aforementioned samples. Moreover, in the case of black-and-white images, one single matrix of these averaged images will be obtained, while in the case of color images of these matrixes of averaged images there will be three, for example, matrices of red, green and blue.

Then, models of the characters to be recognized (for example, letters and / or numbers) are formed, either by reading images of the system of these characters printed on a white background, or by directly using electronic images from computer information files that are currently commercially available for most fonts . In the first case, it is possible to form a model of each symbol to be recognized in the form of an averaged image from a number of copies of the same symbol printed on a white background.

After the models of all the characters and the background model are formed, the first phase of this method, which can be called the "debug phase", is completed.

During the subsequent phase of character recognition, actions are performed in accordance with the following steps:

- carry out the reading of the image subject to recognition of the sample containing unknown characters printed on a specific background in positions that are also unknown (an example is illustrated in Fig. II / 4-a);

- registration of the background model together with the read image using any of the known image registration technologies, for example, using the maximum correlation method;

- subtraction of the registered background model from the read image. The differential image obtained in this way, where the background will be almost completely eliminated, very clearly reveals the position of the characters (an example of the differential image, which is a read image minus the registered background model, is shown in Fig. II / 4-b);

- search for the position of each of the symbols in the difference image. This operation is carried out using any of the known technologies for character localization and segmentation, for example, by analyzing sharp transitions of saturation such as the transition from black to white. Thus, for each position of the symbol, an elementary image will be selected, the sizes of which coincide with the sizes of the models of symbols (Figs. III / 4b show examples of elementary images of segmented symbols);

- selection of the registered background model from the elementary image of the background corresponding to each unknown symbol;

- a combination, for each of the positions of the characters, character patterns with the corresponding elementary image of the background model (see Fig. III / 4-c). Since the background model was registered together with the background of the image containing the characters to be recognized, in elementary images combining the background model with the model of numbers and / or letters, the relative position of the symbol-background is the same as in the unknown image. Thus, in the synthesis for each position of the symbol new prototypes (combined models) of symbols (letters and / or numbers) will be created with the same background as in the unknown image. One of the combination technologies developed will be described in the “Description of Several Preferred Options” section. However, any of the methods proposed by other authors can be used here;

- Comparison of each of the unknown characters with all the models combined in the previous steps. Recognition of a character with a background is thus accomplished by matching patterns of characters with the same background and in the same position. Here you can use any known recognition technologies, for example, the "template-matching" or "features extraction" method, etc.

Brief Description of the Drawings

Fig. I / 4 shows an example of a sequence of characters printed on a highly structured and having a high contrast of the background, and a) shows a sequence of characters printed on a white background, in view b) shows a picture of its own background and in view c) shows the sequence characters a), printed on background b).

In Fig. II / 4, view a) fully corresponds to view c) shown in Fig. I / 4, while view b) shows the result of subtracting the registered background model from the image of the fully printed ticket.

In Fig. III / 4, view a) shows the portion of the ticket from the example shown in the previous figures containing the characters to be recognized, and view b) shows the elementary images corresponding to each position of the character as a result of segmentation. View c) shows, for each position of the symbol, a combination corresponding to elementary images of the registered background with models of all possible symbols, that is, combined models described in the text. This example clearly shows that the characters (b) to be processed can be more effectively recognized when compared with combined models (c) than with models of characters printed on a white background (shown, for example, in FIG. III / 4-d) .

On IV / 4 shows a typical diagram of the construction of the recognition system described in the text.

Description of preferred embodiments of the invention

In the following presentation, one of its preferred variants regarding automatic recognition of series numbers printed on bank notes will be described as a non-limiting example of the implementation of the invention.

Indeed, in most types of bank tickets, the series number is partially or fully printed directly on the background image of this ticket. The printing of bank tickets is carried out, in particular, using a mixture of various technologies, usually at least offset and small notches. This latest technology, in particular, usually represents areas with a large number of lines of very high contrast. When a series number is printed on one of these zones, it is quite difficult to separate the characters from the background using conventional technologies, which means that these characters can be recognized.

In addition, the batch number is usually printed in the final phase of production, after offset printing and fine notching, and on another printing machine. Even if very complex and modern markup systems are used, the relative registration between the series numbers and the background pattern is more likely to be variable, and can usually “move” or move within a few millimeters.

On IV / 4 shows the construction of a system for recognizing series numbers on bank tickets. Here, the CCD 1 linear camera with its lenses 2 and its lighting system 3 is used to read images of bank tickets 4, on which it is necessary to read the series numbers while they are transported using the suction tape 5.

The camera scan lines are sequentially stored in the first circuit of the buffer storage device of the image processing subsystem 6 to form an electronic image of each bank ticket.

The image processing subsystem 6, which can be based both on specialized computing equipment and on programmable computers such as DSP (Digital Signal Processor), high-speed personal computers PC, etc., perform various operations both in the debug phase (formation of a background model and character models), and in the recognition phase.

In the process of implementing the debug phase of the background model, the image processing subsystem:

- provides collection of non-numbered images of bank notes selected as a “Background Debugging Kit” (EAF) and stores this set in a corresponding memory device;

- selects one “reference” image from the set of NOFs for registration and does this either automatically (for example, selects the first image from the set of NOFs) or with the help of a system operator who uses the control panel 7 for this;

- registers all the images from the set of NOF, determining, first of all, the horizontal displacement Δx and the vertical displacement Δy of each image in relation to the reference image, and then applying the corresponding displacement -Δx and -Δy. In this embodiment, the bias is measured using the maximum correlation method. A small rectangular section S ₀ (marking base) of the reference image with the center having coordinates x ₀ , y ₀ , selected, for example, by the operator (outside the zone of printing characters), is compared with the section S _{1 of} the same size, the center of which is shifted step by step by each position (pixel) of the image from the set of NOFs in order to find the position x ₁ , y ₁ , where the correlation coefficient has its maximum value (this corresponds to the most accurate overlap between the two images). In this case, the offset can be determined by the following expression:

Δx = x ₁ -x ₀ and Δy = y ₁ -x ₀

In accordance with this option, the background model M _f is formed in the form of an arithmetic mean of the NOF recorded together with the reference image.

In the process of implementing the debug phase of character models, the image processing subsystem 6:

- provides collection of images of a set of bank notes, where all the numbers and / or letters used in series numbers are printed on a white background, each one by one, and in known positions (set for debugging HOC symbols (EAC);

- then segments the images from the EAC set into elementary images, each of which contains one single character. In accordance with this option, segmentation is provided using standard technology for analyzing the transition from black to white, which is very effective in cases where characters are printed on a white background;

- generates a model M _s for each symbol (letter or number) as the arithmetic average of the set of NOS elementary images of each position registered, for example, together with the symbol of the first ticket from the set of NOS taken as a reference. Registration and averaging are carried out in the same way as in the case of the background model, but the basics of the markup coincide with the whole elementary image of the symbol.

Typically, the series number of a bank ticket uses alphanumeric characters in the same font. Thus, usually one position on tickets from the set of FNL per symbol (one symbol A, one symbol B, etc.) will suffice. In another case, it will be necessary to provide as many provisions per character as the number of different fonts used (for example, A font New York, A font Courrier, A font Geneve, etc.).

In the process of implementing the recognition phase in accordance with the embodiment of the invention described here, the image processing subsystem 6 after reading the image:

- first of all, it records the background image of each bank ticket to be read together with the background model through the same markup framework that was used to debug the model and using the same correlation technology;

- forms a differential image, which is a registered image of a full ticket minus a background model, after which it searches for the positions of the characters. The technology used for this is based on the transition analysis already mentioned above. Usually, a search can be performed on a limited area of the ticket, since the print of the series number can be offset by no more than a few millimeters from the background pattern;

- selects for each position of the symbol marked on the differential image, the corresponding elementary image of the background model. Being registered, the aforementioned elementary image will accurately represent a portion of the background on which an unknown symbol was printed;

- combines for each symbol position the corresponding registered elementary background image M _f with each of the symbol models M _s .

Also, for each position of the symbol, new models will be obtained, symbols plus the background, with the same relative position as on the ticket to be read. In this embodiment of the invention, said combination of M _c is formed, pixel by pixel, using the following equations:

if you print the background first, and then the characters print. Otherwise:

In all cases, the coefficients K _o and K ₁ are constants characterizing the printing inks and paper used. In equations [1] and [2], the first term (namely, the product K _o M _f M _s ) takes into account the transmitting power of the printing inks used and the reflectivity of the paper, while the second term in these equations is related to the reflectivity of the surface of the printing ink, which was applied last;

- for each position of the symbol calculates the correlation coefficient between the corresponding elementary image of the ticket and all new models (symbols plus background). In this case, the symbol to be processed is recognized as a symbol of the combined model corresponding to the maximum value of said correlation coefficient;

- in accordance with this embodiment, the invention further compares the said maximum value of the correlation coefficient with a certain threshold value in order to check the print quality of the symbol and the background of the elementary images corresponding to each symbol position. If the quality is satisfactory (the elementary image to be processed and the combined model are almost identical), the coefficient has a value very close to 1, while poor quality will correspond to a coefficient value closer to zero.

Other preferred embodiments of the invention will include:

a) its use for character recognition on documents other than bank tickets, that is, such as letters, postcards, labels, bank or postal checks, etc .;

b) replacing the conveyor system on a tape with a conveyor system suitable for large sheets, for example, a cylinder of the type used in printing presses or a system in accordance with US Pat. No. 5,598,006 of January 28, 1998;

c) replacement in the recognition system of a linear camera with a matrix camera;

d) using an averaged image of a set of NOF images as a reference image for background recording;

e) automatic allocation of the markup basis for background registration, for example, in accordance with the technology proposed in Patent of the Principality of Monaco No. 2411992479;

f) building a background model using a method other than averaging, for example, using the technology specified in US Patent No. 5778088 of 07/07/1998.

List of references

(1) L. Stringa - "Inspection Automatique de la qualite d’impression par un modele elastique" - Patent for invention No. 2411.99.2479, issued by the State Ministry of the Principality of Monaco (04/27/99).

(2) L. Stringa - "Procedur for Producing A Referance Model etc." - US Patent No. 5,778.088 - July 7, 1998

(3) L. Stringa - "Procede de controle automatique de la qualite d’impression d’une image multichrome" - European Patent Application No. 97810160.8-2304.

(4) L. Stringa - "Installation for Quality Control of Printed Sheets, Especially Security Paper" - US Patent No. 5,598.006 - January 28, 1998

(5) Rice-Nagy-Nartkr - "Optical Character Recognition" - Klumer Academic Publshers - 1999.

Claims (10)

1. The method of automatic recognition by electronic means of characters, such as letters and / or numbers, printed on any material, including structures with strong contrast, even in the case where the background has very contrasting structures, which use an optoelectronic image reader and processing system an image containing the following steps, in which the debugging step is carried out, in which a background model is obtained by reading images of one or more samples in which Only the background is shown, a model of characters is formed (numbers and / or letters, as well as other characters) obtained by reading images of a character set printed on a white background containing at least one instance of each character, after which the recognition step is performed, when which reads the image to be recognized by the sample, containing unknown characters printed on a specific background, records the background model together with the background of the read image, extracts the registered background model from the elementary image expressions of the background corresponding to each unknown symbol, combine for each position of the symbol a model of letters and / or numbers with an elementary image of the corresponding background, create combined models, match unknown symbols with all combined models corresponding to the same symbol position, recognize each unknown symbol as the corresponding symbol, the combined model of which is best superimposed on it in accordance with the “comparison with template" technology. 2. The method according to claim 1, wherein said background model is one of the images of the Background Debugging Kit (NOF). 3. The method according to claim 1, wherein said background model is the average value of the NOF images recorded between each other. 4. The method according to claim 1, wherein said background model is represented by a set of patterns containing either a background or symbols, in accordance with a separation technology, a symbol or background. 5. The method according to claim 4, in which the models of the characters to be recognized are formed as average values of the corresponding images from the Symbol Debugging Kit (NOS). 6. The method according to claim 4, in which the models to be recognized by characters are formed using information computer files. 7. The method according to claim 6, in which a color image reading system is used, the recognition of which is carried out in the color channel that gives the best overlap. 8. The method according to claim 7, in which each image to be processed is represented as separating unknown characters from the background by highlighting a registered background model. 9. The method of claim 8, in which the combination of background models and symbols is carried out in accordance with the following equations:

; Ms≈Mf, if the background is printed first and then the characters are printed, otherwise:

; Ms≈Mf, where Mb is the symbol model b; Mf - background model; Ms - model for each character; By _{about 1} and K - factors which are constant. 10. The method according to claim 9, in which it is used to check print quality by comparing with the threshold value of the correlation coefficient between the elementary image of each of the positions of the characters and the corresponding combined model selected at the recognition level.

Images