RU2707320C1 - Method of recognizing a symbol on a banknote and a coprocessor for a computing system of a banknote processing device - Google Patents

Abstract

FIELD: data processing.

SUBSTANCE: group of inventions relates to a method of recognizing symbols on banknotes and a coprocessor of a computer system of a banknote processing device, which enables to implement said method. In the method of character recognition, scanning a banknote to obtain a digital image consisting of pixels, for which membership in a certain class is determined, based on said class, the location of the symbol to be recognized is determined, character locate predefined shape and size matched with the symbol image, wherein binarization is carried out to extract pixels associated with the symbol. Coprocessor for the banknote processing device computing system comprises an input port for obtaining pixel values corresponding to the character locate, where said value defines either a pixel of a symbol, or a pixel of a space, a decision logic circuit configured to provide a set of numerical characteristics for each verification character locate and an output port for issuing sets of numerical characteristics corresponding to character locate string lines.

EFFECT: technical result consists in broader possibility of using method for recognizing characters of any type of font.

20 cl, 11 dwg

Description

SUBSTANCE: group of inventions relates to a method of recognizing characters on banknotes and to a coprocessor of a computing system of a banknote processing device that allows implementing this method.

A banknote is a valuable document, each copy of which is marked with a unique string of characters, which is usually called a serial number. The serial number, despite its name, can contain not only numbers, but also letters in any combination. In modern technology for processing banknotes, it is necessary to keep track of the movement of individual copies of banknotes along chains of money circulation. This requires equipment that can recognize characters in the serial number for their subsequent use in motion accounting systems.

In addition to accounting for movement, recognition of the serial number allows you to verify the authenticity of banknotes. To this end, the recognized serial number on the banknote is compared with a database of known serial numbers of counterfeit banknotes. In addition, for banknotes in which a string of characters of a serial number is printed on a banknote twice, each of these lines is individually recognized, then the results are compared with each other, and if the characters are different, the banknote is recognized as suspicious. Due to this comparison of two lines of the serial number, it is possible to identify insolvent banknotes made up of parts of various genuine banknotes.

Machine recognition of characters has been known since the late 1960s and to date is a mature field of technology in which, in various combinations, many different recognition methods are used, described both in the scientific and patent literature (IPC groups G06K 9/46, G06K 9/48, G06K 9/50, G06K 9/52, G06K 9/62 and others).

Character recognition is used to obtain text contained in a document as a digital sequence of character codes. As a rule, on a document to be recognized, individual familiarities are first distinguished, and then, separately, characters are recognized in familiarity. Familiarity is understood to mean a section of a document or its image in the form of a parallelogram or rectangle that contains a single character. The position of familiarity may be known in advance, or it may be determined on the basis of characteristic features. For example, if the background is white and the symbol is black, then the borders of familiarity fall entirely on white pixels, and there are black pixels in the interior of the familiarity.

The main area of application of machine recognition of characters is the digitization of print media, as well as documents used in business and administrative circulation. This determined the main goals and technical achievements in improving character recognition.

A font in printing is called a set of characters of a certain size and pattern. The graphic drawing of the characters that make up one font is united by artistic stylistic unity. Fonts can differ from each other in both pattern and character size. The concept of a font must be distinguished from the close concept of a typeface, which is a set of characters that have a stylistic unity of style. Drawing for the same type of symbol, for example, for a certain letter or number, in different fonts can differ in many features, such as: the inclination of the symbol; the ratio of the width and height of the character; line thickness; constancy or change in line thickness; the nature and location of the bends of the lines; the presence of serifs at the ends of the lines, and many others. In printing practice, hundreds of different typefaces with a wide variety of patterns are used.

Recognition documents and prints are usually printed in advance with an unknown font, or in many different fonts. Therefore, in the current state of the art, the main attention is paid to recognition methods, the operability of which does not depend on the pattern and font symbol size. Such methods have high computational complexity, however, they are not subject to stringent performance requirements. As a rule, recognition time within a few seconds per page is considered quite acceptable. In addition, high-performance workstation or server processors with IBM PC architecture are typically used to implement the algorithms used in these methods. The use of high-performance processors compensates for the increased computational cost of character recognition, independent of the font.

Another feature of most modern recognition methods is the high quality digital image of a recognized document. To obtain it, scanners of sufficiently high resolution are used, from 300 dpi and higher. Very high requirements are not imposed on the scanning speed, therefore, the relative speeds of the mutual movement of the document and scanner elements turn out to be small. Due to this, in the digital image there are practically no dynamic geometric distortions of the scan. Further, with the exception of a small number of special types of documents (historical documents, ancient literature), recognized documents have high print quality. As a rule, they are printed in black on a white background, and paper pollution and damage to the paint layer are practically absent. When scanning such documents, a contrasting and complete image of each character is obtained.

As a rule, very high requirements are not imposed on the accuracy of recognition of individual characters on ordinary documents and print media. This is due to the fact that algorithms based on a set of words and grammatical rules of the language in which the document is executed are used to correct errors in the recognition of individual characters. A mistake in one character in a word is corrected based on the correct spelling of the word and the grammatical context of the whole sentence. When errors in the document are completely unacceptable, the result of automatic recognition is additionally checked by a person.

Recognizing serial numbers of banknotes is a completely different task. The pattern and font size of the serial number font is always known. The quality of scanning banknotes in cash processing machines is significantly inferior to the quality of scanning ordinary documents, since the banknote moves in the machine at high speed (up to 10 kilometers per hour), and less than 100 milliseconds are usually allocated to receive a digital image. This leads to a low resolution of scanning, not more than 200 dpi, as well as to large dynamic geometric distortions due to slipping of banknotes in the banknote path.

Further, banknotes are constantly in circulation, which leads to a gradual abrasion of the paint layer, staining and general pollution of the paper. As a result, the contrast and image integrity of the symbol on banknotes can be very noticeably reduced. The situation is also complicated by the fact that the serial number is not always printed on the white section of the banknote. In many cases, it is applied to a previously sealed area, therefore, even on new banknotes, the background of the symbol already contains an additional image that interferes with recognition.

Due to the high speed of banknote movement in the machine, high demands are placed on the speed of recognition of the serial number. Typically, it can take a few milliseconds to recognize a serial number string. This time is approximately comparable with the time of recognition of a word of the same length in modern methods of recognition of ordinary documents executed on high-performance processors. However, the use of high-performance processors in most banknote processing machines is not possible for economic reasons. The same types of processors that are economically viable for use in banknote processing machines have 10 or more times lower performance than the processor performance of a workstation with an IBM PC architecture.

The probability of an error in recognizing the serial number of a banknote should be very low. More precisely, the recognition of the image of one character as another is completely unacceptable. However, the conclusion about the unrecognition of such a symbol is more acceptable. This is due to the peculiarity of processing banknotes: a banknote with an unrecognized symbol can be further rejected for reprocessing, while an incorrectly recognized symbol will lead not to rejection, but to a hidden error in the financial accounting of the movement of banknotes. It should be noted that when recognizing ordinary documents, when there is doubt about the recognition of a particular character, they usually try to find the most likely value for it, and only in extreme cases mark the character as unrecognized. Since the serial number is not a word of any language and is not included in the sentence, there is no way to correct the character recognition error based on the context and grammar of the language.

Thus, the difference between recognizing serial numbers of banknotes and recognition of ordinary documents makes it somewhat easier, but in most aspects complicates the task. The task of recognizing serial numbers is facilitated by the fact that the pattern and size of the font characters are known in advance. However, it is complicated due to the low image quality of the symbol, high requirements for error-freeness and speed of recognition, low processor computing power that is available for implementing the method, and the impossibility of contextual and grammatical error correction. At the current level of technology, the search for the optimal solution to this problem has not yet been completed. To solve it, techniques are used that differ from those used in the recognition of ordinary documents.

The known task of recognizing the registration number of a vehicle from its photograph. This problem arises when taking into account the flow of cars during traffic regulation. Recognition of registration number of the car has a certain similarity with recognition of serial number of a banknote. In both of these tasks, a character string that is not a language word is recognized, and high demands are placed on recognition accuracy.

Also, in both tasks, characters have a known pattern and font character size. The differences between the tasks are performance requirements and the quality of the symbol image. The practical speed of recognition of registration numbers at the current level of technology is one second or several seconds, that is, more than two orders of magnitude worse than required for recognition of serial numbers of banknotes. The image of the vehicle registration number used for recognition has significantly worse quality than the image of the serial number of the banknote. Since photographing a car can be made from different distances and from different angles, the image of the registration number always has scale distortions and perspective distortions that require mandatory correction before recognition. The legibility of the characters is affected by the state of the atmosphere, pollution and damage to the number plate and inaccurate focusing of the camera, as well as uneven external lighting. Because of this, attempts to segment the image of a symbol in a registration number, that is, highlighting pixels related to a symbol, often lead to a complete loss of the shape of the symbol, even when using modern binarization algorithms. As a result, further recognition is not possible. For comparison, segmentation of characters of a serial number of a banknote can lead to the appearance of minor gaps in the outline of the symbol or small noise false clusters of pixels that are not, in fact, related to the symbol. However, these violations do not prevent recognition of a segmented character.

For this reason, in the recognition of registration numbers are used mainly those methods that do not require preliminary segmentation. These include the calculation of two-dimensional correlation with patterns of all the characters in the font, the gradient histogram method, and convolutional neural networks. These methods are well known to those skilled in the art of image recognition. Without going into a detailed technical description, we note that all of them have a high ability to recognize highly noisy symbol images, but differ in very high computational costs. In addition, the problem of preliminary training of the recognition algorithm on the samples arises. To learn the recognition of a particular font character, it is necessary to process many samples of images of this character obtained in slightly different conditions. This leads to a large expenditure of time and labor for the preparation and introduction of samples.

Convolutional neural networks give the best results, among other methods, with relatively small computational costs. However, they require training on a very large number of samples, measured in hundreds of thousands. Correlation methods require the least training, but differ in the largest computational costs. The method of a histogram of gradients gives a fairly high speed and requires an average number of samples for training, but it does not work accurately with contamination or with sudden changes in the registration number lighting. A slight increase in the speed of all three of these methods, in recent times, has been achieved through the use of powerful specialized coprocessors focused on acceleration and parallel execution of typical recognition operations.

When recognizing serial numbers of banknotes, the greatest applicability is found by the gradient histogram method with classification using the support vector machine. This choice is due to the fact that the image quality of the serial number on the banknote is generally higher than the image quality of the vehicle’s registration numbers. A similar method, for example, is used in the banknote serial number recognition method described in WO 2017016240, publ. 02.02.2017, IPC G07D 7/20. After the features are extracted, a classifier based on the support vector machine preliminarily trained by the method of sequential minimal optimization (SMO) is used for classification. Some symbols similar in pattern, such as “B” and “8”, may be detected with an error when using the support vector machine. To more accurately distinguish between these symbols, an additional check is carried out using a correlation comparison of a certain zone of the symbol with the template. The location of the zone is chosen so that it contains the part of the symbol that most distinguishes it from other symbols similar to it.

The gradient histogram method, although it is relatively fast in comparison with correlation methods, still requires a sufficiently high computational power of the processor or the use of a specialized coprocessor. To construct a local histogram of gradients, 8 comparison operations for each pixel of the image are required, to which massive matrix calculations are added. As a result, serial number recognition is difficult to implement in simple equipment designed for mass use. Studying the prior art, the authors of the claimed invention came to the conclusion that for the widespread use of recognition of the symbol of the serial number of banknotes, it is critical to ensure high speed using processors of low processing power, while achieving resistance to image distortion characteristic of banknote processing. This task requires the use of other recognition methods that are less costly in terms of computational complexity.

The prior art methods are known in which an increase in performance is achieved by simplifying the calculation of features, which are then used for classification. One of these methods is described in application CN 108320374, publ. 07.24.2018, IPC G06K 9/62. In it, to characterize a banknote, the so-called perceptual hash (hash of visual perception) is used. A perceptual hash is a binary number, usually 64-bit, that is computed for a particular image. It can be used to compare two images. The more similar the images are, the smaller is the number of binary bits in which the perceptual hashes of these images differ. The number of different binary bits is known as the Hamming distance. In the said application, during training, perceptual hashes are found for all possible types of symbol. For the character to be recognized, a perceptual hash is calculated and the Hamming distances from it to the perceptual hashes of all possible characters obtained during training are determined. This set of distances is used as a feature vector fed to a pre-trained classifier based on a reference vector machine. The speed of execution can be considered an advantage of this recognition, since it uses the fastest known perceptual hashing algorithm, the so-called average hash. The classification is strengthened due to the fact that the classifier makes a decision not only on the degree of similarity of the image with the most likely type of symbol, but also on the degree of difference from other types of symbols. Taking into account the degree of difference from other types of characters improves the classification of characters the stronger, the more differences between different types of characters. The disadvantages of this method are extensions of its advantages. Perceptual hashing, in particular, using the averaging algorithm, emphasizes large image elements and ignores small ones. Therefore, the hashes of similar characters, such as the numbers “8”, “9” and “6”, have very small differences, especially if the font has a thin face and the gaps between the end of the stroke and the circle in the numbers “9” and “6” "Made very narrow. Because of this, it is difficult to distinguish between such characters using a perceptual hash. An example of a font with a thin outline and small gaps is the font of the serial number in Russian rubles issued before 2017. In addition, taking into account the degree of difference with other characters does not improve the classification of similar characters, since the Hamming distance from each of the similar characters to another, very different character is almost the same. Therefore, the applicability of this method is limited to fonts specially optimized for machine recognition.

It should be noted that a common drawback of methods that use the machine of support vectors for classification is the frequent output of the wrong type of character with a recognition error. As already mentioned above, for financial reasons, it is more correct to report the impossibility of recognizing a symbol for a serial number of a banknote than to report the type of symbol that does not correspond to reality. Known US patent US 4,180,800, publ. 12/25/1979, IPC G06K 9/50, which describes a character recognition device based on line-by-line analysis of a digital image of familiarity. It is distinguished by the exceptional simplicity of the formation of features. Familiarity strings are sequentially analyzed, one after another, and the contents of each of them are classified according to a small number of possible variants of codes, otherwise called lowercase characters. So, the code C0 stands for an empty line, C1 stands for a narrow mark in the line on the left, C3 stands for a narrow mark in the right, C6 stands for a wide mark in the center, etc. Then, the stylized number “2”, consisting of segments oriented at right angles to each other, when moving along lines from top to bottom will be encoded as C0 (empty line from above) C6 C6 (upper horizontal bar) C3 C3 C3 C3 (right vertical bar ) С6 С6 (middle horizontal stroke) С1 С1 С1 С1 (left vertical stroke) С6 С6 (lower horizontal stroke) С0 (empty line below).

During string analysis, codes are sequentially fed to the state machine, so that each code causes a transition from one state of the machine to another in accordance with a predefined graph. The operation of the machine starts from the initial state and by the time it passes through all the lines of familiarity, the machine goes into one of the states corresponding to a possible recognized character. Such a device is well adapted to recognize numbers printed in a special font, where each digit has a fixed size and consists of segments oriented to each other at right angles. Such fonts have been used successfully for marking and automatically sorting mailings since the 1970s. The implementation of the device in the form of logic circuits for the string encoder and the state machine provides high performance. Some complication of the transition state graph of the state machine allows the device to successfully recognize characters that have a moderate slope due to incorrect scanner orientation relative to the document.

This device has two main disadvantages. Firstly, it is adapted to only one type of font, which is very different from traditional typographic fonts. Adaptation to another font is possible only with the replacement of the transition graph, and the new font should also consist only of segments oriented at right angles.

Secondly, the device is not resistant to image defects of the symbol. Abrasion of a part of the paint layer or a stain of dirt causes an incorrect sequence of codes, which does not allow the state machine to come to the desired state corresponding to the real symbol.

Due to these shortcomings, the scope of the device is limited to high-quality documents made with a specially adapted font.

US patent 4,180,800 was selected as a prototype of the claimed invention. The technical result consists in expanding the possibility of applying the method to the recognition of characters in a font that has almost any pattern features.

This technical result is achieved due to the fact that in the method for recognizing a character on a banknote relating to a known font,

scanning a banknote to obtain a digital image consisting of pixels for which membership in a particular class is determined, based on the specified class, the location of the character to be recognized is determined,

combine familiarity of a predetermined shape and size with the image of the character to be recognized, so that the pixels related to the character are within the boundaries of familiarity,

while conducting binarization to highlight pixels related to the symbol, moreover,

for familiarity, a set of test straight lines intersecting this familiarity is predefined,

and on each of the test straight lines a one-dimensional coordinate system is set,

so that the position of each test straight line and the origin of the coordinate system on it is set in a predetermined manner relative to the boundaries of familiarity,

then, on each test line, pixels are marked along which this line passes, indicating them in different ways for the pixels of the symbol and for whitespace pixels,

and match each test direct set of numerical characteristics, including

the length of the interval covering all the specified pixels of the character,

the number of intervals of the specified pixels of the symbol, in which the specified pixels of the symbol are arranged in a row one after another,

the coordinate of the center of gravity of the specified pixels of the symbol, expressed in the coordinate system for this test line,

and the total number of all the specified pixels of the character,

then perform the classification of the character to be recognized, using the previously obtained sets of numerical characteristics as features.

On character familiarity, a set of test lines is predefined. At the intersection of these lines with the image, pixels will be selected for the subsequent classification of the symbol. In general, straight lines can be oriented in any way: along image lines, along its columns, or at an arbitrary angle. The use of a set of test lines for classification is based on the feature of a symbol as a set of graphic elements. Symbols of alphabets created by man have only two levels of intensity. One level, usually black, corresponds to the character itself, and another level, usually white, corresponds to a space. The terms “black” and “white” are used here with a certain convention, since banknotes are multi-color printing, and scanning is carried out using certain backlight wavelengths. We call black the level corresponding to optically contrasting paint with a high absorption of backlight radiation, and white the level corresponding to paper or a colorful layer with a low absorption of backlight radiation.

The symbol consists of lines that have a constant or slightly varying thickness. As a result, the intersection of the test line with the lines of the symbol is represented by a set of black segments, where each segment corresponds to a separate line, and the length of the segment is determined by the thickness of the line and the angle of intersection with the test line. The set of check lines allows, therefore, to characterize the image of the symbol not as a two-dimensional array of pixels, but as a set of segments that occur at the intersection of the lines of the symbol with the test lines. Thus, the characterization described here makes it possible to analyze the structure of an unknown symbol, as a set of lines. This feature is important for quick and reliable recognition, since the specific type of symbol to which the recognizable symbol image is related is determined precisely by the number and relative position of its constituent lines.

The prototype uses test lines oriented along the lines of the image. The result of the intersection of the test line with the lines of the symbol is analyzed by rough signs of the presence of a segment and an assessment of its length as short or long. As a result of processing the intersection of the test line with the image of the symbol, a code relating to a small number of possible codes is found in the prototype. The claimed method, in contrast to the prototype, is not limited to the recognition of characters consisting of orthogonal strokes.

In the claimed method, in contrast to the prototype, instead of a code, a set of four numerical characteristics is formed. These characteristics give significantly wider possibilities for describing the features of the shape and arrangement of the lines of the symbol, even if the drawing of the symbol is highly complex. On the other hand, through the permissible deviations of the numerical characteristics, the classification can take into account the influence of various factors that distort the image and interfere with reliable character recognition. Such factors include geometric scanning errors, expressed in the displacement of digital image elements, as well as deviations of the line thickness from the nominal value, arising both from errors in the printing process, and due to wear of banknotes and gradation errors of scanning.

At the initial familiarity, pixels related to the symbol are distinguished, and the remaining familiarity pixels are assigned to whitespace. This operation is known in digital image processing called binarization. There are many binarization algorithms known, of which the Otsu algorithm is most widely used, which ensures high quality separation of the symbol from the background with good speed. The claimed method does not impose restrictions on the binarization algorithm used. As a result, a binarized image is obtained having only two pixel intensity levels, one of which corresponds to a symbol and the other to a space. On each test line, pixels are marked along which this line passes, indicating their type: as pixels of a symbol, or as white pixels. If the line passes along a row or a column of an image, it’s enough to compose a linear array in which each value corresponds to the pixel value of the corresponding row or corresponding column of the binarized image. For an inclined line, it is necessary to compose a linear array, copying into it the pixel values of the binarized image taken along a certain path of their traversal, which is an approximation of the inclined line. Such a bypass path can be compiled using the well-known Bersenham algorithm used in computer graphics to draw lines.

Consider the numerical characteristics in more detail. In this description, we will call the length of the interval on the test line that covers all the pixels of the symbol the width and denote W. The width is measured in pixels and shows the total length of the section where the test line intersects the symbol lines. For different images of the same character belonging to a known font, the width on a certain test line is a stable characteristic, which changes to a small extent due to geometric scanning errors and errors in line thickness.

The number of symbol pixel intervals in which the symbol pixels are arranged one after another will be called the number of strokes and denoted by R. The number of strokes indicates how many lines of the symbol the test line crossed. For different images of the same symbol, the number of strokes on a certain test line is practically unchanged, except when the test line passes near the points of merging or intersection of lines or touches the line. In these special cases, the value of P can vary from the smallest random factors and is not indicative, therefore, it is recommended not to take it into account for analysis. The coordinate of the center of gravity of the pixels of the symbol, expressed in pixels in the coordinate system of the test line, will be called the center and denoted by C. The center shows the location on the test line of the section where this line intersects the symbol lines. The center C on the test line changes slightly with geometric scanning errors, and is almost independent of the errors in the line thickness. However, this is the only one of the four numerical characteristics, which is measured from the origin of the coordinate system of the test line, the position of which is set in some predetermined manner relative to the boundaries of familiarity. Therefore, for stable calculation of C, it is necessary to ensure the position of the origin of the coordinate system with respect to the boundaries of familiarity, taking into account the location of the image of the symbol in familiarity. As will be shown later, for such a positioning, it is best to use the axis of gravity or the center of gravity of the entire familiarity image.

The total number of all pixels of the symbol marked on the test line, we will call the weight and denote WT. For different images of the same symbol, the weight depends little on the geometric errors of the scan, and is almost entirely determined by the errors in the line thickness.

The calculation of all four numerical characteristics in accordance with the description of their meaning is obvious enough for a specialist. Before calculating them, carried out for a certain test line, it is necessary to set P, W, C and WT equal to zero. Moving along the test line from the edge of familiarity in a certain direction, it is necessary to analyze the value of each marked pixel, and each time decide on a possible increase in P, W, WT in accordance with the pixel value. To calculate the center C, it is necessary to calculate the sum of the coordinates for all the pixels on the line related to the symbol and divide it by the weight WT. This corresponds to the well-known method of calculating the center of gravity. The sum of the coordinates for all the pixels on the line related to the symbol is an intermediate digital characteristic, which we will denote by SC.

The calculation of the numerical characteristics of the check string is a quick operation that does not require many comparisons and matrix actions, characteristic, for example, of the well-known gradient histogram method. Moreover, since the calculation of numerical characteristics for different test lines is not interconnected, it can easily be parallelized. Parallelization allows you to speed up the work many times when using a multi-core central processor in the computer system of a banknote processing machine. In General, the claimed method provides a performance not worse than the prototype. Additional work acceleration can be realized by calculating the numerical characteristics in the hardware using a simple coprocessor. Such a coprocessor can be implemented in the structure of a programmable logic chip (FPGA).

The choice of four numerical characteristics P, W, C and WT for encoding the result of the intersection of the symbol image with the test line is justified by the fact that such encoding can be considered as a kind of local frequency analysis. Frequency analysis, for example, Fourier transform, and local analysis, such as wavelet transform, have proven themselves in image processing as reliable methods for detecting the structural features of an image, which is important for character recognition. Coding using four numerical characteristics has significant similarities with the window Fourier transform method, well known in digital signal processing and a combination of frequency and local analysis. We point out here the correspondence between the parameters of the window Fourier transform and numerical characteristics. The width W approximately corresponds to the width of the rectangular window of the window transform. Center C corresponds to the position of the center of the window. The value of the number of strokes P defines the number of periods of the sine component that fit into the window. Namely, the value (2P-1) is approximately equal to the number of half-periods of the sine component within the window width. So, at P = 1, half the period of the sinusoid is laid in the window and forms a single peak corresponding to a single segment on the test line obtained as a result of intersection with a single line of the symbol. At P = 2, one and a half periods fit in the window, which form two peaks corresponding to a pair of segments on the test line and the intersection with two lines of the symbol. At P = 3, two and a half periods fit in the window, which form three peaks corresponding to three segments on the test line and the intersection with three lines of the symbol.

The convenience of using the local frequency conversion is to encode the result of the intersection of the test line with the symbol image in the form of a single, dominant component, which is determined by the width of the window, the position of the center of the window, and the frequency of the periodic function in this window. In terms of numerical characteristics, this corresponds to a set of W, C, and P, respectively. Such an encoding is approximate, since it discards the residual components of the transformation, and leaves only the dominant one. With the inverse transformation of this dominant component to the pixel values on the test line, you can approximately restore the original distribution of black and white pixels. Since the pixels on the test line can take only two possible values, then to map the dominant component represented by a sine wave to the pixel values, you need to apply binarization.

However, just the dominant sine component is not enough to correctly restore the original proportion of black pixels within the width W. To transmit this fraction, you must additionally encode a constant component that raises or lowers the sine wave relative to the zero level. Weight WT nonlinearly sets the level of the DC component. If we take the binarization threshold equal to O, then for the absence of a constant component and P> 1, the width of the black and white segments on the test line turn out to be equal, that is, WT = W / 2. When the DC component changes, the proportion of black pixels will decrease and increase, together with the weight of WT.

The set of numerical characteristics is calculated, as described above, by a single pass through the pixels on a straight line without the use of Fourier transform technologies. Such an encoding is somewhat less accurate than a complete calculation of the dominant component of the Fourier transform, but is performed very quickly. In more detail, the relationship between the window Fourier transform and the set of numerical characteristics will be described later.

As a result of processing the image of an unknown symbol in familiarity, this image is converted into an array, where each check line corresponds to four numerical characteristics P, W, C and WT. High coding efficiency using such a set was shown in experiments conducted by the authors. All lines of familiarity were chosen as test lines. After encoding the image of the real symbol, the image was reversed based on the resulting array of numerical characteristics. To do this, the pixel interval in each row, having center C and width W, was filled with the number P of black segments of the same length WT / P of pixels uniformly spaced with the same W / P pitch. The resulting restored image, except for rare exceptions, retained the stylistic features of the picture and the full recognition of the symbol, although it had certain visual differences from the original. That is, encoding using a set of numerical characteristics P, W, C, and WT, although it loses an insignificant part of the graphic information, generally saves the data necessary for successful character recognition. Numerical characteristics successfully convey features of the symbol pattern, such as line thickness, their slope, change in line thickness along its length, location of intersection and docking lines, and the like.

Unlike the prototype, in the claimed method, the test lines are not limited to a single direction and can be positioned so that their intersection with the lines of the symbol to the greatest extent covers various graphic elements characteristic of the character pattern of a particular font. Due to the fact that the test lines have many intersections with various graphic elements characteristic of the symbol, the amount of useful information about the location of the lines can be increased.

The numerical characteristics found for the test lines are input to the operation of the classifier, which decides what kind of recognizable character belongs. In the prior art there are many classifiers, such as neural networks, statistical classifiers, cascades of weak classifiers, and many others. The construction and training of the classifier is covered in the scientific literature and is well known to specialists. For reliable classification, it is necessary that the signs fully convey the characteristics of the classified objects. An array of numerical characteristics, as indicated above, has a high completeness of representation of the symbol image.

The features of the claimed method are the high efficiency of encoding the shape of the symbol, as well as the ability to use the required number and arbitrary orientation of the test lines to take into account the features of even the most complex symbol patterns. These features make it possible to use the claimed method for recognizing a font having practically unlimited features of a character pattern, and not just for a simple font with orthogonally directed strokes. This expands the possibilities of applying the method to various fonts with almost arbitrary features of the character pattern and provides a technical result.

In a preferred implementation of the method, recognition reliability is further ensured on banknotes having contamination and damage to the paint layer, while maintaining high speed. This result is achieved due to the fact that, during classification,

for each of the possible types of symbol perform a sequential check of a set of criteria for belonging to this type,

where the sets of criteria are predefined, and at least part of the criteria in each set is set based on the numerical characteristics of the test lines,

upon detection of the fulfillment of all criteria in the set corresponding to any possible type of symbol, they interrupt further verification and conclude that the symbol to be recognized belongs to this type,

if, after comparison with the sets of criteria for all possible types of characters, no match is found for any of the possible types of characters, it is concluded that the character is not recognized,

Moreover, for each of the criteria established on the basis of the characteristics of the test lines, they are preliminarily set

requirements of the criterion for the characteristics of the test line, containing a tolerance for the value of at least one numerical characteristic,

and a set of test lines to test the criterion,

moreover, such a criterion is considered fulfilled if the number of test lines in the set that meet the requirements of this criterion is within a predetermined interval.

The decision on what type an unknown symbol belongs to is made on the basis of a sequential check of a set of criteria, where each set of criteria corresponds to a separate type of symbol. The criterion verifies compliance with a certain rule of a set of numerical characteristics found for particular test lines. If it is found that all the criteria in the set are fulfilled for a certain type of symbol, then conclude that the unknown symbol refers to this particular type.

This construction of a set of criteria allows you to individually select criteria depending on the drawing of a certain type of character. Due to this, the speed and quality of recognition is increased. Namely, from the entire set of test lines, only those lines that most characterize the features of the pattern can be selected. This increases speed, since lines that do not have high information content with respect to the characteristics of the drawing of a certain symbol are not checked. Further, in the criteria, it is possible to exclude the use of lines passing near the places of merging and intersection of lines, as well as touching the symbol line and the test line. The characteristics for these lines can be unstable and, if checked, the recognition quality can only deteriorate.

If, in the process of checking the set of criteria for a certain symbol, a mismatch is found for any criterion, then further verification of this set immediately stops, and the verification of the next set corresponding to another type of symbol begins. We will call this approach early termination. Early completion allows you to not perform checks provided for any kind of symbol, as soon as it becomes clear that the unknown symbol does not belong to this type. The early completion, as experience shows, allows many times to reduce the total number of tested criteria, and accordingly increase performance. Verification of the vast majority of criteria is completed almost immediately after its inception. This claimed method differs from the known methods based on convolutional neural networks and on support vector machines, where in order to reach an opinion it is necessary to perform an irreducible large amount of computation.

All of the above features of the claimed method provide it with high performance.

In the criterion, a check can be specified on a set of more than one check line, and, to fulfill the criterion, it may be acceptable to meet the requirements of the criterion not for all lines of this set. This allows the criterion to be fulfilled even in the case of local damage to the image falling on one of the test lines. Such damage may be a stain or local abrasion of the paint layer. For a damaged line, the correspondence condition for a set of numerical characteristics will not be satisfied. However, for the remaining test lines this condition will be fulfilled, which will allow us to consider the criterion, in general, fulfilled. That is, the claimed method provides resistance to local damage to the image. Note that in the prototype local damage to the image and the corresponding incorrect code almost always leads to an incorrect state machine transition and, as a result, to a character recognition error.

The criteria establish requirements for the numerical characteristics of the test line, which provide for a certain tolerance for at least one of the characteristics. Such tolerances allow you to take into account the natural geometric errors of scanning and errors in the thickness of the lines that occur during printing, handling and processing of banknotes. In the image of a symbol of the type to which the criterion belongs, the result of checking the criterion will be positive, despite the geometric errors of scanning and errors of the line thickness. That is, the claimed method allows to the extent necessary to trump up the recognition criteria to avoid false negative results on normal banknotes.

Thus, the claimed invention, while maintaining high speed, provides resistance to defects and distortion of the symbol image, which are characteristic of serial numbers on banknotes.

An additional advantage of the preferred implementation of the claimed method is that in case of an error, as a rule, it issues a conclusion about the unrecognized character. This is due to the fact that the decision-making sequence is based on checking sets of logical criteria, where a conclusion on the recognition of any kind of symbol requires a positive result of each of the criteria in its corresponding set. Therefore, if the sets of criteria are compiled sufficiently complete to confirm the corresponding type of symbol, then the natural result with a recognition error is a negative result of checking each of the sets of criteria, which leads to the final conclusion about the unrecognition of the symbol. The issuance of an incorrect conclusion about the recognition of a symbol as belonging to some kind can take place only with an insufficiently complete set of criteria for this type. The obvious solution to eliminate the possibility of an incorrect conclusion would be to add additional criteria to the set to tighten decision making. In this, the claimed method differs from classifiers based on dividing surfaces, such as a support vector machine. For such classifiers, the ability to control tightened decision making is severely limited.

For alphabetic characters of European alphabets, as well as for most non-European digital systems, the vertical elongation of the characters and mainly the vertical orientation of the lines are characteristic. For these characters, it is most advantageous to place the test lines along the lines of the image of familiarity, since this ensures the intersection of one test line of many lines of the character and, as a result, a higher information content of the numerical characteristics of this line.

Small image defects, such as small spots of pollution and small abrasions of the paint layer, without additional processing lead to erroneous numerical characteristics. To avoid this error, you should, according to certain rules, filter the pixels marked on the test line. The simplest type of correction is to replace short black segments with white and short white segments with black. A short black line usually occurs due to small spots of contamination, and its replacement with white destroys the effect of the spot. Similarly, a short white line appears due to a small abrasion of the colorful standing, which does not damage the line of the symbol to its full width. Replacing this segment with black restores the integrity of the line. The limiting limitation of the length of the segments to be replaced is chosen small enough so that this does not lead to a significant distortion of the undamaged symbol.

Filtration is most conveniently done directly in the process of finding numerical characteristics. As an alternative filtering method, when calculating P, C and WT, you can ignore the short black and short white segments, with the same results as with the replacement of the pixel values described above.

In order to confirm that the symbol belongs to a certain type, the requirements for the characteristics of the test line are set in the form of allowable intervals for each of the characteristics, and, to fulfill the requirements, the values of all characteristics must be in the intervals allowed for them. The width of the allowable intervals should be set in accordance with the geometric errors and errors of the line thickness characteristic of a certain type of banknote and a specific banknote processing device on which scanning is carried out.

Sometimes, two different types of symbols turn out to be very similar, and when checking a set of criteria for compliance with one type, it is necessary to exclude the symbol from another type. For this, the requirements for the characteristics of the test line are set in the form of unacceptable intervals for each of the characteristics, and, in order to fulfill the requirements, the value of at least one characteristic must be outside the interval that is not valid for it.

Checking the numerical characteristics falling within acceptable intervals is a very fast operation, which, for general-purpose processors, requires no more than 8 consecutive comparisons to control each line. There are opportunities to further increase the speed of such a check. These sequential comparisons may be performed in parallel if the processor architecture provides this capability. In processors with the ability to expand a set of instructions, it is possible to create a special verification command that checks all four numerical characteristics simultaneously in one cycle.

If it is possible that the banknote will be significantly skewed during scanning, the image of the symbol may also be skewed, which affects the reliability of its recognition. To combat this phenomenon, it is necessary, even before combining the symbol image with familiarity, to compensate for the skew by rotating the character image by the skew angle in the direction opposite to the skew direction. Image rotation can also be used when the symbol is located on the banknote at a known angle to its side. An alternative to image rotation can be rotation of the familiarity at an angle of skew when combining familiarity with the image of the symbol. This approach, as a rule, is more computationally expensive due to the oblique orientation of the test lines with respect to the image lines.

The position of each test straight line and the origin of the coordinate system on it are set relative to the boundaries of familiarity in a predetermined manner. However, the graphic elements of the symbol can be located relative to the boundaries of familiarity with a significant random error, since combining the image of the symbol with familiarity is a rather crude operation and only ensures that the symbol is inside the familiarity. The specified random error leads to an increased random spread of numerical characteristics and degrades the reliability of recognition. For a detailed reflection of information about the image of the symbol, in some implementations of the method, the test lines have a one-to-one correspondence with the lines of the image of familiarity. The test lines are numbered and use the line number to identify it when checking the criteria.

To eliminate this random scatter, an additional correction is performed for the number of the test line and the position of the origin of its coordinate system based on previously obtained sets of numerical characteristics. Changing the position of the origin of the coordinate system of the test line, in turn, leads to a change in the value of the numerical characteristic of the center C without changing the other three characteristics, since only the numerical characteristic of the center C is measured from the beginning of the coordinate system.

The described correction allows you to link the test lines used in the criteria and the associated numerical characteristics to the actual position of the character within the familiarity. Such a binding, in turn, helps to improve the recognition accuracy of the same symbol in different images, since each criterion is applied to checking exactly that part of the symbol for which it is intended, and the numerical characteristics of the test lines do not depend on the displacement of the symbol image in relation to the boundaries of familiarity.

As experience shows, certain graphic elements of different images of the same symbol have almost the same coordinates in the row and column directions, measured relative to the center of gravity of the image. Therefore, the position of the center of gravity or the axis of gravity of the image inside the familiarity can be successfully used to carry out the binding. In particular, high binding stability is provided by setting the origin of the coordinate system on check lines with respect to the axis of gravity of the symbol image. When the characters are stretched in the column direction and the test lines are directed along the lines of the image, the most preferred correction is in accordance with the position of the axis of gravity of the image within the familiar direction in the column direction, such that the coordinate on the test line as a result of the correction is counted from the point of intersection with the specified axis. Similarly, the correction numbers of the test lines can be carried out in such a way that the distance from the test line with a certain number to the center of gravity of the image, as a result of the correction, would correspond to a predetermined pattern. For example, the test lines, as a result of the correction, can be numbered using the integer value of the distance from the test line to the center of gravity of the familiar image.

For characters of European alphabets, the upper edge of the image, as a rule, is steadily defined by such graphic elements as the end of the line, the horizontal line or the upper point of the arched line. Therefore, high stability of binding in the column direction can be obtained by setting the numbers of test lines in accordance with the distance to the upper side of the surrounding rectangle of the image, which corresponds to its upper edge. This correction of the numbers of the test lines is successfully combined with the correction of the position of the origin of the coordinate system on the test line in accordance with the position of the axis of gravity of the image within the familiarity directed in the column direction.

где суммирование проводится по всем строкам знакоместа. Для уменьшения погрешности вычисления, связанной с округлением значения центра тяжести строки С до целого значения, имеет смысл использовать более точную формулу

Вертикальную координату центра тяжести изображения знакоместа дает формула

где суммирование ведется по всем строкам. В результате коррекции положения начала системы координат для проверочной прямой N, значение характеристики C_N этой прямой необходимо уменьшить на полученное положение Х_М оси тяжести, направленной в столбцевом направлении.In order to carry out the correction in the case of a one-to-one correspondence between the image lines and the test lines, it is necessary to use not the image itself, but the sets of numerical characteristics obtained for each line of this image. This allows you to reduce the amount of calculations, as previously calculated numerical characteristics already carry information that provides a simple calculation of the position of the axes of gravity and the center of gravity, as well as the upper side of the surrounding rectangle of the symbol. We assume that the test lines correspond to the lines and are numbered by the number N from the highest to the lowest, starting from zero. The line number is its vertical coordinate. Horizontal coordinates are counted from the left edge of familiarity. Then, the horizontal position of the axis of gravity of the image directed in the column direction, in accordance with the generally accepted definition of the axis of gravity, can be obtained from the characteristics C and WT as

where the summation is carried out on all lines of familiarity. To reduce the calculation error associated with rounding the value of the center of gravity of the string C to an integer value, it makes sense to use a more accurate formula

The vertical coordinate of the center of gravity of the familiarity image is given by the formula

where the summation is over all lines. As a result of the correction of the position of the origin of the coordinate system for the test line N, the value of the characteristic C _{N of} this line must be reduced by the obtained position X _{M of the} axis of gravity directed in the column direction.

Для коррекции номеров проверочных прямых по положению охватывающего прямоугольника, нужно из номера каждой из проверочных прямых вычесть N_S.If P _N = 0, then the image line N does not contain pixels related to the character. Therefore, the line corresponding to the upper edge of the image, that is, the upper side of the enclosing rectangle, has the smallest number N _S at which

To correct the numbers of the test lines by the position of the enclosing rectangle, subtract N _S from the number of each of the test lines.

Symbols of East Asian syllabic alphabets, such as Korean or Japanese, as a rule, lack a marked line direction. To recognize such characters, it is preferable to orient part of the test lines along the lines of familiarity, and to orient the other part along the columns. This will provide information about the lines of the symbol having different directions.

To increase the accuracy of distinguishing between different types of characters with a similar pattern, criteria that are not based on checking numerical characteristics can be added to the set of criteria. One of such criteria may be checking the similarity of the previously saved reference image with the region in the familiarity, the size and location of which are set in advance. As a measure of similarity, various known image comparison methods may be used. This may be the calculation of the pixel-by-pixel Manhattan distance between the template and the region in the symbol image, correlation matching, one or another type of neural network, and others known to specialists in the field of image recognition. The criterion is satisfied if the value of the similarity index is better than some predetermined threshold value.

Checking the similarities requires significantly more time than checking the numerical characteristics at specified intervals. Since the criterion for checking the similarity is only part of the set of criteria, due to its early completion, it will not always be checked. From the point of view of improving performance, it is desirable to check this criterion after checking all the criteria in the set that are based on numerical characteristics. Then, the time will be spent on checking only with almost complete confidence that the symbol belongs to a certain type.

Similarly, control over the presence of a predetermined structural element in familiarity can be introduced into the set of criteria. For example, a check may be made to determine whether a certain whitespace in a symbol is an island, a peninsula or a strait. These types of checks based on the control of connectivity of areas are widely used in other methods of character recognition. However, since checking connectivity is a rather slow operation, it should also be performed after checking all the criteria in the set that are based on numerical characteristics.

As already mentioned, the calculation of the numerical characteristics of the test line is well suited for hardware implementation in the form of a special coprocessor, which can significantly increase the speed of character recognition while retaining all the advantages of using the claimed method. This is achieved by the fact that in the computing system of the device for processing banknotes a coprocessor is used, containing

input port for receiving pixel values corresponding to familiarity, where the specified value determines either a pixel of a character or a pixel of a space,

a decisive logic circuit configured to provide a set of numerical characteristics for each test direct familiarity, including

the length of the interval that covers all the pixels of the character located on the test line,

the number of pixel intervals of the symbol in which the pixels of the symbol are located on the test line one after another,

the sum of the coordinates of the pixels of the symbol located on the test line, expressed in a one-dimensional coordinate system of the test line,

and the total number of all pixels of the character located on the test line,

and an output port for issuing sets of numerical characteristics corresponding to familiarity strings.

The computer system of the banknote processing device includes a central processor, memory and communication interfaces. The coprocessor connects to the computer system and takes the burden of forming sets of numerical characteristics for the test lines that are set for familiarity. The remaining steps of the claimed method are performed by the central processor of the computing system.

Note that instead of the numerical characteristic of the center C, a sum of SC coordinates of the symbol pixels located on the check line, expressed in a one-dimensional coordinate system of the check line, is formed in the coprocessor. From this sum, the value C is directly obtained by dividing by WT. Division, upon completion of processing the test line, can be performed either in the central processor or in the coprocessor. The ability to remove fission from the coprocessor is due to the fact that high-speed fission schemes are complex in structure, and fission can be faster performed in the central processor due to the higher clock frequency of its core.

The crucial logic circuit, specially designed for the simultaneous formation of all four numerical characteristics, can process one pixel per cycle without additional overhead for data manipulation. This is superior to the performance of a general-purpose central processor, in which, for processing a single pixel, several commands must be executed due to the need for additional internal data transfer between processor elements.

The input port can be performed in any way that provides the desired performance. For example, it can be connected to the transmitting port of interprocessor communication, or to the controller of direct access to memory, or work as a master on the bus with independent generation of the address sequence. If it is necessary to supply pixels located on an oblique straight line, then the corresponding selection of values from the memory can be realized due to the complex modes of address generation with direct access to the memory. Namely, we are talking about two-dimensional generation of addresses or direct access to a linked list of addresses.

The output port can use the same data transfer mechanisms as the input port. Since the amount of transmitted data of the output port is usually less than the amount of input data, it may be reasonable for the output port to read the results by accessing the coprocessor registers from the central processor.

Due to the independence of processing pixels corresponding to individual test lines, the decisive logic circuit can be configured to simultaneously process pixels located on several test lines. This is achieved by multiplying the segment of the decision circuit processing the data of one straight line by several instances. The performance of the most crucial scheme grows in proportion to the number of such instances. The maximum achievable speed, most often, is no longer limited by the speed of the logic circuit, but by the speed of data supply to the input port and the speed of pumping data from the output port.

The logic circuit for processing pixels located on one test line may contain a state machine, in which a sequential input of pixel values located on the test line is provided. The supplied input value determines the transition between the states of the automaton, and the transition graph provides the formation of numerical characteristics of the test line in the internal registers of the automaton. The state machine processes one pixel per cycle.

However, the logic circuit for processing pixels located on the test line can also be implemented as a combinational circuit, since the values of the numerical characteristics are determined solely by the values of the pixels. With this implementation, the circuit processes all the pixels on the test line in one cycle, which gives multiple acceleration compared to the implementation in the form of a finite state machine.

Filtering pixels on the test line can be implemented in a decisive logic circuit in the same way as filtering was previously described in relation to the claimed method. Very often, digital image damage appears in adjacent pixels on adjacent check lines. This feature can be used to more accurately identify defects. To this end, filtering can analyze the ratio of the values of neighboring pixels located on adjacent test lines.

The claimed method is intended primarily for the recognition of serial numbers on banknotes. However, it can be successfully used to recognize other text information made in a pre-known font, for example, indicating the denomination, version of the banknote and the year of its issue. Also, it can be used to process inscriptions in non-monetary documents made in a pre-known font.

In FIG. 1 shows the principle of finding numerical characteristics corresponding to a test line.

FIG. 2 illustrates an array of numerical characteristics encoding character strings and the possibility of the inverse transformation from an array of numerical characteristics to a symbol image.

In FIG. Figure 3 shows the relationship between the windowed Fourier transform and numerical characteristics.

In FIG. Figure 4 shows the transition graph of a finite state machine that implements the calculation of numerical characteristics with filtering of defective pixels. In FIG. 5 shows an image of a symbol with defective pixels.

In FIG. 6 shows a block diagram of a coprocessor implementation for calculating numerical characteristics.

In FIG. 7 shows the correction of the coordinates of the image of the symbol.

In FIG. 8 and FIG. 9 shows a set of criteria and the results of checking criteria for images of various symbols.

In FIG. 10 shows a step-by-step character recognition sequence.

In FIG. 11 shows a step-by-step banknote processing sequence in a banknote processing machine.

An example implementation of the inventive method describes its implementation using a counting and sorting machine, comprising: a tapping mechanism, providing movement of banknotes from the supply pocket into the receiving pockets; a sensor system that allows you to scan a digital image of a banknote as it moves in the banknote mechanism; and embedded computing system. The computing system, in turn, contains a central processor, memory, user interface, as well as peripheral devices and electrical circuits for connecting sensors and drives of the banknote mechanism.

When moving in a banknote mechanism, a banknote is scanned in transmitted red and transmitted infrared radiation, and also on both sides in reflected white radiation. At the same time, low-resolution digital images (25 lines per inch) are obtained. For reflected white radiation, in addition to the above, receive high-resolution digital images (200 lines per inch). When scanning banknotes, the angle and direction of its bias are measured. Next, rotate the low-resolution digital images at the same angle, but in the opposite direction, to compensate for the distortion of the image.

Low resolution digital images are used to determine the currency, denomination and orientation of the banknote, as well as to verify its authenticity. To do this, use the method described in patent RU 2438182, publ. 12/27/2011, IPC G07D 7/00. First, a preliminary classification is performed based on the size of the digital image of the banknote. In this way, a plurality of possible classes are identified that a banknote may relate to. Further, a detailed verification of the banknote belonging to the selected possible classes is performed. Digital images of banknotes are sequentially checked for compliance with individual sets, consisting of many criteria, where each set corresponds to a specific class. In the criteria, it is checked whether the ratio of the average intensities of digital images in the specified zones of the banknote falls into the specified interval of values. The verification of the set of criteria is terminated as soon as the failure to at least one of its constituent criteria is detected. If all the criteria in the set are fulfilled, then the banknote check is stopped and they conclude that it belongs to the class to which this set corresponds. If none of the criteria set failed to fulfill all the criteria, then the banknote is considered unrecognized.

The location of the serial number for each type of banknote is clearly defined by the issuing bank. As soon as the currency, denomination and orientation of the banknote becomes known, we can conclude about the location of the serial number of the banknote in a high-resolution digital image. Upon completion of banknote recognition, a portion containing a serial number image is cut from a high-resolution digital image. Together with cutting, the digital image of the serial number is additionally rotated at a known skew angle in order to compensate for the skew that occurred during the scan.

Then, the digital image of the serial number is divided into rectangular familiarity, where each familiarity corresponds to a separate character. For definiteness, the breakdown of the serial number is described here, the line of which is oriented in the horizontal direction. The location area of each symbol is determined by the projection of the digital image of the serial number on the horizontal axis. In the locations of characters that are darker than the surrounding background, the projection has minimum values, and the maximum values of the projection correspond to the gaps between the symbols. In this way, vertical boundaries are found between the character location areas on the serial number. Further, for each such zone, the projection onto the vertical axis is individually determined, and the vertical borders of the symbol are found from this projection. Based on the found boundaries, familiarity is combined with the image of the symbol so that the symbol is entirely located inside the familiarity. If the line of the serial number is oriented vertically, then the splitting is performed in the same way, but with the mutual replacement of the vertical and horizontal directions for all actions. The separation of lines into characters using projection is widely used in document recognition and is well known to specialists in this field of technology.

Next, binarization of the familiarity image is carried out. The well-known Otsu method is used to find the binarization threshold. Those pixels that relate to characters are assigned a value of 0, and whitespace pixels are assigned a value of 255. An example of the familiarity image 1 resulting from this is shown in FIG. 1. After binarization, an array of numerical characteristics is obtained for the test lines, which in the described implementation coincide with the image lines. There is a one-to-one correspondence between the lines of the familiarity image and the test lines: each line of the familiarity image corresponds to a check line, and each test line corresponds to a familiarity image line. Therefore, in the further description we will not make a significant difference between the lines of the familiarity image and the test lines.

Let us show the receipt of numerical characteristics by the example of one test line. Line 2, which runs along the image line, intersects the symbol in such a way that the interval W, covering all the pixels of the symbol on this line, is 16. Moreover, the number P of pixel intervals in which the pixels of the symbol are located one after another is 2. The total number WT of the pixels of the symbol on the line is 9. The center of gravity C of the pixels of the symbol on the line is 7, and the coordinates of the center of gravity are counted from the left edge of the familiarity.

An array of obtained numerical characteristics for each row of image 1 is shown in FIG. 2. The column of values of N indicates the number of each line, counted from the top line of familiarity. The same figure shows the image of familiarity 4, restored on the basis of an array of numerical characteristics. To do this, the pixel interval in each row, having center C and width W, was filled with the number P of black segments of the same length WT / P of pixels uniformly spaced with the same W / P pitch. Due to rounding to an integer value, the length of the segments in the restored row may differ by one pixel. You can see that the symbol retained its shape and full recognition, despite slight differences from the original digital image 1. This suggests that the encoding of the symbol image using the four numerical characteristics P, W, C and WT has good completeness, and information loss with encoding is not essential for the purpose of character recognition.

Note that for a single segment of the pixels of the symbol on the line, that is, when P = 1, the values of W and WT always coincide.

представляет собой синусоиду, смещенную по вертикали на постоянную составляющую А и ограниченную, в нашем примере, количеством периодов K=2,5. Для чистой синусоиды 7 значение А=0. При А=А₁>0 график 8 базисной функции проходит выше, а при А=А₂<0 график 9 проходит ниже синусоиды 7.As already mentioned, coding a string using the numerical characteristics of P, W, C, and WT is very close to coding a string using the window Fourier transform. The relationship between these encoding methods is shown in FIG. 3 by the example of reducing the basis function of the window Fourier transform to a set of numerical characteristics P, W, C, and WT. A rectangular window 5 with a width of T = 20 pixels defines the window conversion function. For simplicity of explanation, we assume that the left edge of the window corresponds to the coordinate 0 on the test line. Outside the window, the basis function has a zero level, indicated by the position of line 6. Inside the window, the basis function

represents a sinusoid, shifted vertically by a constant component A and limited, in our example, by the number of periods K = 2.5. For a pure sinusoid 7, the value A = 0. For A = A ₁ > 0, graph 8 of the basis function runs higher, and for A = A ₂ <0, graph 9 passes below sinusoid 7.

Между постоянной составляющей А и весом WT существует нелинейная, но монотонная зависимость: при A=1 WT=0, при А=0 WT=Т/2 и при А=-1 WT=T. Положение середины окна соответствует центру С каждой строки пикселей.We quantize and binarize the basis function in such a way that when it is negative, the pixels get black, and for values greater than or equal to zero, the pixels are assigned white. The pixel size is assumed to be 1. When binarizing functions 8, 7, 9, rows of 10, 11, 12 pixels are obtained, respectively. The figure shows the values of the characteristics P, W, C, and WT for each of the obtained rows. As can be seen from these values, the numerical characteristics are determined by the parameters of the basis function. The value of the width W is approximately equal to the size of the window T. The number of strokes is related to the number of periods inside the window:

There is a non-linear but monotonic relationship between the constant component A and the weight WT: for A = 1 WT = 0, for A = 0 WT = T / 2, and for A = -1 WT = T. The middle position of the window corresponds to the center C of each row of pixels.

Thus, with a certain degree of approximation, we can say that encoding a string through numerical characteristics is equivalent to encoding using the most significant basis function of the window Fourier transform. That is, coding with P, W, C, and WT is, in fact, a variant of the local frequency analysis, in which the dominant spatial frequency, the interval of its localization, and the level of the constant component are determined. Unlike a real window Fourier transform, the calculation of numerical characteristics is very fast, which gives a significant speed advantage for the entire method of character recognition.

The calculation of the numerical characteristics for each line of the digital image is performed using a state machine, the transition graph of which is shown in FIG. 4. In the given example, the state machine is implemented in software, however, as will be shown later, it can also be implemented as a hardware coprocessor.

The state machine has an initial state S0, and four operating states S1-S4. After switching on, the machine immediately goes into state S1 (Space). The machine analyzes the pixel values, one after another, sequentially selected from the line of the digital image. The transition from one working state to another is carried out when each pixel is selected, depending on its value.

The automaton has internal variables for the width W, the number of strokes P, the weight WT, and the sum of the coordinates of the black pixels SC. In the course of processing the string, these variables accumulate the corresponding values of numerical characteristics. From the value of SC, the center C is calculated by the formula C = SC / WT at the very end of the line processing. The initial values of these variables are zero. Also, the machine has internal variables for filtering defective pixels. The FC variable is the pixel counter of the filtered interval. It has an initial value of 2 and is equal to the maximum permissible length of the filtered interval minus 1. The variable FA accumulates the sum of the coordinates of the pixels included in the filtered interval. It has a zero initial value.

From the moment the line analysis begins, the arrival of each pixel increases by 1 the value of the internal variable PC, which stores the coordinate of the next pixel. The variable PC has an initial value of 0. The same is true for the intermediate variable WC, which serves to calculate the numerical characteristic of the width.

At the beginning of line analysis, the machine is in state S1 (Space). This state corresponds to the steady state of a detected space in the line. A space may be several pixels long. If a white pixel is selected in this state (transition T1), then it is considered that the space length has increased by one pixel, and the automaton remains in state S1.

If a black pixel is selected (transition T5), then the machine goes into state S3 (beginning of the segment). This state corresponds to the filtering process, in which it is not yet clear whether the black pixel is the beginning of the black segment at the intersection with the symbol line, or the noise pixel due to a black image defect. When entering S3, the FC variable is reset to the initial value, and in FA the coordinate of the current pixel. If the next selected pixel turns out to be white, then it is considered that the black image defect has ended, and upon transition T4, the machine returns to state S1. If the next pixel is black, and the limit value of the filtered interval has not yet been reached, then the automaton remains in state S3 (transition T8). For each transition T8, the counter FC is increased by one, and the coordinate of the incoming pixel is added to the storage variable FA. If the limit value is reached, then upon the arrival of the black pixel (transition T9), it is assumed that there was no image defect, and the machine goes into state S4 (Segment).

At the moment of transition T9, the variable P increases by 1, which corresponds to the discovery of a new black segment. If this is the first detected black line (that is, P = 1), then the value of the counter FC is recorded in the intermediate counter width WC. Additionally, the FC value for this transition is always added to WT. The value of the storage variable FA and the coordinate of the current pixel are added to SC. All these actions allow us to correctly take into account in the numerical characteristics those pixels that were analyzed in the S3 state and turned out to belong to the black segment. The transition of T4 to state S1 ignores the accumulated values of FC and FA, since the filtering interval traversed in state S3 refers to a space.

In state S4 (Segment), the arrival of each black pixel does not change state (transition T10), since the black segment continues. According to this transition, the WT variable is increased by 1, the SC variable is increased by the coordinate value of the current pixel, and the value of the temporary counter of the WC width, written down by 1, is written into the W variable. When the white pixel appears, the machine transits T6 to state S2 (Start space) .

In state S2, filtering is performed with the detection of white noise pixels. Upon entering S3, the FC variable is reset to the initial value, and the coordinate of the current pixel is entered in FA. When a white pixel arrives, if the limit value of the filtering interval has not yet been reached (transition T3), then the automaton remains in state S2. For each transition T3, the counter FC is increased by one, and the coordinate value of this pixel is added to the storage variable FA. If a white pixel arrives and the limit value is reached (transition T2), then the machine goes into state S1 (Space). This confirms the completion of the black segment without white defective pixels. If a black pixel arrives, this indicates that the previous white pixel was defective, therefore, the machine returns (transition T7) to state S4.

At the time of T7 transition, the FC value is always added to WT. The value of the storage variable FA and the coordinate of the current pixel are added to SC. All these actions allow us to correctly take into account in the numerical characteristics those pixels that were analyzed in state S2 and turned out to belong to the black segment. The transition of T2 to state S1 (Space) ignores the accumulated values of FC and FA, since the filtering interval traveled in state S2 refers to a space.

When the last pixel in the line was applied to the input of the machine, the line analysis is stopped, and the accumulated values of P, W, C = SC / WT and WT are stored in an array of numerical characteristics. At P = 0, the values of the remaining numerical characteristics W, C, and WT do not make sense, and they are assigned zero values.

Let us show by example an analysis of the line with the number N = 11 in the familiarity image 13, as shown in FIG. 5. Line N = 11 has defects in pixels with numbers p7 and p13. The filtering interval limit is set to 2, which corresponds to only one valid defective pixel.

The analysis begins in state S1 (Space) with a white pixel p0. The arrival of the next black pixel p1 puts the automaton at the transition of T5 to state S3 (Start of the segment). Cumulative variables get the values FC = 2 (initial) and FA = 1 (coordinate of this pixel). The next pixel p2 is also black, while the maximum length of the filtering interval 2 has already been reached. There is a transition of T9 to state S4 (Line). The value of P increases by 1, that is, P = 1. According to this transition, since it happened the first time, the FC value is written to the intermediate WC width counter: WC = 2. Also, FC is added to WT, so, WT = 2. The SC value is obtained as the sum of the coordinates of the current pixel 3 and FA, that is, SC = 3. When subsequent black pixels p3 and p4 are supplied, state S4 is maintained, and transition T8 occurs. As a result, we get WC = 4, WT = 4 and SC = T0.

The white pixel p5 starts the transition of the automaton T6 to state S2 (the beginning of a space), and the value of the temporary counter of width WC is written into the variable of width W: W = WC-1 = 4. The filtering interval starts again, therefore, set FC = 2 (initial) and FA = 5 (the coordinate of this pixel). The next pixel p6 is also white, and the maximum length of the filtering interval is fixed with the transition of T2 to state S1 (Space). However, the next pixel p7 turns out to be black, and the automaton (T5) goes into state S3 (Beginning of a segment). The further pixel p8 is white, therefore, the T4 transition automaton again returns to state S1 (Space). Thus, the filtering of the defective pixel p7 was checked. Further white pixels p9 and p10 do not change the state of the automaton and are processed by the transition T1.

Pixel p11 turns out to be black, which puts the machine in state S3 (beginning of the segment). Set FC = 2 (initial) and FA = 11 (coordinate of this pixel). On the black pixel p12, the completion of the filter interval is recorded. The transition of T9 to state S4 occurs, and P = 1 + 1 = 2, WT = 4 + 2 = 6, SC = 10 + 11 + 12 = 33. The black pixel p13 does not change state at the T10 transition, and changes the following variables: WT = 6 + 1 = 7, SC = 33 + 13 = 46.

The defective pixel p14 is white, it puts the machine in state S2 (Start space) with the setting FC = 2 (initial) and FA = 14 (the coordinate of this pixel). The next black pixel p15 causes T7 to transition to state S4 (Segment), according to which the following actions occur: WT = 7 + FC = 2, SC = 46 + FA + 15 = 75. Thus, a defective white pixel is filtered.

The white pixel p16 starts the transition of the automaton T6 to state S2 (Start of space), and the value of the temporary counter for the width of the WC is written into the variable of width W: W = WC-1 = 15. Pixel p17, after exceeding the filtering interval, puts the machine in state S1 (Space). The next white pixel p18 performs transition T1 without changing the state of the machine. On this, the analysis of the line is stopped, and the numerical characteristics of the line N = 1 accumulated by this moment are fixed: P = 2, W = 15, WT = 9 and C = SC / WT = 75/9 = 8. Comparing this with the image of familiarity 1 without defects in FIG. 2, it can be verified that the presence of defective pixels did not affect the obtained numerical characteristics of this line.

An alternative implementation of calculating the numerical characteristics of a string involves the use of a hardware coprocessor connected to the central processor of a computing system. The block diagram of the coprocessor is shown in FIG. 6. The coprocessor implements binarization of pixels and the state machine described above for familiarity strings up to 32 pixels long. It contains a status number register 25 and a microprogram memory 26 associated with it at the input and output. Registers 27-30 are provided for storing the accumulated values of the numerical characteristics of W, WT, SC and P. Their contents can be read by the central processor through the output port in the form of a bus driver 31 connected to a 32-bit data bus of the central processor. A temporary WC width counter is implemented in register 38. The pixel counter of the filtered interval FC is located in register 37, and the accumulator of the sum of the coordinates of the filtered interval FA is represented in register 44. The pixel counter of the PC line is in register 35.

The input port 32 is designed to connect to the data bus of the central processor, where a WR pulse is generated when writing to a specific memory address. The Reset signal is generated by the central processor before processing the next line to reset all elements of the coprocessor to the initial values. The Pixel bus is designed to transmit a byte pixel value to it. For pixel transmission, direct memory access is used, in which byte pixel values are sequentially read from the computer system memory and written to the address of input port 32 of the coprocessor, with the generation of a WR pulse.

All elements of the coprocessor, except bus driver 31, are made with synchronous communication, where the WR signal is used as a clock signal. When writing the value of the next pixel to port 32, this value is compared by the comparator 33 with the binarization level of the VT stored in register 34. The binarization level of the VT is calculated by the histogram of the serial number image using the Otsu method, and loaded into the register 34. The output line of the comparator 33 represents is the binarized value of the character (0 or 1). It is connected to the input of the firmware ROM 26.

Before starting work, the central processor also loads into the register 43 the value of the length limit of the filtering segment FL. The FL value is compared by the comparator 42 with the FC value stored in register 37 to determine whether it is necessary to switch from the state S3 (Beginning of a segment) to S4 (Segment), as well as from the state S2 (Beginning of a space) to S1 (Space). The output of the comparator 42 is connected to the input of the firmware ROM 26.

The ROM 26 and the status number register 25 form a coprocessor firmware control device. According to the transition graph of the automaton shown in FIG. 4, transitions between states are determined by the state number, the value of the input pixel, and whether the maximum length of the filtered segment has been reached. Accordingly, the firmware ROM 26 generates reset, write and increment signals for the coprocessor registers, in accordance with the status number in the register 25, the level of the input pixel and with the signal to reach the maximum length of the filtered segment, which are fed to it at the input. The next state number is supplied from the output of the ROM 26 to the input of the status number register 25. The state machine transitions by the difference of the WR signal, which clocks overwriting the status number register 25. The registers and other elements of the coprocessor circuit are made and connected so that each of the possible transitions of the machine performed synchronously in one clock cycle of the WR signal.

The value of the number of strokes P in the register 30 is incremented in accordance with the described logic of the automaton according to the signal from the ROM 26. The FC value of the counter 37 pixels of the filtered interval is used to write the WC width to the counter 38 and to add the weight WT to the register 28 using the adder 39. From counter 38 of the width WC, the value is written into the register 27 of the width W with a hardware-implemented subtraction of one.

The counter 35 of the pixel number PC, the drive 44 of the sum of the coordinates of the filtered interval FA and the register 29 of the sum of the coordinates of SC are interconnected through adders 36, 40 and 41 to implement the accumulation and transmission of the sum of coordinates in accordance with the description of the operation of the state machine described above. The adder 36 implements the addition of the coordinates of the current pixel to the accumulator 44 of the sum of the coordinates of the filtered interval FA. Adders 40 and 41 form a three-input summation circuit and add the coordinates of the current pixel PC and the sum of the coordinates of the filtered interval FA to the sum of the coordinates SC.

After the completion of the transfer of all the pixels in the string to port 32, registers 27-30 contain ready-made values for the numerical characteristics W (5 bits), WT (5 bits), SC (9 bits) and P (3 bits). The bit depth of the registers corresponds to the maximum string length specified earlier 32 pixels. All four characteristics obtained are packaged by the bus driver of the output port 31 into one 32-bit word. The central processor accesses the specified address of the output port 31 to generate a pulse of the RD signal and, thus, reads its contents. After reading, if the value of P is not equal to 0, the central processor independently performs the division of SC by WT to obtain a numerical characteristic of the center of line C.

Processing a single pixel in the described coprocessor circuit takes one transmission cycle on the channel of direct access to memory. Note that the coprocessor performs pixel binarization in the same clock cycle. The same pixel processing in the software implementation of the machine using the central processor requires several tens of clock cycles of the processor core frequency. For modern processors of digital signal processing, the core frequency is 5-10 times higher than the clock frequency of direct memory access. Despite this, the coprocessor, nevertheless, gives a performance gain several times in comparison with the software implementation.

The co-processor is most conveniently implemented as a separate logical unit in the structure of a programmable logic chip (FPGA), which is already available in the computing system of most counting and sorting machines. A particular advantage can be achieved when the central processor and FPGA are combined into a single chip in a computing system. Such microcircuits can provide a higher frequency of data transfer to the input port of the coprocessor than when placing the FPGA as a separate microcircuit.

To increase coprocessor performance, it is possible to supply input data not through the byte 8-bit bus, as shown in FIG. 6, and through a bus with a larger capacity, for example, 32-bit. This will give an acceleration of the coprocessor about four times. There are two options for processing 32-bit words. According to the first of them, they organize the division of a 32-bit word into four bytes, corresponding to four consecutive pixels of the line. To process them, a clock generator is introduced, which generates 4 clock pulses for each 32-bit word record, and these clock pulses are used to synchronize the machine instead of the WR signal. Also, an accelerated shift scheme of a 32-bit word is introduced, which provides a shift of its contents by 8 bits and the output of the next byte to the comparator 33 with each clock pulse. The main limitation for this option may be too high clock speed, at which it will be difficult to implement a coprocessor in FPGA.

In the second embodiment, a 32-bit word is used, the individual bytes of which correspond to the pixels of four lines. These quadruples of pixels are processed in parallel using four identical and independently working machines, similar to those shown in FIG. 6. In this case, the clock frequency of the coprocessor will correspond to the frequency of the direct memory access cycles. The difficulty in implementing this option is due to the fact that when storing images in memory, each word usually contains 4 consecutive pixels in a row. For simultaneous submission to 4 state machines, it is necessary to pack the pixels of four different lines into one 32-bit word. This can be accomplished, for example, using a direct memory access controller having a two-dimensional addressing mode of data and a mode of packing bytes into words. Or, this packaging can be performed in the input port of the coprocessor using a special buffer with a capacity of four lines. Such a buffer should ensure the recording of four consecutive pixels of one line in each word, and sequential reading in the form of four pixels of each of the lines in one word. There should be two similar buffers: while the pixels of four lines from the first buffer are output to the state machines, the pixels of the next four lines are written to the second buffer. When all the pixels from the first buffer are issued, and the second buffer is filled with new lines, the buffers are interchanged by reconnecting their inputs and their outputs, and continue to work.

To reduce the load on the central processor, the output port of the coprocessor can be connected to a direct memory access channel for uploading the results of processing a string from the bus driver 31 to the system memory. Unloading the results should reset the coprocessor registers to the initial values. In addition, a line end detection scheme must be introduced into the coprocessor by the contents of the counter of 35 pixels PC, which, after receiving the last pixel of the line, will set up a direct memory access request for uploading the results. Data can be fed into the input port continuously, line by line. Such a change in the coprocessor will allow the entire image of familiarity to be processed without the participation of the central processor in switching from one line to another. The numerical characteristics of the image will automatically accumulate in the array in memory.

In the described implementation, the binding of test lines to the boundaries of familiarity is carried out taking into account the content of the image in familiarity. This allows you to precisely link the test lines to the graphic elements of the symbol image in order to compensate for random displacements of familiarity with respect to the symbol when they are combined. In addition, taking into account the content of the image is useful for eliminating errors in the quantization of the position of the graphic elements of the symbol that occur during scanning and when the image is rotated.

In the described implementation, the binding in the horizontal direction is performed to the axis of gravity of the image, which is perpendicular to the lines. The intersection of the axis of gravity with the line is considered the starting point of the one-dimensional coordinate system of this line. The binding in the column direction is made to the upper edge of the enclosing rectangle of the symbol, that is, to the very top line, which contains pixels related to the symbol. The numbering of the test lines that correspond to the lines of the image is corrected so that the numbering starts and goes down from this line.

которая направлена в столбцевом направлении, и строку 15 с номером N_S, соответствующую верхней кромке изображения. Значение N_S находят как самый малый номер строки, при котором Р≠0.The analysis of the image content to ensure the binding can be performed as a separate operation, even before finding the numerical characteristics. However, a significantly different computationally expensive path is the other one chosen in the described implementation and illustrated in FIG. 7. A set of numerical characteristics, including an intermediate value SC, is calculated for each line of the familiarity image, and then these sets of numerical characteristics are used to perform the binding. Find the horizontal coordinate of the axis of gravity 14 images

which is directed in the column direction, and row 15 with the number N _S corresponding to the top edge of the image. The value of N _{S is} found as the smallest line number at which P ≠ 0.

When using a hardware coprocessor, the sums ΣSC _N and ΣWT _N can be calculated inside it along with the calculation of the numerical characteristics of the strings. To calculate each such amount, you will additionally need one accumulative register and one adder. The registers must be reset before the familiarity processing begins, and not before the string processing begins, as is done for other coprocessor registers. Summation should be done every time after the processing of the string.

Next, the line number of each line is corrected: N _CORR = NN _S. Due to this, the line with the zero corrected number is attached to the upper edge 15 of the surrounding rectangle of the symbol image in familiarity. The new, corrected line number will then be used to identify the test line. Lines with negative corrected numbers are empty and will not be considered further.

For each row, a new, adjusted value of the center of gravity of this row is calculated C _CORR = _{CX M.} Thus, the coordinate system on each test line turns out to be tied to the gravity axis 14 of the image directed in the vertical direction.

When combining familiarity with the image of a symbol, it is only necessary to ensure that all pixels related to the symbol are located inside the familiarity. Therefore, the graphic elements of the symbol can be positioned relative to the boundaries of familiarity with a certain random error, the scope of which is approximately equal to the difference in the size of familiarity and the size of the symbol. After binding with correction of the line number and the origin on the line itself, the sets of numerical characteristics for the lines making up the image of the character become independent of this random error. Regardless of how the location of the symbol image relative to the boundaries of familiarity turned out to be, sets of numerical characteristics that correspond to test lines in the described implementation will be approximately the same for the same type of symbol. Due to this, the described binding method significantly improves the accuracy of character recognition.

Verification that the symbol image in familiarity refers to a certain type of symbol is performed using a set of criteria. Each possible type of character has its own set of criteria. In FIG. 8 shows the symbol "9" and a set of six criteria 16-21 that correspond to this view. Each criterion is described by a set of admissible intervals for the numerical characteristics of P, W, C, and WT. Hereinafter, we consider the already adjusted values of the center of gravity of line C and line number N. The number of strokes P is the most important for verification, therefore, the interval of valid values for checking it consists of one number.

For each criterion, a set of lines is given on which it is necessary to check the correspondence of the numerical characteristics to the acceptable intervals. So, in criterion 16 lines with numbers 0-3 are checked. For each line, check that P = 1, that W lies in the range from 6 to 11, that C lies in the range from -1 to 0, and that WT lies in the range from 6 to 11. If at least one of the numerical characteristics If it does not fall into the allowable interval, then it is considered that the criterion for this line is not fulfilled. The entire set of rows that is specified in the criterion must have at least MIN rows for which the criterion is met. So, for criterion 16 there should be at least two such lines. In reality, there are three of them: these are lines 1, 2, and 3. In order to reduce the time spent on checking the criterion, as soon as the fulfillment of the criterion for MIN rows is detected, the test of the criterion is completed. For example, when checking criterion 16, lines 0, 1, and 2 are checked. Since two of these lines, with numbers 1 and 2, already meet the criterion, they detect that MIN = 2 is reached, and complete the test of the criterion with a success message without checking line 3.

In the rectangle, which depicts the criterion, indicate the successful verification of the criterion with OK and its failure with NOK. As seen in FIG. 8, all the criteria in the set are marked OK, that is, they were successful. If in the set of criteria the verification of all criteria was successful, then they conclude that the entire set has been successfully checked and that the symbol image in the familiarity belongs to the kind that the criterion corresponds to.

The criteria in the set are checked in a specific order. As soon as it is discovered that the next test criterion failed, the check of this set of criteria is immediately completed and a conclusion is made about the unsuccessful check of the entire set. Failure to check the set of criteria corresponding to a certain type of symbol indicates that the image of the symbol in familiarity does not apply to this type.

In FIG. 9 shows a check of the same set of criteria 16-21 corresponding to the symbol “9” for the image 22 of the symbol “1”. As can be seen from the values of the numerical characteristics of the rows and the admissible ranges of the criteria, only criterion 21 is successfully fulfilled, and criteria 16-20 are not fulfilled. Since the criteria are checked in the order of 16-17-18-19-20-21, already after the unsuccessful result of the test of criterion 16, the verification of the set of criteria ceases with the conclusion that the test failed. That is, there is an early completion of the verification of a set of criteria. If it is simplified to assume that the verification of the criteria takes approximately the same time, then the early completion occurred after approximately 17% of the time required to fully verify all the criteria in the set.

In order to recognize the image of a symbol in familiarity, that is, to find the kind of symbol to which this image belongs, a series of tests of criteria that correspond to all possible types of symbols are carried out. For example, if in a certain position of the serial number of the banknote the familiarity can contain only a digit, then ten sets of criteria are checked, corresponding to the digits from “0” to “9”. As soon as a successful check of any set of criteria is completed, recognition of the character in familiarity is completed and the conclusion is made that this character belongs to the type for which the test of the set of criteria was successful. If, after checking the sets of criteria for all possible types of characters, none of these checks were successful, then they conclude that the character is not recognized.

The order of checking the criteria in the set is chosen in such a way that the criteria that most distinguish the symbol of a given species from other species are first checked. This allows you to complete the verification of a set of criteria for a certain type as soon as possible if the symbol in the familiarity image does not apply to this type. Such an early completion significantly accelerates the classification of images in familiarity, since the sets of most criteria for different types are not fully verified, but only for a small part.

The criteria included in each of the sets can be compiled by the human operator manually, or using machine learning, based on the analysis of many images of the same type of symbol. For drawing up the criteria manually, it is convenient to graphically represent the symbol on a computer screen, where, as in FIG. 8, line numbers and numerical characteristics of each line are displayed. The operator needs to select line spacing characteristic of a certain type of character, which shows the difference between the graphic elements of this type of character from other possible types of characters. For each of these intervals, you need to create a separate criterion that checks the corresponding graphic element. For example, criteria 16, 18, and 21 in FIG. 8 check horizontal strokes, criteria 17 and 20 check vertical strokes, and criterion 19 controls the opening of the line. As experience shows, in most cases, from 5 to 10 intervals and the corresponding criteria are sufficient.

Selecting MIN allows you to make the criterion, to some extent, insensitive to distortion of image lines. For example, the requirements of criterion 16 in FIG. 8 for the character "9" is usually performed for three lines 1-3, and the MIN value in the criteria is set to two. Such a slightly reduced value of MIN allows you to successfully recognize the image of a character in which one of lines 1-3 is damaged due to a spot, a worn paint layer or a scanning defect.

Thus, the claimed method provides two mechanisms for ensuring sustainable recognition in the event of damage to a digital image. Filtering out segments of defective pixels in obtaining the numerical characteristics of the line allows you to reduce the negative impact of small defects that do not lead to line breaks. The appropriate choice of the MIN value allows to reduce the negative impact of more significant defects, leading to the rupture of one line or the merger of several lines.

The use of acceptable intervals in criteria 16-21 allows you to take into account the instability of the numerical characteristics obtained on different images of the same symbol. The thickness of the symbol line can change both on a real banknote due to the instability of the printing process or wear of the banknote in circulation, and on the scanned image due to a change in the focus of the scanning sensor and due to an error in the binarization process. The thickness of the symbol line affects the width W to a small extent, and greatly affects the weight of WT. Geometric errors of scanning, that is, errors in the mutual arrangement of graphic elements, arise during scanning due to rounding of the coordinate to an integer number of pixels and due to slipping of a banknote in the path, as well as due to rounding of the pixel position when performing image rotation to compensate for skew. Geometric errors mainly affect the width W, and to a lesser extent, the center C. They have almost no effect on WT. Allowable intervals in the criteria should be selected so that the natural deviations of the line thickness of the characters and the geometric errors of the scan do not lead to an unsuccessful verification of the criterion. Then, the use of valid intervals provides stable recognition, despite the natural geometric errors of scanning and deviations of the line thickness, characteristic for a particular class of banknotes and for the equipment used for processing banknotes.

To further distinguish between similar characters, criteria with a logical inversion of the result can be used. That is, if the number of MIN or more lines is found for which the numerical characteristics fall in the specified intervals, then the criterion is considered to be not fulfilled. And, on the contrary, if there are fewer such lines than MIN, then the criterion is considered to be successfully fulfilled. The criterion with the inverse of the result allows you to check the absence of a specific graphic element in the symbol.

Criteria based on the numerical characteristics of strings, similar to criteria 16-21, are checked very quickly. To check such a criterion for one row, in the described implementation, you need to perform 7 independent comparisons and combine their result with a logical operation. When using a CPU with a very long instruction word (VLIW) or a superscalar CPU, some of these comparisons can be performed in parallel. If it is possible to add a special command to the central processor instructions, then the criterion can be checked immediately by a single special command. To do this, the intervals of the criterion, as well as numerical characteristics, need to be packed into the words of the processor. For the most practical horizontal size of familiarity, not exceeding 32 pixels, it is possible to allocate for storing the numeric characteristics of a string 3 bits per number of strokes (P), 5 bits per width and weight (W and WT) and 4 bits per center (C), which a total of 17 bits. Then, to encode the limits, you need 3 bits for the number of strokes (P), 10 bits for width and weight intervals (W and WT) and 8 bits for the center interval (C), which in total fits into 31 bits. Thus, a special criterion check command can use a 32-bit word for the numeric characteristics of the string and a 32-bit word for the limits of the criterion. Due to this, such a command in a 32-bit processor with modern RISC architecture can be executed in a single clock cycle.

In the implementation of the claimed method, other types of criteria can be added that are not based on the values of numerical characteristics. Any criteria can be used that evaluate the image of familiarity for similarity with a certain type of symbol and give a binary result of the evaluation in the form of success or failure. Such criteria may be required to distinguish between very similar characters. For example, a similarity check of a previously stored reference image with a region in familiarity whose size and location are predetermined can be applied. For it, it is necessary to calculate the correlation coefficient between the standard and the familiar image for several positions of the standard both at the point of its calculated placement on the familiar image and near it. If, as a result, the correlation coefficient exceeds at least one of the positions a predetermined threshold value, then the criterion is successful. If, in any of the provisions, the threshold value is not exceeded, then the criterion fails.

Such a correlation criterion allows you to detect the difference between very similar characters, for example, “D” and “0”. However, like any correlation method, it is very computationally expensive. Therefore, it is advisable to check the correlation criterion at the end of the set of criteria, so that the correlation criteria are checked only once or twice during character recognition, and in most sets the correlation criterion would not be checked at all due to early completion. Thus, the regularly applied early completion of the verification of a set of criteria is a distinctive feature of the claimed method and allows for high performance, including when using computationally expensive criteria that are not based on the numerical characteristics of the test line.

As another criterion, a connectivity check for whitespace elements can be used to determine the presence of a structural element in a symbol. For example, the symbol "8" contains two isolated white space elements, which are usually called islands. The island is surrounded on all sides by pixels related to the symbol, and is not connected with other whitespace areas. The symbol "9" contains one island and one peninsula, where under the peninsula we mean an uninsulated space element connected by a jumper to another space region. The symbol "3" contains two peninsulas. Checking the connectivity allows you to distinguish among themselves several characters that are similar in pattern and differ only in structural elements, such as "8", "3", "6" and "9".

For criteria that are not based on the numerical characteristics of the test lines, the exact same correction of line numbers and the position of the origin of the coordinate system on the line is used, which was described earlier and illustrated in FIG. 7.

In FIG. 10 shows a general sequence of implementation of character image recognition. At step 101, the image is binarized in familiarity. At step 102, sets of numerical characteristics are found for all lines of familiarity. Then, at step 103, the line numbers and the origin on each line are linked to the boundaries of familiarity, based on the numerical characteristics of the image calculated in the previous step. Next, in step 104, a symbol type is selected to verify that it is a member. At step 105, the fulfillment of all criteria from the set of criteria that corresponds to the type being tested is subsequently checked. If the verification of all criteria in the set is successful (step 107), then it is reported that the symbol image belongs to the type being checked, and the recognition of the symbol is completed. If, when checking all the criteria, a criterion is found, the verification of which was unsuccessful, then the further verification of the set of criteria is stopped and additionally checked (step 106) if there are still unverified types of characters. In the case when there are such types of symbols, they return to step 104, select the next type of symbol for verification, and again verify the set of criteria (step 105).

If, at step 106, it turns out that there are no more character types to check, then they inform (step 108) that the character is not recognized, and complete the recognition.

The described sequence of steps 101-108 can be performed as a whole on the central processor of the computing system, or when the central processor and the coprocessor described herein work together. The coprocessor takes on part of the binarization step (101) and almost the entire step of obtaining numerical characteristics (102).

The banknote processing sequence using the inventive method in a counting and sorting machine is shown in FIG. 11. The machine provides transportation of banknotes from the supply pocket, while scanning and recognition is carried out (step 201). If the banknote is not recognized, then a decision is made (step 210) to send the banknote to the reject pocket. This solution is implemented using an electromechanical redirecting unit located in the banknote path of a counting and sorting machine.

If the banknote is recognized, then a digital image of the serial number is selected and rotated to compensate for skew (step 203). Next, divide the digital image of the serial number into familiarity and combine familiarity with the images of the characters of the serial number (step 204). Images of characters in familiar characters are recognized sequentially. First, one character is recognized (step 205) using the sequence of steps 101-108 described above. If the symbol is not recognized (step 206), then a decision is made on the direction of the banknote in the reject pocket (step 210). If the character is recognized, then remember the corresponding Unicode character code in the array-string of the serial number, and check if there are still unrecognized familiarities (step 207). If there are such familiarities, they return to step 205 and recognize the next familiarity.

When at step 207 it is found that all the familiarities are already recognized, they send the banknote to the receiving pocket (step 208) and write the serial number from the array-string into the electronic sorting report (step 209). The described processing (steps 201-210) is repeated for all banknotes located in the feed pocket of a counting and sorting machine. After that, the recognized banknotes with the successfully recognized serial number are in the receiving pocket of the machine, and the electronic report contains a list of serial numbers of these banknotes. Banknotes that were not recognized, or in the serial number of which not all characters were recognized, are in the pocket of rejection. An electronic report containing a list of serial numbers can then be used to track the movement of banknotes in circulation.

There are many other options for using the information obtained as a result of character recognition of the serial number. These options are well known to those skilled in the art of automated banknote processing. The claimed method relates only to the recognition of a single character, and does not apply to subsequent processing of the recognized serial number.

Claims (54)

1. A method for recognizing a character on a banknote relating to a known font in which scanning a banknote to obtain a digital image consisting of pixels for which membership in a particular class is determined, based on the specified class, the location of the character to be recognized is determined, combine familiarity of a predetermined shape and size with the image of the character to be recognized, so that the pixels related to the character are within the boundaries of familiarity, while conducting binarization to highlight pixels related to the symbol, moreover, for familiarity, a set of test straight lines intersecting this familiarity is predefined, and on each of the test straight lines a one-dimensional coordinate system is set, so that the position of each test straight line and the origin of the coordinate system on it is set in a predetermined manner relative to the boundaries of familiarity, then, on each test line, pixels are marked along which this line passes, indicating them in different ways for the pixels of the symbol and for whitespace pixels, and match each test direct set of numerical characteristics, including the length of the interval covering all of the indicated pixels of the symbol, the number of intervals of the specified pixels of the symbol in which the specified pixels of the symbol are arranged in a row one after another, the coordinate of the center of gravity of the indicated pixels of the symbol, expressed in the coordinate system for this test line, and the total number of all the specified pixels of the symbol, then perform the classification of the character to be recognized, using the previously obtained sets of numerical characteristics as features. 2. The method according to p. 1, in which the classification of the symbol is as follows for each of the possible types of symbol perform a sequential check of a set of criteria for belonging to this type, where the sets of criteria are predefined, and at least part of the criteria in each set is set based on the numerical characteristics of the test lines, upon detection of the fulfillment of all criteria in the set corresponding to any possible type of symbol, they interrupt further verification and conclude that the symbol to be recognized belongs to this type, if, after comparison with the sets of criteria for all possible types of characters, no match is found for any of the possible types of characters, it is concluded that the character is not recognized, Moreover, for each of the criteria established on the basis of the characteristics of the test lines, they are preliminarily set requirements of the criterion for the characteristics of the test line, containing a tolerance for the value of at least one numerical characteristic, and a set of test lines to test the criterion, moreover, such a criterion is considered fulfilled if the number of test lines in the set that meet the requirements of this criterion is within a predetermined interval. 3. The method according to PP. 1, 2, in which each test line corresponds to either a row or a column of a digital image of familiarity. 4. The method according to claim 2, in which each test line corresponds to the line of the image of familiarity and each line of the image of familiarity corresponds to the test line, each test line is preassigned with a number determined by the position of the corresponding line relative to the borders of familiarity, and the number assigned to it is used to identify the test line in each of the criteria established on the basis of the characteristics of the test lines. 5. The method according to p. 2, in which each test line corresponds to a column of the image of familiarity and each column of the image of familiarity corresponds to a test line, each test line is preassigned with a number determined by the position of its corresponding column relative to the borders of familiarity, and the number assigned to it is used to identify the test line in each of the criteria established on the basis of the characteristics of the test lines. 6. The method according to p. 4, in which, when finding the numerical characteristics, to fix possible image defects according to a predetermined rule, the pixels marked on the test line are filtered. 7. The method according to p. 6, in which, when at least one criterion is specified, the predefined requirements for the characteristics of the test line are set in the form of allowable intervals for each of the characteristics, and, to fulfill the above requirements, the values of all characteristics must be in acceptable intervals for them. 8. The method according to p. 7, in which, when at least one criterion is specified, the predefined requirements for the characteristics of the test line are set in the form of unacceptable intervals for each of the characteristics, and, to fulfill the aforementioned requirements, the value of at least one characteristic must be outside the range that is not allowed for it. 9. The method according to p. 7, in which before performing the combination of familiarity with the image of the symbol, the image of the symbol is rotated in order to obtain its predetermined orientation in the familiarity. 10. The method according to any one of paragraphs. 2, 4, 6, 7, 8, in which at least one set of criteria, there is a criterion for which verification, the similarity of the previously saved reference image with the region in familiarity is checked, the size and location of which are set in advance. 11. The method according to any one of paragraphs. 2, 4, 6, 7, 8, in which, in at least one set of criteria, there is a criterion for which verification the presence of a predetermined structural element in familiarity is checked. 12. The method according to p. 7 or 9, in which get a set of numerical characteristics for each line of the image of familiarity, and based on the obtained sets of numerical characteristics, the number and position of the coordinate origin are corrected for at least one check line in order to take into account the position of the symbol image in relation to the borders of familiarity. 13. The method according to p. 12, in which the correction is carried out in accordance with the position of the axis of gravity of the image within familiarity directed in the column direction, such that the coordinate on the test line as a result of the correction is counted from the point of its intersection with the specified axis. 14. The method according to p. 13, in which the correction is additionally carried out in accordance with the position of the center of gravity of the image of familiarity so that the distance from the test line with a certain number to the center of gravity of the image, as a result of the correction, would correspond to a predetermined pattern. 15. The method according to p. 13, in which the correction of the number of the test line is carried out in accordance with the position of the surrounding rectangle of the symbol image. 16. A coprocessor for a computer system of a banknote processing device, comprising input port for receiving pixel values corresponding to familiarity, where the specified value determines either a pixel of a character or a pixel of a space, a decisive logic circuit configured to provide a set of numerical characteristics for each test direct familiarity, including the length of the interval that covers all the pixels of the character located on the test line, the number of pixel intervals of the symbol in which the pixels of the symbol are located on the test line one after another, the sum of the coordinates of the pixels of the symbol located on the test line, expressed in a one-dimensional coordinate system of the test line, and the total number of all pixels of the character located on the test line, and an output port for issuing sets of numerical characteristics corresponding to familiarity strings. 17. The coprocessor according to claim 16, in which the decisive logic circuit is configured to correct possible defects in the digital image by filtering pixels corresponding to the familiarity according to a predetermined rule. 18. The coprocessor according to claim 17, in which the decision logic contains at least two segments, each of which is configured to provide a set of numerical characteristics for a single test direct familiarity. 19. The coprocessor according to claim 18, in which the decision logic includes the implementation of a finite state machine for calculating the numerical characteristics of a separate test line, configured to provide sequential feed to the machine input of pixel values located on the test line, the value supplied to the input determines the transition between the states of the machine, and the transition graph provides the formation of numerical characteristics of the test line in the internal registers of the machine. 20. The coprocessor according to claim 18, in which the decisive logic circuit contains a combinational part for calculating the numerical characteristics of a separate test line, the input of which is supplied with the pixel values located on the test line, and the values of the numerical characteristics of the familiarity string are generated at the output.

Images