JPH0954814A - Analysis of input signal expression and scoring system of possible interpretation of input signal expression - Google Patents

Abstract

PROBLEM TO BE SOLVED: To analyze an input symbol expression by dividing an input data set showing the input symbol expression into plural specified segments, evaluating scores allocated to identified candidate interpretations and generating a normalized score. SOLUTION: Image acquisition A acquires an image I of a character string such as a handwritten character on a recording medium 6 and stores it in a frame buffer. Image processing B preprocesses image normalization, etc., has a boundary capable of designating the input symbol expression by image segment formation D and divides it into plural segments which can be classified. Image segment analysis G analyzes the respective segments and allocates scores to the respective classes of segments attached to a specified symbol in the prescribed input symbol set. Character string interpretation I identifies candidate symbol interpretation based on the allocated score, evaluates the score allocated to this candidate interpretation and while using the evaluated scores concerning these plural kinds of possible analysis, the normalized score concerning each kind of candidate interpretation is generated.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and system for automatically interpreting input symbolic representations, such as handwriting, using a novel inductive probability measure and optimally learned neural information processing network.

[0002]

2. Description of the Related Art Currently, there is a strong demand in the market for development of an apparatus capable of accurately interpreting (i.e., recognizing) a properly combined string of alphabetic characters recorded on various media. For example, the U.S. Department of Posts and Telecommunications to accurately recognize a ZIP code (ie, zip code) handwritten on a mail piece during postal accumulation and national routing operations.
We strongly desire the rapid development of such a device.

Many character recognition systems are currently being developed for use in various environments. Such various systems and related techniques are disclosed in the following technical documents. (1) Y. Le Cun, B. Boser, JS Denker, D. Henderso
n, RE Howard, W. Hubbard, and LD Jackel, "Hand
writtten Digit Recognition with a Back-Propagation
Network ", pp. 396-404 in Advances in Neural Infor
mation Processing2, David Touretzky, ed., Morgan K
aufman (1990), (2) JS Bridle, "Probabilistic Inte
rpretation of Feedforward Classification Network O
utputs, with Relationships to Satistical Pattern R
ecognition ", in Neuro-Computing: Algorithms, Archi
tectures and apications, F. Fogelman and J. Heraul
t, ed., Springer-Verlag (1989), (3) JS Bridle, "T
raining Stochastic Model Recognition Algorithms as
Networks Can Lead To Maximum Mutual Information E
stimulation of Parameters ", in Advances in Neural Inf
ormation Processing 2, David Touretzky, ed., Morga
n Kaufman (1990), (4) O. Matan, J. Bromley, CJC
Burges, JS Denker, LD Jackel, Y. LeCun, EPD
Pednault, WD Satterfield, CE Stenard, and T.
J. Thompson, "Reading Handwritten Digit: A ZIP cod
e Recognition System ", IEEE Computer 25 (7) 59-63 (J
uly 1992), (5) CJC Burges, O. Matan, Y. Le Cun,
JS Denker, LD Jackel, CE Stenard, CR Nohl,
JI Ben, "Shortest Path Segmentation: A Method f
or Training a Neural Network to Recognize Characte
r Strings ", IJCNN Conference proceedings 3, pp.165-
172 (June 1992), (6) CJC Burges, O. Matan, J. Br
omley, CE Stenard, "Rapid Segmentation and Class
ification of Handwritten Postal Delivery Addresses
using Neural Network Technology ", Interim Report,
Task Order Number 104230-90-C-2456, USPS Referenc
e Library, Washington DC (August 1991), (7) Edwin
PD Pednault, "A Hidden Markov Model For Resolvi
ng Segmentation and Interpretation Ambiguities in
Unconstrained Handwriting Recognition ", Ball Labs
Technical Memorandum 11352-090929-01TM, (1992),
And (8) Ofer Matan, CJC Burges, Y. Le Cun, JS
Denker, "Multi-Digit Recognition Using a Space Di
splacement Neural Network ", in Neural Information
Processing System 4, JM Moody, SJ Hanson and
RP Lippman, eds., Morgan Kaufman (1990).

While the conventional systems described in the above references are distinguishable from each other, they are best characterized by the structural and functional features they share in common.

In particular, each prior art system acquires at least one image I of an optionally connected character string which is to be interpreted by the system. In general, for a given alphabet, the number of possible interpretations, including the “best” interpretation that the system must choose, can be stringed together using the letters in the alphabet and applicable morphological constraints. Equal to the number of possible strings. In ZIP code (postal code) recognition applications, each acceptable interpretation is constrained by the length of the ZIP code. That is, the ZIP code must be 5 or 9 digits.

According to the prior art, the captured image of the character string is generally pre-processed to remove underlining, spatial noise, etc. This preprocessed image I is then "cut" or divided into manageable sized sub-images. The sub-images between each set of adjacent cutlines are called image "cells".
In some cases, the boundary between two cells is determined to be a "determined cut" that is explicitly included between the two characters. On the other hand, in other cases, the cut is considered ambiguous and the determination of whether the cut is contained between two characters is deferred until further processing.

Adjacent image cells are then combined to produce an image "segment". The image segments are then stitched together from left to right to produce an acceptable image "consegmentation" that includes almost all the pixels of the preprocessed image. In particular, an acyclic (chain) directed graph is used to construct a model of admissible image "consegmentation". In general, this model is constructed by associating each image segment with a node in a directed acyclic graph.

The nodes in the graph are then connected with directed arcs. In general, two nodes in a graph are connected only if the image segments they represent are legally adjacent within an acceptable image consegmentation.

When the graph is fully constructed, each pass through the graph corresponds to an image concatenation of the preprocessed image and all possible image concatenations correspond to a particular pass through the graph. After the graph has been constructed, a recursive pruning technique is used to remove from the graph the nodes corresponding to image segments that meet a distinct cutline through the preprocessed image.

After the graph has been pruned, each image segment associated with the remaining nodes in the pruned graph is sent to a neural network recognizer for classification and scoring. Based on such classification and scoring, each node in the pruning graph is assigned a "score" that is derived from the recognizer score assigned to the satellite image segment.

Generally, each recognizer score is converted into a probability by a computational procedure consisting of normalizing the recognizer score. Then the pass score (ie, the joint probability)
Is calculated for each pass through the pruning graph. This calculation is done, for example, by simply multiplying the "score" assigned to the node along the path. With this multi-character recognition (MCR) scheme, the highest scoring pass through the pruning graph corresponds to the "best" image consegmentation and string interpretation for the acquired image.

A detailed description of these techniques is given in 19
US Patent Application No. 07 / filed December 31, 1991
No. 8164414 and No. 07/816415.

While conventional methods are useful in the design of commercial and experimental character recognition systems, the performance of such systems is not ideal, especially for demanding real-time applications. In particular, conventional MCR systems generally operate by identifying only one consegmentation that supports a given interpretation. This method presupposes the concept that there is only one "best" consegmentation.

In such conventional methods, this single "best" concatenation score is the only score considered during the recognition process. Therefore, conventional MCR systems use a method that is equivalent to incorrectly assuming that the correct image segmentation is known. Relying on this assumption, individual character scores are normalized to calculate the probability for a particular character in the acceptance code or alphabet.

This irreversibly discards useful information about how the segmentation algorithm was done for a particular segment of the image. Conventional MCR systems based on such assumptions are often referred to as "maximum likelihood sequence estimation" (MLSE) machines.

In addition to the choice of image interpretation, some conventional MCR systems often provide a score which means that the chosen interpretation gives some indication of the correct probability. In many applications it is desirable to have a score that can be interpreted as an exact probability to facilitate combining this result of the MCR system with other sources.
However, conventional MCR systems tended to emphasize the choice of "best" interpretation, but not accurate scoring. Therefore, scores often include significant systematic errors, on the order of several orders of magnitude.

Therefore, it is highly desirable to develop a good method and system for interpreting symbol sequences displayed on various media.

[0018]

Accordingly, it is a general object of the present invention to provide a superior method of interpreting input symbolic representations such as strings displayed or recorded on a medium by, for example, printing or cursive writing techniques. And to provide a system.

Another object of the invention is to provide a method and system for automated string interpretation that uses inductive probabilities in selecting the best string interpretation.

Another object of the present invention is to provide a method and system for automated string interpretation in which each a posteriori probability is derived a posteriori by combining a priori information with a known example pixel image. That is.

Another object of the present invention is to provide an automatic character string interpretation method and system which can interpret a character string of an arbitrary length and can be easily used together with an automatic text interpretation system. Is to provide.

Another object of the present invention is to provide a single adaptive learning process performed by a complex of neural computational networks that has been optimally trained to maximize the correct string interpretation score. The purpose is to provide a method of multi-character recognition that is coupled to column interpretation.

Another object of the invention is to use a new data structure based on a specially modified acyclic directed graph, where each pass through the graph exhibits both image consegmentation and string interpretation. Is to provide.

Another object of the present invention is to assign a score to a selectable interpretation of an image, especially a score that can be interpreted as an accurate estimate of the probability of this selected interpretation.

Another object of the invention is that the inductive probability assigned to each particular string interpretation is defined as a ratio,
The numerator part of this ratio is calculated by adding the path scores along all paths through the graph showing the same string interpretation, and the denominator part of the ratio is the graph showing all possible string interpretations with the same number of characters. It is to provide a system that is calculated by adding the path scores along all paths through it.

Another object of the present invention is to provide a multi-character handwriting recognition system which can be realized as a portable device.

Another object of the present invention is to provide a method of string interpretation which is used to identify which string interpretation has the best pass score.
bi) algorithm, then use the forward algorithm to compute the exact sum of all pass scores that indicate the string interpretation identified by the Viterbi algorithm, and indicate all possible string interpretations. It consists of using a forward algorithm to calculate the normalization constant for the correctly calculated sum by adding all the pass scores through the graph.

Another object of the invention is to provide a method of string interpretation that uses a beam search algorithm to identify the multiple competing string interpretations that have the best pass score set. And then use the forward algorithm to compute the exact sum of all path scores indicating the competing string interpretations identified by the Viterbi algorithm for each string interpretation, and then all possible string interpretations. Consisting of using the forward algorithm to compute a single normalization constant for each competing string interpretation by adding all the pass scores through the graph showing

Another object of the present invention is to use both the graph and neural information processing network complex to train the system by optimally adjusting the parameters of the neural network during one or more learning sessions. To provide an input symbolic representation interpretation system having a learning mode of operation.

Another object of the present invention is to increase the inductive probability of string interpretations known to be correct and decrease the inductive probability of interpretations known to be incorrect, in neural networks. To provide a system in which sensitivity analysis is used during neural network learning to adjust each tunable parameter.

Another object of the invention is to sensitively calculate these scores produced by the overall change in the system in response to the incremental changes made to each adjustable parameter of the neural network. , Providing an input symbolic representation interpretation system which consists of using the Baum-Welch algorithm during the learning mode of operation.

[0032]

In order to solve the above problems, the present invention provides a method and system for generating an interpretation of an input symbolic representation rendered on a medium using a printing or cursive writing technique. To do.

In general, the system of the present invention obtains an input data set representing an input symbolic representation. The acquired input data set is divided into a series of segments. This series of segments is then used to specify a series of segmentations. The system of the present invention then uses the novel data structure to implicitly indicate each concatenation and each possible interpretation of the input symbolic representation.

The data structure can be represented as a directed acyclic (chain) graph consisting of a two-dimensional array of nodes arranged in rows and columns, selectively connected by directed arcs.
Each path that passes through the node and along the directed arc represents one consegmentation and one possible interpretation of the input symbolic representation. All concatenations and all possible interpretations of the input symbolic representation are implied by a series of paths extending through the graph.

For each node row in the graph, a series of scores is generated for a known set of symbols using, for example, an optimally trained neural information processing network. Related to the graph, these scores are implicitly assigned a path score for each path through the graph. Using these pass scores, the system of the present invention identifies the best symbol sequence interpretations and computes the a posteriori probabilities for them.

Highly reliable probabilities for each symbol sequence interpretation are generated by analyzing almost all acquired input data sets to derive each inductive probability. The principles of the present invention can be implemented for almost any symbolic representation sequence, such as a scribbled string of arbitrary length. The system of the present invention can be easily adapted for use with an automatic sentence interpretation system.

The system of the present invention determines the string interpretation that has the highest scoring path through the graph. In order to determine if this interpretation is reliable, the system of the invention also produces as output the recursive probability of this string interpretation. This probability is calculated as the ratio of the numerator part to the denominator part. The numerator is equal to the sum of the path scores of all paths that pass through the graph showing the given string interpretation.

The denominator part is equal to the sum of the pass scores of all the passes through the graph showing all possible string interpretations.
If the probability is less than a predetermined threshold, the user cannot guarantee that this interpretation is reliable, so that the user
Before further action, it is announced that another step must be taken.

In another embodiment of the present invention, the system of the present invention first finds the series of paths through the graph having the highest series of path scores. For each path in this series of paths, the system of the present invention identifies the corresponding string interpretation and then computes the numerical value of the recursive probability of this interpretation (including contributions from other paths that have the same contribution). Ask. The system of the present invention identifies a set of possible string interpretations indicated by the set of discovered paths.

Next, the recursive probability of the possible character string interpretation group is calculated. In order to determine which possible string interpretation has the maximum inductive probability, the system of the present invention analyzes the calculated inductive probability group. Based on this analysis, the system of the present invention produces (i) one or more string interpretations with high recursive probabilities and (ii) an accurate estimate of the recursive probabilities of each string interpretation. .

The inductive probability for each competing string interpretation is
Calculated as the ratio of the numerator part to the denominator part. The numerator part is equal to the sum of the path scores for all paths through the graph showing competing string interpretations. The denominator part is equal to the sum of path scores for all paths through the graph showing all possible string interpretations.

Novel methods and systems for optimally learning the symbol sequence interpretation system of the present invention are also provided.
This is done by giving the system a unique learning mode of operation.

In the learning mode of operation, the system of the present invention processes a large number of learning images showing known input symbolic representations. For each processed learning image, the system of the present invention incrementally adjusts a set of adjustable parameters that characterize the function of each neural network. The direction of each incremental adjustment is such that the average probability for a string interpretation known to be correct increases while the average probability for a symbol sequence interpretation known to be incorrect decreases.

The system and method of the present invention can be used, for example, with electrically passive media or pressure sensitive writing surfaces such as paper, plastic or fabric and "touch screen" writing / display surfaces well known to those skilled in the art. It is used to interpret a displayed character string in virtually any way, including graphic recording on electrically active media such as.

[0045]

DETAILED DESCRIPTION OF THE INVENTION FIG. 1 illustrates a symbol sequence interpretation (ie, "recognition") consisting of a number of integrated system components.
2 is a block diagram of the system 1. FIG. In particular, the system comprises one or more processing units 2 (eg microprocessors) controlled by a program stored in a program memory storage device 3. The program memory storage device 3 also stores an operating system program, application programs, and various image processing routines of the present invention. It also has a data storage memory 4 for storing the data associated with the data structure of the invention.

Generally, the system comprises an input data set acquisition device 5 for acquiring an input data set representing a display sequence of symbols. This device is realized as an image detector for obtaining a density gradation or a colored image of a possible connection sequence of alphabetic characters recorded on a recording medium 6, as shown in FIG.

The strings may be electrically passive recording surfaces such as paper, plastic, wood, fibers, etc., or pressure sensitive digitized surfaces or "touch screen" LCDs well known to those skilled in the art.
It is possible to record on electrically active recording surfaces such as writing and display surfaces. The strings can be represented using conventional printing or cursive (ie, handwriting) writing techniques.

The system of the present invention comprises a random access data storage memory (eg VRAM) 7 for buffering the captured image of the character string to be interpreted. A mass data storage memory 8 is provided for long term storage of these images.

The system of the present invention includes a visual display unit 9 having a visual display screen or screen (LCD), a keyboard or other data input device 10, pointing, dragging and displaying graphical icons visually displayed on the display screen. It also includes a device 11 for selecting, an input / output device 12, and a system interface 13 for interfacing with one or more external host systems 14 using the information provided by the system 1.

The system components 2, 3, 4, 7 and 8 are housed in a compact housing suitable for the particular application at hand. The other components are housed in their respective housings. Each of these components can operate in association with the processor 2 via one or more system buses 15. For ZIP code (so-called postal code) recognition applications, the system of the present invention is suitably interfaced with a conventional mail storage and routing device 14.

As shown in FIG. 2, the character string interpretation system 1 uses the graphically recorded "characters" based on the analysis of the pixel information included in the acquired image I of the graphically recorded character string. Perform a number of functions in order to arrive at the interpretation of the sequence "(indicated by Ci)". These image processing stages will be described in order below with reference to other related figures.

In general, the system and method of the present invention is applicable to machine-printed or handwritten strings of any length. Therefore, the present invention is useful in handwriting recognition applications. In this case, the handwriting writer can write one or more characters on various types of writing screens, or one or more sentences for automatic recognition.

The embodiments shown in FIGS. 14 and 17 examine the problem of interpretation (ie, classification) of handwritten ZIP codes, where the string length is 5 or 9 digits. However, the method and system of the present invention can also be used to interpret strings (ie words) of arbitrary length, such as the long context of automatic sentence recognition systems known to those skilled in the art.

In FIG. 2, blocks A to I schematically show various stages performed during the character string interpretation processing of the present invention. As shown in block A in FIG. 2, the first stage of the process is to obtain an image I of the string. Generally, each image I acquired by the system 1
Consists of a matrix of pixels.

Each pixel in the image matrix has a density gradation brightness that indicates the intensity of the image at a pixel location within the image. It can also indicate pixel saturation. Each acquired image is stored in the frame buffer 7. As indicated by block B, the second stage of processing is the "pre-processing" of the stored image I. Suitable image pre-processing operations performed by processor 2 include "desired area" positioning, underline removal, image deslanting and deskewing, small dots (ie, small connected components) and penetration. Stroke removal, normalization of the image to a standard size (e.g. normalization to 20 pixel height with width selected such that the aspect ratio of the image does not change), etc.

Image normalization is performed to allow the preprocessed image I'to be sent to subsequent stages of the system without the need for further image normalization. The resampling performed during the normalization process yields an effective grayscale image even if the original image is black and white. Then use the top and bottom contours of the normalized image,
Clip the long tail of characters in both horizontal and vertical image directions. A more detailed description of the above image pre-processing operations is disclosed in U.S. patent application Ser. No. 07 / 816,414 filed December 31, 1991.

The next stage in the recognition process, shown in block C, is to cut the preprocessed image I'into a sub-image called a "cell". The purpose of generating the image cells is to combine the image cells so that an image "segment" S _i can be generated during the image segment generation stage shown in block D.

According to the invention, image cells are generated by first performing a "connective component analysis" on the preprocessed image in order to detect the presence of large "connective components". Then, "smart" wave cut line drawing processing is performed on these large connected component-containing sub-images. Both the connected component analysis and the smart wave cutline drawing sub-process are performed by the programmed processor 2 using the attached RAM 4.

More specifically, connected component analysis analyzes the intensities of the pixels of the preprocessed image to determine the presence of character components (ie, groups of pixels) that have been combined together.
Connected component analysis appears to produce large character components that have intensity values within a predetermined range along the vertical and horizontal image directions and are combined together and appear to be associated with one or more characters. , Search for pixel clusters. The combined character component is, for example, the second and third Z shown in FIGS.
For example, an IP code image.

More than one character may be included in a sub-image containing a large connected component. It is important to draw cut lines through such sub-images so that no more than one character is indicated by the pixels of the image cell. This is done by creating a "wave" cutline in the discriminated large connected component.

In general, this cut line generation processing makes it possible to subdivide the character indicated by the large connected component into two or more image cells simply by drawing a cut line in the pixel group indicating the character. The number of methods of combining adjacent image cells to form an image segment grows rapidly with the number of image cells generated during this recognition processing stage.

The system of the present invention cuts a pre-processed image into minute image cells by using a complex heuristic that identifies a series of good cut lines and removes redundant lines and the like. Avoid things. This sub-processing operation is illustrated by the cut lines drawn and selectively removed for the pre-processed images shown in FIGS.

At the end of this sub-processing, the pixels between each adjacent pair of remaining cutlines define an image "cell". The image cells generated during the image cell generation process are shown in the table of FIG. As shown in this table, each image cell is identified by a cell number (eg, 0, 1, 2, 3, 4 etc.).
For a more detailed description of automatic cutline generation during this stage of the recognition process, see US patent application Ser. No. 07/8164.
No. 14 specification.

As shown in block D of FIG. 2, the next stage of processing combines adjacent (ie, consecutive) image cells in left-to-right order, as shown in the table of FIG. Generate a series of image "segments". As shown in this table, each image segment has a number (eg, 0, 01, 1, 2, 2) assigned to its constituent image cells.
3 etc.). Ideally,
Each image segment contains pixels that represent only one character. But this is not always the case.

It is important that the final set of image segments include the correct image segment. A complex heuristic is used to determine the number of image cells and which image cells should be combined to form an image segment. In general, heuristics are described in terms of "explicitly bounded" cuts, "interconnect components" cuts, "interconnect components" cuts, and the like. The parameters and adjustment factors for these heuristics are determined empirically.

Each image segment consists of a series of image pixels. The image pixels are analyzed by an assigned neural information processing network included in the system. As will be described in detail below, the function of each neural network is to analyze a series of pixels in each assigned image segment, and the pixel set can actually be shown or classified as possible. To produce as output a score for each of the numeric characters (ie symbols).

The next stage of processing, indicated by block E, is to combine successive image "segments" from left to right to produce a series of acceptable (ie, legal) image "consegmentations". It is to connect the beads. Each such image consegmentation is
All the pixels in the preprocessed image I'must be accounted for.

It is desirable to consider as few consegmentations as possible. This still ensures that the correct segmentation is included in the sequence of all image concatenations made up of the generated image cells. In the table of FIG. 11, three examples of legal image segmentation for a five character ZIP code example are illustrated. As shown in block E, the consegmentation is generated by the "directed acyclic assignment graph" of FIG.

The structure of this graph guarantees that each of these image consegmentations consists of 5 image segments. There are rules governing how the image segments can be spliced together in order to capture the substance of the spatial structure of the input image I. For example, the right edge of one segment must touch the left edge of the next image segment. (Ie, skipping a bundle of pixels and / or combining pixels in the wrong spatial order is not allowed.)

However, if desired, some of these constraints may be relaxed under appropriate conditions. For a more detailed description of lacing consecutive image segments S _i together, see US patent application Ser. No. 07/81641.
No. 5 discloses it. If desired, the selected image consegmentation can be explicitly displayed in block F.

The directed acyclic (chain) graph of the present invention shows the possible image concatenation group {S} of the preprocessed image I'and the alphabetic characters enabled or the recorded character string is displayed. It also provides a novel means of simultaneously modeling both string interpretation (ie, taxonomy) groups {C} that are constrained by the language or code syntax that is used.

This data structure, which can be represented as a "directed acyclic graph", as described in detail in connection with FIG. 12, addresses both image consegmentation and string interpretation problems in "optimal paths within the graph.""Used by the system of the present invention to formulate as a matter in a uniform manner. Intuitively, this problem formulation has a geometrical appeal.

The alignment graph, the data structure that implements this graph, and the method of using this graph are described in detail below. The method of using this graph is then described in detail in the image segment analysis stage shown in block B, the pass score and probability calculation stage shown in block H and the string interpretation stage shown in block I of FIG.

As shown in FIG. 12, the graph of the present invention consists of a two-dimensional array of nodes. This graph, in a high-level description, resembles a conventional graph called a lattice or trellis diagram. The alignment graph of the present invention is implemented with a data structure that performs a number of important modeling functions. This data structure is created, modified and managed by the programmed processor 2 in a manner well known in the programming industry.

Each node in the alignment graph is realized as an individual data structure. This is a substructure of the "main data structure". The data structure for each node has a number of "local" information fields. This information column can store the following information items and is marked with a special mark. A unique node identifier (ie, a code that identifies the column / row address of the node), a possible numerical digit that can indicate the pixels of the satellite image segment, a calculated score for each of the letters, a possible digit that can indicate the pixels of the satellite image segment. The calculated "denormalized" score for each of the characters, the node identifier of the ancestor node, and the node identifier of the descendant node.

In order to store the information generated at each stage of the method, the main data structure is a large number of "global".
It has an information column. This information column can store the following information items and is marked with a special mark.
Same as a series of codes identifying which particular image segment is represented by a node in each particular row in the alignment graph, a series of addresses identifying where each image segment is stored in memory, and a selected path Sum of scores along a series of paths in the alignment graph showing string interpretation.

The number of columns in the alignment graph is equal to the number of digits of characters in the possible string interpretation (eg, 5 digits in the ZIP code of FIG. 3). Also, the number of rows in the alignment graph is equal to the number of image segments constructed during the image segment generation stage of the method of the present invention. For example, the size of the alignment graph typically changes the row size for each image I acquired for interpretation (ie analysis and classification).

Thus, for each acquired image I, the programmed processor 2 routinely produces a graph of the type shown in FIG. 12 specially made for the acquired image.
Each such alignment graph is physically realized by generating a data structure corresponding to that stored in the RAM 4.

The information on the image I and the image segmentation on its possible character string interpretation is stored in the information field of the data structure specially generated for this information. Finally, this organized information is used by the programmed processor 2 to select the most promising string interpretation C from the candidate set of interpretations {C}.

As shown in FIG. 12, the alignment graph of the present invention has a number of precise structural features. The main part of the graph has rows and columns. Each string corresponds to one character position in the string interpretation C. This example is a 5-character ZIP
As it relates to the code, it requires 5 columns as shown. Each row corresponds to an image segment. This case has 11 segments, so 1
You need one line.

In each interpretation of a certain row of a certain column, there is a node indicated by a pair of dots (...). The left dot indicates the "morning" part of the node and the right dot indicates the "evening" part of the node. Each such node is identified by its row index and column index. Furthermore, before the first character position, there is a special start node 17 located at the left end of the leftmost image segment. Similarly, the special end node 1 placed to the right of the last character position and below the leftmost image segment.
There are eight.

As shown in FIG. 12, there are 10 recognition arcs connecting the morning and evening parts of each node. For clarity, only three of the ten recognition arcs are shown in FIG. During the interpretation process, each recognition arc 19 is labeled with an "r-score".

This "r-score" is assigned to the character indicated by the recognition arc. These recognition arcs indicate the denormalized r-score assigned to the numeric characters that make up the ZIP code. However, in word and sentence recognition applications, these recognition arcs generally represent denormalized scores assigned to symbols in a given alphabet or vocabulary. As shown in FIG. 12, in order to show the node descendant system and the ancestor system between such nodes, between each evening part of a certain node and the morning part of the immediately adjacent node,
A straightening glue-arc 19 is also drawn.

Unlike the recognition arc, this glue arc is not assigned an r-score by the neural network. In this embodiment, a simple system is used, although in other embodiments complex glue arc scores may be used. That is, a score of 1.0 is assigned to the allowable arc and held. However, the non-permissive arc is assigned a score of 0.0 and is discarded from the alignment graph.

The morning portion of a node can also have more than one glue arc entering it. Similarly, the evening portion of a node can have more than one glue arc out of it. As a result of the constraints imposed on the composition of image consegmentation, there may be glue arcs in the alignment graph that make sense locally but not globally. Therefore, in order to improve the calculation efficiency of the interpretation process,
Specific glue arcs can be removed or pruned. FIG.
The next "glue arc" pruning process can be performed on the alignment graph before proceeding to the image segment analysis stage shown in block G.

The first step in the glue arc pruning process is to compute the "forward cone" of a node that is a descendant of the start node by iteratively marking the descendants of the node already marked as members of the forward cone. It is to be.
The second step in the process is to compute the "back cone" of the node that is the ancestor of the end node by iteratively marking the ancestors of the nodes already marked as members of the back cone.

The third step of the process is to determine which node
Determine if they are not in the logical intersection of two cones, then mark these nodes as "dead". Then, the glue arcs that extend to or from the node marked as "dead" are removed (ie, pruned) from the list of allowed glue arcs. Each node within the intersection of these cones is considered an "alive" and has a score assigned to its series of recognition arcs during the image segment analysis stage.

Satisfying this global constraint results in a large number of nodes in the upper right and lower left corners of the alignment graph that have no legal ancestors or descendants. This fact is shown by the absence of input and output glue arcs within these regions of the alignment graph, as shown in FIG. Furthermore, if necessary or desired, the alignment graph can also be pruned by using the presence of distinct cuts.

Each path in the graph represents both consegmentation and interpretation. Glue arcs in the path specify consegmentation, and cognitive arcs in the path specify the interpretation. To understand how the method of the present invention selects the "correct" string interpretation from either all of the possible string interpretations or a minority of competing string interpretations,
First of all, one must understand some sub-processes that precede the final choice of "correct" string interpretation.

The first sub-process involves the calculation of the denormalized r-score assigned to the recognition arc of each node. The second sub-process involves the calculation of the sum of r-scores associated with all string passes through the alignment graph showing the same string interpretation. These sub-processes will be described below.

As shown in FIG. 13, the image segment analysis stage of the interpretation process uses a complex neural computation network 21. The basic function of each i-th neural computing network is to analyze the pixels of the image segment S _i that are co-indexed with the i-th row in the graph and find the recognition arc at each node in the i-th row in the graph. Computing a series of "scores" (ie, r-scores) that are assigned.

When there is one segment, one neural network per row and all nodes in the same row receive the same set of 10 r-scores. For clarity, 3 out of 10 recognition arcs for each node
Only one is shown in FIG. In essence, each neural computation network has its input (a group of pixels represented by a series of numbers) 1 called the r-score.
0-piece number r ₀ , r ₁ ,. . . Map to r ₉ .

The network architecture ensures that these r-scores are positive and accept their interpretation as denormalized probabilities. A large value of r ₀ indicates a high probability that the input segment represents the number “0”, and likewise, the other 9 r-scores respectively correspond to the other 9 numbers. Also, a large r-score reflects the high probability that the input segment is part of the correct segmentation of the image.

Conversely, if a segment is produced by cutting numbers in half (sometimes it happens sometimes), then all 10 r-scores for this segment are of undesired properties of the segment. Must be small to indicate detection.

According to the present invention, the mapping function of each neural calculation network is such that the weight vectors W ₁ , W ₂ ,. . . It features a set of adjustable parameters that can be shown in vector form as W _m . at first,
Adjust a set of adjustable parameters for each neural computation network to a set of initial values.

However, as will be described in detail below, the neural network parameter adjustment stage, indicated by block J in FIG. 2, serializes the input / output mapping functions of each neural computational network during one or more learning sessions. Are provided to allow incremental adjustment of these parameters in such a way that they are conditioned to accommodate the learning data of. This training dataset consists of hundreds of thousands of validated training images with ZIP codes handwritten by different people across the country.

The r-score generated from each i-th neural network is r = r ₁ , r ₂ ,. . . , R
Displayed in vector form as _N , it is assigned to the 10 corresponding recognition arcs (ie, information columns) at all nodes in the i-th row of the alignment graph.

In general, each neural computing network can be implemented as a computer program, an electrical circuit, or a microscopic or macroscopic device capable of implementing the input / output mapping function of the neural computing network. However, each neural network is a well-known LeNet
It is realized by executing a (registered trademark) computer program.

This LeNet® computer program is described by Y. Le Cun et al., "Handwritten Digit Reco.
gnition with a Back-Propagation Network ", pp 396-4
04, Advances in Neural Information Processing 2,
(David Touretzky, Editor), Morgan Kaufman (1990)
Are detailed in. In addition, a detailed description of the construction and learning of neural computing networks can be found in John Denker et al.
al., "Automatic Learning, Rule Extraction, and G
eneralization ", pp 877-922, Complex Systems, Vol.
1, October, 1987.

In the alignment graph, there may be more than one path (indicating different consegmentation) indicating the same character string interpretation. Paths that exhibit a given interpretation must be considered "groups." The score assigned to a given interpretation must depend on the sum of the scores of all paths in the group. This disregards the contributions of other paths in the group, unlike conventional recognizers which generally consider the score for only one path in this group.

For images containing 5 numbers, there are typically 10 ⁵ possible individual interpretations, and the number of passes through the alignment graph may be even higher. Therefore, it is not practical to show these clearly or examine each probability individually. The data structure and algorithm of the present invention allows the system of the present invention to identify a particular important path group (eg, the path group corresponding to a given interpretation or all path groups), and also the score of this group (ie, , The sum of the scores of the paths in this group) can be efficiently evaluated.

The system of the present invention analyzes the pixels of the acquired image I and calculates the sum of all the paths through the graph showing the candidate interpretations (ie, classifications) for which the probabilities have been calculated. Each item in the sum is the product of the scores assigned to the arcs of the particular path in the alignment graph. Normalization is performed only after the sum is calculated. This is called "column by column" normalization.

In contrast, conventional recognizers that compute probabilities generally normalize scores at relatively early stages of processing. In general, in a sense, it is equivalent to "character by character" normalization. This discards valuable information about the quality of the consegmentation. The neural computation network learning process described below trains a complex of neural networks to produce r-scores that contain information about the probability that a given segmentation is correct, rather than the probability that a given character interpretation of a segment is correct. This is very important.

The normalized score generated by the system and method of the present invention represents an estimate of the posterior probability P (C / I). In contrast, the maximum likelihood sequence estimation probabilities used in prior art multi-character recognition MCR systems are generally P
Use (I / C) type prior probabilities. These different probability measures are acceptable for many purposes, as they can be associated with each other certain additional information. The actual advantage of posterior probabilities is that the internal computations of the system and method of the present invention rely on an estimate of the combined posterior probabilities P (C, S / I) of interpretation and segmentation.

Corresponding prior (maximum likelihood) display P (I / C,
S) cannot be easily related to useful posterior forms. This is because it is generally not easy to estimate the marginal probability P (S). As a result, conventional recognizers are able to identify the interpretation of the highest score, but cannot assign a properly normalized score. The correctly normalized scores of the present invention can be very easily interpreted as probabilities, and thus can be combined with information from other sources very easily.

In general, the goal of the procedure shown in FIG. 14 is to find a new posterior probability P (C / C / for each competing string interpretation shown by the alignment graph shown in FIG.
I) is to be calculated. Each such probability is calculated as a ratio displayed as the numerator part divided by the denominator part. Mathematically, the probability measure of the present invention is given by:

[0107]

[Equation 1]

First term of the molecular part

[Equation 2] Denotes a series of multiplications of the r-scores along the arc of each path (S _i '), the total numerator part

(Equation 3) Indicates the addition of such path score products over all paths exhibiting the same string interpretation (i.e. consegmentation S ').

The first expression in the denominator part,

(Equation 4) Is the sum of path score products over all paths showing the same string interpretation, the total denominator part,

(Equation 5) Indicates the addition of all path score products over all character strings {C} indicated by the alignment graph.

Since the denominator part contains contributions from all possible interpretations, its value depends only on the acquired image I, not on the particular interpretation C. The purpose of the denominator part is
It is to ensure that the probabilities are properly normalized.
Thus, according to the general principle of probability, P (C _i /
The sum of I) (ie, all C _i ) is equal to 1.

When the numerator part is calculated for a particular string interpretation, the probability for this string interpretation is obtained by dividing its calculated denominator by the common denominator. There are a number of different ways in which the above probabilistic calculation procedure can be used to arrive at a "correct" string interpretation, for example by incorporating it into a larger procedure. An example of this method is shown in the flow chart of FIGS. Yet another method is shown in the flow chart of FIGS. These two methods are described in detail below.

The steps of the first character string interpretation procedure of the present invention are shown in the flow charts of FIGS. As shown in block A, the first step of this procedure is to compute the set of r-scores for each node along the i-th row in the graph to compute the i-th neural set shown in FIG. Is to use a computational network. Then, as shown in block B, the procedure uses the well-known Viterbi algorithm to identify the paths (as a series of codes indicating glue and recognition arcs) through the alignment graph with the maximum path score. use.

The processor then responds to this path.
Identifies a string interpretation. The path for this string path
The core is an approximation that is not a reliable measure by itself.
This score is discarded as it is nothing more. This path
Character string interpretation C indicated by _(V)Information that identifies the
For example, only the 5 character ZIP code 35733) is saved.
Be held.

The procedure then uses the well-known "forward algorithm" to compute the common denominator part D (I) of the probability measure for the identified string interpretation, as shown in block C of FIG. To do. This number is stored in the main data structure used to implement the alignment graph of FIG.

By using the forward algorithm, the exact value of the (over all paths) sum of products (along each path) of the denormalized r-scores for all possible string interpretations indicated by the alignment graph. Is obtained. Glue arcs contribute to the pass score only by their presence or absence. Scores can be assigned to glue arcs (as well as recognition arcs), and all such scores are factored into the product along each path.

After computing said common denominator part D (I), as shown in block D of FIG. 14, the correct string interpretation C already identified by the Viterbi algorithm is obtained.
_A "forward algorithm" is used to compute the numerator part N (C _(V) I ₎ of the probability measure of _(V) . Then, FIG.
Store this number in the main data structure used to implement the alignment graph of 2.

The forward algorithm accepts as input a code identifying the selected character string interpretation identified by the Viterbi algorithm, and this selected character string interpretation C
Generate as output the exact numerator value (i.e., restricted sum) of _(V) . The calculated numerator portion of the string interpretation is equal to the sum (of the entire path) of the denormalized r-score products along each path through the alignment graph showing the string interpretation C _(V) . During the calculation of this numerator part, the glue arcs are processed in the same way as during the calculation of the denominator part.

After the denominator and numerator parts have been calculated, as shown in block E of FIG. 14, the string interpretation C _(V)
An improved probability P (C _V / I) for is calculated.
This probability is then stored in the main data structure. Finally, as shown in block F of FIG. 15, the processor determines whether the calculated probability in block E is greater than a threshold.

If so, the processor determines that the character string interpretation selected by the Viterbi algorithm is the analyzed image I.
I'm sure it is the highest probability string interpretation for. Thereafter, in block G, the processor produces as output from the system both (i) a string interpretation (e.g., 35733) and (ii) an associated calculation probability. These two items (along with other information) can be used together as a basis for determining how to route mail.

At this stage of processing, there are a number of reasons why it may be desirable to make additional calculations to identify alternative high score interpretations. For example, it may be desirable to reliably identify the highest probability interpretation even if the probability assigned to C _{(V )} in block F is less than 0.5. In this case, a set of competing string interpretations are identified and the probabilities for each member of the set are calculated.

The present invention also provides multiple interpretations (and probabilities).
Can also be used as part of a larger system used in subsequent processing. In particular, it can also be excluded by an interpretation given a high probability according to the invention on the basis of the acquired pixel image, a later stage in a larger system. Therefore, another interpretation is needed. For this or other reasons, the alternative procedure shown in the flow charts of FIGS. 16 and 17 is used.

As shown in block A of FIG. 16, the first step in this procedure is to compute a series of r-scores for each node along the ith row in the graph, i
The second computational network is also used. Then, as shown in block B, the procedure uses a beam search algorithm to identify a relatively small set of paths through the alignment graph (as a series of codes indicating glue and recognition arcs). The set of competing string interpretations {C _j } corresponding to this set of paths is then identified.

As shown in block C of FIG. 16, the processor uses the well-known forward algorithm to calculate the denominator D (I). The denominator D (I) serves as the denominator part of the probability P (C _j I) for each interpretation C _j in the competitive interpretation set {Cj}. This number is stored in the main data structure. The forward algorithm gives the exact value of the (over the entire path) sum of the denormalized r-score products of the arcs along each path. In the case of the denominator part, the sum outlines all possible interpretations.

To compute the score for the identified interpretation, the processor uses a forward algorithm, as shown in block D, to compute the numerator portion N (C _j / I) of the probabilities of each competing string interpretation C _j. To calculate. These numbers are stored in the main data structure. The forward algorithm gives the exact value of the (over the entire path) sum of the denormalized r-score products of the arcs along each path.

The sum calculated by the forward algorithm is the sum of all paths. One path is the path identified by the beam search algorithm in block B. In fact, this pass produces the largest term in the sum. If the sum is considered to be properly approximated by its maximal term, then no forward algorithm needs to be performed to evaluate the molecule. The results of the beam search algorithm are satisfactory. This is called the "unary sum" approximation.

However, this sum is not necessarily properly approximated by its maximum term. Therefore, it is preferable to discard the score calculated by the beam search algorithm, retain the interpretation identified by the beam search algorithm, and evaluate the retained interpretation score using the forward algorithm.

It is generally not possible to calculate the molecule for all possible interpretations. This is why it is preferable to identify a relatively small set of interpretations in block B. This interpretation is expected to have a large probability due to the large molecule and its facts, due to its large "one-item" score.

The above description describes the operation of the system after it has been learned. Next, the learning mode of the system will be described.

For optimal performance, the string interpretation system of the present invention is provided with an automatic learning mode of operation that allows the system to automatically learn during one or more learning sessions. This mode of operation is described in detail below with reference to the system illustrated in block J of FIG. 2 and FIG.

As shown in block J of FIG. 2 and FIG. 13, the character string interpretation system of the present invention has a neural network parameter adjustment module 29. This module 29 is a graph of the system 3 shown in FIG.
It interacts with both 0 and the complex of the neural network 21. In general, the learning process of the present invention is
Based on the concept of supervised learning.

That is, for each image I ^* in the learning set, there is an interpretation C ^* assigned from the beginning. The neural network parameter adjustment module uses the predicted (ie, average) probability P (C ^* /
I ^* ) increases during the processing of all images I ^* in the training set, while the prediction probability P (C / C /
It is designed to ensure that I) drops during the learning process.

In summary, the purpose of the learning mode, and thus the purpose of the neural network parameter adjustment module, is to ensure that the probability of correct string interpretation C is maximized while minimizing the average probability of incorrect interpretation. Is. Since the log function has a steep slope near zero, log [P
(C / I)] is selected as the objective function.

This causes the learning process to emphasize low score pixel patterns (ie, image segments). This emphasis is preferable because these patterns are the most problematic and therefore the most learning-intensive. To implement the selected objective function, the processor uses the gradient of the function, shown below in Equation 6.

[0134]

(Equation 6)

In the above equation 6, W = W ₁ ,
W ₂ ,. . . , W _m are m-dimensional neural network weight vectors, and r _i = r ₁ , r ₂ , ..., R _n are i generated as outputs from the i-th neural network.
The th n-dimensional r-score vector. Typically,
The weight vector w has 10,000 or more components. The r-score vector has exactly 10 components for digit recognition. The dot product on the right side of the above gradient equation means the sum of all components of r.

In general, for each neural network, ie for each row in the alignment grid,
There is a gradient formula of this form. At times, it may be preferable to control more than one network with the same weight vector w. In this case, the gradient of w contains the contributions from each such network. As shown in FIG. 13, the weight vector is stored in the register 31. Register 31 provides the same weight vector to each and every neural network in the system.

Prior to starting the learning process of the multi-character recognizer described herein, the neural network weight vector must be initialized. It can be initialized with random values that follow some rational distribution, or it can be initialized with selected values that a priori appear to be particularly suitable.

In many cases, it is preferable to temporarily separate the neural network from the alignment graph and pretrain it on the manually fragmented image, as if it were used as a single character recognizer. . The obtained weight vector values serve as a starting point for the multi-character recognizer learning process described below.

The left side of the gradient equation is called the system sensitivity vector. Because it is a slope that gives information about the sensitivity of the output of the whole system with respect to changes in the weight vector w. Each component of the system sensitivity vector belongs to the corresponding component of the weight vector.

In particular, if the predetermined component of the system sensitivity vector is greater than zero, a small increase in the corresponding component of the weight vector increases the probability P (CI) that the system assigns to the interpretation C for the image I in question. In short, the system sensitivity vector can be used to optimize the above objective function.

To better understand the underlying principles of this learning process, it is useful to recognize the true value of the properties of the numbers that make up the gradient function.

According to the above formula, the system sensitivity vector is calculated as the dot product (vector matrix product) of two other numbers shown on the right side of the formula. Such a first number is the vector, ∂logP / ∂r.
This is the r-score r ₁ . . . Gives information on the sensitivity of the graph output for changes in r _n .

This can be regarded as a graph sensitivity vector. The second number is the N × N matrix ∂r / ∂W. This gives information about the sensitivity of the neural network output for changes in the weight vector controlling all neural networks.

The above-mentioned three types of items can be considered to be functionally interrelated as follows. During processing of each training image I ^* , the evaluated neural network sensitivity matrix is used to transform the evaluated graph sensitivity vector to produce an evaluated system sensitivity vector. The individual components of the evaluated system sensitivity vector then adjust the corresponding components of the weight vector, so that the objective function P (C ^* /
I ^* ) is optimized.

Theoretically, the system sensitivity vector can be obtained by numerically evaluating the items to the right of the gradient function and then performing the specified mathematical operation. However, during the learning session, each image / interpretation pair {I ^* , C ^* }
There is a simpler way to operatively evaluate the system sensitivity vector for.

As described below with respect to the flow charts of FIGS. 19 and 20, there is a well-known technique for evaluating system sensitivity vectors in a computer-efficient manner without the need to explicitly evaluate neural network sensitivity matrices. The back-propagation ("Back-Prop") algorithm of can be used.

When operating the system of the present invention in its learning mode, the learning process of FIGS.
This is done for each image I ^* in the learning set database, as shown in block k. Each image I ^* is
Associated with the known string interpretation C ^* . In general, a very large amount (eg, tens of thousands) of image / interpretation pairs {I ^* , C ^* } are used to train the system in the course of a particular learning session.

As shown in block B of the figure, each image I ^* is preprocessed in much the same way as is done during the interpretation process of the present invention. In addition, blocks C to E in FIG.
As shown in, the image segment and the image consegmentation are created for the image I ^* , respectively, in much the same way as done during the interpretation process of the present invention.

Then, as shown in block F of FIG. 2, a graph model is created for the generated image consegmentation and possible string interpretation associated with the image I ^* . At this stage of the learning process, the learning method of the present invention develops the following facts.

First, each probability P (C ^* / I ^* ) has a numerator part N (C ^* / I ^* ) and a common denominator part DI. Second,
Using the well-known properties of logarithm and derivative, the graph sensitivity vector (ie, log [P
The partial derivative of (C ^* / I ^* )] can be re-expressed by the following equation 7.

[0151]

(Equation 7)

The graph sensitivity vector, which is conspicuously shown on the left side of the equation, can be easily obtained by the procedure shown in FIGS. 19 and 20 described below.

As shown in block A of FIG. 19, the processor executes a forward propagation algorithm to compute the numerical value of the numerator part of the probability P (C ^* / I ^* ) for the image / interpretation pair {I ^* , C ^* }. Then, the numerical value of the denominator part is calculated. Then, this numerical value is stored. In this step of the process, the forward algorithm computes the graph created for the image / interpretation pair {I ^* , C ^* } with the associated probability P.
We develop the fact of implicitly showing the analytical (ie algebraic) formulas used to mathematically represent the numerator and denominator parts of (C ^* / I ^* ).

In block B of FIG. 19, the processor determines the numerical value of the partial derivative of the numerator part of the probability P (C ^* / I ^* ) with respect to the variable r by the well-known Baum-Welch algorithm. To execute. At block C, the processor uses a forward algorithm to calculate the value of the denominator part of the probability P (C ^* / I ^* ).

In block D, the processor determines the numerical value of the partial derivative of the denominator part of the probability P (C ^* / I ^* ) for the variable by the well known Baum-Welch method.
lch) Run the algorithm. Then, in block E of FIG. 20, the processor uses the numerically evaluated numerator and denominator parts and their partial derivatives to numerically determine the graph sensitivity vector according to the above equation.

To efficiently determine the numerical value of the system sensitivity vector for the image / interpretation pair {I ^* , C ^* }, see FIG.
The learning process sets the output layer gradient vector of each neural network equal to the corresponding component of the numerically determined graph sensitivity vector, as shown in block F of.

Then, in block G, the processor uses the backpropagation algorithm to calculate the components of the system sensitivity vector according to the above equation. A detailed description of the process of the backpropagation algorithm used to calculate the desired result is given in Denker et al., "Auto.
matic Learning, Rule Extraction, and Generalizatio
n ".

The back-propagation algorithm is not used to unambiguously evaluate the neural network sensitivity matrix, but rather to determine the vector-matrix product of the neural network sensitivity matrix and the graph sensitivity vector.

The result is a clear evaluation of the overall system sensitivity vector. This suggests a valid direction to update each of the components in the weight vector of each neural network. After processing each image I ^* , as shown in block H, the processor uses the individual components of the numerically determined system sensitivity vector to update the individual components of the weight vector. The preferred update procedure is described below.

Before updating, each i-th component of the weight vector is shown as W _i , and after updating each i-th component is shown as W _i ′. After processing each image I ^* , the weight vector is updated according to Equation 8 below.

[0161]

(Equation 8)

In Expression 8, δ _i is a “step size control parameter”, W _i ′ represents an updated weight vector, and ∂log (C ^* / I ^* ) / ∂W _i is a log relating to W _i. It is a partial derivative of (I ^* / C ^* ). In principle, there are many different step size control parameters δ _i for each component of the weight vector, but in practice it is preferable to set them all equally.

In general, the value of the step size control parameter is (i) the normalization factor selected for the pixel inputs to the neural network, and (ii) the median value of the neural network (ie, some layer in the neural network). To the next layer) and can be re-estimated during training, depending on the normalization factor chosen.

In summary, there are two main concerns when choosing an appropriate value for the step size control parameter. If the selected value for this control parameter is too small, the convergence of the weight vector w to its optimum value proceeds very slowly. On the other hand, if the selected value for this control parameter is too large, the learning process is very likely to skip over the optimal value of w. This phenomenon of the weight space W is called “oscillation branch”. This tends to reduce the overall quality of system performance and completely disrupts the learning procedure.

The above learning process is repeated for each image / interpretation pair {I ^* , C ^* } in the learning set. As more and more learning data is processed by the system in learning mode, the values of the individual components of the neural network weight vector converge towards an optimal value that satisfies the objective function governing the learning process of the present invention. There is no need to perform beam search or Viterbi algorithms during the learning process.

Once the learning process has generated a satisfactory weight vector, the system can perform its recognition and scoring tasks without further reference to the learning database. This means learning is “in the lab”
Recognition and scoring can be done in
means that you can do it in the field) ”.

The results obtained in the field do not require having a facility for storing learning databases or learning algorithms. In some cases, it may be desirable for the results obtained in the field to be re-learned or incrementally learned. In such cases, equipment may be needed to store the selected learning examples.

In particular, in the case of a "personal" recognizer as shown in FIG. 18, retraining the system maximizes system performance and is based on the implementation provided by this recognizer, either single-user or small. The specificity of the user group can be adapted.

When the method and system of the present invention is implemented in a portable handwriting recognizer, the bitmapped image of a word, number sequence, etc., as seen by the user, is stored in a non-volatile memory structure within the device. It is preferable to store it. The function of this memory structure is that the image / interpretation pair {I ^* ,
To store both bitmapped and ASCII formatted information corresponding to C ^* }. Over the life of the device, a training dataset is constructed from such collected information.

If the learning data set is of sufficient size, the portable device can be operated in its "learning mode". After each image / interpretation pair {I ^* , C ^* } has been reprocessed, the individual components of the weight vector are incrementally adjusted in such a way that the above objective function is performed.

Many types of additional embodiments of the present invention can also be readily constructed. For example, instead of a preprocessed image derived from the image information, the input to the system could be a preprocessed image derived from the pen stroke information or a list (not an image form) derived from the stroke information. . In another example, the input can also consist of pre-processing information derived from a voice signal (eg, speech).

Similarly, other forms of output can be realized. Output symbols are not only numbers, but also alphabet letters,
Phonemes, whole words, ellipsis or groups of these can also be indicated. It is easy to imagine applications such as decoding and error correction coded symbols transmitted over high noise communication channels.

In another embodiment, the function performed by the neural network complex is (1) capable of receiving input and (2) interpreted as a score or vector of scores, according to a set of parameters. Possible outputs can be generated, and (3) based on a given derivative vector, performed by a device that can adjust the parameters in a way that changes the output in the direction specified by the derivative vector.

The functions performed by the "alignment graph" can be performed by a conventional dynamic programming grid or a device that processes a series of information in the required manner. This method is specifically (1) receiving scores describing various entities that are part of the sequence, (2) efficiently identifying various high scoring sequences and corresponding interpretations, and (3) predetermined It consists of efficiently computing the total score for all sequences that match the interpretation, and (4) efficiently computing the sensitivity of the result to the input score.

Also, the number of modules in the processing chain can be two or more. Each module has (i) a sensitivity output (if the preceding module has adjustable parameters), (ii) a sensitivity input (if this module or the preceding module has adjustable parameters), and (iii) normal data input and data. Must have an output.

The probabilities just described need not be indicated in the processor and memory by numbers between zero or one. For example, it is preferable to store this probability as a logarithmic probability in the range between a slightly larger negative number and zero, and adapt the calculation steps describing the series and parallel combinations of probabilities.

The systems and methods of this invention can be used to interpret input symbolic representations. Such input symbolic representations can be used for a variety of different media {eg, electrically passive (graphic) recording media such as paper, wood, glass, pressure sensitive writing surface and touch screen writing and display surface. Such as electrically active recording media, audio recording media such as human voice and machine-generated voice, and media such as air (where pen strokes waving in air are, for example, RF position sensing, Coded by an electrically active contactless method such as optical position sensing, capacitive position sensing, etc.), and then transmitted, stored and using the system and method of the present invention. / Or be recognized. In such an application, the sequence of symbols need not be displayed graphically on the surface, but merely be displayed.

The system and method of the present invention can also be used in conventional voice recognition systems. Such applications include, for example:
The input data set is the recorded voice pronunciation (ie, voice signal) shown in the time domain. According to the invention, the recorded voice pronunciation is divided into small voice samples (e.g. voice cells), each of which has a very short duration. Each voice cell is preprocessed and divided into velocity cells.

The voice cells are then combined to produce a "voice segment". Each segment contains spectral information that indicates at least one phoneme in the phonetic pronunciation.
These speech segments are then combined to produce the consegmentation shown using the acyclic graph of the present invention. Then, using this consegmentation and the set of all possible phoneme string interpretations, the system and method of the present invention begins to compute the a posteriori probability for the highest scoring phoneme string interpretation. The details of this speech recognition process will be apparent to those skilled in the art of speech recognition.

[0180]

As described above, according to the present invention,
Superior methods and systems are provided for interpreting input symbolic representations such as strings displayed or recorded on media by printing or cursive writing techniques. According to the present invention, the recursive probabilities are used to select the best string interpretation, and each recursive probability is
It can be derived a posteriori by combining a priori information with a pixel image of a known example, and a character string of arbitrary length can be accurately interpreted.

[Brief description of drawings]

FIG. 1 is a system block diagram illustrating various components used to implement a string interpretation system according to an example of the present invention.

FIG. 2 is a block diagram of a character string interpretation system of the present invention.

FIG. 3 is a diagram of a preprocessed image of a handwritten ZIP code using a cursive writing technique.

FIG. 4 is a diagram of a pre-processed image of the ZIP code in FIG. 3 with a series of overlay cut lines generated during the image cell generation stage of the character string interpretation method of the present invention.

FIG. 5 is a diagram of a preprocessed image of the ZIP code in FIG. 3, having a series of overlay cutlines generated during the image cell generation stage of the character string interpretation method of the present invention.

FIG. 6 is a diagram of a pre-processed image of the ZIP code in FIG. 3 with a series of overlay cut lines generated during the image cell generation stage of the character string interpretation method of the present invention.

7 is a diagram of a pre-processed image of the ZIP code in FIG. 3 having a series of overlay cutlines generated during the image cell generation stage of the character string interpretation method of the present invention.

FIG. 8 is a diagram of a pre-processed image of the ZIP code in FIG. 3 with a series of overlay cut lines generated during the image cell generation stage of the character string interpretation method of the present invention.

9 is a table of image "cells" (ie, sub-images) generated between the cut lines shown in FIGS. 4-8.

10 is a table of image "segments" produced by combining adjacent image cells shown in FIGS. 4-9. FIG.

11 is a table showing three types of concatenation of the number of legal image “consegmentation” generated by the combined set of spatially continuous image segments shown in FIG.

FIG. 12 graphically illustrates the novel data structure of the present invention used to graphically represent image segments, possible image consegmentation generated therefrom, possible string interpretations and scores assigned to possible string interpretations. It is a schematic diagram shown in.

FIG. 13: ZIP analyzed into 11 image segments
FIG. 3 is a schematic diagram of a character string interpretation system of the present invention that is adapted to recognize a code image.

FIG. 14 is a high level flow chart illustrating the steps performed in a method of interpreting a string according to the present invention, which is integrally combined with FIG. 15 below.

FIG. 15 is a high level flow chart illustrating the steps performed in a method of interpreting a string according to the present invention, which is integrally combined with FIG. 14 above.

FIG. 16 is a high level flow chart illustrating the steps performed in a method of interpreting a string according to the present invention, which is integrally combined with FIG. 17 below.

17 is a high level flow chart illustrating the steps performed in a method of interpreting a string according to the present invention, which is integrally combined with FIG. 16 above.

FIG. 18 is a schematic perspective view of a handheld type character string interpretation system of the present invention.

FIG. 19 is a high level flow chart illustrating steps performed in a method of training a string interpretation system of the present invention, which is integrally combined with FIG. 20 below.

FIG. 20 is a high level flow chart illustrating the steps performed in a method for training a string interpretation system of the present invention, which is integrally combined with FIG. 19 above.

[Explanation of symbols]

 1 Symbol Sequence Interpretation System of the Present Invention 2 Processor 3 Program Storage Memory 4 Data Storage Memory 5 Image Acquisition Device 7 Frame Buffer 8 Mass Storage Memory 9 Visible Display 10 Keyboard 11 Pointing Device (Mouse) 12 Input / Output Device 13 System Interface 14 Host system 15 System bus

Front Page Continuation (72) Inventor John Steward Denker USA, 07737 New Jersey, Leonardo, Coosman Drive 6

Claims (34)

[Claims] 1. A system for analyzing an input symbol representation and scoring possible interpretations of the input symbol representation, analyzing an input data set representing the input symbol representation, and dividing the input data set into a plurality of segments. Segment generating means, wherein each segment has a definable boundary and can be classified as indicating any one of a plurality of symbols within a given symbol set,
Segment scoring means for analyzing each segment in the plurality of segments and assigning a score to each possible classification of the segment associated with a particular symbol in the predetermined symbol set; and a plurality of possible interpretations of the input symbol representation. And a display means showing a plurality of image concatenations, wherein each possible interpretation consists of a different sequence of the symbols and each consegmentation consists of a different sequence of the segments, based on a score assigned to the segment, Consegmentation scoring means for assigning scores to the plurality of consegmentations; candidate interpretation identifying means for identifying one or more candidate symbol interpretations from the plurality of possible interpretations based on the scores assigned to the plurality of segments; The plurality of segments A symbol sequence scoring means for assigning a score to the one or more candidate interpretations based on a score assigned to one or more of them, and a first score for evaluating the score assigned to the one or more candidate interpretations Evaluating means, second score evaluating means for evaluating the scores assigned to the plurality of candidate interpretations, and normalized score generation for generating a normalized score for each candidate interpretation using the evaluation scores for the plurality of possible interpretations. And a scoring system for possible interpretation of the input symbolic representation, characterized by comprising: 2. The input data set comprises a series of pixels associated with a captured image of a graphically represented symbol sequence, the segment generating means analyzing the group of pixels to define the series of pixels as a plurality of pixels. Split into image segments,
Thereby, each of the image segments has a specified boundary and can be classified as indicating any one or more characters of the plurality of characters in a given character set. Item 1 system. 3. The segment scoring means analyzes each image segment in the plurality of image segments,
The system of claim 2, then assigning a score to each possible classification of the image segment, each assigned score being associated with a particular character within the predetermined character set. 4. The system of claim 3, wherein said display means indicates a plurality of character sequences and a plurality of image consegmentations, each possible character sequence comprises said character sequence, and each said concatenation comprises said image segment sequence. . 5. The consegmentation scoring means assigns scores to the plurality of image segmentations based on the scores assigned to the image segments, and the candidate symbol sequence identifying means assigns scores to the image segments. 5. The system of claim 4, based on which one or more candidate character sequences are identified. 6. The symbol sequence scoring means assigns scores to the one or more candidate character sequences based on the scores assigned to the image segments, and the first score evaluating means includes the one or more candidates. The system of claim 5, wherein the score assigned to the character sequence is evaluated. 7. The second score evaluation means evaluates the scores assigned to the plurality of possible character sequences, and the score normalization means uses the evaluation scores for the plurality of possible character sequences to obtain the candidate characters. 7. The system of claim 6, normalizing the scores assigned to the sequences. 8. The display means is arranged in columns and rows,
And a data structure that can be displayed by a graph consisting of a two-dimensional node array selectively connected by directed arcs, each node column is indicated by one character position, and each node row is the acquired image. Indexed by one said image segment in an order corresponding to the spatial structure of
Each path that passes through the node and that extends along the directed arc represents one of the image consegmentation and one of the possible character sequences, and approximately all of the image consegmentation and approximately all of the possible character sequences. 8. The system of claim 7, indicated by a series of paths extending through the graph. 9. The system of claim 8, wherein each node further comprises a series of recognition arcs, each recognition arc indicating one of the letters and associated with one of the assignment scores. 10. The system of claim 1, wherein said display means is indicative of said plurality of possible interpretations and said plurality of image consegmentations. 11. The display means comprises a data structure which can be displayed by a graph which is arranged in columns and rows and which is composed of a two-dimensional node array which is selectively connected by directed arcs. Pointed to by one symbolic position, each said node row being pointed by one said segment in an order generally corresponding to the sequential structure of said input data set, passing through said node and along said directed arc. Each extending path represents one said segmentation and one said possible interpretation of said input symbolic representation, wherein substantially all said consegmentation and substantially all said possible interpretations are represented by a series of paths extending through said graph. 10 systems. 12. The display means has a data structure that can be displayed by a graph that is arranged in columns and rows and that is composed of a two-dimensional node array that is selectively connected by directed arcs. Pointed to by one symbolic position, each said node row being pointed by one said segment in an order generally corresponding to the sequential structure of said input data set, passing through said node and along said directed arc. 3. Each path extending represents one said segmentation and one said possible interpretation for said input symbolic representation, all said consegmentations and all said possible interpretations being indicated by a series of paths extending through said graph. system. 13. A method of generating an interpretation of an input symbolic representation, the input symbolic representation being represented in a medium, the interpretation being a sequence of symbols, each symbol being an element within a predetermined symbol set, The method comprises: (a) obtaining an input data set representing the input symbolic representation, and (b) processing the input data set to generate a series of segments, where:
The segment is at least a partial subset of the acquired input data set and can be classified as indicating any one symbol in the predetermined symbol set,
(c) generating a data structure showing a set of concatenations and a set of possible interpretations for the input symbolic representations, where each said concatenation is
Collectively showing the input datasets, consisting of the segments arranged in an order that generally preserves the sequential structure of the input datasets, each possible interpretation of the input symbolic representations consisting of possible symbol sequences, Each symbol in the possible symbol sequence is selected from a predetermined symbol set and occupies a symbol position in the possible symbol sequence, the data structure being arranged in columns and rows, selectively linked by directed arcs. Graphically represented by a graph consisting of a two-dimensional array of nodes, each column of nodes can be pointed to by one of the symbolic positions, and each row of nodes can be represented in an order corresponding to the logical structure of the acquired input data set. Each path that can be pointed to by one of the image segments and that extends through the node and along the directed arc is One said possible segmentation and one said possible interpretation for said input symbolic representation, all said segmentation and all said possible interpretations for said input symbolic representation are indicated by a series of paths extending in said graph, (d ) Generating, for each node row in the graph, a series of scores for the predetermined symbol set represented by each node in the row, wherein generating the series of scores produces the series of scores. (E) implicitly or explicitly assigning a path score to a path through the graph, including (f) analyzing one or more possible interpretations of the input symbolic representation, including analysis of segments pointing to the node rows Analyzing the path scores attributed to the paths passing through the graph in step (e) to make a selection; Method of generating an interpretation of the input expression, characterized by comprising. 14. Each node further comprises a series of recognition arcs, each recognition arc representing one of the characters, and
14. The method of claim 13, associated with one of the scores generated in (d). 15. The method of claim 14, wherein step (d) comprises using a plurality of adjustable parameters to generate the series of scores. 16. The information processing means characterized by said plurality of adjustable parameters is used in step (d) for analyzing each said segment and for generating said score group for this segment. Item 15 method. 17. Step (f) comprises computing a quantity corresponding to an inductive probability for at least one of the possible interpretations of the input symbolic representation, where:
Each said quantity is calculated as the ratio of the numerator part to the denominator part, the numerator part corresponding to the sum of the path scores for almost all paths through the graph showing one said possible interpretation of the said input symbolic representation, each said The path score corresponds to the product of the scores associated with the recognition arcs along one of the paths, and the denominator part is the path score for almost all paths through the graph showing almost all of the possible interpretations for the input symbolic representation. 15. The method of claim 14 corresponding to the sum of the scores, each path score corresponding to the product of the scores associated with the recognition arcs along one of the paths. 18. In step (f), (1) determining the path through the graph having the highest path score, and (2) relating to the input symbolic representation represented by the path determined in substep (1). Identify possible interpretations, (3) calculate the quantity for the possible interpretations for the input symbolic representation identified in substep (2), and (4) calculate the quantity and substeps calculated in substep (3). 2)
18. Providing as an output an indication indicating the possible interpretation of the input symbolic representation identified in.
the method of. 19. In step (f), further comprising: (1) determining a series of paths through the graph having a high series of path scores, and (2) the series of paths determined in substep (1). Identifying a set of possible interpretations for the input symbolic representation indicated by the path, and (3) calculating a set of said quantities for said set of possible interpretations for the input symbolic representation identified in substep (2), Analyzing the set of quantities calculated in sub-step (3) to determine which of the possible interpretations of the input symbolic representation have a high scoring recursive probability, and The method of claim 17, comprising providing as outputs possible indications for the input symbolic representation identified in substep (2) and indications of high scoring recursive probabilities determined in substep (4). 20. Each of the inductive probabilities is calculated as a ratio of the numerator part to the denominator part, and step (f) further comprises (1) determining a series of paths through the graph having a high series of path scores. , (2) identifying a set of possible interpretations for the input symbolic representation represented by the sequence of paths determined in substep (1), and (3) relating the input symbolic representation identified in substep (2). Compute a series of the quantities for the set of possible interpretations, and (4) output a series of possible interpretations for the input symbolic representation identified in substep (2) and the quantity calculated in substep (3). 18. The method of claim 17, comprising: 21. In step (d), the set of adjustable parameters is between the segment supplied to the information processing means for analysis and the series of scores generated from the information processing means. 16. The method of claim 15, wherein the relationship is specified. 22. (1) processing a large number of known symbol sequences using said information processing means, and (2) incrementally adjusting said series of adjustable parameters for each known sequence, whereby 22. The method of claim 21, comprising training the information processing means by averaging the probabilities assigned to the correct interpretations and decreasing the probabilities assigned to the incorrect interpretations. 23. The method of claim 22, wherein said information processing means comprises a neural information processing network. 24. The method of claim 13, wherein the input symbolic representation is displayed using a printing or cursive writing technique and is graphically recorded on a recording medium. 25. A system for generating an interpretation of an input symbolic representation, the input symbolic representation being represented in a medium, the interpretation being a symbol sequence, each symbol being an element within a predetermined symbol set, The system comprises: (a) a dataset acquisition means for obtaining an input dataset indicative of the input symbolic representation; and (b) a data processing means for processing the obtained dataset to generate a plurality of segments, wherein: Each segment has a specifiable boundary and can be classified as indicating any one of a plurality of symbols within a given symbol set,
(c) A segmentation specifying means for generating data specifying a series of consegmentation, and here,
Each said segmentation collectively represents said acquired input data and consists of a series of said segments arranged in an order generally preserving the sequential structure of said acquired input data set, and (d) a series of said input symbolic representations. Symbol sequence interpretation designating means for generating data designating possible interpretations, wherein each possible interpretation of the input symbolic representation comprises a possible sequence of symbols, wherein each said symbol in said possible sequence of symbols is said predetermined Storing in a data structure selected data from a set of symbols and occupying a symbol position within said possible sequence of symbols, (e) generating each said consegmentation and said each possible interpretation of said input symbolic representation. The data storage means and the data structure are arranged in columns and rows and selectively connected by directed arcs. Graphically represented by a graph consisting of a two-dimensional array of nodes, each column of nodes can be pointed to by one of the symbolic positions, and each node row is in an order corresponding to the sequential structure of the acquired input data set. Each path that can be pointed to by one of the image segments and that extends through the node and along the directed arc represents one of the series of concatenations and one of the possible interpretations of the input symbol representation, The set of concatenations and the set of possible interpretations for the input symbolic representations are indicated by a set of paths extending through the graph, (f) analyzing the data in each segment, and each row of nodes in the graph. , Generate a series of scores for the symbol set represented by each node in the row And segment analysis means that a path score calculating means for calculating a path score for each path through the (g) the graph, and
(h) An interpretation generating system for an input symbolic expression, comprising: a path score analyzing means for analyzing a calculation path score in order to select one or more possible interpretations of the input symbolic expression. 26. The system of claim 25, wherein each node further comprises a series of recognition arcs, each recognition arc representing one of the known symbols and associated with one of the calculated scores. 27. The system of claim 26, wherein said pass score analysis means further comprises means for calculating a quantity corresponding to an inductive probability of each of said possible interpretations of said input symbolic representation. 28. Each said quantity is calculated as a ratio of a numerator part to a denominator part, the numerator part being the sum of path scores for almost all paths through a graph showing one said possible interpretation of said input symbolic representation. Correspondingly, each path score corresponds to the product of the scores associated with the recognition arcs along one of the paths, and the denominator part is approximately all that passes through the graph showing almost all the possible interpretations of the input symbolic representation 28. The system of claim 27, which corresponds to the sum of the path scores for the paths, each path score corresponding to the product of the scores associated with the recognition arcs along one of the paths. 29. (1) means for determining a path through the graph having the highest path score; and (2) means for identifying possible interpretations for the input symbolic representations indicated by the determined path having the highest path score. And (3) means for calculating the quantity of each of the possible interpretations of the input symbolic representation, and (4) means for supplying as output an indication indicating the calculated quantity and the possible interpretations of the input symbolic representation. The system of claim 25. 30. The path score analysis means is represented by (1) means for determining a series of paths through the graph having the highest series of path scores, and (2) the determined series of paths. Means for identifying a set of possible interpretations for the input symbolic representation, (3) means for calculating the series of quantities for the series of identifiable interpretations for the input symbolic representation, and (4) the calculated series of quantities. And (5) determining which of the possible interpretations of the input symbolic representation has the highest inductive probability of the highest set of path scores, and (5) the input having the highest inductive probability. 30. The system of claim 29, further comprising means for providing, as an output, an indication of the possible interpretations of symbolic representations and the determined highest a posteriori probability. 31. The segment analysis means includes the segment supplied to the information processing means for analysis,
28. The system of claim 27, comprising a set of the adjustable parameters that specify a relationship between the set of scores generated from the information processing means. 32. A system learning means for training the system using a plurality of learning data sets, wherein the learning data set includes an acquisition data set of an input symbol representation and a known and correct interpretation of the input symbol representation. Including the system learning means to increase the average interpretation measure for the known-correct interpretations and decrease the average interpretation measure for the set of incorrect-known interpretations. 32. The system of claim 31, further comprising parameter adjustment means for incrementally adjusting the adjustable parameter of. 33. A system for generating an interpretation of an input symbolic representation, the input symbolic representation being represented in a medium, the interpretation being a sequence of symbols, each symbol being an element within a predetermined symbol set, The system comprises (a) image acquisition means for acquiring an image of the input symbolic representation, and (b) image processing means for processing the image to generate a series of image segments, wherein the image segment is the A sub-image of the acquired image, (c) image concatenation specifying means for generating data specifying a series of image concatenations, wherein each of the image consegmentation indicates the acquired image collectively, Consisting of a series of said image segments arranged in an order that preserves the spatial structure of the acquired image, (d)
Symbol sequence interpretation designating means for generating data designating a series of possible interpretations of the input symbolic representation, wherein each possible interpretation of the input symbolic representation consists of a possible sequence of symbols A symbol is selected from the predetermined symbol set and occupies a symbol position within the symbol sequence, (e) a data structure of each of the image consegmentation and generated data indicating each of the possible interpretations of the input symbolic representation. Data storage means for storing therein, said data structure being graphically represented by a directed acyclic graph consisting of a two-dimensional array of nodes arranged in columns and rows, selectively connected by directed arcs, Each of the node columns can be pointed to by one of the symbol positions, and each of the node rows is in an order corresponding to the spatial structure of the acquired image. Each path that can be pointed to by one of the image segments and that extends through the node and along the directed arc represents one of the image consegmentation and one of the possible interpretations of the input symbolic representation. All of the consegmentation and all of the possible interpretations of the input symbolic representation are represented by a series of paths extending through the graph,
(f) image segment analysis means for analyzing each of the image segments and generating, for each row of nodes in the graph, a series of scores for the predetermined symbol set represented by each node in the row, (g) Path score calculating means for calculating a path score for each path passing through the graph, and (h) path score analyzing means for analyzing the calculated path score to select one or more of the possible interpretations for the input symbolic representation. An interpretation and generation system for input symbolic representations, which consists of and. 34. A system for generating an interpretation of an input symbolic representation, the input symbolic representation being represented in a medium, the interpretation being a sequence of symbols, each symbol being an element within a predetermined symbol set, The system comprises: (a) a means for providing an input data set and a confirmed symbol sequence for each of a plurality of known input symbol representations; and (b) analyzing each input data set to generate a plurality of said input data sets. Segment generation means for dividing into a plurality of segments, wherein the segment has a definable boundary and can be classified as indicating any one of a plurality of symbols in the predetermined symbol set, (c) analyze each segment by one or more adjustable parameters, and a set of scores such that the segmentation depends on the one or more adjustable parameters. Segment scoring means characterized by means assigned to each of the possible interpretations of each, where each score in each of the assigned series of scores is associated with a particular symbol in the predetermined symbol set, (d) display means showing a plurality of possible symbol sequences and a plurality of image consegmentations, wherein each possible symbol sequence comprises a different sequence of the symbols and each consegmentation comprises a different sequence of the segments, (e) Consegmentation scoring means for assigning scores to the plurality of consegmentations based on the scores assigned to the segments; and (f)
Symbol sequence scoring means for assigning a score to each of said confirmed symbol sequences based on a score assigned to one or more of said plurality of consegmentations; and (g) assigned to said confirmed symbol sequences. A first score evaluating means for evaluating the score,
(h) second score evaluation means for evaluating the scores assigned to the plurality of possible symbol sequences, and (i) a normalized score for each confirmed symbol sequence using the evaluation scores for the plurality of possible interpretations. A normalized score generation means for generating, (j) a sensitivity estimation means for estimating the sensitivity of the generated normalized score with respect to the one or more adjustable parameters, and (k) an average probability that each segment is accurately classified. And adjusting the one or more adjustable parameters to reduce the average probability of each segment being incorrectly classified. Interpretation generation system.