WO2002095667A1

WO2002095667A1 - Method for recognition of a hand-written pattern

Info

Publication number: WO2002095667A1
Application number: PCT/SE2002/000914
Authority: WO
Inventors: Frederik Mattisson; Kalle ÅSTRÖM
Original assignee: Decuma Ab
Priority date: 2001-05-23
Filing date: 2002-05-15
Publication date: 2002-11-28
Also published as: SE0101826D0; SE519380C2; SE0101826L; TW540005B

Abstract

A method for creating a template for hand-written pattern recognition comprises the steps of detecting (2) a plurality of variants of one hand-written pattern and extracting (4) individual features of the hand-written pattern. The method further comprises the steps of identifying (6) variations among the individual features in the variants of the hand-written pattern. The variations comprise variations of combinations of features in the variants of the hand-written pattern. The method further comprises the step of creating (8) the template for the hand-written pattern by establishing a cost function, said cost function weighting the individual features in accordance with the variations among the individual features. A device for hand-written pattern recognition comprises a detection means for detecting a hand-written pattern, a digitizer for transforming the hand-written pattern into a digital representation, and a reference database of templates for different hand-written patterns. The templates have been created according to the method for creating templates. The device further comprises a comparison means for comparing the hand-written pattern to templates of the reference database.

Description

METHOD FOR RECOGNITION OF A HANDWRITTEN PATTERN

Technical Field

The present invention relates to a method for creating a template for recognition of a handwritten pattern and a use of the templates created by the method for signature verification.

The invention also relates to a method for recognition of a handwritten pattern and a method for signature verification. Additionally, the invention relates to a device for handwritten pattern recognition.

Background of the Invention

Today, there is an increasing need for recognition of handwriting. Portable data handling devices, such as PDAs (Personal Digital Assistants) , mobile phones and portable computers, are becoming increasingly popular. In order to enter data into these portable devices, a text insertion unit is needed.

Text insertion units have formerly been implemented as keyboards. This, however, sets limits to the minimum size of the portable device, and therefore a different way of inserting text into the device is needed in order to enable smaller devices.

In such devices, the keyboards can now be replaced by some kind of device for recognition of handwritten text. A common solution is to arrange a pressure- sensitive area where a user can write characters . The characters are then interpreted and handled by the portable device. It is, of course, also of interest to replace keyboards of regular stationary computers. Thus, recognition of handwritten characters is an important task in portable and stationary devices. The recognition of a handwritten character is based on the fact that the user writes the character in a certain manner. The handwriting must not be untidy, since this would lead to an incorrect recognition. The user will have to learn to write in a common manner and fit his handwriting to the recognition device to enable the device to recognize the handwriting. Another method is based on teaching the recognition device to recognize the handwriting of a user. However, the teaching procedure takes a long time and tries the patience of the user.

In EP 689 153 a method for recognizing a handwritten character is disclosed. Features of the handwritten character are extracted and compared to reference features of reference characters. Different features could be weighted differently in order to enable more correct recognition. However, the weighting could suit certain characters better than others, which implies that some characters are hard to recognize.

Another aspect of handwriting recognition is signature verification. Signatures are often used for personal identification purposes. It's used every day in connec- tion with usage of credit cards, and without any control it is very easy to withdraw money from someone else's account. The need for an automatic signature verification system is therefore very great.

The signature verification puts some other demands on the recognition. A forgery must not be accepted, whereas a correct signature should always be accepted. On the other hand, the recognition device will often know which signature is supposed to be entered. The entered signature will therefore not have to be compared to several references.

A method for verifying a person's identity is disclosed in Yamazaki, Y. and Komatsu, N. , Extraction of Personal Features from On-Line Handwriting Information in Context-Independent Characters, IEICE Trans. Fundamen- tals, Vol. E83-A, No. 10, October 2000, p. 1955-1962. Here, a person's identity is decided by identifying specific characteristics of the handwriting of different persons. The identification is performed by extracting text-independent personal features from ordinary characters. Each writer is assigned different weighting factors for different features in order to enable distinction between different writers. This method requires that writers can be separated by means of text-independent features, which implies that features such as pen- pressure or pen-azimuth angle must be collected. These features are not as easily detected as features corres- ponding to the shape of the character. Furthermore, the method is used for identifying people when writing ordinary text and not while writing their specific signature. The text-independent personal features of a writer are not necessarily the same when the writer writes ordinary text as when he writes his signature.

Summary of the Invention

An object of the invention is to enable an improvement of a method for recognition of handwritten patterns. A special object of the invention is to provide an improved method for signature verification.

The objects of the invention are achieved by a method for creating a template according to claim 1, a method for recognition of a handwritten pattern according to claim 6, a method for signature verification according to claim 13 and a device for recognition of a handwritten pattern according to claim 14. Specific advantages of the methods can be derived from the dependent claims 2-5 and 7-12. Thus, a method for creating a template for recognition of a handwritten pattern according to the invention comprises the steps of detecting a plurality of variants of the handwritten pattern and extracting individual features of the handwritten pattern. The method further comprises the step of identifying variations among the individual features in the variants of the handwritten pattern. The variations comprise varia- tions of combinations of features in the variants of the handwritten pattern. The method further comprises the step of creating the template for the handwritten pattern by establishing a cost function, said cost function weighting the individual features in accordance with the variations among the individual features.

This method for creating a template could be implemented in a method for recognition of a handwritten pattern according to claim 6 or in a method for signature verification according to claim 13.

The invention is based on the idea that variations that normally occur for a character should be allowed to occur. These variations are hard to relate to one feature and in many cases the variations are related to the interrelations of features. According to the invention, the cost function of a template will allow normal variations, such as the vertical positioning of the strokes relative to each other in a t', by assigning a small weight factor to this variation. On the other hand, features and combinations of features that normally are stable and have a low variation are assigned a large weight factor, such as the angle between the strokes in the ^Λt' .

Thanks to the invention, the features of each template are weighted in the cost function according to the normal variations of the features in the handwritten pattern corresponding to the template. This implies that the cost function of a template is tailored for the specific pattern to which it corresponds. As a result, the recognition of the handwritten patterns could be improved and a user will not have to adjust his handwriting in order for it to be correctly recognized by means of templates in a recognition device.

In the context of this application, a template comprises information of what a pattern should look like. Thus, a template holds the information that is needed for calculation of a cost function. This information could be both mean values for the variants and variations in the variants .

When the invention is used for signature verification, the method comprises the steps of creating templates for different signatures. The step of creating a template for a signature comprises the steps of detecting a plurality of variants of the signature and extracting individual features of the signature. The step of creating a template further comprises the steps of identifying variations among the individual features in the variants of the signature, and creating the template for the signature by establishing a cost function. The cost function weights the individual features in accordance with the variations among the individual features. The method further comprises the steps of detecting a signature, which is to be verified, comparing the signature to the templates using the cost functions of the templates, and verifying the signature as a signature corresponding to a template if the cost function of that template is within a predefined interval.

Here, the features in the cost function should be weighted in such a manner that it is impossible to forge the signature, while the natural variations of a_. person's signature should be allowed. This could be achieved by means of the invention, since the natural variations of a person' s signature are recorded when the template is created and could thus be assigned low weighting factors. It is, of course, very hard to ensure that a good copy of the signature will be refused. However, by combining the shape features of the signature with, for example, a time feature, i.e. the time taken to write the signature, a very secure verification could be achieved. Thus, the invention could accept the natural variations of a person's signature without decreasing the security against forgeries.

Preferably, the step of identifying variations among the features is performed by means of a principal compo- nent analysis. Through principal component analysis the variations between the variants of a pattern are analyzed. The combinations of features that correspond to the largest contributions to the variations are identi- fied. Thus, the combinations of features that should be allowed to vary could easily be found. These combinations could be assigned low weighting factors in order to allow these combinations to vary, while combinations of features that have low or no variation are assigned high weighting factors.

• According to a preferred embodiment, the method further comprises the step of creating several cost functions of the template for different types of features. As a result, the cost functions could consider different sets of features. Thus, a handwritten pattern could be required to comply with the template according to each set of features in order to be accepted. This is especially useful for signature verification, since the chance to expose a forger increases . In recognition of a pattern, the step of comparing preferably comprises the step of calculating a value of the cost function of a template for the handwritten pattern. The calculated value of the cost function gives an indication whether the handwritten pattern corresponds to the template. Preferably, the method further comprises the step of comparing the calculated value of the cost function to a threshold value, below which the handwritten pattern is accepted as corresponding to the template. Thus, the comparison of the calculated value to the threshold value immediately gives an answer to whether the pattern should be identified as the template or not .

In a preferred embodiment, the step of comparing the handwritten pattern to the templates uses all the cost functions of each template and the handwritten pattern is only accepted as corresponding to the template if the value of each one of the cost functions is below the threshold value. Usage of all cost functions of a template gives a higher security for correctly recognizing a pattern.

Brief Description of the Drawings

A presently preferred embodiment will now be described by way of example only with reference to the accompanying drawings, on which

Fig. 1 is a flow chart of a method for creating a template for handwritten pattern recognition according to the invention,

Fig. 2 is a flow chart of a method for handwritten pattern recognition using the template created according to the method shown in Fig. 1, and Fig. 3 is a schematic view of a device for handwritten pattern recognition according to the invention.

Detailed Description of a Preferred Embodiment of the Invention Referring to Figs 1-2, a method for handwritten pattern recognition will now be described. The handwritten pattern recognition is based on two procedures, i.e. the procedure of creating templates that the handwritten pattern should be compared to and the procedure of comparing the handwritten pattern to the templates. Now referring to Fig. 1, the implementation of the procedure of creating templates will be described. First, the pattern to which the template should correspond is written by hand several times and detected, step 2, on a device for recognition of handwritten patterns. The handwriting of the pattern several times creates a plurality of variants of the pattern, since two handwritten patterns are never exactly identical . The variants of the pattern hold information of how the pattern normally varies. Features of the variants are extracted, step 4, and analyzed, step 6. The analysis aims to find the features or combinations of features that cause the largest variations. Thus, a principal component analysis is performed. The result of the principal component analysis is used for creation of the template by establishing the cost function, step 8. The templates are stored in a database, step 10, for use in recognition of handwritten patterns .

The analysis of the variants using the principal component analysis and the creation of the cost function will now be described in detail . As input to the ana- lysis, n variants of the pattern are entered. Then, the analysis should estimate how N features, f_lk, f kι —, fm. vary for the pattern k. The mean value for feature i is

A stochastic variable corresponding to the observations of feature i is denoted F . Further, a stochastic variable vector

is created.

The mean feature vector is

where E [Z] is the expectation of Z . The covariance matrix C is the symmetric N x N matrix containing all covarian- ces between all N features in all n patterns, and is defined by

Next, the covariance matrix is transformed into a new basis defined by the eigenvectors of the matrix. The eigenvectors represent the main directions of the statistical variations among the features. The one corresponding to the largest eigenvalue represents the largest variation.

As the variations have been analyzed, the cost function can be defined. The cost function is defined as

r = y^TCr^ly, y = (z-m) ,

where z represents the handwritten pattern that is to be recognized and is a vector of the selected features, m is the corresponding mean feature vector for the variants of the pattern and C is the covariance matrix. A problem arises if the rang r of the matrix C is smaller than the number of features N, since the new basis will not span

R N This will occur if the number of variants used is less than the number of features that are extracted. In the new basis, C will then be a semi-diagonal, and hence non-invertable, matrix, which has the interpretation in the new basis that N-r dimensions have collapsed. The covariance matrix could be expressed as

Thus, the covariance matrix needs to be extended to a diagonal matrix. This is accomplished by setting the rest of the singular values to a value σ , which gives

The diagonal elements are inverted when an inverse of a diagonal matrix is calculated. Thus, small values of σ will correspond to large elements in the inverse matrix used in T . The value l/σ is a weighting factor for the corresponding direction. If σ is large, deviations in the corresponding features or combinations of features of the pattern to be compared with the template will give a large contribution to the result of the cost function.

Since σ _ow is chosen, this value could be trimmed in order to get a good behavior of the cost function. The best choice of χ_ow is normally close to zero, which implies that variations in the first r directions (i.e. the directions where variations occur normally) in practice do not contribute to the result of the cost function.

When the cost function has been established, a threshold for accepting or rejecting the handwritten pattern as corresponding to the template is set. The rejection threshold, that is the value above which the pattern is rejected as not corresponding to the template, could be set by using the entered variants for training the database. Then, one variant is used as the input pattern and T is calculated for this variant. This is repeated with all the variants used as input pattern and the variations of the result of T are studied. However, a new basis has to be built for every "simulation", since the pattern used as input data cannot be a part of the calculation of the basis. If it were, the pattern would coincide with the collapsed dimensions and the result of the cost function would be small. This implies that the basis for the principal component analysis has to be built with N- l patterns out of the N in the database. The result of simulation k is denoted T_k and K = (r_ι,...,r_iV) . The rejection threshold r is set as

r_r= +:-σ_κ ,

where is the mean value and σ^ is the standard deviation of . The number of features is typically N » 1000. Hence, the calculation of eigenvalues of the N x N covariance matrix will take a long time. Therefore, this calculation is only performed once while the database is built, and only the components needed for calculating the cost function T are saved. Thus, the cost function Y is simplified according to the following. The covariance matrix C could be expressed as C = UΣU^T , where U = (U_X U₂) is a unitary matrix and Ui and U₂ correspond to the first r singular values and to the rest of the singular values, respectively . Then C -1

U∑- -l^U/2v∑-^~l^U/2U¹ BB¹ where

B = UΣ^~112 and B^τ = Σ^~U2U^T . Thus , the cost function could be expressed as

r = y^TC-^ly = y^TBB^Ty = \\B^Ty\ -1/2 U^τy\

Here, ∑^~1/2 could be written aε

∑ •1/2 D 0 -'^/z =

where D = diag(l/Jσ^, ..., \l τ_r ) and σ₀ = l/ψτ_h This could be used to simplify T -.

(D - σj)U_x ^τ yt ' + σ²\\y\\ ) + 2σ₀y^τU₁ {D - σ_ύl)ϋ_x ^τy

Now, an expression of T has been created that does not include U₂ . For typical values of N and r, e . g . N « 1000 and r=10, the required memory could be decreased by a factor 100. For this expression of the cost function, the template contains mean value data (m) and the variations in data (Ui, D and σ₀) , that is all data which is needed to form the cost function T.

Above, no indications have been given to what features should be detected for the patterns. Since the principal component analysis is a general tool it could be used for any set of features . The choice of features could vary between the fields of application of the method, i.e. if a character should be recognized or if a signature should be verified. Three different suggestions of sets of features will now be given.

A first set of features could be to simply use the positions of all points in a discrete point representation of the pattern that is detected. This feature selection gives a pictorial judgment of the pattern and is suitable for use in character recognition.

Another selection of features is to use the centre of mass of the strokes in the pattern. The idea of this selection of features is to quickly sort out patterns with an incorrect stroke order. If a pattern has S strokes, the 2 S coordinates for the mass centres are chosen as features . Yet another selection of features is the velocity profile, i.e. the velocity or direction of the curve along the arc length. The velocity profile is calculated from the discrete point representation of the pattern as

^■■ (^χk -^χk- )² + (y_k -y_k-ι)²

for point k. The intention of this feature selection is to study the dynamic properties for a pattern.

These sets of features could individually be used to create a cost function for a template. The template could have one or more cost functions. If several cost func- tions are assigned to one template, the result of each cost function has to be below the threshold value if the handwritten pattern is to be recognized as the template. When the cost functions of the templates have been established, they could be used for recognition of handwritten patterns. Referring to Fig. 2, the procedure for handwritten pattern recognition will now be described. First, a user enters a handwritten pattern, step 20. Next, the handwritten pattern is detected, step 22, and features of the handwritten pattern are extracted, step 24. The features that are extracted correspond to those features that were used for creation of the template. Next, the handwritten pattern is compared to the templates in the database, step 26, in order to find the template to which the handwritten pattern corresponds.

Thus, the cost functions of the templates are calculated, step 26a, for the handwritten pattern, and the results of the cost functions are compared, step 26b, to the preset rejection thresholds. If all cost functions of a template return results for the handwritten pattern below the rejection thresholds, the handwritten pattern is recognized as the template. If one cost function returns a result above the rejection threshold, the handwritten pattern is rejected. The recognition procedure is then continued by comparing the handwritten pattern to another template .

When the method described above is used for character recognition it is suitable that the entered patterns for creation of the template are written by several persons. This would give the method the possibility of recognizing characters regardless of the normal differences between the handwriting of different people.

When the method is used for signature verification the entered patterns for creation of the template should be written by the person to whom the signature belongs. As a result, the normal variations of the signature of the person will not prevent the signature from being verified. However, careful consideration should be taken so that a forgery is not accepted. Therefore, the rejection thresholds might be set lower than for character recognition, since an incorrect acceptance of a signature could have serious consequences. Alternatively, other persons could be allowed to try to copy the signature. These patterns could be used while creating the template for ascertaining that the template does not accept these forgeries . Referring to Fig. 3, a device for handwritten recognition 100 according to the invention will now be described. A user writes a handwritten pattern 102 on a pressure-sensitive area 104 using a pen or stylus 106. The device for handwritten pattern recognition 100 comprises a detection means 108 that detects and digitizes the handwritten pattern 102 that is entered on the pressure-sensitive area 104. An extraction means 110 extracts the features of the handwritten pattern 102 that are relevant for recognition of the pattern. The device 100 further comprises a database 112. The database 112 comprises templates for different handwritten patterns that are to be recognized. Each template is established by one or more cost functions. The components needed for calculation of the cost functions are stored in the database 112. A rejection threshold is also associated with each cost function. The device 100 comprises a comparison means 114 for comparing the handwritten pattern to the templates of the database. The comparison means 114 calculates the cost functions of a template for the handwritten pattern 102. The comparison means 114 then compares the calculated result of the cost functions to the preset rejection thresholds in the database 112. If the result of a cost function is above the rejection threshold the handwritten pattern 102 is decided not to correspond to that template.

The comparison means 114 communicates the template to which the handwritten pattern 102 corresponds to a display 116. The display 116 then presents the result of the recognition of the handwritten pattern 102. In case of character or text recognition, the interpretation of the entered pattern is displayed. For signature verifi- cation, the identity of the writer could be displayed or verified.

It should be emphasized that the preferred embodiment described herein is in no way limiting and that many alternative embodiments are possible within the scope of protection defined by the appended claims. For example, a template could correspond to any kind of handwritten pattern, such as a single character, a combination of characters, a word or a signature.

The rejection threshold could alternatively be a lowest accepted value. Thus, if the result of the cost function for a handwritten pattern is above the rejection threshold the handwritten pattern is accepted. According to another alternative embodiment, two rejection thresholds could be used to define an interval, within which the result of the cost function should be if the handwritten pattern is to be accepted.

Claims

1. A method for creating a template for recognition of a handwritten pattern, comprising the steps of detecting (2) a plurality of variants of the handwritten pattern, extracting (4) individual features of the handwritten pattern, identifying (6) variations among the individual features in the variants of the handwritten pattern, said variations comprising variations of combinations of features in the variants of the handwritten pattern, and creating (8) the template for the handwritten pattern by establishing a cost function, said cost func- tion weighting the individual features in accordance with the variations among the individual features.

2. The method according to claim 1, wherein the step of identifying (4) variations among the features is performed by means of a principal component analysis.

3. The method according to claim 1 or 2 , further comprising the step of creating several cost functions of the template for different types of features.

4. The method according to any one of the preceding claims, wherein the handwritten pattern is a signature.

5. Use of the templates created by the method accor- ding to any one of claims 1-4 for signature verification.

6. A method for recognition of a handwritten pattern, comprising the steps of creating templates for different handwritten patterns, said step of creating a template for a handwritten pattern comprising the steps of detecting (2) a plurality of variants of the handwritten pattern, extracting (4) individual features of the handwritten pattern, identifying (6) variations among the individual features in the variants of the handwritten pattern, said variations comprising variations of combinations of features in the variants of the handwritten pattern, and creating (8) the template for the handwritten pattern by establishing a cost function, said cost function weighting the individual features in accordance with the variations among the individual features, and comparing (26) the handwritten pattern to the templates using the cost functions of the templates in order to recognize the pattern.

7. The method according to claim 6, wherein the step of identifying (4) the variations among the features is performed by means of a principal component analysis.

8. The method according to claim 6 or 7, wherein the step of comparing (26) comprises the step of calculating (26a) a value of the cost function of a template for the handwritten pattern.

9. The method according to claim 8, wherein the step of comparing (26) the handwritten pattern to a template further comprises the step of comparing (26b) the calculated value of the cost function to a threshold value, below which the handwritten pattern is accepted as corresponding to the template.

10. The method according to any one of claims 6-9, wherein the step of creating (8) a template comprises creating several cost functions for different types of features .

11. The method according to claim 10, wherein the step of comparing (26) the handwritten pattern to the templates uses all the cost functions of each template and the handwritten pattern is only accepted as corresponding to the template if the value of each one of the cost functions is below the threshold value.

12. The method according to any one of the claims 6- 11, wherein the handwritten pattern is a signature and the recognition is performed for signature verification.

13. A method for signature verification, comprising the steps of creating templates for different signatures, said step of creating a template for a signature comprising the steps of detecting (2) a plurality of variants of the signature, extracting (4) individual features of the signature, identifying (6) variations among the individual features in the variants of the signature, said variations comprising variations of combinations of features in the variants of the signature, and creating (8) the template for the signature by establishing a cost function, said cost function weighting the individual features in accordance with the variations among the individual features, detecting (22) a signature, which is to be verified, comparing (26) the signature to the templates using the cost functions of the templates, and verifying the signature as a signature corresponding to a template if the cost function of that template is within a predefined interval .

14. A device for handwritten pattern recognition, comprising a detection means (108) for detecting a handwritten pattern, a digitizer for transforming the handwritten pattern into a digital representation, a reference database (112) of templates for different handwritten patterns, wherein a template has been created by detection of several variants of a handwritten pattern by means of the detection means (108) , extraction of individual features of the handwritten pattern and identification of variations among combinations of individual features in the variants of the handwritten pattern for creation of a cost function where the features are weighted in accordance with the variations, and a comparison means (114) for comparing the handwritten pattern to templates of the reference database (112) .