US20070154079A1

US20070154079A1 - Media validation

Info

Publication number: US20070154079A1
Application number: US11/639,597
Authority: US
Inventors: Chao He; Gary Ross
Original assignee: Individual
Current assignee: NCR Voyix Corp
Priority date: 2005-12-16
Filing date: 2006-12-15
Publication date: 2007-07-05
Also published as: BRPI0619926A2; BRPI0620625A2; WO2007068867A1; US20070154099A1; US20070140551A1; US8086017B2; EP1964075A1; JP2009527029A; JP5175210B2; EP1964073A1; JP2009527027A; US20070154078A1; JP5177817B2; BRPI0619845A2; WO2007068928A1; BRPI0620308A2; JP2009527028A; WO2007068923A1; JP2009519532A; WO2007068930A1

Abstract

A banknote validator which classes media items into three or more classes is described. Information from all of a set of training images from genuine media items is used to form one or more segmentation maps which are then used to segment each of the training set images. Features are extracted from the segments and used to form one or more classifiers. Classifiers can be quickly and simply formed for media items such as different currencies and denominations of banknotes in this way and without the need for examples of counterfeit banknotes. In some examples, the classifier(s) are arranged to operate at a plurality of pre-specified confidence levels. In other examples, a plurality of classifiers are formed from feature information obtained from different segments. In other examples, segmentation maps are associated with different regions of an image of a media item. The media validator may be incorporated in a self-service apparatus such as an automated teller machine.

Description

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation-in-part application of U.S. patent application Ser. No. 11/366,147, filed on Mar. 2, 2006, which is a continuation-in-part application of U.S. patent application Ser. No. 11/305,537, filed on Dec. 16, 2005. Application Ser. No. 11/366,147, filed on Mar. 2, 2006 and application Ser. No. 11/305,537, filed on Dec. 16, 2005 are hereby incorporated by reference.

TECHNICAL FIELD

The present invention relates to a method and apparatus for media validation. It is particularly related to, but in now way limited to, validation of media such as banknotes, passports, checks, bonds, share certificates and the like.

BACKGROUND

There is a growing need for automatic verification and validation of banknotes of different currencies and denominations in a simple, reliable, and cost effective manner. This is required, for example, in self-service apparatus which receives banknotes, such as self-service kiosks, ticket vending machines, automated teller machines arranged to take deposits, self-service currency exchange machines and the like.
Previously, manual methods of currency validation have involved image examination, transmission effects such as watermarks and thread registration marks, feel and even smell of banknotes. Other known methods have relied on semi-overt features requiring semi-manual interrogation. For example, using magnetic means, ultraviolet sensors, fluorescence, infrared detectors, capacitance, metal strips, image patterns and similar. However, by their very nature these methods are manual or semi-manual and are not suitable for many applications where manual intervention is unavailable for long periods of time. For example, in self-service apparatus.
There are significant problems to be overcome in order to create an automatic currency validator. For example, many different types of currency exist with different security features and even substrate types. Within those different denominations also exist commonly with different levels of security features. There is therefore a need to provide a generic method of easily and simply performing currency validation for those different currencies and denominations.
Previous automatic validation methods typically require a relatively large number of examples of counterfeit banknotes to be known in order to train the classifier. In addition, those previous classifiers are trained to detect known counterfeits only. This is problematic because often little or no information is available about possible counterfeits. For example, this is particularly problematic for newly introduced denominations or newly introduced currency.
In an earlier paper entitled, “Employing optimized combinations of one-class classifiers for automated currency validation”, published in Pattern Recognition 37, (2004) pages 1085-1096, by Chao He, Mark Girolami and Gary Ross (two of whom are inventors of the present application) an automated currency validation method is described (Patent No. EP1484719, US2004247169) for classifying banknotes as either genuine or counterfeit. This involves segmenting an image of a whole banknote into regions using a grid structure. Individual “one-class” classifiers are built for each region and a small subset of the region specific classifiers are combined to provide an overall decision. (The term, “one-class” is explained in more detail below.) The segmentation and combination of region specific classifiers to achieve good performance is achieved by employing a genetic algorithm. This method requires a small number of counterfeit samples at the genetic algorithm stage and as such is not suitable when counterfeit data is unavailable.
Previously, currency validation has typically involved classifying banknotes as either genuine or counterfeit. However, more recently, a need has arisen to classify banknotes into more than the two classes of counterfeit or genuine. For example, an additional class includes whether a banknote is “suspect” that is, falls between the genuine and counterfeit classes. Various regulatory requirements in different jurisdictions typically specify the classes that are to be used in banknote validation systems. For example, cash-accepting or cash-recycling automated teller machines or other self-service apparatus such as vending machines, kiosks and the like.
Classification of a banknote as “suspect” as opposed to genuine or counterfeit may have financial implications for users of automated banknote validating apparatus. In addition, regulatory and commercial requirements heighten the need to make a distinction between suspect banknotes and those which are genuine or counterfeit.
There is also a need to perform automatic currency validation in a computationally inexpensive manner which can be performed in real time. Many of the issues mentioned above also apply to validation of other types of media such as passports and cheques.

SUMMARY

A media validator which classes media into three or more classes is described. Information from all of a set of training images from genuine media is used to form one or more segmentation maps which are then used to segment each of the training set images. Features are extracted from the segments and used to form one or more classifiers. Classifiers can be quickly and simply formed for different types of media items such as currencies and denominations of banknotes in this way and without the need for examples of counterfeit media items. In some examples, the classifier(s) are arranged to operate at a plurality of pre-specified confidence levels. In other examples, a plurality of classifiers are formed from feature information obtained from different segments. In other examples, segmentation maps are associated with different regions of an image of a media item. The media validator may be incorporated in a self-service apparatus such as an automated teller machine.
The method may be performed by software in machine readable form on a storage medium. The method steps may be carried out in any suitable order and/or in parallel as is apparent to the skilled person in the art.
This acknowledges that software can be a valuable, separately tradable commodity. It is intended to encompass software, which runs on or controls “dumb” or standard hardware, to carry out the desired functions, (and therefore the software essentially defines the functions of the media validator, and can therefore be termed a media validator, even before it is combined with its standard hardware). For similar reasons, it is also intended to encompass software which “describes” or defines the configuration of hardware, such as HDL (hardware description language) software, as is used for designing silicon chips, or for configuring universal programmable chips, to carry out desired functions.
The preferred features may be combined as appropriate, as would be apparent to a skilled person, and may be combined with any of the aspects of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the invention will be described, by way of example, with reference to the following drawings, in which:
FIG. 1 is a flow diagram of a method of creating a classifier for banknote validation;
FIG. 2 is a flow diagram of a method of creating a banknote validator for classifying banknotes into three or more classes;
FIG. 3 is a flow diagram of a method of classifying banknotes into three or more classes using a plurality of classifiers, each associated with a segment of a segmentation map;
FIG. 4 is a schematic diagram of using the same classifier with different significance levels to classify banknotes;
FIG. 5 is a flow diagram of a method of classifying banknotes into three or more classes using the same classifier at each of two significance levels;
FIG. 6 is a schematic diagram of a banknote divided into regions;
FIG. 7 is a flow diagram of a method of classifying banknotes into three or more classes using a plurality of classifiers each associated with a different region of a banknote;
FIG. 8 is a flow diagram of a method of classifying banknotes into three or more classes using a combination of localized segmentation maps and different significance levels of classifiers;
FIG. 9 is a flow diagram of a method of classifying banknotes into three or more classes using a combination of classifiers based on segments and different significance levels of classifiers;
FIG. 10 is a flow diagram of a method of classifying banknotes into three or more classes using a combination of classifiers based on segments and banknote regions as well as different significance levels of classifiers;
FIG. 11 is a schematic diagram of an apparatus for creating a classifier for banknote validation;
FIG. 12 is a schematic diagram of a banknote validator;
FIG. 13 is a flow diagram of a method of validating a banknote;
FIG. 14 is a schematic diagram of a self-service apparatus with a banknote validator.

DETAILED DESCRIPTION

Embodiments of the present invention are described below by way of example only. These examples represent the best ways of putting the invention into practice that are currently known to the Applicant although they are not the only ways in which this could be achieved.
Although the present examples are described and illustrated herein as being implemented in a banknote validation system, the system described herein is provided as an example and not a limitation. As those skilled in the art will appreciate, the present examples are suitable for application in a variety of different types of media validation systems, including but not limited to passport validation systems, check validation systems, bond validation systems and share certificate validation systems.
The term “one class classifier” is used to refer to a classifier that is formed or built using information about examples only from a single class but which is used to allocate newly presented examples either to that single class or not. This differs from a conventional binary classifier which is created using information about examples from two classes and which is used to allocate new examples to one or other of those two classes. A one-class classifier can be thought of as defining a boundary around a known class such that examples falling out with that boundary are deemed not to belong to the known class

As mentioned above, a need has arisen to classify banknotes into more than the two classes of counterfeit or genuine. For example, an additional class includes whether a banknote is “suspect” that is, falls between the genuine and counterfeit classes. Examples of four categories are given in the table below. In this example, a banknote is either classified as not recognized (category 1), as counterfeit (category 2), as genuine (category 4) or as suspect (category 3).



Category	Classification	Properties

1	No banknote, not	Not detected as a banknote because of:
	recognized	Wrong image or format
		Transportation error (e.g. double feeds, etc.)
		Large, dog-eared or missing sections
		Hand-written notes, separating cards, etc.
		- Wrong currency
2	Element(s) identified as	Image and format recognized, but one or
	counterfeit	more authentication features missing or
		clearly out of tolerance.
3	Elements not clearly	Image and format recognized, but not all
	authenticated. Suspect	authentication features recognized because
	banknotes	of quality and/or tolerance deviations. In most
		cases damaged or soiled banknotes.
4	Banknotes authenticated	All authentication checks delivered positive
	as genuine ones	results.

FIG. 1 is a high level flow diagram of a method of creating a classifier for banknote validation.
First we obtain a training set of images of genuine banknotes (see box 10 of FIG. 1). These are images of the same type taken of banknotes of the same currency and denomination. The type of image relates to how the images are obtained, and this may be in any manner known in the art. For example, reflection images, transmission images, images on any of a red, blue or green channel, thermal images, infrared images, ultraviolet images, x-ray images or other image types. The images in the training set are in registration and are the same size. Pre-processing can be carried out to align the images and scale them to size if necessary, as known in the art.
We next create a segmentation map using information from the training set images (see box 12 of FIG. 1). The segmentation map comprises information about how to divide an image into a plurality of segments. The segments may be non-continuous, that is, a given segment can comprise more than one patch in different regions of the image. Preferably, but not essentially, the segmentation map also comprises a specified number of segments to be used.
Using the segmentation map we segment each of the images in the training set (see box 14 of FIG. 1). We then extract one or more features from each segment in each of the training set images (see box 16 of FIG. 1). By the term “feature” we mean any statistic or other characteristic of a segment. For example, the mean pixel intensity, median pixel intensity, mode of the pixel intensities, texture, histogram, Fourier transform descriptors, wavelet transform descriptors and/or any other statistics in a segment.
A classifier is then formed using the feature information (see box 18 of FIG. 1). Any suitable type of classifier can be used as known in the art. In a particularly preferred embodiment of the invention the classifier is a one-class classifier and no information about counterfeit banknotes is needed. However, it is also possible to use a binary classifier or other type of classifier of any suitable type as known in the art. For example, if it is required to classify banknotes into three or more classes (such as genuine, counterfeit and suspect) then a classifier which classifies into the appropriate number of classes may be used.
The method in FIG. 1 enables a classifier for validation of banknotes of a particular currency and denomination to be formed simply, quickly and effectively and automatically. To create classifiers for other currencies or denominations the method is repeated with appropriate training set images.
In a particular example, a one-class classifier is formed which provides classification into only two classes: genuine or counterfeit. In this situation it is sometimes required to provide a means whereby additional classes are possible, such as the class “suspect” mentioned above. In order to enable this we modify the method of FIG. 1 to form more than one classifier, each classifier being associated with only one segment from the segmentation map (see FIG. 2). This results in two or more classifiers (assuming there are two or more segments in the segmentation map). The outputs of the classifiers are then combined to provide a classification into more than one class as described below with reference to FIG. 3.
FIG. 2 shows how the method of FIG. 1 is modified to produce more than one classifier. The method is the same as that of FIG. 1 except that a plurality of classifiers are formed rather than one classifier. Each classifier is formed using feature information from a single segment.
shown in FIG. 3 this allows us to classify banknotes into more than two classes. A banknote to be classified (or validated) is input to an automated banknote validator (see box 30). One or more images of the banknote are captured and pre-processed as described above. A segmentation map (that has already been formed using any of the methods described herein or other suitable methods) is then used to segment the images of the banknote into K segments (see box 32) where K is an integer value of 2 or more.
Information is extracted from the K segments (see box 33) and input to each of K classifiers which have already been formed as described herein or in any other suitable way. If the output from all of the classifiers indicates that the banknote is genuine then an indication is made that the banknote is genuine (see box 35). If the output from all the classifiers indicates that the banknote is counterfeit, then an indication is made that the banknote is counterfeit (see box 36). If one or more classifiers indicates that the banknote is genuine whilst one or more of the other classifiers indicates that it is counterfeit, then an indication is made that the banknote is “suspect” (see box 37).
More detail about forming the segmentation map is now given.
Previously in EP1484719 and US2004247169, (as mentioned in the background section) we used a segmentation technique that involved using a grid structure over the image plane and a genetic algorithm method to form the segmentation map. This necessitated using some information about counterfeit notes, and incurring computational costs when performing the genetic algorithm search.
The present invention uses a different method of forming the segmentation map which removes the need for using a genetic algorithm or equivalent method to search for a good segmentation map within a large number of possible segmentation maps. This reduces computational cost and improves performance. In addition the need for information about counterfeit banknotes is removed.
We believe that generally it is difficult in the counterfeiting process to provide a uniform quality of imitation across the whole note and therefore certain regions of a note are more difficult than others to be copied successfully. We therefore recognized that rather than using a rigidly uniform grid segmentation we could improve banknote validation by using a more sophisticated segmentation. Empirical testing that we carried out indicated that this is indeed the case. Segmentation based on morphological characteristics such as pattern, color and texture led to a better performance in detecting counterfeits. However, traditional image segmentation methods, such as using edge detectors, when applied to each image in the training set were difficult to use. This is because varying results are obtained for each training set member and it is difficult to align corresponding features in different training set images. In order to avoid this problem of aligning segments we used, in one preferred embodiment, a so called “spatio-temporal image decomposition”.
Details about the method of forming the segmentation map are now given. At a high level this method can be thought of as specifying how to divide the image plane into a plurality of segments, each comprising a plurality of specified pixels. The segments can be non-continuous as mentioned above. For example, this specification is made on the basis of information from all images in the training set. In contrast, segmentation using a rigid grid structure does not require information from images in the training set.
For example, each segmentation map comprises information about relationships of corresponding image elements between all images in the training set.
Consider the images in the training set as being stacked and in registration with one another in the same orientation. Taking a given pixel in the note image plane this pixel is thought of as having a “pixel intensity profile” comprising information about the pixel intensity at that particular pixel position in each of the training set images. Using any suitable clustering algorithm, pixel positions in the image plane are clustered into segments, where pixel positions in those segments have similar or correlated pixel intensity profiles.
In a preferred example we use these pixel intensity profiles. However, it is not essential to use pixel intensity profiles. It is also possible to use other information from all images in the training set. For example, intensity profiles for blocks of 4 neighboring pixels or mean values of pixel intensities for pixels at the same location in each of the training set images.
A particularly preferred embodiment of our method of forming the segmentation map is now described in detail. This is based on the method taught in the following publication “EigenSegments: A spatio-temporal decomposition of an ensemble of images” by Avidan, S. Lecture Notes in Computer Science, 2352: 747-758, 2002.
Given an ensemble of images {I_i}i=1, 2, . . . , N which have been registered and scaled to the same size r×c, each image I_ican be represented by its pixels as [a_1i, a_2i, . . . , a_Mi]^Tin vector form, where a_ji(j=1,2, . . . , M) is the intensity of the jth pixel in the ith image and M=r·c is the total number of pixels in the image. A design matrix Aε
^M×Ncan then be generated by stacking vectors I_i(zeroed using the mean value) of all images in the ensemble, thus A=└I₁, I₂, . . . , I_N┘. A row vector └a_ji, a_j2, . . . , a_jN┘ in A can be seen as an intensity profile for a particular pixel (jth) across N images. If two pixels come from the same pattern region of the image they are likely to have the similar intensity values and hence have a strong temporal correlation. Note the term “temporal” here need not exactly correspond to the time axis but is borrowed to indicate the axis across different images in the ensemble. Our algorithm tries to find these correlations and segments the image plane spatially into regions of pixels that have similar temporal behavior. We measure this correlation by defining a metric between intensity profiles. A simple way is to use the Euclidean distance, i.e. the temporal correlation between two pixels j and k can be denoted as d(j,k)=√{square root over (Σ_i=1 ^N(a_ji−a_ki)²)}. The smaller d(j,k), the stronger the correlation between the two pixels.
In order to decompose the image plane spatially using the temporal correlations between pixels, we run a clustering algorithm on the pixel intensity profiles (the rows of the design matrix A). It will produce clusters of temporally correlated pixels. The most straightforward choice is to employ the K-means algorithm, but it could be any other clustering algorithm. As a result the image plane is segmented into several segments of temporally correlated pixels. This can then be used as a map to segment all images in the training set; and a classifier can be built on features extracted from those segments of all images in the training set.
In order to achieve the training without utilizing counterfeit notes, one-class classifier is preferable. Any suitable type of one-class classifier can be used as known in the art. For example, neural network based one-class classifiers and statistical based one-class classifiers.
Suitable statistical methods for one-class classification are in general based on maximization of the log-likelihood ratio under the null-hypothesis that the observation under consideration is drawn from the target class and these include the D²test (described in Morrison, D F: Multivariate Statistical Methods (third edition). McGraw-Hill Publishing Company, New York, 1990) which assumes a multivariate Gaussian distribution for the target class (genuine currency). In the case of an arbitrary non-Gaussian distribution the density of the target class can be estimated using for example a semi-parametric Mixture of Gaussians (described in Bishop, C M: Neural Networks for Pattern Recognition, Oxford University Press, New York, 1995) or a non-parametric Parzen window (described in Duda, R O, Hart, P E, Stork, D G: Pattern Classification (second edition), John Wiley & Sons, INC, New York, 2001) and the distribution of the log-likelihood ratio under the null-hypothesis can be obtained by sampling techniques such as the bootstrap (described in Wang, S, Woodward, W A, Gary, H L et al: A new test for outlier detetion from a multivariate mixture distribution, Journal of Computational and Graphical Statistics, 6(3): 285-299, 1997).
Other methods which can be employed for one-class classification are Support Vector Data Domain Description (SVDD) (described in Tax, D M J, Duin, R P W: Support vector domain description, Pattern Recognition Letters, 20(11-12): 1191-1199, 1999), also known as ‘support estimation’ (described in Hayton, P, Schölkopf, B, Tarrassenko, L, Anuzis, P: Support Vector Novelty Detection Applied to Jet Engine Vibration Spectra, Advances in Neural Information Processing Systems, 13, eds Leen, Todd K and Dietterich, Thomas G and Tresp, Volker, MIT Press, 946-952, 2001) and Extreme Value Theory (EVT) (described in Roberts, S J: Novelty detection using extreme value statistics. IEE Proceedings on Vision, Image & Signal Processing, 146(3): 124-129, 1999). In SVDD the support of the data distribution is estimated, whilst the EVT estimates the distribution of extreme values. For this particular application, large numbers of examples of genuine notes are available, so in this case it is possible to obtain reliable estimates of the target class distribution. We therefore choose one-class classification methods that can estimate the density distribution explicitly in a preferred embodiment, although this is not essential. In a preferred embodiment we use one-class classification methods based on the parametric D²test).
In a preferred embodiment, the statistical hypothesis tests used for our one-class classifier are detailed as follows:
Consider N independent and identically distributed p-dimensional vector samples (the feature set for each banknote) x₁, . . . , x_NεC with an underlying density function with parameters θ given as p(x|θ). The following hypothesis test is given for a new point x_N+1such that H₀:x_N+1εC vs. H₁:x_N+1∉C, where C denotes the region where the null hypothesis is true and is defined by p(x|θ). Assuming that the distribution under the alternate hypothesis is uniform then the standard log-likelihood ratio for the null and alternate hypothesis $\begin{matrix} λ = \frac{\sup_{θ \in Θ} L_{0} (θ)}{\sup_{θ \in Θ} L_{1} (θ)} = \frac{\sup_{θ} \prod_{n = 1}^{N + 1} p (x_{n} | θ)}{\sup_{θ} \prod_{n = 1}^{N} (x_{n} | θ)} & (1) \end{matrix}$
can be employed as a test statistic for the null-hypothesis. In this preferred embodiment we can use the log-likelihood ratio as test statistic for the validation of a newly presented note.
Feature vectors with multivariate Gaussian density: Under the assumption that the feature vectors describing individual points in a sample are multivariate Gaussian, a test that emerges from the above likelihood ratio (1), to assess whether each point in a sample shares a common mean is described in (Morrison, D F: Multivariate Statistical Methods (third edition). McGraw-Hill Publishing Company, New York, 1990). Consider N independent and identically distributed p-dimensional vector samples x₁, . . . , x_Nfrom a multivariate normal distribution with mean μ and covariance C, whose sample estimates are {circumflex over (μ)}_Nand Ĉ_N. From the sample consider a random selection denoted as x₀, the associated squared Mahalanobis distance
D ²=(x ₀−{circumflex over (μ)}_N)^T Ĉ _N ⁻¹(x ₀−{circumflex over (μ)}_N) (2)
can be shown to be distributed as a central F-distribution with p and N−p−1 degrees of freedom by $\begin{matrix} F = \frac{(N - p - 1) {ND}^{2}}{{p (N - 1)}^{2} - {NpD}^{2}} . & (3) \end{matrix}$
Then, the null hypothesis of a common population mean vector x₀and the remaining x_iwill be rejected if
F>F_α;p,N−1, (4)
where F_{α;p,N−p−1}is the upper α·100% point of the F-distribution with (p,N−p−1) degrees of freedom.
Now suppose that x₀was chosen as the observation vector with the maximum D²statistic. The distribution of the maximum D²from a random sample of size N is complicated. However a conservative approximation to the 100α percent upper critical value can be obtained by the Bonferroni inequality. Therefore we might conclude that x₀is an outlier if $\begin{matrix} F > F_{\frac{α}{N}; p, N - p - 1} . & (5) \end{matrix}$
In practice, either equations (4) or (5) can be used for outlier detection.
We can make use of the following incremental estimates of the mean and covariance in devising a test for new examples which do not form part of the original sample when an additional datum x_N+1is made available, i.e. the mean $\begin{matrix} {\hat{μ}}_{N + 1} = \frac{1}{N + 1} {N {\hat{μ}}_{N} + x_{N + 1}} & (6) \end{matrix}$
and the covariance $\begin{matrix} {\hat{C}}_{N + 1} = \frac{N}{N + 1} {\hat{C}}_{N} + \frac{N}{{(N + 1)}^{2}} (x_{N + 1} - {\hat{μ}}_{N}) {(x_{N + 1} - {\hat{μ}}_{N})}^{T} . & (7) \end{matrix}$
By using the expression of (6), (7) and the matrix inversion lemma, Equation (2) for an N-sample reference set and an N+1′th test point becomes
D²=σ_N+1 ^TĈ_N+1 ^{31 1}σ_N+1, (8)
where $\begin{matrix} σ_{N + 1} = (x_{N + 1} - {\hat{μ}}_{N + 1}) = \frac{N}{N + 1} (x_{N + 1} - {\hat{μ}}_{N}), and & (9) \\ {\hat{C}}_{N + 1}^{- 1} = \frac{N + 1}{N} ({\hat{C}}_{N}^{- 1} - \frac{{\hat{C}}_{N}^{- 1} (x_{N + 1} - {\hat{μ}}_{N}) {(x_{N + 1} - {\hat{μ}}_{N})}^{T} {\hat{C}}_{N}^{- 1}}{N + 1 + {(x_{N + 1} - {\hat{μ}}_{N})}^{T} {\hat{C}}_{N}^{- 1} (x_{N + 1} - {\hat{μ}}_{N})}) . & (10) \end{matrix}$
Denoting (x_N+1−{circumflex over (μ)}_N)^TĈ _N ⁻¹(x_N+1−{circumflex over (μ)}_N) by D_N+1,N ², then $\begin{matrix} D^{2} = \frac{{ND}_{N + 1, N}^{2}}{N + 1 + D_{N + 1, N}^{2}} . & (11) \end{matrix}$
So a new point X_N+1can be tested against an estimated and assumed normal distribution for a common estimated mean {circumflex over (μ)}_Nand covariance Ĉ_N. Though the assumption of multivariate Gaussian feature vectors often does not hold in practice, it has been found as an appropriate pragmatic choice for many applications. We relax this assumption and consider arbitrary densities in the following section.
2) Feature Vectors with arbitrary Density: A probability density estimate {circumflex over (p)}(x; θ) can be obtained from the finite data sample S={x₁, . . . , x_N}ε
^ddrawn from an arbitrary density p(x), by using any suitable semi-parametric (e.g. Gaussian Mixture Model) or non-parametric (e.g. Parzen window method) density estimation methods as known in the art. This density can then be employed in computing the log-likelihood ratio (1). Unlike the case of the multivariate Gaussian distribution there is no analytic distribution for the test statistic (λ) under the null hypothesis. So to obtain this distribution, numerical bootstrap methods can be employed to obtain the otherwise non-analytic null distribution under the estimated density and so the various critical values of λ_critcan be established from the empirical distribution obtained. It can be shown that in the limit as N→∞, the likelihood ratio can be estimated by the following $\begin{matrix} λ = \frac{\sup_{θ \in Θ} L_{0} (θ)}{\sup_{θ \in Θ} L_{1} (θ)} \to \hat{p} (x_{N + 1}; {\hat{θ}}_{N}) & (12) \end{matrix}$
where {circumflex over (p)}(x_N+1; {circumflex over (θ)}_N) denotes the probability density of x_N+1under the model estimated by the original N samples.
After generating B sets bootstrap of N samples from the reference data set and using each of these to estimate the parameters of the density distribution {circumflex over (θ)}_N ⁱ, B bootstrap replicates of the test statistic λ_crit ⁱ, i=1, . . . , B can be obtained by randomly selecting an N+1'th sample and computing {circumflex over (p)}(x_N+1; {circumflex over (θ)}_N ⁱ)≈λ_crit ⁱ. By ordering λ_crit ⁱin ascending order, the critical value α can be defined to reject the null-hypothesis at the desired significance level if λ≦λ_α, where λ_αis the jth smallest value of λ_crit ⁱ, and α=j/(B+1).
Preferably the method of forming the classifier is repeated for different numbers of segments and tested using images of banknotes known to be either counterfeit or not. The number of segments giving the best performance is then selected and the classifier using that number of segments used. We found that the best number of segments to be from about 2 to 15 although any suitable number of segments can be used.
described above, in a group of embodiments, a one-class classifier is used. This type of classifier can be thought of as defining a boundary around a known class such that examples falling outside that boundary are deemed not to belong to the known class. However, a one-class classifier typically classifies items into only two classes. This is problematic in situations where it is required to classify banknotes as either counterfeit, genuine, or suspect for example. We propose a method of addressing this by varying a significance level or confidence level used by a one-class classifier.
FIG. 4 is a schematic diagram showing the influence of different significance levels on a one-class classifier. Suppose that a given one-class classifier has a significance level of α1 indicated by the oval boundary 41 in FIG. 4. Banknotes are represented in FIG. 4 by either dots or crosses depending on whether they are actually genuine or actually counterfeit. The majority of genuine banknotes in this example fall within the boundary 41 and are classes as genuine by the one-class classifier. Suppose that the significance level of the one class classifier is now lowered to α2 indicated by the boundary 40 in FIG. 4. Now some counterfeit banknotes fall within the boundary 40 and so are wrongly classified as being genuine. We can also use the two significance levels to introduce a third classification. Those banknotes falling between the boundary 40 and the boundary 41 may be classified as suspect. In this way, be introducing a plurality of different significance levels for the one-class classifier we are able to increase the number of classes into which classification is made.
Advantageously, it is not necessary to retrain the example one-class classifier described in detail herein when the significance level is changed.
FIG. 5 is a flow diagram of a method of validating a banknote using a one-class classifier having different significance levels. Two significance levels, one higher than the other, are pre-defined and stored (see box 50) for example, by manual configuration. Banknote validation is performed as described herein, using a one-class classifier having the higher significance level (see box 51). If the banknote is classified as genuine an output is made indicating this (see boxes 52 and 53). If the banknote is classified as not genuine then the validation is repeated using the same one-class classifier but having the lower significance level (see box 54). If the banknote is classified as counterfeit an output is made to this effect (see boxes 55 and 57). However, if the banknote is classified as genuine then an indication is made that it is “suspect” (see box 56). That is, the automated validation process is repeated for the same banknote but with different significance levels. If the results of the one-class classifier are different for that banknote in each case then the banknote is classed as “suspect”. The one-class classifier is thought of as effectively carrying out a test on a statistical distribution of morphological characteristics of genuine notes. A boundary in this statistical distribution is, for example, defined by a significance level which sets a targeted false rejection rate of genuine notes.
In another embodiment, we enable classification of banknotes into more than two classes by forming two or more segmentation maps (the segmentation maps may or may not have the same number of segments). Each segmentation map is associated with a region of a banknote as now described in more detail with reference to FIG. 6. This results in a plurality of classifiers, one for each segmentation map, so that the classifiers are each associated with a different region of a banknote. These classifiers are referred to herein as localized classifiers.
FIG. 6 is a schematic representation of a face of a banknote of a particular denomination and currency. It is divided into three regions 61, 62, 63 indicated by dotted lines in FIG. 6. Two or more regions are used and these are positioned, sized and arranged in any suitable manner. In a preferred example, the regions are selected such that they each contain one or more security features 64 of the banknote, such as holograms, thread marks, and watermarks. However, this is not essential. The regions may be uniform and contiguous as indicated in FIG. 6 although this is not essential. Advantageously, by selecting the regions such that they each contain one or more security features, we are able to assess likelihood of one or more of those security features being absent. This assists in enabling classification of banknotes into a plurality of categories including, counterfeit, genuine and “suspect”. The regions may be selected in any suitable manner, such as by using an image processing or image recognition system to identify the security features. For example, infra-red or thermal imaging may be used to pick out appropriate security features such as watermarks. Also, tailored illumination may be used to pick out holograms or other complex diffraction grating security features. Alternatively, the regions may be manually configured for different currencies and denominations in advance.
FIG. 7 is a flow diagram of a method of using localized classifiers for banknote validation. A banknote to be validated is input to the validator (see box 70) and images of the banknote captured (see box 71). The images are divided into R specified regions (see box 72). Those R regions are the same regions as already used to form segmentation maps and corresponding classifiers. Each region of the image is then segmented using the segmentation map for that region (see box 73) and information is extracted from each segment of each region. This information is input to the appropriate R classifiers (see box 75). If all the classifiers indicate a pass, i.e. that the banknote is genuine then it is indicated as genuine (see box 76). If all the classifiers indicate a fail then the banknote is indicated as counterfeit (see box 77). Otherwise the banknote is indicated as suspect (see box 78).
It is also possible to combine one or more of the methods described herein for classifying banknotes into two or more categories.
mentioned above, one method involves using a plurality of classifiers, each classifier being associated with one segment of a segmentation map. This is now referred to as method A.
Another method involves using a single classifier but with a plurality of significance levels. This is now referred to as method B.
Another method involves using a plurality of localized classifiers, each associated with a different region of a banknote image. This is now referred to as method C.
Possible combinations of these methods comprise (but are in no way limited to):

A and then B
C and then B (as illustrated in FIG. 8)
C and then A
C and then A and then B.

FIG. 8 is a flow diagram of an example of combining method C and then method B. Method C steps are indicated in FIG. 8 by boxes 82, 83 and 84 and method B steps are indicated by boxes 85, 86, 87, 88 and 89. The banknote to be tested is input (box 80), images are captured (box 81), and the images partitioned into S regions (82). S localized segmentation maps are then created using the methods described herein (box 83) and information is extracted from the S regions using the appropriate segmentation maps (box 84). The classifier test is run, for all S classifiers, using a higher significance level (box 85). If all classifiers indicate a genuine note a genuine note is indicated (see box 87). Otherwise the classifiers repeat the tests using a lower significance level. If all classifiers indicate a genuine note, a suspect note is indicated (box 88). Otherwise a counterfeit note is indicated (box 89). In this way we are able to provide an extra confidence for customers. It is not unusual for a genuine note to become worn and torn after a certain length of circulation. Such a note is very likely to be categorized as counterfeit by all S localized classifiers if using a tight (high) significance level. This may result in a financial loss for the customer. Therefore, by testing again using a looser (lower) significance level, this note may be recognized as genuine by all S classifiers, and thus can be categorized as suspect for further investigation. This avoids the customer's loss. Meanwhile since real counterfeits will not be affected and will still be recognized, the security of the self service apparatus or other component using the process is still kept. The method also provides flexibility for banks to customize their tightness and standardize what quality of notes will be put into the suspect category. This is achieved because both S and the significance levels are adjustable.
FIG. 9 is a flow diagram of an example of combining method A and then method B. Method A steps are indicated by boxes 92 to 93 and method B steps are indicated by boxes 94 through 98. Steps 90 and 91 correspond to steps 80 and 81 of FIG. 8.
FIG. 10 is a flow diagram of an example of combining method C and then A and then B. Method C steps are indicated by boxes 100 and 101. Method A step is 102. In this case, many classifiers are used, one for each banknote region S and segment of each banknote region K. The tests are carried out at the two significance levels (see boxes 103 through 107) using each of the S×K classifiers.
An advantage of the banknote validation methods using a plurality of classes (e.g. counterfeit, genuine, suspect) is that they can increase consumer trust, appreciation and confidence in the automated banknote validator. If a note is classed as suspect it may be accepted and credited to a customer account in the short term, whilst manual or other off-line investigations are made about the validity of the note.
FIG. 11 is a schematic diagram of an apparatus 110 for creating a classifier 112 for banknote validation. It comprises:

an input 111 arranged to access a training set of banknote images;
a processor 113 arranged to create a segmentation map using the training set images;
a segmentor 114 arranged to segmenting each of the training set images using the segmentation map;
a feature extractor 115 arranged to extract one or more features from each segment in each of the training set images; and
classification forming means 116 arranged to form the classifier using the feature information;
wherein the processor is arranged to create the segmentation map on the basis of information from all images in the training set. For example, by using spatio-temporal image decomposition described above.

FIG. 12 is a schematic diagram of a banknote validator 121. It comprises:

an input arranged to receive at least one image 120 of a banknote to be validated;
a segmentation map 122;
a processor 123 arranged to segment the image of the banknote using the segmentation map;
a feature extractor 124 arranged to extract one or more features from each segment of the banknote image;
a classifier 125 arranged to classify the banknote as being either valid or not on the basis of the extracted features;
wherein the segmentation map is formed on the basis of information about each of a set of training images of banknotes. It is noted that it is not essential for the components of FIG. 12 to be independent of one another, these may be integral.

FIG. 13 is a flow diagram of a method of validating a banknote. The method comprises:

accessing at least one image of a banknote to be validated (box 130);
accessing a segmentation map (box 131);
segmenting the image of the banknote using the segmentation map 9 box 132);
extracting features from each segment of the banknote image (box 133);
classifying the banknote as being either valid or not on the basis of the extracted features using a classifier (box 134);
wherein the segmentation map is formed on the basis of information about each of a set of training images of banknotes. These method steps can be carried out in any suitable order or in combination as is known in the art. The segmentation map can be said to implicitly comprise information about each of the images in the training set because it has been formed on the basis of that information. However, the explicit information in the segmentation map can be a simple file with a list of pixel addresses to be included in each segment.

FIG. 14 is a schematic diagram of a self-service apparatus 141 with a banknote validator 143. It comprises:

a means for accepting banknotes 140,
imaging means for obtaining digital images of the banknotes 142; and
a banknote validator 143 as described above.

The methods described herein are performed on images or other representations of banknotes, those images/representations being of any suitable type. For example, images on any of a red, blue and green channel or other images as mentioned above.
The segmentation may be formed on the basis of the images of only one type, say the red channel. Alternatively, the segmentation map may be formed on the basis of the images of all types, say the red, blue and green channel. It is also possible to form a plurality of segmentation maps, one for each type of image or combination of image types. For example, there may be three segmentation maps one for the red channel images, one for the blue channel images and one for the green channel images. In that case, during validation of an individual note, the appropriate segmentation map/classifier is used depending on the type of image selected. Thus each of the methods described above may be modified by using images of different types and corresponding segmentation maps/classifiers.
The means for accepting banknotes is of any suitable type as known in the art as is the imaging means. Any feature selection algorithm known in the art may be used to select one or more types of feature to use in the step of extracting features. Also, the classifier can be formed on the basis of specified information about a particular denomination or currency of banknotes in addition to the feature information discussed herein. For example, information about particularly data rich regions in terms of color or other information, spatial frequency or shapes in a given currency and denomination.
Any range or device value given herein may be extended or altered without losing the effect sought, as will be apparent to the skilled person.
It will be understood that the above description of a preferred embodiment is given by way of example only and that various modifications may be made by those skilled in the art.

Claims

1. A media validator comprising:

(i) an input arranged to receive at least one image of a media item to be validated;

(ii) a segmentation map comprising information about relationships of corresponding image elements between all images in a set of training images of media items;

(iii) a processor arranged to segment the image of the media item using the segmentation map;

(iv) a feature extractor arranged to extract one or more features from each segment of the image of the media item; and

(v) one or more classifiers together arranged to classify the banknote into one of at least three classes on the basis of the extracted features.

2. A media validator as claimed in claim 1 comprising only one classifier, that classifier being arranged to operate at each of a plurality of pre-specified confidence levels.

3. A media validator as claimed in claim 1 comprising a plurality of classifiers each formed from feature information extracted from different ones of the segments.

4. A media validator as claimed in claim 1 comprising means for dividing the image of the media item to be validated into a plurality of regions and further comprising a plurality of segmentation maps, each segmentation map associated with a different one of the regions.

5. A media validator as claimed in claim 4 comprising a plurality of classifiers, each classifier being associated with a different one of the segmentation maps.

6. A media validator as claimed in claim 3 wherein each of the classifiers is further arranged to operate at each of a plurality of pre-specified confidence levels.

7. A media validator as claimed in claim 5 wherein each of the classifiers is further arranged to operate at each of a plurality of pre-specified confidence levels.

8. A media validator as claimed in claim 4 comprising a plurality of classifiers, each classifier being associated with a different one of the segmentation maps and a different segment of that segmentation map.

9. A media validator as claimed in claim 8 wherein each of the classifiers is further arranged to operate at each of a plurality of pre-specified confidence levels.

10. A media validator as claimed in claim 1 wherein the image of the media item is of a particular type and which further comprises a plurality of segmentation maps, each segmentation map being for a different type of media item image.

11. A media validator as claimed in claim 1 wherein the classifier is a one-class classifier.

12. A media validator as claimed in claim 1 comprising means for combining results from a plurality of classifiers.

13. A method of validating a media item comprising:

(i) accessing at least one image of a media item to be validated;

(ii) accessing a segmentation map comprising information about relationships of corresponding image elements between all images in a set of training images of media items;

(iii) segmenting the image of the media item using the segmentation map;

(iv) extracting features from each segment of the image of the media item; and

(v) classifying the media item into one of at least three classes on the basis of the extracted features using one or more classifiers together.

14. A method as claimed in claim 13 which comprises classifying the media item using only one classifier, that classifier being arranged to operate at each of a plurality of pre-specified confidence levels.

15. A method as claimed in claim 13 which comprises classifying the media item using a plurality of classifiers each comprising feature information extracted from different ones of the segments.

16. A method as claimed in claim 13 which further comprises dividing the image of the media item into a plurality of regions and accessing a plurality of segmentation maps, each segmentation map associated with a different one of the regions.

17. A method as claimed in claim 16 which further comprises classifying the media item using a plurality of classifiers, each classifier being associated with a different one of the segmentation maps.

18. A method as claimed in claim 15 which further comprises operating each of the classifiers at a plurality of pre-specified confidence levels.

19. A method as claimed in claim 17 which further comprises operating each of the classifiers at a plurality of pre-specified confidence levels.

20. A method as claimed in claim 16 which further comprises classifying the media item using a plurality of classifiers, each classifier being associated with a different one of the segmentation maps and a different segment of that segmentation map.

21. A method as claimed in claim 20 which further comprises operating each of the classifiers at a plurality of pre-specified confidence levels.

22. A method as claimed in claim 13 wherein the image of the media item is of a particular type and which comprises accessing a plurality of segmentation maps, each segmentation map being for a different type of media item image.

23. A method as claimed in claim 13 comprising combining results from a plurality of classifiers.

24. A computer program comprising computer program code means adapted to perform all the steps of a method of validating a banknote comprising:

(i) accessing at least one image of a banknote to be validated;

(ii) accessing a segmentation map comprising information about relationships of corresponding image elements between all images in a set of training images of banknotes;

(iii) segmenting the image of the banknote using the segmentation map;

(iv) extracting features from each segment of the banknote image; and

(v) classifying the banknote into one of at least three classes on the basis of the extracted features using one or more classifiers together,

when said program is run on a computer.

25. A computer program as claimed in claim 24 embodied on a computer readable medium.

26. A self-service apparatus comprising:

(i) a means for accepting media items,

(ii) imaging means for obtaining digital images of the media items; and

(iii) a media validator comprising:

(iv) a feature extractor arranged to extract one or more features from each segment of the media item image; and

(v) one or more classifiers together arranged to classify the media item into one of at least three classes on the basis of the extracted features.