CN101366061B - Detecting improved quality counterfeit media items - Google Patents

Detecting improved quality counterfeit media items Download PDF

Info

Publication number
CN101366061B
CN101366061B CN2006800473687A CN200680047368A CN101366061B CN 101366061 B CN101366061 B CN 101366061B CN 2006800473687 A CN2006800473687 A CN 2006800473687A CN 200680047368 A CN200680047368 A CN 200680047368A CN 101366061 B CN101366061 B CN 101366061B
Authority
CN
China
Prior art keywords
mapping
cut apart
segmentation
image
sorter
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN2006800473687A
Other languages
Chinese (zh)
Other versions
CN101366061A (en
Inventor
何超
佳里·罗斯
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
NCR Voyix Corp
Original Assignee
NCR Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US11/366,147 external-priority patent/US20070140551A1/en
Application filed by NCR Corp filed Critical NCR Corp
Publication of CN101366061A publication Critical patent/CN101366061A/en
Application granted granted Critical
Publication of CN101366061B publication Critical patent/CN101366061B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Abstract

A method of creating a classifier for media validation is described. Information from all of a set of training images from genuine media items is used to form a segmentation map which is then used to segment each of the training set images. Features are extracted from the segments and used to form a classifier which is preferably a one-class statistical classifier. Classifiers can be quickly and simply formed, for example when the media is a banknote for different currencies and denominations in this way and without the need for examples of counterfeit banknotes. A media validator using such a classifier is described as well as a method of validating a banknote using such a classifier. In a preferred embodiment a plurality of segmentation maps are formed, having different numbers of segments. If higher quality counterfeit media items come into the population of media items, the media validator is able to react immediately by switching to using a segmentation map having a higher number of segments without the need for re-training.

Description

The counterfeit media items of detection improvement quality
The cross reference of related application
The application is the part continuation application of No. 11/366,147, the U.S. Patent application submitted on March 2nd, 2006, and U.S. Patent application is for 11/366, No. 147 the part continuation application of No. 11/305,537, the U.S. Patent application submitted on Dec 16th, 2005.The application that No. 11/366,147, the application that on March 2nd, 2006 submitted to and on Dec 16th, 2005 submit to is incorporated herein by reference for 11/305, No. 537.
Technical field
This instructions relates to a kind of method and apparatus that is used for media verify.Relating to particularly can be to the counterfeit media items (for example, passport, check, bank note, bond, stock) that improves quality or this type of method and apparatus that other this media work.
Background technology
More and more need be with mode automatic inspection and the bank note of verifying different Currency Types and denomination simple, reliable and the saving cost.For example, this is essential in the self-service equipment (for example, self-service retail kiosk, ticket machine, the ATM (Automatic Teller Machine) that is arranged to handle deposit, self-service money changing machine etc.) that receives bank note.The automatic checking of the valuable medium of other types (for example, passport, check etc.) also is essential.
In the past, the manual method of media verify relate to the image inspection of bank note, passport, check etc., such as transparent effect and the feel even the smell of watermark and lines alignment mark.Other known methods rely on the semi-over feature that needs half artificial inquiry.For example, use magnetic devices, UV sensor, fluorescence, infrared detector, electric capacity, bonding jumper, image model etc.Yet with regard to itself, these methods are artificial or are half artificial, and are not suitable for many application of manual intervention for a long time.For example, in self-service equipment.
To have major issue to have to be overcome in order to create the automated media validator.For example, there is many dissimilar currency with different security features even matrix type.The security feature that also has different brackets at different denominations usually.Therefore, need provide a kind of be used for those different Currency Types and denomination, easily and be convenient to carry out the conventional method of currency validation.
In brief, the task of cash inspecting machine be determine given bank note be true or false.Automatic verification method in the past needs a large amount of relatively known counterfeit money samples with training classifier usually.In addition, those previous sorters are trained only to detect known counterfeit money.This is problematic, and reason is, exists for possible counterfeit money seldom or not to have available information.For example, this is problematic for the denomination or the new Currency Type of introducing of new introducing especially.
Be published in the 37th phase of pattern-recognition (Pattem Recognition 37) (2004) 1085-1096 pages or leaves, the author is Chao He, in MarkGirolami and Gary Ross (wherein two people are present inventors), the early stage paper of exercise question for " Employing optimized combinations ofone-class classifiers for automated currency validation (adopting the optimum combination of one-class classifier for automatic banknote validation) ", (patent No. is EP1484719, US20042447169) to have described the automatic currency verification method.This relates to the image segmentation one-tenth zone of use network with whole bank note.Be independent " single class (the one-class) " sorter of each areal structure, and the little subclass of combination zone specific classification device is to provide comprehensive description (explaining term " single class " in more detail below).Cutting apart and make up by the regional specific classification device that adopts genetic algorithm to be embodied as to realize superperformance.This method needs to forge on a small quantity sample in genetic algorithm in the stage, similarly, in the time can't obtaining data falsification, this method is inapplicable.
Also need carry out the automatic currency checking in the little mode of computing cost of energy executed in real time.
Other problem relates to automatic currency verification system situation of operation relatively successfully in position and under given environment.For example, described environment comprises the genuine notes with given quantitative range and distribution and overall (population) of counterfeit money.If described environment is undergone mutation, then this automatic currency verification system is difficult to adapt to usually.For example, it is overall that the counterfeit money of supposing new better quality begins to enter bank note suddenly, and police's wisdom, manually checking and other information sources may be indicated the appearance of the counterfeit money of better quality.In this case, if bank or other suppliers find that counterfeit money just is accepted at the automatic currency proof machine, then makes commercial decision usually and stops using those machines.Yet this is expensive, because need artificial checking to replace, and consumer's inconvenience.Also need to expend important time and money and upgrade the automatic currency verification system to handle the counterfeit money of better quality.
Many problems above-mentioned also are applied to the checking of the valuable medium (for example, passport, check etc.) of other types.
Summary of the invention
The method that a kind of establishment is used for the sorter of media verify has been described.Only use to form and cut apart mapping, then, uses and describedly cut apart mapping and cut apart each and train integrated images from the information of all images in the training image set of genuine dielectric object (media items).Feature is extracted from segmentation, and uses described feature to form sorter, the preferably single class statistical sorter of described sorter.Can also form sorter simply fast to different Currency Types and denomination by this way, and not need the sample of counterfeit media items object.The method of having described the media verify device that uses this sorter and having used this sorter checking bank note.In a preferred embodiment, form a plurality of mappings of cutting apart, shine upon the segmentation with varying number described cutting apart.If the counterfeit media items object of better quality enters the overall of dielectric object, then the media verify device uses the mapping of cutting apart of the segmentation with comparatively high amts to need not just to train again and can make a response immediately by switching to.
Can carry out described method by the software of the machine-readable form on storage medium.Can be by the step of proper order that it will be apparent to those skilled in that and/or the described method of executed in parallel.
This expression, software can be commodity valuable, that can separately buy and sell.This means the software that comprises operation or control " dummy argument (dumb) " or standard software, with the function of carry out desired, (therefore, software is the function of definition register in fact, even therefore before its and its standard hardware is made up, also can be called as register).Owing to similar reason, also comprise the software of the structure of " description " or definition hardware, for example design silicon or the structure employed HDL of universal programmable chips (hardware description language) software, with the function of carry out desired.
Understand as the technician, but the preferable feature appropriate combination, and can with any characteristics combination in the many-side of the present invention.
Description of drawings
To by by way of example embodiments of the invention be described with reference to following accompanying drawing, wherein:
Fig. 1 is a process flow diagram of creating the method for the sorter that is used for banknote validation;
Fig. 2 is a synoptic diagram of creating the equipment of the sorter that is used for banknote validation;
Fig. 3 is the synoptic diagram of biil validator;
Fig. 4 is the process flow diagram of the method for checking bank note;
Fig. 5 is the process flow diagram of method that the appearance of the counterfeit money of improving quality is dynamically made a response;
Fig. 6 is the synoptic diagram of cutting apart mapping that is used for two segmentations;
Fig. 7 is about cutting apart the curve map of the quantity of segmentation in the mapping to three kinds of different Currency Type false acceptance rate/false rejection rates;
Fig. 8 is and the similar curve map of Fig. 7 the selection of its indication number of fragments;
Fig. 9 is that the counterfeit money for the improvement quality of Fig. 8 enters the curve map under the overall situation;
Figure 10 is for the curve map under the situation of the false rejection rate of the demonstration exaggeration of Fig. 8;
Figure 11 is for Fig. 9's but is to use curve map under the situation of cutting apart mapping of the segmentation with comparatively high amts;
Figure 12 is the synoptic diagram with self-service equipment of biil validator.
Embodiment
Below only as example, embodiments of the invention are described.The best way that applies the present invention to put into practice known to these example shown applicant is present, rather than can realize the mode that only has of the present invention.Realize although here this example is described and is shown in the banknote validation system, provide described system as example, rather than restriction.It will be appreciated by those skilled in the art that this example is suitable for the application in the media verify system of number of different types, described media verify system comprises passport verification system, check verification system, bond verification system and stock verification system, but is not limited thereto.
Use term " one-class classifier " to represent to use the sorter that forms or construct from the information of the sample of single class about only, but it is used for emerging sample dispensing is given described single class or do not distributed to described single class.These are different with the traditional binary sorter, and described traditional binary sorter is by using the information about the sample of two classes to be created, and are used to new samples is distributed in described two classes one or another.One-class classifier can be considered to define known class border on every side, thereby the sample that breaks away from described border is considered to not belong to described known class.
Fig. 1 is an outline flowchart of creating the method for the sorter that is used for banknote validation.
At first, we obtain the training set (referring to the square frame 10 of Fig. 1) of the image of genuine notes.These are images of same type of the bank note of the identical Currency Type obtained and denomination.The type of image relates to how obtaining image, and this can use any way as known in the art.For example, reflected image, image, heat picture, infrared image, ultraviolet image, x light image or other image types on the transmission image, red, blue or green any passage.Image in the training set is aligned, and size is identical.As known in the prior art, if necessary, can carry out pre-service and come registration image and zoomed image.
Then, we cut apart mapping (referring to the square frame 12 of Fig. 1) by using from the information creating of training integrated images.Describedly cut apart mapping and comprise the information that about how image division is become a plurality of segmentations (segment).Described segmentation can be discontinuous, promptly given segmentation can comprise in the zones of different of image more than one sheet (patch).Form in any suitable manner and cut apart mapping, and the following example that provides certain methods in detail.For example, the distribution of the amplitude of each pixel of a plurality of images that in based on the training of image set, use and form described segmentation with the relation of the amplitude of other pixels of composing images.Preferably, but nonessential, cut apart the segmentation that mapping also comprises the specific quantity that will use.For example, Fig. 6 has the synoptic diagram of cutting apart mapping 60 that label is two segmentations of 1 and 2 in the drawings.Cut apart the corresponding surf zone (area) with bank note of segmentation 1 and segmentation 2 of mapping, described segmentation 1 comprises that those are designated as 1 zone, and segmentation 2 comprises that those are designated as 2 zone.Cut apart the expression that mapping will comprise the whole surf zone of bank note for one.When segmentation was based on Pixel Information, the maximum quantity of segmentation was the total quantity of the pixel in the image of bank note.
Mapping is cut apart in use, and we cut apart each image (referring to the square frame 14 of Fig. 1) in the training set.Then, we are from each stage extraction one or more features (referring to the square frame 15 of Fig. 1) of each training integrated images.For term " feature ", we represent any statistic or other characteristics of segmentation.For example, any other statistic in the pattern of average pixel luminance (intensity), intermediate value pixel intensity, pixel intensity, texture, histogram, Fourier transform descriptor, wavelet transformation descriptor and/or the segmentation.
Then, form sorter (referring to the square frame 16 of Fig. 1) by use characteristic information.Can use the sorter of any adequate types as be known in the art.In concrete preferred embodiment of the present invention, sorter is an one-class classifier, need be about the information of counterfeit money.Yet, also can use the sorter of the other types of binary classifier or any adequate types as be known in the art.
The method of Fig. 1 makes that the sorter of checking of the bank note be used for specific Currency Type and denomination can formation simply, fast and effectively.In order to create the sorter that is used for other Currency Types or denomination, repeat described method with the suitable training integrated images.
The mapping of cutting apart of using varying number to cut apart produces different results.In addition, along with the quantity increase of segmentation, the needed processing of each bank note also increases.Accordingly, in a preferred embodiment, we carry out to follow the tracks of (if about the information of counterfeit money can with) be used to cut apart the segmentation of mapping with selection optimal number in training and inspection period.
This is illustrated among Fig. 1.Inspection-classification device (referring to square frame 17) is with the performance of visit according to the sorter of false acceptance rate and/or false rejection rate.False acceptance rate is that sorter indication counterfeit money is the indication of genuine frequency.False rejection rate is that how long to indicate genuine notes be the indication of false frequency to sorter.Known counterfeit or " prosthese (the dummy) " counterfeit for checking purpose to create are used in this check.
Then, repeat the method for Fig. 1 for the segmentation of cutting apart varying number in the mapping and select the optimal number (referring to square frame 18) of segmentation.For example, this is undertaken by formation and Fig. 7 curve similar with Fig. 8.If there is not available counterfeit to be used for check, quantity that then can segmentation is set to the good quantity of most of Currency Type work.Our experimental result shows that the Currency Type with good Safety Design only needs 2 to 5 segmentations to accept and the False Rejects performance with the mistake that realizes; And the Currency Type with poor Safety Design may need about 15 segmentations.
Store optimal segmentation mapping and one or more other then and optionally cut apart mapping (referring to the square frame 19 of Fig. 1).Cut apart in the mapping each for these, can calculate and store relevant sorting parameter set.
Fig. 7 is to use the curve map of cutting apart the quantity of segmentation in the mapping of false acceptance rate/three kinds of Currency Types of false rejection rate contrast of banknote validation method described herein.Use curve a, b, c represent the false acceptance rate of three kinds of Currency Types.To every kind of Currency Type, false rejection rate is similar and represent with line 70.
As can be seen, along with the increase of cutting apart number of fragments in the mapping, the chance that mistake is accepted counterfeit reduces.Yet, in the risk of refusal genuine notes, less increase is arranged.
In a preferred embodiment, we select the segmentation of minimum number, thereby false acceptance rate is almost nil.For example, Fig. 8 similar to Fig. 7 shows the quantity of the segmentation X that uses this Standard Selection.
Yet, may there be a certain moment at the life period of Currency Type, the quality of counterfeit money improves.For example, this Currency Type may become the target of more organized forgery clique (counterfeit ring).In addition, more senior reproduction technology or skill can become available.In this case, counterfeit money may be by described automatic system as really accepting.Shown in 90 among Fig. 9, this receptance that leads to errors increases.If the automatic currency verification system only have use small number of segments X cut apart mapping (referring to Fig. 9 and Figure 10), then all possible then be with make the false rejection rate increase very high (referring to Figure 10 100).This expression counterfeit money can not be accepted, but cost be the refusal vast scale genuine notes (be 100% under extreme case, promptly do not support this Currency Type/denomination temporarily, this is common in current reality.)。In order not need to cut off service or to address this problem under the situation of training classifier again, we are cutting apart to shine upon and replace the original mapping of cutting apart with a kind of predefined selectable segmentation with comparatively high amts simply.Available and predefined selectable another relevant sorting parameter of mapping of cutting apart is gathered and is replaced cutting apart the set of relevant first sorting parameter of mapping with original.
This is illustrated in Figure 11.The quantity of cutting apart segmentation in the mapping is the Y bigger than X now.As can be seen, the false rejection rate at the Y place the same with false acceptance rate remain low.
By replacing the sorting parameter set by this way, needn't train again.Therefore, can be fast and adjust the system that is used for the automatic currency checking simply, with introducing in response to the better quality counterfeit money.In presents, with reference to Fig. 5 this is explained in more detail after a while.
Provide more details now about the example of cutting techniques.
Before, in EP1484719 and US20042447169, (as mentioning in the background technology part) our use relates to used the cutting techniques and the genetic algorithm method of network to cut apart mapping with formation to the plane of delineation.This must use the information about counterfeit money, and when carrying out the genetic algorithm search, causes the increase that assesses the cost.
The present invention uses and forms the distinct methods of cutting apart mapping, and described method does not need to use genetic algorithm or equivalent method, to search for the good mapping of cutting apart in the mapping in a large amount of possible cutting apart.This has reduced to assess the cost, and has improved performance.In addition, need be about the information of counterfeit money.
We think, in pseudo-fabrication technique, are difficult to provide the consistent quality of whole bank note to be imitated usually, and therefore, the specific region of bank note more is difficult to success than other zones and duplicates.Therefore, we recognize, do not use the mesh segmentation of strict conformance, and we can complicated dividing improves banknote validation by using more.The empirical test that we carry out the above-mentioned situation of indication is exactly this situation in fact.Cause better performance based on being segmented in the detection counterfeit money of morphological characteristic such as pattern, color and texture.Yet, in the time will being applied to train each image in the set, be difficult to use described traditional images dividing method such as the traditional images dividing method that uses edge detector.This is because each training set entry is obtained different results, and is difficult to aim at the individual features in the different training integrated images.For fear of the problem of aiming at segmentation, in a preferred embodiment, we use and are called as " time-space image decomposition ".
Provide the details of cutting apart the method for mapping about formation now.On summary, this method can be considered to specify how the plane of delineation is divided into a plurality of segmentations, and each segmentation comprises a plurality of specified pixels.As mentioned above, described segmentation can be discontinuous.In the present invention, based on writing this instructions from the information of all images in the training set.On the contrary, use the cutting apart of strict network need be from the information of image in the training set.
For example, each cut apart mapping comprise with all images during training is gathered between the relevant information of relation of respective image element.
Think that the image in the training set piles up mutually and aims in same orientation.Obtain the given pixel in the banknote image plane, this pixel is considered to have " pixel intensity profile (profile) ", and described pixel intensity profile comprises the information about the pixel intensity of the specific pixel location in each training integrated images.Use any suitable cluster (clustering) algorithm, the location of pixels in the plane of delineation is clustered into segmentation, the location of pixels in described segmentation has similar or relevant pixel intensity profile.
In preferred exemplary, we use these pixel intensity profiles.Yet, be not to use the pixel intensity profile.Also can use other information from all images in the training set.For example, the mean value of the pixel intensity of the pixel of same position in the brightness profile of the piece of 4 neighbors or each the training integrated images.
The concrete preferred embodiment of the method for mapping is cut apart in the formation of describing us now in detail.This is based on the method for instructing in the following publication: Lecture Notes in ComputerScience (computer science teaching materials), 2352:747-758, Avidan in 2002, " EigenSegments:A spatio-temporal decomposition of an ensemble ofimages (peculiar segmentation: the space-time of image assemblage decomposes) " of S..
Given image assemblage (the ensemble) { I that has aimed at and be scaled to identical big or small r * c iI=1,2, Λ, N, each image I iCan be represented as [α with vector form by its pixel 1i, α 2i, Λ, α Mi] T, wherein, α Ji(j=1,2, Λ M) is the brightness of j pixel in i the image, M=rc is the total quantity of pixel in the image.Then can be by piling up the vectorial I of all images in (stacking) assemblage i(using average to make zero) produces design matrix
Figure S2006800473687D00101
Therefore, A=[I 1, I 2, Λ, I N].Row vector [α among the A J1, α I2, Λ, α JN] can be regarded as the brightness profile of the specific pixel (j) of N image.If two pixels are from the model identical zone of image, then they may have similar brightness value, and therefore have strong temporal correlation.Notice that the term here " time " does not need accurately corresponding with time shaft, but using this term " time " indicates the axle that passes different images in the assemblage.Our algorithm is attempted finding these correlativitys, and will be divided into the zone of the pixel with similar time behavior on the plane of delineation space.We measure this correlativity by the matrix between the definition brightness profile.Plain mode is to use Euclidean distance, and promptly the temporal correlation between two pixel j and the k can be represented as d ( j , k ) = Σ i = 1 N ( a ji - a ki ) 2 . (j, k) more little, the correlativity between two pixels is strong more for d.
In order to use the temporal correlation between the pixel spatially to decompose the plane of delineation, we carry out clustering algorithm to pixel intensity profile (row of design matrix A).This with related pixel on the generation time bunch.The most direct selection is to adopt the K mean algorithm, but can be any other clustering algorithm.As a result, the plane of delineation is divided into the plurality of segments of last related pixel of time.Then, can use this segmentation to cut apart all images in the training set as template; And the feature of extracting in can those segmentations about all images from training set is come the structural classification device.
Realize training, preferably one-class classifier in order not utilize counterfeit money.Can use the one-class classifier of any adequate types well known in the prior art.For example, based on the one-class classifier of neural network with based on the one-class classifier of adding up.
The suitable statistical method that is used for the classification of single class is based on usually from target class and extracts log-likelihood ratio maximization under the null hypothesis of the observed reading of being considered, and these methods comprise that the hypothetical target class is the D of multivariate Gaussian distribution 2Check (at Morrison, is described among the DF:MultivariateStatistical Methods (Multivariable Statistical Methods) (third edition).McGrawHill publishing company, New York, 1990).Under the situation of any non-Gaussian distribution, can mix (at Bishop by half parameter of using Gauss for example, CM:Neural Networks for PatternRecognition (neural network that is used for pattern-recognition) describes, the Oxford University Press, New York, 1995) or the nonparametric handkerchief once (Parzen) window (at Duda, RO, Hart, PE, Stork, describe among the DG:Pattern Classification (pattern classification) (second edition), John Wiley and Sons, INC, New York, 2001) come the density of estimating target class, and can pass through such as sample certainly (bootstrap) (at Wang, S, Woodward, WA, Gary, people such as HL: describe among the A newtest for outlier detcetion from a multivariate mixture distribution (the new check that exceptional value detects from Multivariate Mixed distributes), Journal of Computationaland Graphical Statistics (calculate and figure add up periodical), 6 (3): 285-299,1997) Sampling techniques obtain the distribution of the log-likelihood ratio under the null hypothesis.
Can be that the support vector data field is described (SVDD) (at Tax to single class adoptable additive method of classifying, DMJ, Duin, describe among the RPW:Support vector domain description (description of support vector territory), Pattern Recognition Letters (pattern-recognition wall bulletin), 20 (11-12): 1191-1199,1999), also have known " supporting to estimate (supportestimation) " (at Hayton, P, Scholkopf, B, Tarrassenko, L, Anuzis, describe among the P:Support Vector Novelty Detection Applied to Jet Engine VibrationSpectra (support vector that is applied to the jet engine vibrational spectra newly detects), Advances in Neural Information Processing Systems (the neural network information handling system is entered rank), 13, eds Leen, Todd K and Dietterich, Thomas G and Tresp, Volker, MIT Press, 946-952,2001) and extreme value theory (EVT) (at Roberts, .IEE Proceedings on Vision, Image ﹠amp are described among the SJ:Novelty detection using extreme value statistics (using the new detection of statistics of extremes); Signal Processing (about the IEE minutes of vision, image and signal Processing), 146 (3): 124-129,1999).In SVDD, estimate the DATA DISTRIBUTION of support, EVT estimates the distribution of extreme value simultaneously.To this application-specific, use a large amount of genuine notes samples, therefore, in this case, can obtain the reliable estimation that target class distributes.Accordingly, in a preferred embodiment, we select clearly to estimate single class sorting technique of Density Distribution, although this is dispensable.In a preferred embodiment, we use the D based on parameter 2Single class sorting technique of check.
For example, it is as follows the statistic test of hypothesis of the one-class classifier be used for us to be described in detail in detail:
Suppose that N is independent, equally distributed p dimensional vector sample (characteristic set of each bank note) x 1, Λ, x N∈ C has the basis density function p (x| θ) about parameter θ.To new some x N+1Provide following test of hypothesis, so that H 0: x N+1∈ C, and H 1 : x N + 1 ∉ C , Wherein, C represents that null hypothesis is genuine zone, and defines C by p (x| θ).Suppose that distribution under the alternative hypothesis is uniformly, the normal state log-likelihood ratio of so invalid and alternative hypothesis
λ = sup θ ∈ Θ L 0 ( θ ) sup θ ∈ Θ L 1 ( θ ) = sup θ Π n = 1 N + 1 p ( x n | θ ) sup θ Π n = 1 N ( x n | θ ) - - - ( 1 )
The test statistics that can be used as null hypothesis.In the preferred embodiment, we can use the test statistics of log-likelihood ratio as the checking of the bank note that is used for up-to-date appearance.
Proper vector with multivariate gaussian density: the proper vector of supposing the indivedual points in the description sample is multivariate Gauss, whether the every bit from the check assessment sample that above likelihood ratio (1) occurs shares COMMON MEAN ((is being described among the Morrison, DF:Multivariate StatisticalMethods (Multivariable Statistical Methods) (third edition).McGraw-Hill publishing company, New York, 1990)).Suppose that N is independent, equally distributed p dimensional vector sample x1, Λ, x NFrom the multivariate normal distribution with average μ and covariance C, its sample estimation is
Figure S2006800473687D00123
With
Figure S2006800473687D00124
The option table at random of sample is shown x 0, relevant square Ma Shi (Mahalanobis) distance
D 2 = ( x 0 - μ ^ N ) T C ^ N - 1 ( x 0 - μ ^ N ) - - - ( 2 )
Can be expressed as being distributed as the center F with p and N-p-1 degree of freedom distributes
F = ( N - p - 1 ) ND 2 p ( N - 1 ) 2 - Np D 2 - - - ( 3 )
Then, if
F>F α;p,N-p-1 (4)
Common population mean vector x then 0With residue x iNull hypothesis will be rejected, wherein, F α; P, N-P-1Be to have degree of freedom (p, last α 100% point that F N-p-1) distributes.
Hypothesis is selected x now 0As having maximum D 2The observation vector of statistic.From size is the maximum D of the random sample of N 2Complex distribution.Yet, can obtain the conservative approximate value of percent 100 α on this critical value by Bao Falongni (Bonferroni) inequality.Therefore, if
F > F α N ; p , N - p - 1 - - - ( 5 )
Then we can conclude x 0It is exceptional value.
In fact, equation (4) and equation (5) all can be used for the exceptional value detection.
As additional data x N+1But the time spent, in the check of the new samples that designs the part that does not form crude sampling, we can use the increment of following average and covariance to estimate, i.e. average
μ ^ N + 1 = 1 N + 1 { N μ ^ N + x N + 1 } - - - ( 6 )
And covariance
C ^ N + 1 = N N + 1 C ^ N + N ( N + 1 ) 2 ( x N + 1 - μ ^ N ) ( x N + 1 - μ ^ N ) T . - - - ( 7 )
By using expression formula (6) and (7) and matrix inversion theorem, the equation (2) that is used for N sampling reference set and N+1 check point becomes
D 2 = σ N + 1 T C ^ N + 1 - 1 σ N + 1 - - - ( 8 )
Wherein,
σ N + 1 = ( x N + 1 - μ ^ N + 1 ) = N N + 1 ( x N + 1 - μ ^ N ) | - - - ( 9 )
C ^ N + 1 - 1 = N + 1 N ( C ^ N - 1 - C ^ N - 1 ( x N + 1 - μ ^ N ) ( x N + 1 - μ ^ N ) T C ^ N - 1 N + 1 + ( x N + 1 - μ ^ N ) T C ^ N - 1 ( x N + 1 - μ ^ N ) ) - - - ( 10 )
Pass through D N+1, N 2Represent
Figure S2006800473687D00137
Then
D 2 = ND N + 1 , N 2 N + 1 + D N + 1 , N 2 - - - ( 11 )
So, can be according to the average of common estimation And covariance
Figure S2006800473687D00142
Estimation and the normal distribution of hypothesis check new some x N+1Although find that the hypothesis of multivariate Gauss feature vector is suitable suitable selections to many application, the hypothesis of multivariate Gauss feature vector is false in fact usually.In the part below, we abandon this hypothesis, and consider any density.
Proper vector with any density: by (for example using any half suitable parameter well known in the prior art, gauss hybrid models) or nonparametric (for example, handkerchief is the window method once) density estimation method, can be from the finite data sample that from any density p (x), extracts
Figure S2006800473687D00143
Figure S2006800473687D00144
The acquisition probability density is estimated
Figure S2006800473687D00145
Then, in calculating log-likelihood ratio (1), can adopt this density.Different with the situation of multivariate Gaussian distribution, under null hypothesis, there is not the analysis distribution of test statistics (λ).So, in order to obtain this distribution, can adopt the other non-analysis ineffective distribution of numeral under the density that the methods of sampling obtains to estimate, so, various critical value λ set up can distributing from the experience that obtains CirtAs can be seen, at ultimate value N → ∞, likelihood ratio can be by with the estimation of getting off
λ = sup θ ∈ Θ L 0 ( θ ) sup θ ∈ Θ L 1 ( θ ) → p ^ ( x N + 1 ; θ ^ N ) - - - ( 12 )
Wherein,
Figure S2006800473687D00147
Be illustrated in the x under the model of estimating by original N sample N+1Probability density.
Gather from sampling and using it to estimate the parameter of Density Distribution at the B that produces N sample from reference data set
Figure S2006800473687D00148
Afterwards, can be by selecting N+1 sample at random and calculating p ^ ( x N + 1 ; θ ^ N i ) ≈ λ crit i Obtain the test statistics λ that B duplicates from sampling Crit i, i=1, K, B.By in the mode of ascending order to λ Crit iOrdering can limit critical value α, thus if λ≤λ α, then refuse null hypothesis in the level of signifiance of expectation, wherein, λ αBe λ Crit iJ minimum value, and α=j/ (B+1).
Preferably, be different segments, repeat to form the method for sorter, and use the image of the bank note of the known true and false to test.Then, select to provide the quantity of segmentation of the set of optimum performance and corresponding sorting parameter.We find that although can use the segmentation of any suitable quantity, for most Currency Types, the optimal number of segmentation approximately is from 2 to 15.
Fig. 2 is a synoptic diagram of creating the equipment 20 of the sorter 22 that is used for banknote validation.It comprises:
Import 21, be configured to visit the training set of banknote image;
Processor 23 is configured to use that training set creation of image is a plurality of cuts apart mapping, and each is cut apart mapping and has different segments;
Dispenser 24 is configured to use one of the selection cut apart in the mapping to cut apart each training integrated images;
Feature extractor 25 is configured to extract one or more features in each segmentation from each training integrated images;
Processor 23 also can be configured to the result who uses dispenser 24 and feature extractor 25 and cut apart the set of mapping calculating sorting parameter for each.
Classification forms device 26, is configured to use first set of selecting of sorting parameter set; And
Adapter 27 is configured to replace the first sorting parameter set of selecting with one in other sorting parameter set,
Wherein, configuration processor is to cut apart mapping based on creating from the information of all images in the training set.For example, by using above-mentioned picture breakdown when empty.
Selectively, the equipment that is used to create described sorter also comprises selector switch, described selector switch is cut apart the classification performance of mapping by estimating each, selects optimal segmentation mapping and/or relevant classification parameter sets, and one or more optional mapping and/or relevant classification parameter sets cut apart.
Fig. 3 is the synoptic diagram of biil validator 31.It comprises:
Import, be configured to receive at least one image 30 of the bank note that to be verified;
A plurality ofly cut apart mapping 32, each is cut apart mapping and has the segmentation of varying number, by optimal segmentation mapping determining in the training stage and one or more selectable cut apart to shine upon form.
Processor 33 is configured to use first to cut apart the image that bank note is cut apart in mapping;
Feature extractor 34 is configured to extract one or more features from each segmentation of banknote image;
Sorter 35 is configured to bank note is categorized as effective or invalid based on the feature of being extracted; And
Adapter 36 is configured to replace first to cut apart mapping with other of cutting apart in the mapping, and uses and cut apart the relevant sorter of mapping with described other and replace this sorter;
Wherein, cut apart mapping and be based on that each relevant information in the set with the training image of bank note forms.Notice that the device of Fig. 3 is not must be separate, these devices can be one.
Fig. 4 is the process flow diagram of the method for checking bank note.Described method comprises:
Visit is with at least one image (square frame 40) of the bank note that is verified;
Mapping (square frame 41) is cut apart in visit;
The image (square frame 42) that bank note is cut apart in mapping is cut apart in use;
From each segmentation of banknote image, extract feature (square frame 43);
Use sorter bank note to be categorized as effective or invalid (square frame 44) based on the feature of being extracted;
Wherein, cut apart mapping be based on the training image of bank note set in each relevant information form.Can or make up these steps of carrying out described method with any suitable order well known in the prior art.Cut apart mapping can imply comprise with training set in each image-related information, this is to cut apart mapping because can form based on described information.Yet the information that implies in cutting apart mapping can be the simple files with pixel address tabulation, and it will be included in each segmentation.
Fig. 5 is a process flow diagram of dynamically adjusting the method for biil validator.Receive the relevant information (referring to square frame 50) of appearance with the counterfeit money that may be accepted by system.Receive this information at the biil validator place or in the central management location that then this information is conveyed to one or more biil validator.For example, the centre management node is published to biil validator by communication network or with any other suitable manner with instruction.
The instruction triggers of described information or reception is the activation of cutting apart mapping (referring to square frame 51) of storage optionally.This is cut apart mapping and has different quantity (segmentation of comparatively high amts usually) with the previous mapping of using of cutting apart.This optionally can be cut apart mapping and be stored in the self-service equipment in advance localizedly, or be stored in the server, if necessary, described server is concentrated the affected equipment that is distributed to by network remote then.In case activate the described mapping of optionally cutting apart, then replace the previous mapping of cutting apart as sending with reference to the described branch of Fig. 4.That is, use shine upon split image 52 described optionally cutting apart.From each segmentation, extract feature (referring to square frame 53), and come bank note classify (referring to square frame 54) based on the feature of being extracted.The mapping of cutting apart of each storage also can be relevant with sorting parameter set precalculated, storage.In this case, the information of reception (square frame 50) can trigger and will activate in the selectable sorting parameter set that is used for as described herein the sorter that dielectric object is classified uses.
Although now mapping is optionally cut apart in use, the developer can create the new attack that counterfeit is resisted in mapping of cutting apart, and describedly new cut apart mapping and optionally cuts apart the segmentation that lesser amt is used in mapping than described.Therefore, in the issue (distribution) that any training again, template development and synthetic material take place, the described use of optionally cutting apart mapping makes the automatic currency checking handle and can carry out.
In said method, only create and store one and optionally cut apart mapping.Yet that can create and store a plurality of segmentations with varying number this optionally cuts apart mapping.Then, can be based on test and error, or, select to use that optionally to cut apart mapping based on the former experience and/or the details of attacking about the specific counterfeit that is just experiencing.
In addition, method described herein is paid close attention to the situation of the quantity increase of segmentation.Yet the quantity of segmentation also can reduce.For example, suppose that the optional template of just using has 15 segmentations.This causes high relatively processing cost and burden.After a while, thus the template of cutting apart with less segmentation can be returned in the source that prevents counterfeit money.
In the past, cutting apart was separately based on the locus, and we are by improving this based on cutting apart to come such as the eigenwert of the pixel intensity profile of pixel in the training set.By this way, each training integrated images is all influential to cutting apart.Yet, in the past, when using mesh segmentation, be not this situation.
Figure 12 is the synoptic diagram with self-service equipment 121 of biil validator 123.It comprises:
Be used to accept the device 120 of bank note;
Be used to obtain the imaging device 122 of the digital picture of bank note; And
Aforesaid biil validator 123.
The same with imaging device, the device that is used to accept bank note is any adequate types well known in the prior art.But the use characteristic selection algorithm selects one or more features to use in extracting characterization step.In addition, except that characteristic information discussed here, also can form sorter based on the customizing messages relevant with the special denomination of bank note or Currency Type.For example, with the information in the zone of significantly enriching, shape or the spatial frequency in given Currency Type and denomination in data aspect the color.
Can carry out method described herein to image or other expressions of bank note, described image/expression is any suitable type.For example, image or aforesaid other images on redness, blueness and the green channel.
Can be only form and cut apart based on the image of a type (such as red channel).Selectively, can form based on the image of all types (such as red, blueness and green channel) and cut apart mapping.Also can form a plurality of mappings of cutting apart, to every kind of image or being combined to form one and cutting apart mapping to multiple image type.For example, may exist three to cut apart mapping, cut apart mapping for one and be used for the red channel image, cut apart mapping for one and be used for the blue channel image, cut apart mapping for one and be used for the green channel image.In this case, during independent banknote validation, can use the suitable mapping/sorter of cutting apart according to the type of selected image.Therefore, can revise above-mentioned every kind of method by using dissimilar images and cutting apart mapping/sorter accordingly.
For the technician is obvious: under the situation of not losing effect, can expand or change given any scope or device value here.
Should be appreciated that only provided the above description of preferred embodiment as example, those skilled in the art can carry out various modifications.

Claims (18)

1. an establishment is used for the method for the sorter of media verify, and described method comprises step:
(i) training of the image of access medium object set;
Use (ii) that training set image creation is a plurality of cuts apart mapping, each cut apart mapping comprise with described training set in the relevant information of relation of respective image element between all images, and each cuts apart the segmentation that mapping has varying number;
(iii) each is cut apart mapping, describedly cut apart mapping and cut apart each training integrated images and calculate sorting parameter set by using, and from each segmentation of each described training integrated images, extract one or more features;
(iv) use first sorting parameter of selecting in the sorting parameter set to gather and form described sorter; And
(v) replace the described first selection sort parameter sets with one in other sorting parameter set.
2. the method for claim 1 is wherein selected described first selection sort parameter sets about the information of known counterfeit to the check of sorter based on using.
3. the method for claim 1 is wherein selected the described first selection sort parameter sets based on the information relevant with the classification performance of cutting apart mapping of the segmentation with a plurality of varying numbers.
4. the method for claim 1 wherein replaces the step of the described first selection sort parameter sets based on the information relevant with the overall change of dielectric object.
5. the method for claim 1 is wherein saidly cut apart mapping and is based on all images that passes in the described training set and is arranged in cluster by using clustering algorithm that the pixel of the plane of delineation creates.
6. the method for claim 1, also comprise feature that the use characteristic selection algorithm selects one or more types with the step of extracting feature (iii) in use.
7. the method for claim 1, wherein said sorter is used for banknote validation, and described method also comprises: form described sorter based on the specifying information relevant with Currency Type with the special denomination of bank note.
8. the method for claim 1 also comprises: at the step that forms described sorter (necessary assembled classifier v).
9. equipment of creating the bank note sorter comprises:
(i) input is configured to the training set of the image of access medium object;
(ii) processor, be configured to use that training set image creation is a plurality of cuts apart mapping, each cut apart mapping comprise with described training set in the relevant information of relation of respective image element between all images, and each cuts apart the segmentation that mapping has varying number;
(iii) dispenser is configured to use and cuts apart in the mapping first and cut apart mapping and cut apart each described training integrated images;
(iv) feature extractor is configured to extract in each segmentation from each described training integrated images one or more features;
(v) classification forms device, is configured to use described characteristic information to form described sorter;
(vi) selector switch is configured to select optimal segmentation to shine upon and one or morely optionally cut apart mapping with each described mapping respective classified performance of cutting apart of creating in (ii) in step by estimating.
10. media verify device comprises:
(i) import, be configured to receive at least one image of the dielectric object that to be verified;
(ii) a plurality ofly cut apart mapping, each cuts apart the segmentation that mapping has varying number, and each cuts apart the relevant information of relation that mapping comprises the respective image element between all images in the training set with dielectric object;
(iii) processor is configured to use and cuts apart in the mapping first and cut apart the image that described dielectric object is cut apart in mapping;
(iv) feature extractor is configured to extract one or more features from each segmentation of the image of described dielectric object;
(v) sorter is configured to described dielectric object is categorized as effective or invalid based on the feature of being extracted; And
(vi) adapter is configured to cut apart in the mapping one with other and cuts apart mapping and replace first to cut apart mapping, and correspondingly revises described sorter.
11. media verify device as claimed in claim 10 is wherein saidly cut apart mapping and is comprised morphologic information.
12. media verify device as claimed in claim 10, wherein said cut apart mapping comprise with each described training integrated images in the relevant information of pixel of same position.
13. media verify device as claimed in claim 10 is wherein saidly cut apart mapping and is comprised the pixel intensity profile.
14. media verify device as claimed in claim 10, wherein said sorter is an one-class classifier.
15. a method of verifying dielectric object comprises:
(i) at least one image of the dielectric object that visit will be verified;
(ii) visit a plurality of mappings of cutting apart, described a plurality of cut apart mapping comprise with training set in the relevant information of relation of respective image element between all images, and each is cut apart and shines upon the segmentation with varying number;
(iii) select a plurality ofly to cut apart first in the mapping and cut apart mapping;
(iv) use described first of selection to cut apart the image that described dielectric object is cut apart in mapping;
(v) from each segmentation of the image of described dielectric object, extract feature;
(vi) use sorter described dielectric object to be classified based on the feature of extracting; And
(vii) use adapter to cut apart mapping and replace described first to cut apart mapping, and correspondingly revise described sorter to cut apart in the mapping one with other.
16. method as claimed in claim 15 wherein selects step described first in (iii) to cut apart mapping according to the information relevant with the overall change of dielectric object.
17. method as claimed in claim 16, wherein said information comprise the information relevant with the quality of counterfeit media items object.
18. a self-service equipment comprises:
(i) be used to accept the device of dielectric object;
(ii) be used to obtain the imaging device of the digital picture of dielectric object;
(iii) the media verify device comprises
(i) import, be configured to receive at least one image of the dielectric object that to be verified;
(ii) a plurality ofly cut apart mapping, each cuts apart the segmentation that mapping has varying number, and each cuts apart the relevant information of relation that mapping comprises the respective image element between all images in the training set with dielectric object;
(iii) processor is configured to use and cuts apart in the mapping first and cut apart the image that described dielectric object is cut apart in mapping;
(iv) feature extractor is configured to extract one or more features from each segmentation of the image of described dielectric object;
(v) sorter is configured to described dielectric object is categorized as effective or invalid based on the feature of being extracted; And
(vi) adapter is configured to cut apart in the mapping one with other and cuts apart mapping and replace first to cut apart mapping, and correspondingly revises described sorter.
CN2006800473687A 2005-12-16 2006-12-14 Detecting improved quality counterfeit media items Active CN101366061B (en)

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
US30553705A 2005-12-16 2005-12-16
US11/305,537 2005-12-16
US11/366,147 2006-03-02
US11/366,147 US20070140551A1 (en) 2005-12-16 2006-03-02 Banknote validation
PCT/GB2006/004670 WO2007068928A1 (en) 2005-12-16 2006-12-14 Detecting improved quality counterfeit media

Publications (2)

Publication Number Publication Date
CN101366061A CN101366061A (en) 2009-02-11
CN101366061B true CN101366061B (en) 2010-12-08

Family

ID=40206435

Family Applications (4)

Application Number Title Priority Date Filing Date
CN2006800473583A Expired - Fee Related CN101331526B (en) 2005-12-16 2006-09-26 Banknote validation
CN2006800473687A Active CN101366061B (en) 2005-12-16 2006-12-14 Detecting improved quality counterfeit media items
CN2006800475165A Expired - Fee Related CN101331527B (en) 2005-12-16 2006-12-14 Processing images of media items before validation
CN2006800472788A Expired - Fee Related CN101366060B (en) 2005-12-16 2006-12-14 Media validation

Family Applications Before (1)

Application Number Title Priority Date Filing Date
CN2006800473583A Expired - Fee Related CN101331526B (en) 2005-12-16 2006-09-26 Banknote validation

Family Applications After (2)

Application Number Title Priority Date Filing Date
CN2006800475165A Expired - Fee Related CN101331527B (en) 2005-12-16 2006-12-14 Processing images of media items before validation
CN2006800472788A Expired - Fee Related CN101366060B (en) 2005-12-16 2006-12-14 Media validation

Country Status (1)

Country Link
CN (4) CN101331526B (en)

Families Citing this family (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102010055974A1 (en) 2010-12-23 2012-06-28 Giesecke & Devrient Gmbh Method and device for determining a class reference data set for the classification of value documents
CN102110323B (en) * 2011-01-14 2012-11-21 深圳市怡化电脑有限公司 Method and device for examining money
CN102959588A (en) * 2011-04-28 2013-03-06 中国科学院自动化研究所 Method for detecting tampering with color digital image based on chroma of image
CN102306415B (en) * 2011-08-01 2013-06-26 广州广电运通金融电子股份有限公司 Portable valuable file identification device
CN102565074B (en) * 2012-01-09 2014-02-05 西安印钞有限公司 System and method for rechecking images of suspected defective products by small sheet sorter
US8983168B2 (en) * 2012-04-30 2015-03-17 Ncr Corporation System and method of categorising defects in a media item
US9299225B2 (en) * 2014-06-23 2016-03-29 Ncr Corporation Value media dispenser recognition systems
CN105184954B (en) * 2015-08-14 2018-04-06 深圳怡化电脑股份有限公司 A kind of method and banknote tester for detecting bank note
DE102015016716A1 (en) * 2015-12-22 2017-06-22 Giesecke & Devrient Gmbh Method for transmitting transmission data from a transmitting device to a receiving device for processing the transmission data and means for carrying out the method
CN108074320A (en) * 2016-11-10 2018-05-25 深圳怡化电脑股份有限公司 A kind of image-recognizing method and device
CN108806058A (en) * 2017-05-05 2018-11-13 深圳怡化电脑股份有限公司 A kind of paper currency detecting method and device
CN107705417A (en) * 2017-10-10 2018-02-16 深圳怡化电脑股份有限公司 Recognition methods, device, finance device and the storage medium of bank note version
CN111480167A (en) * 2017-12-20 2020-07-31 艾普维真股份有限公司 Authenticated machine learning with multi-digit representation
CN110910561B (en) * 2018-09-18 2021-11-16 深圳怡化电脑股份有限公司 Banknote contamination identification method and device, storage medium and financial equipment
TWI709188B (en) * 2018-09-27 2020-11-01 財團法人工業技術研究院 Fusion-based classifier, classification method, and classification system
CN111599081A (en) * 2020-05-15 2020-08-28 上海应用技术大学 Method and system for collecting and dividing RMB paper money
CN113538809B (en) * 2021-06-11 2023-08-04 深圳怡化电脑科技有限公司 Data processing method and device based on self-service equipment

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1484719A2 (en) * 2003-06-06 2004-12-08 Ncr International Inc. Currency validation

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5729623A (en) * 1993-10-18 1998-03-17 Glory Kogyo Kabushiki Kaisha Pattern recognition apparatus and method of optimizing mask for pattern recognition according to genetic algorithm
JP3369088B2 (en) * 1997-11-21 2003-01-20 富士通株式会社 Paper discrimination device
TWI256226B (en) * 2001-10-30 2006-06-01 Matsushita Electric Ind Co Ltd Data processing system, method and device, computer-readable program recording medium having recorded thereon a data processing program, and recording medium in a data processing system
WO2004023403A1 (en) * 2002-08-30 2004-03-18 Fujitsu Limited Device, method and program for identifying paper sheet_____
US7194105B2 (en) * 2002-10-16 2007-03-20 Hersch Roger D Authentication of documents and articles by moiré patterns
JP2005018688A (en) * 2003-06-30 2005-01-20 Asahi Seiko Kk Banknote validator with reflecting optical sensor

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1484719A2 (en) * 2003-06-06 2004-12-08 Ncr International Inc. Currency validation

Also Published As

Publication number Publication date
CN101331526B (en) 2010-10-13
CN101366060B (en) 2012-08-29
CN101331527A (en) 2008-12-24
CN101331526A (en) 2008-12-24
CN101366061A (en) 2009-02-11
CN101331527B (en) 2011-07-06
CN101366060A (en) 2009-02-11

Similar Documents

Publication Publication Date Title
CN101366061B (en) Detecting improved quality counterfeit media items
JP5219211B2 (en) Banknote confirmation method and apparatus
JP6878575B2 (en) Banknote management methods, systems, programs and recording media
US7639858B2 (en) Currency validation
US8611665B2 (en) Method of recognizing a media item
US8625876B2 (en) Validation template for valuable media of multiple classes
Bruzzone et al. A novel context-sensitive semisupervised SVM classifier robust to mislabeled training samples
CN104298989A (en) Counterfeit identifying method and counterfeit identifying system based on zebra crossing infrared image characteristics
EP1944737A1 (en) Validation template for valuable media of multiple classes
US10438436B2 (en) Method and system for detecting staining

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant