CN104298987A - Handwritten numeral recognition method based on point density weighting online FCM clustering - Google Patents

Abstract

The invention discloses a handwritten numeral recognition method based on point density weighting online FCM clustering. The method is used for processing the large-scale offline handwritten numeral recognition problem. The method includes the steps that (1), all handwritten numeral image sets are preprocessed; (2), clustering centers are initialized, and data points are made to sequentially enter processing procedures; (3), the membership degree of the current data point and all the clustering centers is calculated; (4), if the membership degree reaches a threshold value, the position of the nearest clustering center is updated; (5), if the membership degree does not reach the threshold value, the current data point is not processed and is temporarily placed in a to-be-processed region; (6), when the to-be-processed region reaches certain standards, data in the to-be-processed region are clustered through a point density weighting FCM algorithm, and the clustering centers are updated; (7), circulation continues until all the data points are processed; (8), the membership degrees of all the data points are calculated through acquired clustering center blocks, the data points are divided into different classes, and data classification is finished through scanning at a time. According to the method, the space complexity and the time complexity can be lowered from the aspect of processing the large-scale handwritten numeral recognition problem.

Description

Based on the Handwritten Digit Recognition method of the online FCM cluster of dot density weighting

Technical field

The invention belongs to electronic information technical field, relate to the Handwritten Digit Recognition of On-line Fuzzy C average (Weighted-FCM) cluster based on dot density weighting, specifically a kind of Handwritten Digit Recognition method based on the online FCM cluster of dot density weighting.

Background technology

Handwritten Digit Recognition belongs to a branch of optical character recognition, and how it is computing machine if mainly being studied recognizes that people is handwritten in the arabic numeral on the media such as paper automatically.This technology has wide practical use at numerous areas such as bank, finance and postcode self-identifyings.In addition, Handwritten Digit Recognition, as a major issue of area of pattern recognition, also has important theory value.This technology current has become the focus of image procossing and area of pattern recognition research, the method of Handwritten Digit Recognition has two classes substantially, Classification and Identification and clustering recognition, clustering recognition is according to certain method for measuring similarity of digital picture characteristic use, identical or close for feature is classified as a class, realizes clustering.Fuzzy clustering has ambiguity and the fault-tolerance of height, but still has certain limitation in actual applications, time as comparatively huge in clustering object quantity, often exceeds the ability to bear of computer hardware, and therefore original fuzzy clustering algorithm just lost efficacy.Current method be mostly to data sampling cluster again post-processed determine the classification of non-sampled data, or piecemeal process is carried out to extensive sample, then finally cluster centre is obtained to each fritter successively cluster.But it is large all also to there is calculated amount, the shortcomings such as space complexity is high, thus more massive data acquisition cannot be tackled.

Summary of the invention

The object of the invention is to overcome above-mentioned the deficiencies in the prior art, propose a kind of Handwritten Digit Recognition method based on the online FCM cluster of dot density weighting.

Technical scheme of the present invention is: based on the Handwritten Digit Recognition method of the online FCM cluster of dot density weighting, comprise the steps:

Step 101: pre-service handwritten numeral picture normalization data, its objective is high dimensional data gray level image being converted into applicable cluster computing;

Step 102: the parameter of setting needed for clustering algorithm: make the set of x representative data, N is the sum of data, and c is classification number, and m is the blur level of FCM Algorithms, and ζ is Studying factors, and U is degree of membership, U _thfor degree of membership threshold value, V (k), k=1,2,3 ... c is an initialized c cluster centre, and i is greater than the integer that 0 is less than or equal to N, x _irepresent i-th data point, pool is temporal data pool, and it is pool higher limit that the satisfied data point upgrading cluster centre condition temporarily puts into pool num, and w is data point weights, and e is clustering operator iteration precision judgment value;

Step 103: read a data point x _i, calculate the degree of membership U of this data point and initialized cluster centre, judge the maximal value U of degree of membership _maxif, U _maxbe greater than U _th, go to step 104, otherwise go to step 105;

Step 104: upgrade distance x _inearest cluster centre position, goes to step 107;

Step 105: by x _iput into pool, in pool, data point number adds 1, judges in pool, whether data amount check reaches preset value num.If reach num to go to step 106, otherwise go to step 107;

Step 106: use existing cluster centre for initial cluster center, calculates each data point weight w in pool _j, by all data points in wFCM cluster pool, upgrade Clustering Model, after obtaining new Clustering Model, data in data pool pool emptied;

Step 107: judge whether i is less than data total amount N, if be less than N, go to step 103 after making i=i+1, otherwise goes to step 108;

Step 108: call wFCM operator cluster again and newly obtain but not yet participate in the final cluster centre of data acquisition in the pool of cluster, calculate a little with the degree of membership of existing cluster centre, determine each data point class mark.

Above-mentioned steps 101, comprises the steps:

Step 201: preprocessing image data: by the gray level image of each 28*28 and be set to one 784 dimension feature value vector;

Step 202: the normalization of data, because image intensity value is the integer between 0 ~ 255, then by and institute's directed quantity of postpone with divided by 255, the scope of obtaining is the normalization data of 0 ~ 1;

Step 203: upset data matrix order at random, cause the randomness of reading in data category.

Renewal distance x described in above-mentioned steps 104 _inearest cluster centre position, comprises the following steps:

Step 301: calculate range data point x _inearest cluster centre V (k _near);

Step 302: upgrade nearest cluster centre position, V (k _near)=ζ (x (i)-V (k _near))+V (k _near), ζ is Studying factors, value ζ=0.03;

Data point weight w in pool is calculated in upper step 106 _j, comprise the steps:

Step 401: the Euclidean distance calculating data point between two in pool, obtains the adjacency matrix d of data point, computing formula:

d _ij＝||x(i)-x(j)||,1≤i≤num,1≤j≤num；

Step 402: utilize adjacency matrix d calculation level density

Step 403: dot density matrix z is normalized, obtains weighting coefficient $w_i=\frac{z_i}{\underset{j=1}{\overset{\mathrm{num}}{\mathrm{\Σ}}}{z}_j},1\≤i\≤\mathrm{num};$

Use data point in wFCM cluster pool to obtain new Clustering Model in above-mentioned steps 106, comprise the steps:

Step 501: the degree of membership calculating each point and current cluster centre in pool;

Step 502: upgrade cluster centre:

$v_i=\frac{\underset{j=1}{\overset{n}{\mathrm{\Σ}}}{w}_j{\left(u_{\mathrm{ij}}\right)}^m{x}_j}{\underset{j=1}{\overset{n}{\mathrm{\Σ}}}{w}_j{\left(u_{\mathrm{ij}}\right)}^m},\∀i;$

Step 503: calculate e=max _1≤i≤c|| v _{i, new}-v _{i, old}|| ²value;

Step 504: if e is greater than ε, go to step 501, otherwise go to step 505;

Step 505: return current cluster centre, is the new Clustering Model after renewal.

Beneficial effect of the present invention: this method is according to existing flow data clustering method, each reading data point, each circulation only calculates degree of membership to a point and existing cluster centre, determines whether making this point participate in the renewal of Clustering Model directly according to the maximal value of degree of membership.Committed step of the present invention is exactly devise a cluster framework based on flow data, in conjunction with the FCM algorithm of online k-means algorithm model update method and dot density weighting, the on-line talking achieving one-by-one carries out the Handwritten Digit Recognition without monitor mode.Present invention, avoiding and process all Digital Image Data simultaneously, requirement to computer hardware when dramatically reducing large-scale data process, simultaneously compared with the wFCM algorithm of existing piecemeal, because major part point has participated in the renewal of Clustering Model directly, decrease the number of times calling wFCM operator, thus save computing time, reduce the complexity of time, be more suitable for processing large-scale data.

Accompanying drawing explanation

Fig. 1 is the general flow chart that the present invention realizes;

Fig. 2 is data prediction process flow diagram;

Fig. 3 is that data point directly upgrades Clustering Model process flow diagram;

Fig. 4 is the process flow diagram calculating weights process in wFCM operator;

Fig. 5 is the process flow diagram that in wFCM operator, iteration obtains stable cluster centre process;

Fig. 6 is experimental result schematic diagram of the present invention.

Embodiment

The present invention realizes the problem of Handwritten Digit Recognition by unsupervised clustering method, mainly be on a grand scale for destination object number, memory headroom needed for computing machine cannot meet the memory requirements of original algorithm, by online method scan-data completed the determination of classification by single pass one by one, thus realize the identification problem of handwritten numeral, realizing environment is MATLAB2008b.The method being realized the problem of fairly large Handwritten Digit Recognition by the method for Unsupervised clustering is a lot, the present invention adopts the clustering method based on dot density Weighted Fuzzy C-Means, with flow data scan mode, the data point that order enters is processed, temporal data pool pool is put into for the data point not meeting update condition, the specific implementation of pool defines the dimension matrix identical with data point in a program, by data point stored in matrix, again cluster is carried out to the data in pool after pool piles, avoid and all cluster operation is carried out to total data, object is the space complexity reducing to calculate, reduce reunion class number of times to reduce time complexity.

As shown in Figure 1, main flow chart step of the present invention is as follows:

Step 101: pre-service handwritten numeral picture, is the high dimensional data in cluster process by greyscale image transitions, often opens the data line after image corresponding conversion, be convenient to follow-up clustering processing;

Step 102: the parameter of setting needed for clustering algorithm: make the set of x representative data, N is the sum of data, and c is cluster classification number, and m is the blur level in FCM Algorithms cluster process, ζ upgrades the Studying factors in cluster centre process, and U is degree of membership, U _thfor degree of membership threshold value, V (k), k=1,2,3 ... c is initialized cluster centre, and i is greater than the integer that 0 is less than or equal to N, x _irepresent i-th each data point, pool is that num is pool higher limit, and w is data point weights, and e is that the iteration stopping of dot density weighted FCM judges precision not by the data pool of the data of degree of membership threshold decision after storage calculates;

Step 103: read a data point x _i, calculate the degree of membership U of this data point and each initialized cluster centre:

$u_{\mathrm{ij}}={\left[\underset{k=1}{\overset{c}{\mathrm{\Σ}}}{\left(\frac{\left|\right|x_j-v_i\left|\right|}{\left|\right|x_j-v_k\left|\right|}\right)}^{\frac{2}{m-1}}\right]}^{-1},\∀i,j$

Judge the maximal value U of degree of membership _maxif, U _maxbe greater than U _th, go to step 104, otherwise go to step 105;

Step 104: this point meets renewal cluster centre condition, very near with a certain cluster centre distance, this dot-dash is grouped into this classification, and then upgrade the cluster centre position nearest apart from this point, computing formula is:

V(k _near)＝ζ·(x(i)-V(k _near))+V(k _near)，

Go to step 107;

Step 105: this point is all close with existing cluster centre distance, then think that this point cannot determine that classification belongs under current cluster centre distribution situation, then temporary transient by this point data stored in data pool pool, in pool, data point number adds 1, judges in pool, whether data amount check reaches preset value num.If reach num to go to step 106, otherwise go to step 107;

Step 106: calculate each data point weight w in pool _jweights size is determined by the dense degree of this ambient data point, the larger then weights of density are larger, and the possibility that this point becomes cluster centre is larger, substitute into weights wFCM clustering algorithm and use existing cluster centre V (k), k=1,2,3..., c are as initial cluster center, all data points in cluster pool, upgrade c cluster centre V (k _new), data pool pool can be emptied after upgrading cluster centre, wait and enter to be placed into follow-up but the data point that classification belongs to cannot be determined;

Step 107: judge whether i is less than data total amount N, if be less than N, go to step 103 after making i=i+1, otherwise goes to step 108;

Step 108: call available data point in a wFCM operator cluster pool again, so far all data points have calculated complete all, obtain final cluster centre V (k), calculate a little with the degree of membership of existing cluster centre, degree of membership computing formula:

$u_{\mathrm{ij}}={\left[\underset{k=1}{\overset{c}{\mathrm{\Σ}}}{\left(\frac{\left|\right|x_j-v_i\left|\right|}{\left|\right|x_j-v_k\left|\right|}\right)}^{\frac{2}{m-1}}\right]}^{-1},\∀i,j$

Determine each data point class mark according to degree of membership size, calculate ARI (Adjusted Rand Index).

As shown in Figure 2,

Described step 101, comprises the steps:

Step 201: preprocessing image data: by the gray level image of each 28*28 and be set to one 784 dimension feature value vector;

Step 202: the normalization of data, because image intensity value is the integer between 0 ~ 255, then by and institute's directed quantity of postpone with divided by 255, the scope of obtaining is the normalization data of 0 ~ 1;

Step 203: in order to avoid producing the phenomenon that flocks together when data are read in, namely mass data all belongs to same class and causes cluster centre skewness continuously, so upset data matrix order at random, causes the randomness of reading in data category.

As shown in Figure 3, affiliated step 104, comprises the following steps:

Step 301: calculate range data point x _inearest cluster centre V (k _near);

Step 302: upgrade nearest cluster centre position, V (k _near)=ζ (x (i)-V (k _near))+V (k _near), ζ is Studying factors, the amplitude that this value effect cluster centre upgrades, and have certain influence to Clustering Effect, need to regulate as the case may be, this value is a smaller value, generally can get ζ=0.03;

As shown in Figure 4, data point weight w in pool is calculated in affiliated step 106 _j, comprise the steps:

Step 401: the Euclidean distance calculating data point between two in pool, obtains the adjacency matrix d of data point, Euclidean distance computing formula:

d _ij＝||x(i)-x(j)||,1≤i≤num,1≤j≤num；

Step 402: utilize adjacency matrix d to calculate the dot density of each point in pool, obtains dot density matrix z, dot density computing formula:

$z_i=\underset{j=1,j\≠i}{\overset{\mathrm{num}}{\mathrm{\Σ}}}\frac{1}{d_{\mathrm{ij}}},1\≤i\≤\mathrm{num};$

Step 403: dot density matrix z is normalized, obtains weight matrix, weights coefficient formulas:

$w_i=\frac{z_i}{\underset{j=1}{\overset{\mathrm{num}}{\mathrm{\Σ}}}{z}_j},1\≤i\≤\mathrm{num};$

As shown in Figure 5, use data point in wFCM cluster pool to obtain new Clustering Model in affiliated step 106, comprise the steps:

Step 501: the degree of membership calculating each point and current cluster centre in pool obtains subordinated-degree matrix U:

$u_{\mathrm{ij}}={\left[\underset{k=1}{\overset{c}{\mathrm{\Σ}}}{\left(\frac{\left|\right|x_j-v_i\left|\right|}{\left|\right|x_j-v_k\left|\right|}\right)}^{\frac{2}{m-1}}\right]}^{-1},\∀i,j,$

Step 502: upgrade cluster centre according to subordinated-degree matrix U:

$v_i=\frac{\underset{j=1}{\overset{n}{\mathrm{\Σ}}}{w}_j{\left(u_{\mathrm{ij}}\right)}^m{x}_j}{\underset{j=1}{\overset{n}{\mathrm{\Σ}}}{w}_j{\left(u_{\mathrm{ij}}\right)}^m},\∀i;$

Step 503: calculate e=max _1≤i≤c|| v _{i, new}-v _{i, old}|| ²value;

Step 504: if e is greater than ε, go to step 501, otherwise go to step 505;

Step 505: return current cluster centre, is the new Clustering Model after renewal;

Be illustrated in figure 6 experimental result schematic diagram of the present invention, can find out that the present invention obtains good cluster centre, ten class digital pictures can be found out clearly from Fig. 5, be followed successively by 0,2,5,6,4,8,7,3,9,1 from top to bottom from left to right.The present invention tests total degree more than 50 times, and calculating ARI (Adjusted Random Index) mean value is 0.3401, and maximal value is 0.4002, and minimum value is 0.2813, and variance is 0.0159.

To sum up, the present invention is according to existing flow data clustering method, and each reading data point, each circulation only calculates degree of membership to a point and existing cluster centre, determines whether making this point participate in the renewal of Clustering Model directly according to the maximal value of degree of membership.Such as now read a data xi from data centralization and each cluster centre calculates degree of membership U, if to be greater than value much bigger compared with its residual value for degree of membership maximal value, exceed default threshold value U _th, then this point participates in Renewal model directly.If U maximal value does not exceed threshold value, then think that this point wouldn't belong to any one class, and this point is temporary in pool, when the number of data points in pool reaches a certain numerical value, the convenient fuzzy C-means clustering carrying out dot density weighting with the initial classes center that existing cluster centre is total data in pool, obtain the cluster centre after upgrading and be current Clustering Model, proceed the process of follow-up data.Committed step of the present invention is exactly devise a cluster framework based on flow data, in conjunction with the FCM algorithm of online k-means algorithm model update method and dot density weighting, the on-line talking achieving one-by-one carries out the Handwritten Digit Recognition without monitor mode, is more applicable to the large-scale data acquisition of process.

Present invention, avoiding and process all Digital Image Data simultaneously, requirement to computer hardware when dramatically reducing large-scale data process, simultaneously compared with the wFCM algorithm of existing piecemeal, because major part point has participated in the renewal of Clustering Model directly, decrease the number of times calling wFCM operator, thus save computing time, reduce the complexity of time, be more suitable for processing large-scale data.

The part do not described in detail in present embodiment belongs to the known conventional means of the industry, does not describe one by one here.More than exemplifying is only illustrate of the present invention, does not form the restriction to protection scope of the present invention, everyly all belongs within protection scope of the present invention with the same or analogous design of the present invention.

Claims (5)

1., based on the Handwritten Digit Recognition method of the online FCM cluster of dot density weighting, it is characterized in that: comprise the steps:

Step 101: pre-service handwritten numeral picture normalization data, its objective is high dimensional data gray level image being converted into applicable cluster computing;

Step 102: the parameter of setting needed for clustering algorithm: make the set of x representative data, N is the sum of data, and c is classification number, and m is the blur level of FCM Algorithms, and ζ is Studying factors, and U is degree of membership, U _thfor degree of membership threshold value, V (k), k=1,2,3 ... c is an initialized c cluster centre, and i is greater than the integer that 0 is less than or equal to N, x _irepresent i-th data point, pool is temporal data pool, and it is pool higher limit that the satisfied data point upgrading cluster centre condition temporarily puts into pool num, and w is data point weights, and e is clustering operator iteration precision judgment value;

Step 103: read a data point x _i, calculate the degree of membership U of this data point and initialized cluster centre, judge the maximal value U of degree of membership _maxif, U _maxbe greater than U _th, go to step 104, otherwise go to step 105;

Step 104: upgrade distance x _inearest cluster centre position, goes to step 107;

Step 105: by x _iput into pool, in pool, data point number adds 1, judges in pool, whether data amount check reaches preset value num, if reach num to go to step 106, otherwise goes to step 107;

Step 106: use existing cluster centre for initial cluster center, calculates each data point weight w in pool _j, by all data points in wFCM cluster pool, upgrade Clustering Model, after upgrading Clustering Model, data in data pool pool emptied;

Step 107: judge whether i is less than data total amount N, if be less than N, go to step 103 after making i=i+1, otherwise goes to step 108;

Step 108: call wFCM operator cluster again and newly obtain but not yet participate in the final cluster centre of data acquisition in the pool of cluster, calculate a little with the degree of membership of existing cluster centre, determine each data point class mark.

2. the Handwritten Digit Recognition method based on the online FCM cluster of dot density weighting according to claim 1, is characterized in that: described step 101, comprises the steps:

Step 201: preprocessing image data: by the gray level image of each 28*28 and be set to one 784 dimension feature value vector;

Step 202: the normalization of data, because image intensity value is the integer between 0 ~ 255, then by and institute's directed quantity of postpone with divided by 255, the scope of obtaining is the normalization data of 0 ~ 1;

Step 203: upset data matrix order at random, cause the randomness of reading in data category.

3. the Handwritten Digit Recognition method based on the online FCM cluster of dot density weighting according to claim 1, is characterized in that: the renewal distance x described in step 104 _inearest cluster centre position, comprises the following steps:

Step 301: calculate range data point x _inearest cluster centre V (k _near);

Step 302: upgrade nearest cluster centre position, V (k _near)=ζ (x (i)-V (k _near))+V (k _near), ζ is Studying factors, value ζ=0.03;

4. the Handwritten Digit Recognition method based on the online FCM cluster of dot density weighting according to claim 1, is characterized in that: calculate data point weight w in pool in described step 106 _j, comprise the steps:

Step 401: the Euclidean distance calculating data point between two in pool, obtains the adjacency matrix d of data point, computing formula:

d _ij＝‖x(i)-x(j)‖,1≤i≤num,1≤j≤num；

Step 402: utilize adjacency matrix d calculation level density

Step 403: dot density matrix z is normalized, obtains weighting coefficient

$w_i=\frac{z_i}{\underset{j=1}{\overset{\mathrm{num}}{\mathrm{\Σ}}}{z}_j},1\≤i\≤\mathrm{num};$

5. the Handwritten Digit Recognition method based on the online FCM cluster of dot density weighting according to claim 1, is characterized in that: use data point in wFCM cluster pool to obtain new Clustering Model in described step 106, comprise the steps:

Step 501: the degree of membership calculating each point and current cluster centre in pool;

Step 502: upgrade cluster centre:

$v_i=\frac{\underset{j=1}{\overset{n}{\mathrm{\Σ}}}{w}_j{\left(u_{\mathrm{ij}}\right)}^m{x}_j}{\underset{j=1}{\overset{n}{\mathrm{\Σ}}}{w}_j{\left(u_{\mathrm{ij}}\right)}^m},\∀i;$

Step 503: calculate e=max _1≤i≤c{ ‖ v _{i, new}-v _{i, old}‖ ²value;

Step 504: if e is greater than ε, go to step 501, otherwise go to step 505;

Step 505: return current cluster centre, is the new Clustering Model after renewal.