CN103679218B - A kind of handwritten form keyword detection method - Google Patents

Abstract

The invention discloses a kind of handwritten form keyword detection method, set up keyword feature storehouse including extracting the characteristic point of key word image in key word image library；Extract the characteristic point of text image Chinese version to be detected, obtain the characteristic point storehouse of text image to be detected；Extracting the sliding window of text image to be detected, extract the some set of sliding window characteristic of correspondence from characteristic point storehouse, comparison characteristic point set obtains initial matching double points set；Screen initial matching double points set and obtain matching double points accurately；Screen the sliding window in text image to be detected according to matching double points and integrate, obtaining testing result.In characteristic extraction procedure, this method have employed SIFT feature and characterizes.The method is applicable to the key word detection of a large amount of handwritten form document, such as historical document, letter, notes etc., on the premise of setting up blacklist image library, can not only effectively detect the key word in blacklist, more can distinguish the person's handwriting of different authors.

Description

A kind of handwritten form keyword detection method

Technical field

The present invention relates to image detecting technique, particularly relate to a kind of based on word local feature specific write people's handwritten form Key word detection method.

Background technology

The detection of handwritten form key word is to detect specific key word in a large amount of handwritten text images.Handwritten form is crucial at present The method of word detection is typically to carry out character retrieval on the basis of Text region again.

But Chinese character classification is many, writing style is changeable, existing based on the handwritten form key word error detection side identifying word Method is firstly the need of setting up huge Chinese character template base, and takes a significant amount of time training and the classification carrying out feature, the image of early stage Pretreatment and Text segmentation the most largely affect the other result knowing word, and then affect key word testing result；The most this The writing style relying on the method for result identified that difference is not write people is included in and is considered, thus cannot be effectively to handwritten form literary composition The identification of word.

Summary of the invention

Instant invention overcomes and prior art cannot efficiently identify the specific defect writing people's handwritten text, it is proposed that A kind of handwritten form keyword detection method.

The present invention proposes a kind of handwritten form keyword detection method, comprises the steps: step one: obtain key word figure As storehouse, extract the characteristic point of key word image in described key word image library and set up keyword feature storehouse；Step 2: extract to be checked Survey the characteristic point of text image Chinese version, obtain the characteristic point storehouse of described text image to be detected；Step 3: extract literary composition to be detected The sliding window of this image, extracts described sliding window characteristic of correspondence point set, by described feature from described characteristic point storehouse Point set contrasts with described keyword feature storehouse, obtains initial matching double points set；Step 4: according in described sliding window Word geometry information, screens described initial matching double points set and obtains matching double points accurately；Step 5: according to described Matching double points screens the sliding window in described text image to be detected and integrates, and obtains testing result.

In the handwritten form keyword detection method that the present invention proposes, as the spy in described characteristic point set in described step 3 Levy a little with described keyword feature storehouse difference less than threshold value time, this feature point is detected as initial matching double points, described feature Point is shown below:

$\begin{array}{c}|{\mathrm{\&theta;}}_s-{\mathrm{\&theta;}}_w|<p1\\ p2<\frac{{\mathrm{\&sigma;}}_s}{{\mathrm{\&sigma;}}_w}<\frac{1}{p2}\\ {\left|\right|{\stackrel{\&RightArrow;}{d}}_s-{\stackrel{\&RightArrow;}{d}}_w\left|\right|}^2<p3\end{array};$

In formula, θ_sRepresent the characteristic point deflection of key word image, σ_sRepresent the characteristic point yardstick of key word image,Table Show the characteristic point characteristic vector of key word image, θ_wRepresent the characteristic point deflection of text image to be detected, σ_wRepresent literary composition to be detected The characteristic point yardstick of this image,Representing the characteristic point characteristic vector of text image to be detected, p1, p2, p3 represent threshold value respectively.

In the handwritten form keyword detection method that the present invention proposes, in described step 4 by word geometry information about Bundle screens described initial matching double points set, and its process comprises the steps:

Step b1: set up the geological information between the characteristic point set of described characteristic point set and described key word image about Shu Tu；

Step b2: screened described geological information constraints graph by Clique lookup algorithm, deletes described initial The matching double points of error hiding in matching double points set.

In the handwritten form keyword detection method that the present invention proposes, the constraints such as following formula of described geological information constraints graph Represent:

|(x_si-x_sj)-(x_wi-x_sj) | ＜ p4 × Avg (σ_s)；

|(y_si-y_sj)-(y_wi-y_wj) | ＜ p5 × Avg (σ_s)

In formula, x_sRepresent characteristic point abscissa in key word image, x_wRepresent that characteristic point is in sliding window image Abscissa, y_sRepresent characteristic point vertical coordinate in key word image, y_wThen represent the characteristic point vertical seat in sliding window image Mark, Avg (σ_s) represent key word image average characteristics point scale, p4 and p5 represents threshold value.

In the handwritten form keyword detection method that the present invention proposes, described step 5 is screened described text image to be detected In sliding window comprise the steps:

Step c1: if the centre-of gravity shift ratio of the center of gravity of the matching double points of described sliding window and former key word is more than 0.15, delete this slip sliding window, otherwise continue next step；

Step c2: if the match point of described sliding window is inclined with the distribution proportion of former key word in the ratio of left half of distribution Difference, more than 0.18, is deleted this slip sliding window, is otherwise retained this sliding window.

This method will detect the specific hand-written key word writing people by calculating images match degree, i.e. to reserved specific The key word image of person writing and text image to be detected carry out feature extraction respectively, use sliding window at text diagram to be detected As sliding, detect specific hand-written key word.In characteristic extraction procedure, this method have employed SIFT feature and characterizes.Should Method is applicable to the key word detection of a large amount of handwritten form document, such as historical document, letter, notes etc., is setting up blacklist figure As, on the premise of storehouse, can not only effectively detecting the key word in blacklist, more can distinguish the person's handwriting of different authors.

Accompanying drawing explanation

Fig. 1 is the flow chart of handwritten form keyword detection method of the present invention；

Fig. 2 is the schematic diagram of text image to be detected in the present embodiment；

Fig. 3 is the flow chart of key word detection-phase in the present invention；

Fig. 4 is the schematic diagram of matching double points preliminary in the present embodiment；

Fig. 5 is the schematic diagram of geological information figure in the present embodiment；

Fig. 6 is the testing result schematic diagram of the present embodiment；

Fig. 7 be the present embodiment testing result in the schematic diagram of center of gravity；

Fig. 8 be the present embodiment testing result in the schematic diagram of error hiding；

Fig. 9 is the testing result schematic diagram of the present embodiment.

Detailed description of the invention

In conjunction with specific examples below and accompanying drawing, the present invention is described in further detail.Implement the present invention process, Condition, experimental technique etc., outside the lower content mentioned specially, be universal knowledege and the common knowledge of this area, this Bright content is not particularly limited.

The flow process of this method is as it is shown in figure 1, be broadly divided into two stages: feature extraction phases and key word detection-phase. Feature extraction phases is extracted the characteristic point set of key word image in key word image library, and text image to be detected Chinese This characteristic point set, forms keyword feature point storehouse and characteristic point set to be detected respectively after extraction.Key word detects Both are compared by the stage, thus obtain the matching double points between text and key word image, by constantly screening thus Obtain final matching double points, thus complete the detection of key word.

In feature extraction phases, the SIFT feature that first key word image library carries out batch is extracted.Each envelope image can To be described as the set S={F of SIFT feature point_i, each characteristic pointMainly it is made up of 5 data, its Concrete meaning is shown in Table 1.The characteristic point set of each key word image is stored in text, forms feature database.

Table 1SIFT characteristic point data table

For text image to be detected, in advance it is gone segmentation.Document Segmentation to be detected is become some Individual sliding window, comprises several words in each sliding window.Preferably, the text image of every a line is being extracted it During SIFT feature point set, arrange storage by the abscissa x value ascending order of characteristic point, be designated asSo that follow-up slip is sliding Dynamic window extracts.

Due to the repetitive structure that Chinese character is special, the key word characteristics of image direct matching result on text image is very poor , obtain is substantially error hiding characteristic point.So being cut by text image, and use slip sliding window, by feature Coupling reduces in the range of several Chinese characters, can obtain more correct match point.

In the present embodiment, (p, q) represents sliding window, and wherein p and q represents rising of sliding window respectively to utilize set Wind Beginning coordinate and termination coordinate.If text image width is W_d, sliding window width is w, and sliding window moving step length is s, slides Number of windows is n, as in figure 2 it is shown, split text image according to following three kinds of methods and extract slip sliding window:

If 1 W_d＜ w, then n=1, p₁=0, q₁=W_d；

2, otherwise, the coordinate of sliding window is calculated according to equation below (1)

If 3 q_i＞ W_d, then q_i=W_d, i.e. final stage deficiency width also becomes a sliding window.

After obtaining sliding window set, each characteristic matching is carried out just for some sliding window, from text diagram The left side of picture starts to right end, just as a sliding window slided from left to right.Coupling in sliding window is only every time Fall at p for abscissa x_iAnd q_iBetween text feature point set.

In above formula, sliding window width w and sliding window moving step length s represents such as below equation (2) respectively:

$\begin{array}{c}w=W_s\&times;\frac{\mathrm{Avg}\left({\mathrm{\&sigma;}}_d\right)}{\mathrm{Avg}\left({\mathrm{\&sigma;}}_s\right)}\\ s=\frac{w}{6}\end{array}---\left(2\right)$

In formula, W_sFor key word picture traverse, Avg (σ_s) it is the average of key word characteristics of image point scale, Avg (σ_d) it is The average of text image characteristic point.Regulate sliding window width by characteristic point yardstick, be adapted to the dimensional variation of image.

Key word detection-phase is the core of this method.The main process in this stage is as shown in Figure 3.At each sliding window In, key word image characteristic point can use Kd-tree algorithm to find and the spy of 5 arest neighbors in characteristic point set in sliding window Levy a little, form initial matching double points.

Matching double points can be carried out preliminary screening, if not examining first with information such as the deflection of characteristic point, yardsticks Consider the rotation of file and picture, distort and scale, then deflection between two similar characteristic points, scale-value difference would not The biggest.It addition, the king-sized matching double points of the Euclidean distance of characteristic vector also can be deleted, because characteristic vector represents feature Point gradient statistical information around.Utilize formula (3) to matching characteristic point to carrying out Preliminary screening

$\begin{array}{c}|{\mathrm{\&theta;}}_s-{\mathrm{\&theta;}}_w|<p1\\ p2<\frac{{\mathrm{\&sigma;}}_s}{{\mathrm{\&sigma;}}_w}<\frac{1}{p2}\\ {\left|\right|{\stackrel{\&RightArrow;}{d}}_s-{\stackrel{\&RightArrow;}{d}}_w\left|\right|}^2<p3\end{array}---\left(3\right)$

In formula, θ_s、σ_s、And θ_w、σ_w、Represent respectively key word image and the characteristic point deflection of sliding window image, Yardstick, characteristic vector.P1, p2, p3 are corresponding threshold value respectively.Match point result after Preliminary screening is as shown in Figure 4

Fig. 4 has intercepted a slip sliding window fragment in text image, and rectangle frame is the position of sliding window, under Face is shown that the image of key word, and thin straight line is connected in two width images the characteristic point of coupling.

Owing to repetitive structure Fig. 5 of Chinese character still existing a lot of Mismatching point.Characteristic point is utilized to divide in the space of word Geometrical constraint set up by cloth, can delete Mismatching point pair further.The first step first sets up geometrical constraint figure, and its step is as follows:

1, S={s is set₁..., s_mIt is key word characteristics of image point set, W={w₁..., w_mIt is in sliding window image The feature point set matched.Non-directed graph G=(V, E) represents geometrical constraint figure.V={v₁... v_mIt is the vertex set of G, its Middle v_i=(s_i, w_i), represent a summit in the most corresponding G of every a pair match point in S and W.Set for G limit.

2, the limit of G is added as follows, for any two feature point pairs v_i=(s_i, w_i) and v_j=(s_j, w_j), full Foot constraints

|(x_si-x_sj)-(x_wi-x_wj) | ＜ p4 × Avg (σ_s) (4)

|(y_si-y_sj)-(y_wi-y_wj) | ＜ p5 × Avg (σ_s)

Then in vertex v_iAnd v_jBetween add a limit.X in above formula_s、x_wRepresent that characteristic point is at key word image and sliding window Abscissa in mouth image, and y_sAnd y_wThen represent its vertical coordinate, with reference to Fig. 5 example.P4 and p5 is two threshold values, in order to adapt to The size of different key word images, the average dimension value of key word image characteristic point to be multiplied by.Although geometrical constraint figure is to compare The conventional method setting up image geometry contact, different constraints can build different constraints graphs, but the present invention carries The unique constraints gone out, as shown in formula (4), meets the architectural feature of word, is suitable for the screening of feature point pairs, to carrying The accuracy rate of the high present invention has the biggest effect.

After geometrical constraint figure established above, it is assumed that the most all of characteristic point is the most correctly mated, then all in GCG figure Summit will interconnect, form close agglomerate.A maximum son is found when there is error hiding it is necessary in GCG figure Figure, in this subgraph, all of summit is all to interconnect, and Clique lookup algorithm can be used to realize, its concrete steps As follows:

1, candidate vertices collection C=V, Clique vertex set M=φ are initialized；

2, calculate the degree on each summit in V, be designated as gathering Deg (V)；

3, vertex v maximum for Deg (V) in C is selected₀, it is added into M, i.e. M=M ∪ { v₀}；

4, by v₀Delete from C, and only retain and v in C₀The summit being connected, i.e. C=C v₀, C=C ∩ N (v₀)。 Wherein N (v₀) represent v₀Adjacent vertex collection.

If 5 C=φ, algorithm terminates, and otherwise forwards 3 repetition said process to.

By setting up geometrical constraint figure and Clique lookup algorithm, a large amount of Mismatching point pair can be deleted, initial in Fig. 4 Mate through geometrical constraint process after result as shown in Figure 6.With Fig. 4 contrast seen from delete the matching double points of many error hiding. Seeing the contrast of Fig. 8 Yu Fig. 9 again, sliding window covers " promoting " two word, due to the similarity of some strokes, this sliding window with Key word image " food " creates a large amount of error matching points pair.According to text geometry information delete Mismatching point to After, in Fig. 9, the matching double points of display just decreases a lot, it can be seen that the present invention can interpolate that out it is different character images. Geometrical constraint figure is effectively utilized the structural information of word, it is possible to the correctness of feature point pair matching is greatly improved, in the present invention Method plays central role.

After the geometrical constraint screening to match point, each sliding window can the quantity of a corresponding match point, The very few sliding window of quantity is then deleted.Owing to Chinese character complicated and simple degree difference causes himself characteristic point quantity the most different, The match point quantity variance making different Chinese character is very big, therefore uses adaptive threshold to screen sliding window.Formula (5) shows root Threshold value is adjusted according to the quantity of SIFT feature point in key word image

Tn=k × S_fnum (5)

In formula, tn is threshold value, S_fnumCounting for key word characteristics of image, k takes empirical value 0.25.

When some word in key word occurs in text image, the point often mated is the most more.This method uses Two kinds of strategies judge: one is judgement based on centre-of gravity shift, and two is based on a judgement for distribution.

When a certain sliding window characteristic point set is mated with key word image characteristic point, initial matching point can be obtained Collect and after screening, mate point set two set, if remaining center of gravity abscissa and the correspondence of match point after Mismatching point screens When the coordinate offset amount of key word image characteristic point is more than threshold value, delete this sliding window；If it is surplus after Mismatching point screens (on the left of including or right side) distribution proportion and the key word image characteristic point in key word image of remaining match point are at key word figure When distribution proportion in Xiang is more than threshold value, delete this sliding window.I.e. when meeting below equation

|XCent(S₀)-XCent(S_x) | ＞ tc (6)

|LPoi(S₀)-LPoi(S_x) | ＞ tp

Time, then current sliding window mouth is deleted.In formula, XCent (S) represents the center of gravity abscissa of key word image characteristic point, LPoi (S) represents the key word image characteristic point distributive law at left one side of something.Tc and tp is two threshold values, takes empirical value 0.15 He 0.18.As it is shown in fig. 7, two black circles represent the center of gravity of key word image, it is moved to the position in left side from middle position Put, because after screening Mismatching point, match point has the most all concentrated on Chinese character and " eaten ", although match point number also compares Many, but can be seen that the distribution of its match point of judgement through centre-of gravity shift is very big with key word image deviations, this sliding window Still can be deleted, " property food " also will not be detected as " food ", thus reduces the error rate of handwriting recongnition.

Finally need remaining sliding window is merged, because the sliding window of some overlap all contains same Key word.When two sliding window laps account for more than the 60% of whole sliding window, then they are merged into a slip Window.Sliding window after merging is exactly the candidate keywords sliding window eventually detected.

The present embodiment is tested in 2764 envelope text images, 50 authors write, everyone 50 texts.Carry Take key word 15,99 key word positions to be detected, carry out 10 groups of experiments.Recall rate R, false alarm rate E and new F value is used to evaluate Experimental result.Assuming to detect image W envelope, S key word, the most correctly detection Y detected altogether, flase drop is N number of, S=Y+N, should When the key word detected has T, then above three judgment criteria calculates according to formula (7)

$\begin{array}{c}R=\frac{P}{T}\\ E=\frac{N}{W}\\ F=\frac{2}{0.8\&times;\frac{1}{R}+1.2\&times;\left(\frac{1}{1-E}\right)}\end{array}---\left(7\right)$

This method can be embodied to key word power of test by R seen from formula (7), the highest more good；E then to try one's best low, Detection to great amount of images is lacked to introduce erroneous judgement as far as possible, embodies the method distinguishing ability to key word.New F value is then Balance to two above standard, is R and the E harmonic-mean according to different weight calculation, and concrete outcome is shown in Table 2.

Table 2 the present embodiment key word testing result

	R	E	F
				Optimum	91.92%	0.65%	94.91%
Worst-case value	85.86%	3.12%	93.18%
				Meansigma methods	88.64%	1.85%	94.09%

The protection content of the present invention is not limited to above example.Under the spirit and scope without departing substantially from inventive concept, this Skilled person it is conceivable that change and advantage be all included in the present invention, and with appending claims for protect Protect scope.

Claims (4)

1. a handwritten form keyword detection method, it is characterised in that comprise the steps:

Step one: obtain key word image library, the characteristic point extracting key word image in described key word image library sets up key Word feature database；

Step 2: extract the characteristic point of text image Chinese version to be detected, obtains the characteristic point storehouse of described text image to be detected；

Step 3: extract the sliding window of text image to be detected, extracts described sliding window corresponding from described characteristic point storehouse Characteristic point set, described characteristic point set and described keyword feature storehouse are contrasted, the feature in described characteristic point set When point is less than threshold value with described keyword feature storehouse difference, this feature point is detected as initial matching double points, obtains initial Matching double points set；Described characteristic point is shown below:

|θ_s-θ_w|<p1

$p2<\frac{{\mathrm{\&sigma;}}_s}{{\mathrm{\&sigma;}}_w}<\frac{1}{p2};$

$\left|\right|{\stackrel{\&RightArrow;}{d}}_s-{\stackrel{\&RightArrow;}{d}}_w|{|}^2<p3$

In formula, θ_sRepresent the characteristic point deflection of key word image, σ_sRepresent the characteristic point yardstick of key word image,Represent and close The characteristic point characteristic vector of keyword image, θ_wRepresent the characteristic point deflection of text image to be detected, σ_wRepresent text diagram to be detected The characteristic point yardstick of picture,Representing the characteristic point characteristic vector of text image to be detected, p1, p2, p3 represent threshold value respectively；

Step 4: according to word geometry information in described sliding window, screens described initial matching double points set and obtains Matching double points accurately；

Step 5: screen the sliding window in described text image to be detected according to described matching double points and integrate, being detected Result.

2. handwritten form keyword detection method as claimed in claim 1, it is characterised in that several by word in described step 4 Described initial matching double points set is screened in what structural information constraint, and its process comprises the steps:

Step b1: set up the geological information constraint between the characteristic point set of described characteristic point set and described key word image Figure；

Step b2: screened described geological information constraints graph by Clique lookup algorithm, deletes described initial coupling Point is to the matching double points of error hiding in set.

3. handwritten form keyword detection method as claimed in claim 2, it is characterised in that the pact of described geological information constraints graph Bundle condition such as following formula represents:

$\begin{array}{c}|(x_{si}-x_{sj})-(x_{wi}-x_{wj})|<p4\&times;Avg\left({\mathrm{\&sigma;}}_s\right)\\ |(y_{si}-y_{sj})-(y_{wi}-y_{wj})|<p5\&times;Avg\left({\mathrm{\&sigma;}}_s\right)\end{array};$

In formula, x_sRepresent characteristic point abscissa in key word image, x_wRepresent the characteristic point horizontal seat in sliding window image Mark, y_sRepresent characteristic point vertical coordinate in key word image, y_wThen represent characteristic point vertical coordinate in sliding window image, Avg(σ_s) represent key word image average characteristics point scale, p4 and p5 represents threshold value.

4. handwritten form keyword detection method as claimed in claim 1, it is characterised in that treat described in screening in described step 5 Sliding window in detection text image comprises the steps:

Step c1: if the centre-of gravity shift ratio of the center of gravity of the match point of described sliding window and former key word is more than 0.15, delete This sliding window, otherwise continues next step；

Step c2: if the match point of described sliding window is big with the distribution proportion deviation of former key word in the ratio of left half of distribution In 0.18, delete this sliding window, otherwise retain this sliding window.