CN104008119A - One-to-many mixed string comparison method - Google Patents
One-to-many mixed string comparison method Download PDFInfo
- Publication number
- CN104008119A CN104008119A CN201310746846.5A CN201310746846A CN104008119A CN 104008119 A CN104008119 A CN 104008119A CN 201310746846 A CN201310746846 A CN 201310746846A CN 104008119 A CN104008119 A CN 104008119A
- Authority
- CN
- China
- Prior art keywords
- string
- matching
- matching degree
- character
- source
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/903—Querying
- G06F16/90335—Query processing
- G06F16/90344—Query processing by using string matching techniques
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a one-to-many mixed string comparison method which is characterized in that a most similar or matched target string is found out for a source string in a group of strings to be compared. The one-to-many mixed string comparison method includes that a GST * algorithm is improved, a POC (partial order comparison) algorithm is then utilized; matching degree value acquired from the two kinds of algorithms is subjected to weight fusion to acquire final matching degree by combining respective characteristics of the algorithms in aspects of disorder and partial order matching of the strings; besides, synonymy substrings in the source string and the strings to be compared are equivalently substituted into the same string by adopting an equivalent string substitution strategy according to different ways of expression of synonymy strings in different occasions, and the matching degree of the two strings can be greatly improved. The source string and strings to be compared are matched separately, the matching degrees are then sorted, the string with the highest matching degree is taken as the target string, and better matching of one-to-many mixed strings is realized.
Description
Technical field
The invention belongs to character string intelligence comparison technology field, the mixed characters string that is specifically related to a kind of novel one-to-many merges comparison method.
Background technology
Character string comparison problem is a basic problem in computer science, and its research contents all has important using value in various fields such as information retrieval, pattern-recognitions
[1]-[4].
Document 1 is studied respectively Approximate String Matching for Chinese Text, and document 2 has been studied a kind of Chinese character string similarity calculating method based on Chinese characters clustering feature.3 couples of LCS of document and GST algorithm compare, GST algorithm is a kind of greedy character string alignment algorithm, is also a kind of unordered matching algorithm, and application is wider at present, but this algorithm has adopted two character strings method of charactor comparison one by one, so the time complexity of algorithm is larger.Document 4 has been studied GST algorithm has been improved to rear RKR-GST algorithm, improved the operational efficiency of GST algorithm, but in RKR-GST algorithm, the selection of hash function is very large to the influence on system operation of algorithm.
Existing character string comparison method often only adopts a kind of algorithm, there is no to make full use of unordered character substring and the features of partial order character substring when matching degree is calculated, and often their comparison effect is unsatisfactory.In the practical application of some some mixed characters strings, not only require the accuracy of comparison high, and require the speed of comparison fast.At present, by single matching degree computing method, be often difficult to the similarity degree of accurately expressing character string.
In addition, existing character string comparison method is not considered the situation that synonym character string may different expression waies, makes existing character string comparison method be difficult in such cases reach requirement more accurate, high matching rate.
List of references:
[1] old building canals, Zhao Jie, Peng Zhiwei. Fast Approximate String Matching for Chinese Text [J]. Journal of Chinese Information Processing, 2003,18 (2): 58-65
[2] Wang Jingting. the Chinese character string similarity based on Chinese characters clustering feature is calculated research [J]. modern Library technology, 2011,20 (2): 48-53
[3] Yu Haiying. LCS and GST algorithm comparison [J] in similarity of character string tolerance. electronics technology, 2011,24 (3): 101-103
[4] ox analysis and the realization [J] of clean .RKR_GST algorithm in _ NET forever. infotech, 2012,3:171-174
Summary of the invention
In view of the above deficiency of prior art, the object of this invention is to provide a kind of string of mixed characters more accurately and merge comparison method.Solved in practical application and with single matching degree computing method, to be difficult to reach similarity degree between accurate expression character string, synonym character string and to have under different expression way situations the existing character string comparison method problem such as almost lost efficacy.
The object of the invention is to realize by following means:
A kind of mixed characters string of one-to-many merges comparison method, similarity to the mixed characters string being comprised of Chinese character, numeral, English alphabet based on Chinese characters clustering feature is carried out fusion ratio pair, to improve the similar degree of accuracy of expressing character string, comprise following key step:
1) take out source string and one group of matching string;
2) read the character string building the in advance dictionary of replacing of equal value in storer, partial character (substring) in this group matching string is carried out to equivalence and replace; Utilize the dictionary of replacing of equal value, by above-mentioned, in source string occasion and matching string occasion, there is different describe but two kinds of identical substrings of implication are unified;
3) take out source string, take out according to this matching string in this matching string array after of equal value replacement;
4) utilize GST* algorithm to calculate the matching degree a of source string and this matching string:
Adopt traditional GST algorithm, obtain each public substring in two character strings, they are deposited in public substring chained list.If the character length of certain public substring is more than or equal to 0.33 with the ratio of longer character string character length, when calculating matching degree, the character number of this public substring is multiplied by weight, this weight is to be greater than 1 constant; If the ratio of the character length of certain public substring and longer character string character length be less than 0.33 and the character number of public substring be greater than smallest match length, while calculating matching degree, directly bring the character number of this public substring into calculating;
5) partially orderly string matching algorithm POC(Partial Order Comparison, POC of utilization) calculate the matching degree b of source string and matching string:
Two to be matched mixed characters strings that contain Chinese character, numeral and English alphabet are called to source string and matching string,
First, first search out source string and character or Chinese character identical in matching string, record their number;
Secondly, the longer character string in source string and matching string of take is standard, asks matching degree 1 (match_degree1):
Take is wherein standard compared with short character strings, asks matching degree 2 (match_degree2):
In formula (1), (2), [] represents to round;
Again, respectively the 1st or the 2nd numeral and letter in reference source character string and matching string, last 1 or second-to-last numeral and letter, if wherein 1 equate, the match_degree2 numerical value of adjusting matching degree 2 is match_degree2+1:
To matching degree 1 and matching degree 2, give different weight 0.41,0.59, ask the final matching value b of source string and matching string:
b＝match_degree1×0.41+match_degree2×0.59 （3）
6) the matching degree b of the matching degree a of step 4) GST* calculating gained and step 5) POC calculating gained is weighted to fusion, fusion method is, if matching degree a is greater than matching degree b, final matching degree is a; If matching degree a is less than matching degree b, final matching degree equals (a+b)/2;
7) each matching string in source string and matching string array is calculated to the matching degree that obtains and sort, matching string corresponding to maximum matching degree, as the target string mating most with source string.
In step 4), first search out each public substring that source string is identical with matching string, then give different weights to the public substring of different length, increased the weight of longer common characters substring.
GST* algorithm of the present invention, the larger phenomenon of matching degree of the long public substring of shorter public substring matching degree possibility existing for traditional GST algorithm, it is improved: if the character length of public substring is more than or equal to 0.33 with the ratio of longer character string character length, when calculating matching degree, the character number of this public substring is multiplied by weight (being greater than 1 constant); If the ratio of the character length of public substring and longer character string character length be less than 0.33 and the character number of public substring be greater than smallest match length, while calculating matching degree, directly bring the character number of this public substring into calculating.
In step 5), using two mixed characters strings that contain numeral, letter, Chinese character respectively as source string and matching string; Take respectively wherein longer character string, compared with short character strings, be standard, obtain matching degree 1 and matching degree 2; And then relatively whether first or a plurality of numeral equate with letter with alphabetical, last or a plurality of numeral, and matching degree 2 is modified.Finally to two kinds of matching degrees, give respectively different weights, obtain two matching degree values between character string.
Character string alignment algorithm POC partially in order of the present invention considers that matching degree 2 more can reflect actual match situation, therefore gives the weight that matching degree 2 is larger a little.
The present invention has provided character string replacement policy of equal value.Such as, " high-pressure side " and " 220KV side ", " kilovolt " and " kV " are of equal value in implication.Adopt existing all kinds of alignment algorithm all can not reflect exactly the relation of equivalence between them, therefore propose character string replacement policy of equal value.Build in advance the character substring dictionary of replacing of equal value, adopt: the form of the source substring of middle substring=equivalence to be matched, such as kilovolt=kV, it represents that the character substring of equal sign both sides is identical in implication, equal sign left side substring represents certain substring in matching string, substring in the source string of the representative of equal sign right side substring and left side equivalence.
Doing before matching degree calculates, first checking in matching string, whether to contain that character substring is of equal value replaces in dictionary the character substring in left side in each row, if had, replacing it for the source character substring on equal sign right side.On this basis, then use this fusion alignment algorithm to compare, calculate corresponding matching degree, so greatly improved the degree of accuracy of coupling, can reflect and participate in relatively real match condition between two character strings.
The present invention is applicable to the comparison of one-to-many mixed characters string.Calculate respectively the matching degree of source string and one group of matching string, and each matching degree obtaining is sorted, therefrom find out the matching string with source string matching degree maximum, it is defined as to target string, thereby has realized the best match of one-to-many character string.
Accompanying drawing explanation:
Fig. 1 is the process flow diagram of the fusion comparison method of novel one-to-many character string.
Fig. 2 is the application example of the fusion comparison method of one-to-many mixed characters string.
Embodiment
Below in conjunction with accompanying drawing, method of the present invention is described in further detail
Below in conjunction with accompanying drawing, the present invention is done further and described in detail.The present invention is specifically related to a kind of fusion comparison method of mixed characters string.First character string to be matched is called to source string and matching string.The present invention more can be suitable for finding the object matching character string of mating most with source string from one group of matching string.
Embodiment is as follows.
1. take out source string and one group of matching string;
2. the character string dictionary of replacing of equal value of reading prior structure, carries out equivalence to partial character in this group matching string and replaces.For example " high-pressure side " and " 220KV side " equivalence, " kilovolt " and " kV " equivalence.Before carrying out the calculating of string matching degree, utilize the dictionary of replacing of equal value above-mentioned different description can be unified;
3. take out source string, take out according to this matching string in this group matching string array after of equal value replacement;
4. utilize GST* algorithm to calculate the matching degree of source string and matching string.
The improvement effect of GST* algorithm and traditional GST algorithm, by illustrating below.
For example " abcde " is two groups of character strings to be compared with " qbcio ", " abcde " with " qbico ", utilizes GST algorithm to calculate two groups of string matching degree and is 40%.
And adopting GST* algorithm to calculate two groups of string matching degree, result is respectively 43.2% and 40%.The comparison result that can find out GST* algorithm is more accurate.
The matching degree of two character strings that GST* algorithm makes to have longer public substring is higher.
5. utilize partially orderly string matching algorithm POC to calculate the matching degree of source string and matching string.
Two to be matched contain Chinese character, numeral and alphabetical mixed characters strings are called to source string and matching string.
First, first search out the character that source string is identical with matching string, record their number.
Secondly, the longer character string in source string and matching string of take is standard, asks matching degree 1 (match_degree1):
Take is wherein standard compared with short character strings, asks matching degree 2 (match_degree2):
In formula (1), (2), [ ] represents to round.
Again, in reference source character string and matching string, the 1st (or the 2nd) is digital and alphabetical respectively, last 1 (or second-to-last) numeral and letter, if wherein 1 equate, the match_degree2 numerical value of adjusting matching degree 2 is match_degree2+1.
Finally, due in actual applications, matching degree 2 more can reflect actual match situation, therefore gives the weight that matching degree 2 is larger.
To matching degree 1 and matching degree 2, give different weight 0.41,0.59, ask the final matching value b of source string and matching string:
b＝match_degree1×0.41+match_degree2×0.59 （3）
6) the matching degree b of the matching degree a of step 4) GST* calculating gained and step 5) POC calculating gained is weighted to fusion, fusion method is, if matching degree a is greater than matching degree b, final matching degree is a.If matching degree a is less than matching degree b, final matching degree equals (a+b)/2.
Obtain final comparison result, take full advantage of the feature of two kinds of algorithms;
7. check whether circulation is finished;
8. pair each matching degree sorts, and finds out the maximum corresponding character string of matching degree, as the target string mating most.
Fig. 2 is the application example of mixed characters string one-to-many comparison method of the present invention.Calculated the match condition of one group of mixed characters string.In Fig. 2, list respectively and utilize GST* algorithm, the matching degree of GST*_POC after string matching algorithm (POC algorithm), two kinds of algorithm weights fusion methods partially in order.
Can see, in Fig. 2, in the 1st comparison, the 2nd comparison, the matching string of the matching string of the 1st than the 2nd is closer to the source string of the 1st row.
From Fig. 2, result can be reached a conclusion, and utilizes algorithm of the present invention to obtain comparatively desirable comparison result.
Claims (3)
1. the mixed characters string of an one-to-many merges comparison method, similarity to the mixed characters string being comprised of Chinese character, numeral, English alphabet based on Chinese characters clustering feature is carried out fusion ratio pair, to improve the similar degree of accuracy of expressing character string, comprise following key step:
1) take out source string and one group of matching string;
2) read the character string building the in advance dictionary of replacing of equal value in storer, partial character (substring) in this group matching string is carried out to equivalence and replace; Utilize the dictionary of replacing of equal value, by above-mentioned, in source string occasion and matching string occasion, there is different describe but two kinds of identical substrings of implication are unified;
3) take out source string, take out according to this matching string in this matching string array after of equal value replacement;
4) utilize GST* algorithm to calculate the matching degree a of source string and this matching string:
Adopt traditional GST algorithm, obtain each public substring in two character strings, they are deposited in public substring chained list.If the character length of certain public substring is more than or equal to 0.33 with the ratio of longer character string character length, when calculating matching degree, the character number of this public substring is multiplied by weight, this weight is to be greater than 1 constant; If the ratio of the character length of certain public substring and longer character string character length be less than 0.33 and the character number of public substring be greater than smallest match length, while calculating matching degree, directly bring the character number of this public substring into calculating;
5) partially orderly string matching algorithm POC(Partial Order Comparison, POC of utilization) calculate the matching degree b of source string and matching string:
Two to be matched mixed characters strings that contain Chinese character, numeral and English alphabet are called to source string and matching string,
First, first search out source string and character or Chinese character identical in matching string, record their number;
Secondly, the longer character string in source string and matching string of take is standard, asks matching degree 1 (match_degree1):
Take is wherein standard compared with short character strings, asks matching degree 2 (match_degree2):
In formula (1), (2), [] represents to round;
Again, respectively the 1st or the 2nd numeral and letter in reference source character string and matching string, last 1 or second-to-last numeral and letter, if wherein 1 equate, the match_degree2 numerical value of adjusting matching degree 2 is match_degree2+1:
To matching degree 1 and matching degree 2, give different weight 0.41,0.59, ask the final matching value b of source string and matching string:
b＝match_degree1×0.41+match_degree2×0.59 （3）
6) the matching degree b of the matching degree a of step 4) GST* calculating gained and step 5) POC calculating gained is weighted to fusion, fusion method is, if matching degree a is greater than matching degree b, final matching degree is a; If matching degree a is less than matching degree b, final matching degree equals (a+b)/2;
7) each matching string in source string and matching string array is calculated to the matching degree that obtains and sort, matching string corresponding to maximum matching degree, as the target string mating most with source string.
2. according to the mixed characters string of the one-to-many described in claim, merge comparison method, it is characterized in that, in step 4), first search out each public substring that source string is identical with matching string, to the public substring of different length, give different weights again, increase the weight of longer common characters substring.
3. according to the mixed characters string of the one-to-many described in claim, merge comparison method, it is characterized in that, in step 5), using two mixed characters strings that contain numeral, letter, Chinese character respectively as source string and matching string; Take respectively wherein longer character string, compared with short character strings, be standard, obtain matching degree 1 and matching degree 2; And then relatively whether first or a plurality of numeral equate with letter with alphabetical, last or a plurality of numeral, and matching degree 2 is modified.Finally to two kinds of matching degrees, give respectively different weights, obtain the matching degree value of two character strings.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310746846.5A CN104008119B (en) | 2013-12-30 | 2013-12-30 | A kind of one-to-many mixed characters string fusion comparison method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310746846.5A CN104008119B (en) | 2013-12-30 | 2013-12-30 | A kind of one-to-many mixed characters string fusion comparison method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104008119A true CN104008119A (en) | 2014-08-27 |
| CN104008119B CN104008119B (en) | 2017-09-26 |
Family
ID=51368778
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310746846.5A Expired - Fee Related CN104008119B (en) | 2013-12-30 | 2013-12-30 | A kind of one-to-many mixed characters string fusion comparison method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104008119B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104732041A (en) * | 2015-04-13 | 2015-06-24 | 国网四川省电力公司电力科学研究院 | Automatic virtual terminal generation method based on multiple SCD templates |
| CN105184713A (en) * | 2015-07-17 | 2015-12-23 | 四川久远银海软件股份有限公司 | Intelligent matching and sorting system and method capable of benefitting contrast of assigned drugs of medical insurance |
| CN106484678A (en) * | 2016-10-13 | 2017-03-08 | 北京智能管家科技有限公司 | A kind of short text similarity calculating method and device |
| CN106919663A (en) * | 2017-02-14 | 2017-07-04 | 华北电力大学 | Character string matching method in the multi-source heterogeneous data fusion of power regulation system |
| WO2017143907A1 (en) * | 2016-02-22 | 2017-08-31 | 阿里巴巴集团控股有限公司 | Character string distance calculation method and device |
| CN109741745A (en) * | 2019-01-28 | 2019-05-10 | 中国银行股份有限公司 | A kind of transaction air navigation aid and device |
| CN112215216A (en) * | 2020-09-10 | 2021-01-12 | 中国东方电气集团有限公司 | Character string fuzzy matching system and method for image recognition result |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060179052A1 (en) * | 2003-03-03 | 2006-08-10 | Pauws Steffen C | Method and arrangement for searching for strings |
| CN101329680A (en) * | 2008-07-17 | 2008-12-24 | 安徽科大讯飞信息科技股份有限公司 | Large scale rapid matching method of sentence surface |
-
2013
- 2013-12-30 CN CN201310746846.5A patent/CN104008119B/en not_active Expired - Fee Related
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060179052A1 (en) * | 2003-03-03 | 2006-08-10 | Pauws Steffen C | Method and arrangement for searching for strings |
| CN101329680A (en) * | 2008-07-17 | 2008-12-24 | 安徽科大讯飞信息科技股份有限公司 | Large scale rapid matching method of sentence surface |
Non-Patent Citations (2)
| Title |
|---|
| JONGKYU SEO等: "A Unified Approach for Computing Document Similarity with Fingerprinting and Alignments", 《2012 IEEE 12TH INTERNATIONAL CONFERENCE ON COMPUTER AND INFORMATION TECHNOLOGY》 * |
| 牛永洁 等: "多种字符串相似度算法的比较研究", 《计算机与数字工程》 * |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104732041A (en) * | 2015-04-13 | 2015-06-24 | 国网四川省电力公司电力科学研究院 | Automatic virtual terminal generation method based on multiple SCD templates |
| CN104732041B (en) * | 2015-04-13 | 2017-09-29 | 国网四川省电力公司电力科学研究院 | A kind of empty terminal table automatic generation method based on many SCD templates |
| CN105184713A (en) * | 2015-07-17 | 2015-12-23 | 四川久远银海软件股份有限公司 | Intelligent matching and sorting system and method capable of benefitting contrast of assigned drugs of medical insurance |
| WO2017143907A1 (en) * | 2016-02-22 | 2017-08-31 | 阿里巴巴集团控股有限公司 | Character string distance calculation method and device |
| TWI659358B (en) * | 2016-02-22 | 2019-05-11 | 香港商阿里巴巴集團服務有限公司 | Method and device for calculating string distance |
| US11256756B2 (en) | 2016-02-22 | 2022-02-22 | Advanced New Technologies Co., Ltd. | Character string distance calculation method and device |
| CN106484678A (en) * | 2016-10-13 | 2017-03-08 | 北京智能管家科技有限公司 | A kind of short text similarity calculating method and device |
| CN106919663A (en) * | 2017-02-14 | 2017-07-04 | 华北电力大学 | Character string matching method in the multi-source heterogeneous data fusion of power regulation system |
| CN109741745A (en) * | 2019-01-28 | 2019-05-10 | 中国银行股份有限公司 | A kind of transaction air navigation aid and device |
| CN112215216A (en) * | 2020-09-10 | 2021-01-12 | 中国东方电气集团有限公司 | Character string fuzzy matching system and method for image recognition result |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104008119B (en) | 2017-09-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104008119A (en) | One-to-many mixed string comparison method | |
| CN103970722B (en) | A kind of method of content of text duplicate removal | |
| CN104050247B (en) | The method for realizing massive video quick-searching | |
| US8943091B2 (en) | System, method, and computer program product for performing a string search | |
| Van Opdenbosch et al. | Camera-based indoor positioning using scalable streaming of compressed binary image signatures | |
| EP3072076B1 (en) | A method of generating a reference index data structure and method for finding a position of a data pattern in a reference data structure | |
| CN109299086B (en) | Optimal sort key compression and index reconstruction | |
| CN105574212A (en) | Image retrieval method for multi-index disk Hash structure | |
| CN109271487A (en) | A kind of Similar Text analysis method | |
| CN101329680B (en) | Large scale rapid matching method of sentence surface | |
| CN103473307A (en) | Cross-media sparse Hash indexing method | |
| CN113741858B (en) | Memory multiply-add computing method, memory multiply-add computing device, chip and computing equipment | |
| CN103020321B (en) | Neighbor search method and system | |
| CN104021234A (en) | Large-scale image library retrieval method based on self-adaptive bit allocation Hash algorithm | |
| CN106570166B (en) | Video retrieval method and device based on multiple locality sensitive hash tables | |
| Vulinović et al. | Neural networks for file fragment classification | |
| CN105183792A (en) | Distributed fast text classification method based on locality sensitive hashing | |
| CN103761298B (en) | Distributed-architecture-based entity matching method | |
| WO2017157038A1 (en) | Data processing method, apparatus and equipment | |
| CN104731889B (en) | A kind of method for estimating query result size | |
| Chen et al. | Sorting local descriptors for lowbit rate mobile visual search | |
| CN104699701A (en) | Parallel nearest node computing method and distributed system based on sensitive hashing | |
| Yang et al. | Supervised deep quantization for efficient image search | |
| Chen et al. | Smaller text classifiers with discriminative cluster embeddings | |
| Ma et al. | BCH–LSH: a new scheme of locality‐sensitive hashing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20170926 Termination date: 20181230 |