CN111209447A - Chinese character string similarity calculation method and device based on sound-shape codes - Google Patents

Chinese character string similarity calculation method and device based on sound-shape codes Download PDF

Info

Publication number
CN111209447A
CN111209447A CN201910146570.4A CN201910146570A CN111209447A CN 111209447 A CN111209447 A CN 111209447A CN 201910146570 A CN201910146570 A CN 201910146570A CN 111209447 A CN111209447 A CN 111209447A
Authority
CN
China
Prior art keywords
codes
similarity
sound
character strings
shape
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201910146570.4A
Other languages
Chinese (zh)
Inventor
刘卫国
宋红磊
张�浩
殷泽坤
张雯
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shandong University
Original Assignee
Shandong University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shandong University filed Critical Shandong University
Priority to CN201910146570.4A priority Critical patent/CN111209447A/en
Publication of CN111209447A publication Critical patent/CN111209447A/en
Pending legal-status Critical Current

Links

Images

Landscapes

  • Document Processing Apparatus (AREA)

Abstract

The invention discloses a sound-shape code which comprises a sound code and a shape code, wherein the sound code consists of numerical codes of initial consonants and vowels, and the shape code consists of four-corner codes, structural codes and stroke numbers of Chinese characters; pre-storing the mapping rule of the sound-shape code and the pronunciation similarity of partial initial consonants/vowels, wherein the method comprises the following steps: receiving two character strings to be compared; reading a mapping rule of the sound-shape codes, and converting each Chinese character in the two character strings into sound-shape code representation according to the mapping rule; based on the pronunciation similarity of the initial consonants and the vowels, the editing distance between every two corresponding substrings of the two character strings is calculated by adopting the editing distance; and calculating the similarity of the two character strings according to the editing distance. The invention converts the character string into the sound-shape code digital string for comparison, improves the matching precision of the Chinese characters, and on the other hand, replaces the weight of the editing distance with the editing distance of the Chinese characters, and can more accurately calculate the similarity of the character string.

Description

Chinese character string similarity calculation method and device based on sound-shape codes
Technical Field
The invention belongs to the technical field of text similarity calculation, and particularly relates to a Chinese character string similarity calculation method and device based on a sound-shape code.
Background
The similarity of character strings is a basic technique in String matching (String matching), Text Comparison (Text Comparison) and Information Extraction (Information Extraction) as a measure for measuring the degree of approximation between two character strings, and the input of the basic technique is usually two identical or different character strings, and the output of the basic technique is a determined value. The higher the similarity of two character strings, the larger the corresponding return value. There are many methods for measuring the similarity of character strings, including cosine similarity (cosine similarity), Euclidean distance (Euclidean distance), edit distance (edit distance), hamming distance (hamming distance), Dice distance, Jaccard distance, J-W distance (Jaro-Winkerdistance), and so on. The edit distance algorithm, also called Levenshtein distance, represents the minimum number of edits required to convert a character string into another character string, i.e., replacing one character in the character string with another character or inserting a deleted character, and the calculation of the minimum number of edits between a pair of character strings is the core of the edit distance. The Smith-Waterman algorithm is an algorithm for local sequence alignment (as opposed to global alignment) and is often applied to similarity calculation between nucleotide or protein sequences, and the aim of the algorithm is not to perform full sequence alignment but to find fragments of two sequences with high similarity. When the specific problem of Chinese character string matching in a Chinese language environment is faced, the practicability of a classic edit distance algorithm is reduced; the common font-based algorithm is based on the five-stroke coding of the Chinese characters, and cannot well mine the structural characteristics of the Chinese characters; the Smith-Waterman algorithm is better suited to find segments with high similarity in two strings.
In order to realize the quantification of the similarity between Chinese characters, a coding mode, namely a sound-shape code, capable of simultaneously describing the pronunciation and the structure of the Chinese characters is proposed at present, but the inventor finds that when the sound-shape code encodes the initial consonant and the final consonant, part of the initial consonant/the final consonant adopts the same coding, such as an and ang, and z and zh, and the same coding is adopted for conversion, so that the difference between the initial consonant and the final consonant is weakened, but the direct equivalent substitution mode can not reflect the difference, the problem that some fuzzy sounds possibly existing in Chinese pronunciation in reality are similar is ignored, such as pronunciation habits of flat tongue warping in the square and nose sounds before and after the square, and the accuracy of similarity comparison is necessarily influenced.
Disclosure of Invention
In order to overcome the defects of the prior art, the invention provides a Chinese character string similarity calculation method and a Chinese character string similarity calculation device based on a sound-shape code, which simplify the representation of the existing sound-shape code, comprise initial consonants, final consonants, four-corner codes, structures and stroke number information of Chinese characters, and replace the editing distance between character strings by the editing distance between the Chinese characters, thereby calculating the editing distance between two characters more comprehensively and obtaining more accurate Chinese character string similarity.
In order to achieve the above object, one or more embodiments of the present invention provide the following technical solutions:
a Chinese character string similarity calculation method based on a sound-shape code comprises the sound code and the shape code, wherein the sound code consists of numerical codes of initial consonants and vowels, and the shape code consists of four-corner codes, structural codes and stroke numbers of Chinese characters; pre-storing the mapping rule of the sound-shape code and the pronunciation similarity of partial initial consonants/vowels, wherein the method comprises the following steps:
receiving two character strings to be compared;
reading a mapping rule of the sound-shape codes, and converting each Chinese character in the two character strings into sound-shape code representation according to the mapping rule;
based on the pronunciation similarity of the initial consonants and the vowels, the editing distance between every two corresponding substrings of the two character strings is calculated by adopting the editing distance;
and calculating the similarity of the two character strings according to the editing distance.
Further, the phonetic-shape code comprises 12 bits: 2-bit initial consonant, 2-bit vowel, 5-bit four-corner coding, 1-bit structure code and 2-bit stroke number.
Further, the mapping rule of the phonetic-shape code comprises: the mapping rules of Chinese characters to pinyin, strokes, structures and four-corner codes and the mapping rules of initials, finals and structures to numerical codes.
Further, the calculating the edit distance between each two corresponding substrings of the two character strings based on the edit distance includes:
initializing an editing distance matrix;
and sequentially calculating the editing distance between every two Chinese characters corresponding to the two character strings according to a dynamic planning strategy, and writing the editing distance into an editing distance matrix.
Furthermore, an edit distance from a sub-string with the length i of the character string A to a sub-string with the length j of the character string B is represented by edge _ char (i, j), and the dynamic programming strategy is as follows:
1)if i==0&&j==0,edit_char(i,j)=0;
2)if i==0&&j>0,edit_char(i,j)=j;
3)if i>0&&j==0,edit_char(i,j)=I;
4)if 0<i<=m&&0<j<=n,
an edge _ char (i, j) ═ min { edge _ char (i-l, j) +1, edge _ char (i, j-1) +1, edge _ char (i-l, j-l) + f (i, j) }, wherein f (i, j) ═ l when the ith character of the character string a is not equal to the jth character of the character string B; otherwise, f (i, j) is 0.
Further, f (i, j) ═ α × SSound+β*SShape of
Wherein S isSoundDistance of phonetic code, i.e. pronunciation similarity, SShape ofFor the shape code distance, α and β are adjustment coefficients.
Further, SShape ofThe four-corner coding editing distance + theta 2 structural code comparison value + theta 3 stroke number difference/max (the maximum stroke number of two Chinese characters), wherein theta 1, theta 2 and theta 3 are all weight coefficients.
Further, calculating the similarity of the two character strings according to the edit distance includes:
taking the value of the lower right corner in the edit distance matrix as the shortest edit matrix distance, and calculating the Similarity between the two character strings A and B by adopting the following formula:
Similarity=1-distance/max(length(A),length(B))
where length (a) and length (B) represent the lengths of strings a and B, respectively, and max (·) represents a maximum function.
One or more embodiments provide a computing device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the method for phonographic code based calculation of similarity of chinese character strings when executing the program.
One or more embodiments provide a computer-readable storage medium on which a computer program is stored, which, when executed by a processor, implements the phonographic code-based chinese character string similarity calculation method.
The above one or more technical solutions have the following beneficial effects:
the invention simplifies the existing expression of the sound-shape codes, including initial consonants, vowels, four-corner codes, structures and stroke number information of Chinese characters, and the conversion efficiency from the Chinese characters to the sound-shape codes is higher; the four-corner coding with complementary codes is adopted, the font can be more accurately represented, the influence of pronunciation habits of flat-warped tongues and front and back nasal sounds in dialects on the measurement of the pronunciation similarity of the Chinese characters is considered, for initial consonants and vowels of part of similar pronunciations such as an and ang, en and eng and the like, the similarity is set while different codes are adopted for representation, namely the similarity between pronunciations is considered, the difference is also embodied, the existing algorithm does not consider the point and is only simple, identical and different, and therefore, in comparison, the sound-shape codes can more accurately and comprehensively reflect the similarity between the Chinese characters.
The invention improves the calculation method of the editing distance of the character string, replaces the weight of the editing distance with the editing distance of the Chinese character, and can reasonably consider the conditions of different positions of the same character, such as common character string dislocation, so that the comparison between the character strings is more comprehensive, the matching precision of the character strings is improved, and the similarity of the character strings can be calculated more accurately.
The similarity calculation method introduces adjustable weight in the process of calculating the editing distance between Chinese characters, can adjust the weight of the font part and the pronunciation part according to the actual situation, for example, can improve the proportion of the font part in the process of character recognition in a natural scene, and can improve the proportion of the similarity of the pronunciation in the process of voice recognition, thereby being applied to a database retrieval system supporting Chinese character input and voice input.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, are included to provide a further understanding of the application, and are incorporated in and constitute a part of this specification, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application.
FIG. 1 is a flowchart illustrating an overall method for calculating similarity of Chinese character strings based on phonographic codes according to an embodiment of the present invention;
FIG. 2 is a block diagram of an idea of performing similarity calculation between Chinese characters based on phonetic and font codes according to an embodiment of the present invention.
Detailed Description
It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the disclosure. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments according to the present application. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, and it should be understood that when the terms "comprises" and/or "comprising" are used in this specification, they specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof, unless the context clearly indicates otherwise.
The embodiments and features of the embodiments in the present application may be combined with each other without conflict.
Example one
The embodiment discloses a Chinese character string similarity calculation method based on a sound-shape code, wherein the sound-shape code comprises a sound code and a shape code, the sound code consists of numerical codes of an initial consonant and a final sound, and the shape code consists of four-corner codes, structural codes and stroke numbers of Chinese characters; pre-storing the mapping rule of the sound-shape code in a database;
as shown in fig. 1, the method comprises the steps of:
step 1: receiving two character strings A and B to be compared;
step 2: reading a mapping rule of the sound-shape codes from a database, and converting each Chinese character in the two character strings into the sound-shape code expression according to the mapping rule;
and step 3: calculating the editing distance between every two corresponding substrings of the two character strings based on the editing distance;
and 4, step 4: and calculating the similarity of the character strings according to the editing distance.
The phono configurational code comprises 12 bits: 2-bit initial consonant, 2-bit vowel, 5-bit four-corner coding, 1-bit structure code and 2-bit stroke number. The expression method is shown in table 1.
TABLE 1 phono configurational code composition
Initial consonant (2 position) Finals (2 bit) Four corner coding (5 bit) Structure code (1 bit) Stroke number (2 bit)
The mapping rule of the phonetic-shape codes comprises the following steps: the mapping rules of Chinese characters to pinyin, strokes, structures and four-corner codes and the mapping rules of initials, finals and structures to numerical codes. The mapping rule from the initial consonant and the final vowel to the numerical code is as follows:
TABLE 2 mapping rules of initials and finals to numerical codes
a 01 ai 07 ie 13 un 19
o 02 ei 08 ve 14 vn 20
e 03 ui 09 er 15 ang 21
i 04 ao 10 an 16 eng 22
u 05 ou 11 en 17 ing 23
v 06 iu 12 in 18 ong 24
The step 3 comprises the following steps:
initializing an editing distance matrix;
and sequentially calculating the editing distance between every two corresponding substrings of the two character strings according to a dynamic programming strategy, and writing the editing distance into an editing distance matrix.
Specifically, the editing distance between the Chinese characters is adopted to replace the editing distance between the substrings.
Wherein the dynamic planning strategy is as follows:
1)if i==0&&j==0,edit_char(i,j)=0;
2)if i==0&&j>0,edit_char(i,j)=j;
3)if i>0&&j==0,edit_char(i,j)=I;
4) if0< i < ═ m & &0< j < ═ n, edge _ char (i, j) ═ min { edge _ char (i-l, j) +1, edge _ char (i, j-1) +1, edge _ char (i-l, j-l) + f (i, j) }, when the ith character of the first character string is not equal to the jth character of the second character string, f (i, j) } l; otherwise, f (i, j) is 0.
And adopting an edge _ char (i, j) function to represent the edit distance from the substring with the length i of the first character string A to the substring with the length j of the first character string B.
f(i,j)=α*SSound+β*SShape of
Wherein SSoundα and β are coefficient parameters, α + β is 1, and can be adjusted according to similarity comparison requirements, when the character strings A and B are both texts, the value of the coefficient β can be correspondingly increased, emphasis is placed on font comparison, and if texts obtained by voice conversion exist in the character strings A and B, the value of α is properly increased, and emphasis is placed on pronunciation comparison.
TABLE 3 initial/final distance mapping table
Distance between two adjacent plates Distance between two adjacent plates
c ch 0.9 h f 0.7
s sh 0.9 eng ang 0.7
z zh 0.9 eng ong 0.7
an ang 0.9 eng ing 0.7
en eng 0.9 ang ong 0.7
in ing 0.9 ang ing 0.7
n l 0.7 ing ong 0.7
SShape ofθ 1 × four-corner coding edit distance + θ 2 × configuration code comparison value + θ 3 × stroke number difference/max (stroke number);
wherein, θ 1, θ 2, θ 3 are coefficient values of each comparison part (which can be adjusted according to comparison requirements); the quadrangle code edit distance is obtained by table lookup.
The step 4 comprises the following steps:
taking the value of the lower right corner in the edit distance matrix as the shortest edit matrix distance, and calculating the Similarity between the two character strings A and B by adopting the following formula:
Similarity=1-distance/max(length(A),length(B))
where length (a) and length (B) represent the lengths of strings a and B, respectively, and max (·) represents a maximum function.
As an example, a mapping Table is established to obtain a mapping Table Table _1 from Chinese characters to pinyin, a mapping Table Table _2 from Chinese characters to strokes, a mapping Table Table _3 from Chinese characters to structures and a mapping Table Table _4 from Chinese characters to four-corner numbers, and the sound-shape codes of the Chinese characters can be obtained by inquiring the mapping Table, wherein the specific structure comprises the following steps: 2-bit initial consonant, 2-bit vowel, 5-bit four-corner coding, 1-bit structure code and 2-bit stroke number.
For example: the character string A is 'Shanxi Taiji', the character string B is 'Shandong university', and the sound-shape codes of all Chinese characters are obtained based on the mapping table, as follows:
TABLE 4 phono configurational code for character string A and character string B
Chinese characters Sound-shape code Chinese characters Sound-shape code
Mountain 171622770003 Mountain 171622770003
East 052440904005 Western medicine 140410604011
Big (a) 050140800013 Taiwan (Chinese character of 'tai') 060740030014
Study the design 141490407208 Study the design 141490407208
If the edit distance of the character strings A and B is required, the edit distance between the Chinese characters is calculated first.
First, the character string a, B has a length of m-n-4.
Edit distance matrix, as in table 5:
TABLE 5 initialized edit distance matrix
Figure BDA0001980215610000061
Figure BDA0001980215610000071
Calculating an edit distance from a substring with the length i of the character string A to a substring with the length j of the character string B, and firstly calculating the edit distance from a mountain to a mountain of the character string A, wherein f (i, j) is obviously 0, and edge _ char (1,1) is obviously 0;
at this time, as shown in table 6, the edit distance matrix is updated to:
TABLE 6 write "mountain" to "mountain" edit distance to edit distance matrix
Mountain East Big (a) Study the design
0 1 2 3 4
Mountain 1 0
Western medicine 2
Taiwan (Chinese character of 'tai') 3
Study the design 4
Then, edit _ char (1,2), i.e., the edit distance of "mountain" and "east of mountain", is calculated. From the dynamic programming formulation we need to calculate f (1, 2).
f(i,j)=α*SSound+β*SShape of
Wherein SSoundAcquiring the distance of the initial consonant and the final consonant from the constructed distance table of the initial consonant and the final consonant;
Sshape ofθ 1 × four-corner coding edit distance + θ 2 × configuration code comparison value + θ 3 × stroke number difference/max (stroke number);
α and β are coefficient parameters which can be adjusted according to the similarity comparison requirement, in this case, the similarity of the weight bias font is calculated, so the parameter values are 0.2 and 0.8 respectively.
The S of mountain and east can be known by looking up the tableSound=1,SShape ofThe edit distances of the four middle corner codes are as follows:
TABLE 7 "mountain" and "east" quadrangle code edit distances
Figure BDA0001980215610000072
Figure BDA0001980215610000081
As can be seen from the table, the edit distance of the "mountain" and "east" four corner codes is 5/5 ═ 1.0;
the structures are the same, so the distance of the structures is 0, the stroke number difference is 2, SShape of0.8 × 1+0.05 × 0+0.15 × 2/5 ═ 0.86. By the formula, it _ char (i, j) ═ min { edit _ char (i-l, j) +1, edit _ char (i, j-1) +1, edit _ char (i-l, j-l) + f (i, j) }, when it _ char (i-l, j) +1 ═ 3, edit _ char (i, j-1) +1 ═ 1, edit _ char (i-l, j-l) + f (i, j) ═ 1.888; by analogy, the final results are shown in table 8:
TABLE 8 edit distance of character strings "Shanxi Taiji" and "Shandong university
Mountain East Big (a) Study the design
0 1 2 3 4
Mountain 1 0 1 2 3
Western medicine 2 1 0.476 1.476 2.476
Taiwan (Chinese character of 'tai') 3 2 1.476 0.962 1.962
Study the design 4 3 2.476 1.962 0.962
As can be seen from the table, the shortest editing distance improvement algorithm based on the phonetic codes of the character strings "shanxi tai chou" and "shandong university" results in 0.962, and the similarity of the character strings is 1-0.962/4-0.760.
In the embodiment, the editing distance of the Chinese character string is replaced by the editing distance of the Chinese character, so that the conditions of different positions of the same character can be reasonably considered, the comparison between the character strings is more comprehensive, the matching precision of the character strings is improved, and the similarity of the character strings can be more accurately calculated.
In one or more embodiments, the similarity calculation method may be used in a database retrieval system, where one of the character string a and the character string B is a text to be retrieved, and the other is a text in a database. And the text to be searched can be directly input through characters or can be input through language, and the system firstly converts the voice into the text and then executes the search.
Example two
An object of the present embodiment is to provide a computing device.
A computing device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the memory having pre-stored therein mapping rules for the pictographic codes, the processor implementing the following steps when executing the program:
receiving two character strings to be compared;
reading a mapping rule of the sound-shape codes, and converting each Chinese character in the two character strings into sound-shape code representation according to the mapping rule; the sound-shape codes comprise sound codes and shape codes, wherein the sound codes comprise numerical codes of initials and finals, and the shape codes comprise four-corner codes, structural codes and stroke numbers of Chinese characters;
calculating the editing distance between every two corresponding substrings of the two character strings based on the editing distance;
and calculating the similarity of the two character strings according to the editing distance.
EXAMPLE III
An object of the present embodiment is to provide a computer-readable storage medium.
A computer-readable storage medium having stored thereon a mapping rule of the phonetic-to-shape codes and a computer program for calculating text similarity, which program, when executed by a processor, performs the steps of:
receiving two character strings to be compared;
reading a mapping rule of the sound-shape codes, and converting each Chinese character in the two character strings into sound-shape code representation according to the mapping rule; the sound-shape codes comprise sound codes and shape codes, wherein the sound codes comprise numerical codes of initials and finals, and the shape codes comprise four-corner codes, structural codes and stroke numbers of Chinese characters;
calculating the editing distance between every two corresponding substrings of the two character strings based on the editing distance;
and calculating the similarity of the two character strings according to the editing distance.
The steps involved in the second and third embodiments correspond to the first embodiment of the method, and the detailed description thereof can be found in the relevant description of the first embodiment. The term "computer-readable storage medium" should be taken to include a single medium or multiple media containing one or more sets of instructions; it should also be understood to include any medium that is capable of storing, encoding or carrying a set of instructions for execution by a processor and that cause the processor to perform any of the methods of the present invention.
One or more of the above embodiments have the following technical effects:
the existing expression of the sound-shape codes is simplified, the sound-shape codes comprise initial consonants, vowels, four-corner codes, structures and stroke number information of the Chinese characters, and the conversion efficiency from the Chinese characters to the sound-shape codes is higher; the four-corner coding with the complementary codes is adopted, the character patterns can be more accurately represented, and for partial consonants and vowels with similar pronunciations, such as an and ang, en and eng, and the like, the similarity is set while different codes are adopted for representation, namely the similarity between pronunciations is considered, and the difference is also reflected, so that the similarity between Chinese characters can be more accurately identified.
The method for calculating the editing distance of the character string is improved, the weight of the editing distance is replaced by the editing distance of the Chinese characters, the conditions of different positions of the same character can be reasonably considered, such as common character string dislocation, so that the comparison between the character strings is more comprehensive, the matching precision of the character strings is improved, and the similarity of the character strings can be calculated more accurately.
Adjustable weight is introduced in the process of calculating the editing distance between Chinese characters, the weight of a character shape part and a character sound part can be adjusted according to actual situations, for example, the proportion of the character shape part can be improved in the process of character recognition in natural scenes, and the proportion of the similarity of the character sound can be improved in the process of voice recognition, so that the method can be applied to a database retrieval system supporting Chinese character input and voice input.
Those skilled in the art will appreciate that the modules or steps of the present application described above can be implemented using general purpose computing devices, or alternatively, they can be implemented using program code executable by computing devices, such that they are stored in a storage device and executed by computing devices, or they are separately fabricated into individual integrated circuit modules, or multiple modules or steps thereof are fabricated into a single integrated circuit module. The present application is not limited to any specific combination of hardware and software.
The above description is only a preferred embodiment of the present application and is not intended to limit the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Although the embodiments of the present application have been described with reference to the accompanying drawings, it is not intended to limit the scope of the present application, and it should be understood by those skilled in the art that various modifications and variations can be made without inventive effort by those skilled in the art.

Claims (10)

1. A Chinese character string similarity calculation method based on sound-shape codes is characterized in that the sound-shape codes comprise sound codes and shape codes, wherein the sound codes comprise numerical codes of initials and finals, and the shape codes comprise four-corner codes, structural codes and stroke numbers of Chinese characters; pre-storing the mapping rule of the sound-shape code and the pronunciation similarity of partial initial consonants/vowels, wherein the method comprises the following steps:
receiving two character strings to be compared;
reading a mapping rule of the sound-shape codes, and converting each Chinese character in the two character strings into sound-shape code representation according to the mapping rule;
based on the pronunciation similarity of the initial consonants and the vowels, the editing distance between every two corresponding substrings of the two character strings is calculated by adopting the editing distance;
and calculating the similarity of the two character strings according to the editing distance.
2. The method for calculating the similarity of Chinese character strings based on the phonographic codes as claimed in claim 1, wherein the phonographic codes comprise 12 bits: 2-bit initial consonant, 2-bit vowel, 5-bit four-corner coding, 1-bit structure code and 2-bit stroke number.
3. The method for calculating the similarity of Chinese character strings based on the phonographic codes as claimed in claim 1, wherein the mapping rule of the phonographic codes comprises: the mapping rules of Chinese characters to pinyin, strokes, structures and four-corner codes and the mapping rules of initials, finals and structures to numerical codes.
4. The method for calculating the similarity of Chinese character strings based on phonographic codes as claimed in claim 1, wherein said calculating the edit distance between two corresponding substrings of two character strings based on the edit distance comprises:
initializing an editing distance matrix;
and sequentially calculating the editing distance between every two Chinese characters in the two character strings according to a dynamic programming strategy, and writing the editing distance into an editing distance matrix.
5. The method for calculating the similarity of Chinese character strings based on phonographic codes as claimed in claim 4, wherein the edit distance from the sub-string with the length i in the character string A to the sub-string with the length j in the character string B is represented by edge _ char (i, j), and the dynamic programming strategy is as follows:
1)if i==0&&j==0,edit_char(i,j)=0;
2)if i==0&&j>0,edit_char(i,j)=j;
3)if i>0&&j==0,edit_char(i,j)=I;
4)if 0<i<=m&&0<j<=n,
an edge _ char (i, j) ═ min { edge _ char (i-l, j) +1, edge _ char (i, j-1) +1, edge _ char (i-l, j-l) + f (i, j) }, wherein f (i, j) ═ l when the ith character of the character string a is not equal to the jth character of the character string B; otherwise, f (i, j) is 0.
6. The method for calculating the similarity of Chinese character strings based on phonographic codes according to claim 5,
f(i,j)=α*Ssound+β*SShape of
Wherein S isSoundDistance of phonetic code, i.e. pronunciation similarity, SShape ofα, β are adjustment systems for the code distanceAnd (4) counting.
7. The method for calculating the similarity of Chinese character strings based on phonographic codes as claimed in claim 6, wherein S isShape ofThe method comprises the steps of dividing a four-corner coding editing distance into theta 1 and theta 2, comparing a structural code comparison value into theta 3 and stroke number difference, and dividing the stroke number difference into the maximum stroke number of two Chinese characters, wherein the theta 1, the theta 2 and the theta 3 are all weight coefficients.
8. The method of claim 6, wherein calculating the similarity of two strings according to the edit distance comprises:
taking the value of the lower right corner in the edit distance matrix as the shortest edit matrix distance, and calculating the Similarity between the two character strings A and B by adopting the following formula:
Similarity=1-distance/max(length(A),length(B))
where length (a) and length (B) represent the lengths of strings a and B, respectively, and max (·) represents a maximum function.
9. A computing device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the method for calculating the similarity of a chinese string based on a phonographic code according to any one of claims 1 to 8 when executing the program.
10. A computer-readable storage medium on which a computer program is stored, the program, when executed by a processor, implementing the method for calculating the similarity of a chinese character string based on a phonographic code according to any one of claims 1 to 8.
CN201910146570.4A 2019-02-27 2019-02-27 Chinese character string similarity calculation method and device based on sound-shape codes Pending CN111209447A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910146570.4A CN111209447A (en) 2019-02-27 2019-02-27 Chinese character string similarity calculation method and device based on sound-shape codes

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910146570.4A CN111209447A (en) 2019-02-27 2019-02-27 Chinese character string similarity calculation method and device based on sound-shape codes

Publications (1)

Publication Number Publication Date
CN111209447A true CN111209447A (en) 2020-05-29

Family

ID=70789533

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910146570.4A Pending CN111209447A (en) 2019-02-27 2019-02-27 Chinese character string similarity calculation method and device based on sound-shape codes

Country Status (1)

Country Link
CN (1) CN111209447A (en)

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111753147A (en) * 2020-06-27 2020-10-09 百度在线网络技术(北京)有限公司 Similarity processing method, device, server and storage medium
CN111767422A (en) * 2020-06-30 2020-10-13 平安国际智慧城市科技股份有限公司 Data auditing method, device, terminal and storage medium
CN112214535A (en) * 2020-10-22 2021-01-12 上海明略人工智能(集团)有限公司 Similarity calculation method and system, electronic device and storage medium
CN112613522A (en) * 2021-01-04 2021-04-06 重庆邮电大学 Method for correcting recognition result of medicine taking order based on fusion font information
CN112765422A (en) * 2021-01-18 2021-05-07 深轻(上海)科技有限公司 Table look-up method for two-dimensional data table
CN112861844A (en) * 2021-03-30 2021-05-28 中国工商银行股份有限公司 Service data processing method and device and server
CN112966475A (en) * 2021-03-02 2021-06-15 挂号网(杭州)科技有限公司 Character similarity determining method and device, electronic equipment and storage medium
CN113536786A (en) * 2021-06-22 2021-10-22 深圳价值在线信息科技股份有限公司 Method for generating confusing Chinese characters, terminal device and computer readable storage medium
CN113642563A (en) * 2021-08-31 2021-11-12 平安医疗健康管理股份有限公司 Drug use rechecking method, device, equipment and storage medium
CN114386385A (en) * 2022-03-22 2022-04-22 北京创新乐知网络技术有限公司 Method, device, system and storage medium for discovering sensitive word derived vocabulary
CN116303731A (en) * 2023-05-22 2023-06-23 四川互慧软件有限公司 Code matching method and device for hospital standard main data and electronic equipment
CN116757189A (en) * 2023-08-11 2023-09-15 四川互慧软件有限公司 Patient name disambiguation method based on Chinese character features

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103399907A (en) * 2013-07-31 2013-11-20 深圳市华傲数据技术有限公司 Method and device for calculating similarity of Chinese character strings on the basis of edit distance
CN108009253A (en) * 2017-12-05 2018-05-08 昆明理工大学 A kind of improved character string Similar contrasts method

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103399907A (en) * 2013-07-31 2013-11-20 深圳市华傲数据技术有限公司 Method and device for calculating similarity of Chinese character strings on the basis of edit distance
CN108009253A (en) * 2017-12-05 2018-05-08 昆明理工大学 A kind of improved character string Similar contrasts method

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
数据中国: "中文相似度算法", 《HTTPS://BLOG.CSDN.NET/CHNDATA/ARTICLE/DETAILS/41114771》 *
陈正铭等: "编辑距离算法在中文文本相似度计算中的优化与实现", 《韶关学院学报》 *
陈鸣等: "基于音形码的汉字相似度比对算法", 《信息技术》 *

Cited By (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111753147A (en) * 2020-06-27 2020-10-09 百度在线网络技术(北京)有限公司 Similarity processing method, device, server and storage medium
US12079258B2 (en) 2020-06-27 2024-09-03 Baidu Online Network Technology (Beijing) Co., Ltd. Similarity processing method, apparatus, server and storage medium
CN111767422A (en) * 2020-06-30 2020-10-13 平安国际智慧城市科技股份有限公司 Data auditing method, device, terminal and storage medium
CN112214535A (en) * 2020-10-22 2021-01-12 上海明略人工智能(集团)有限公司 Similarity calculation method and system, electronic device and storage medium
CN112613522A (en) * 2021-01-04 2021-04-06 重庆邮电大学 Method for correcting recognition result of medicine taking order based on fusion font information
CN112765422B (en) * 2021-01-18 2024-04-05 深轻(上海)科技有限公司 Table lookup method for two-dimensional data table
CN112765422A (en) * 2021-01-18 2021-05-07 深轻(上海)科技有限公司 Table look-up method for two-dimensional data table
CN112966475A (en) * 2021-03-02 2021-06-15 挂号网(杭州)科技有限公司 Character similarity determining method and device, electronic equipment and storage medium
CN112861844A (en) * 2021-03-30 2021-05-28 中国工商银行股份有限公司 Service data processing method and device and server
CN113536786A (en) * 2021-06-22 2021-10-22 深圳价值在线信息科技股份有限公司 Method for generating confusing Chinese characters, terminal device and computer readable storage medium
CN113642563A (en) * 2021-08-31 2021-11-12 平安医疗健康管理股份有限公司 Drug use rechecking method, device, equipment and storage medium
CN114386385A (en) * 2022-03-22 2022-04-22 北京创新乐知网络技术有限公司 Method, device, system and storage medium for discovering sensitive word derived vocabulary
CN116303731B (en) * 2023-05-22 2023-07-21 四川互慧软件有限公司 Code matching method and device for hospital standard main data and electronic equipment
CN116303731A (en) * 2023-05-22 2023-06-23 四川互慧软件有限公司 Code matching method and device for hospital standard main data and electronic equipment
CN116757189A (en) * 2023-08-11 2023-09-15 四川互慧软件有限公司 Patient name disambiguation method based on Chinese character features
CN116757189B (en) * 2023-08-11 2023-10-31 四川互慧软件有限公司 Patient name disambiguation method based on Chinese character features

Similar Documents

Publication Publication Date Title
CN111209447A (en) Chinese character string similarity calculation method and device based on sound-shape codes
JP7280382B2 (en) End-to-end automatic speech recognition of digit strings
CN111310443B (en) Text error correction method and system
CN107220235B (en) Speech recognition error correction method and device based on artificial intelligence and storage medium
JP6204959B2 (en) Speech recognition result optimization apparatus, speech recognition result optimization method, and program
US6738741B2 (en) Segmentation technique increasing the active vocabulary of speech recognizers
US9384730B2 (en) Pronunciation accuracy in speech recognition
CN111199726B (en) Speech processing based on fine granularity mapping of speech components
CN112507734B (en) Neural machine translation system based on romanized Uygur language
JP6778655B2 (en) Word concatenation discriminative model learning device, word concatenation detection device, method, and program
CN110870004A (en) Syllable-based automatic speech recognition
KR20230009564A (en) Learning data correction method and apparatus thereof using ensemble score
CN113626563A (en) Method and electronic equipment for training natural language processing model and natural language processing
CN114036957B (en) Rapid semantic similarity calculation method
KR101483947B1 (en) Apparatus for discriminative training acoustic model considering error of phonemes in keyword and computer recordable medium storing the method thereof
JP6577900B2 (en) Phoneme error acquisition device, phoneme error acquisition method, and program
JP2010164918A (en) Speech translation device and method
JP6301794B2 (en) Automaton deformation device, automaton deformation method and program
US20190155902A1 (en) Information generation method, information processing device, and word extraction method
CN113536776B (en) Method for generating confusion statement, terminal device and computer readable storage medium
JP3950957B2 (en) Language processing apparatus and method
CN110399608B (en) Text error correction system and method for dialogue system based on pinyin
KR102299269B1 (en) Method and apparatus for building voice database by aligning voice and script
Lei et al. Data-driven lexicon expansion for Mandarin broadcast news and conversation speech recognition
US20070055515A1 (en) Method for automatically matching graphic elements and phonetic elements

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20200529