CN116227471A - Error correction method and device and electronic equipment - Google Patents

Error correction method and device and electronic equipment Download PDF

Info

Publication number
CN116227471A
CN116227471A CN202310518790.1A CN202310518790A CN116227471A CN 116227471 A CN116227471 A CN 116227471A CN 202310518790 A CN202310518790 A CN 202310518790A CN 116227471 A CN116227471 A CN 116227471A
Authority
CN
China
Prior art keywords
distance
word
node
corrected
search word
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202310518790.1A
Other languages
Chinese (zh)
Other versions
CN116227471B (en
Inventor
孙玉霖
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Honor Device Co Ltd
Original Assignee
Honor Device Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Honor Device Co Ltd filed Critical Honor Device Co Ltd
Priority to CN202310518790.1A priority Critical patent/CN116227471B/en
Publication of CN116227471A publication Critical patent/CN116227471A/en
Application granted granted Critical
Publication of CN116227471B publication Critical patent/CN116227471B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00Handling natural language data
    • G06F40/20Natural language analysis
    • G06F40/232Orthographic correction, e.g. spell checking or vowelisation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/31Indexing; Data structures therefor; Storage structures
    • G06F16/316Indexing structures
    • G06F16/322Trees
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/33Querying
    • G06F16/3331Query processing
    • G06F16/334Query execution
    • G06F16/3344Query execution using natural language analysis
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00Handling natural language data
    • G06F40/20Natural language analysis
    • G06F40/279Recognition of textual entities
    • G06F40/284Lexical analysis, e.g. tokenisation or collocates
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10Character recognition
    • G06V30/12Detection or correction of errors, e.g. by rescanning the pattern
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10Character recognition
    • G06V30/19Recognition using electronic means
    • G06V30/19007Matching; Proximity measures
    • G06V30/19093Proximity measures, i.e. similarity or distance measures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/40Document-oriented image-based pattern recognition
    • G06V30/41Analysis of document content
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computational Linguistics (AREA)
  • Multimedia (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Software Systems (AREA)
  • Character Discrimination (AREA)

Abstract

The application provides an error correction method, an error correction device and electronic equipment. The method comprises the steps of utilizing a trained OCR model to identify at least one character of each word in a preset dictionary, and obtaining a first word vector of each word; establishing a font distance matrix according to the first character vector of any two characters; determining a first query mode based on the font distance matrix and the metric Tree BK-Tree algorithm, and determining a second query mode based on the font distance matrix and the dynamic programming algorithm; and inquiring the search word to be corrected based on the first inquiry mode and/or the second inquiry mode to obtain a search result corresponding to the search word to be corrected, wherein the search result comprises a first target search word or a second target search word. The technical scheme can be used for correcting the shape near-word errors, so that a user can obtain correct search results when the search engine or the application program inputs search words with the shape near-word errors.

Description

Error correction method and device and electronic equipment
Technical Field
The present disclosure relates to the field of computer technologies, and in particular, to an error correction method, an error correction device, and an electronic device.
Background
Chinese input methods generally include pinyin input methods, handwriting input methods, five-stroke input methods, and the like. When a user uses a handwriting input method, a word-approaching error is likely to occur. Because the input method has limited error correction capability and the input method cannot recognize the specific use scene of the vocabulary, when a user inputs the search word in the search engine or the application program, the search engine or the application program usually needs to perform further error correction on the search word.
Currently, a search engine or an application program usually corrects only word-sound errors generated by a search word input through a pinyin input method, but does not correct shape-near word errors generated by a search word input through a handwriting input method. Thus, when a search engine or application searches for a search term having a near word error, the search engine or application cannot obtain a correct search result.
Disclosure of Invention
The application provides an error correction method, an error correction device and electronic equipment, which can solve the problem that a user cannot obtain a correct search result when searching for a search word with a shape and a near word error in a search engine or an application program.
In a first aspect, the present application provides an error correction method, including: recognizing at least one character of each character in a preset dictionary by utilizing the trained optical character recognition OCR model to obtain a first character vector of each character, wherein the fonts of the at least one character are different, and the first character vector comprises the average value of the feature vectors of the at least one character of the character; establishing a font distance matrix according to the first character vectors of any two characters, wherein the font distance matrix comprises font distances between any two characters, and the font distances are obtained according to cosine similarity between the first character vectors of any two characters; determining a first query mode based on the font distance matrix and the BK-Tree algorithm, and determining a second query mode based on the font distance matrix and the dynamic programming algorithm, wherein the first query mode is used for querying a first target search word corresponding to a search word to be corrected, which is input by a user, the first target search word has the same text length as the search word to be corrected, and the second query mode is used for querying a second target search word corresponding to the search word to be corrected, and the second target search word has the same text length as the search word to be corrected or different text lengths from the search word to be corrected; and inquiring the search word to be corrected based on the first inquiry mode and/or the second inquiry mode to obtain a search result corresponding to the search word to be corrected, wherein the search result comprises a first target search word or a second target search word. According to the error correction method, the first query mode and the second query mode can be configured in the search engine or the application program to perform quick query or supplementary query, so that the search engine or the application program can correct search words with shape and near word errors input by a user, correct search results can be obtained under the condition that the search words have multiple words, wrong words or missed words, and the problem that the recall rate of the search results is low when the user searches the search words with shape and near word errors is solved.
In one implementation, the method further includes, before identifying at least one character of each word in the preset dictionary by using the trained OCR model to obtain the first word vector of each word: acquiring a training data set, wherein the training data set comprises at least one single-word image data, and the single-word image data is used for recording a character in each image; at least one single-word image data is input into an initial OCR model for training, and a trained OCR model is obtained. By adopting the implementation mode, a specific training mode of the OCR model is shown, so that the feature vector of each character can be obtained according to the trained OCR model and used for constructing a subsequent query mode.
In one implementation, identifying at least one character of each word using the trained OCR model to obtain a first word vector for each word, comprising: deleting the softmax layer from the trained OCR model, and deleting at least one full connection layer connected with the softmax layer to obtain a feature extraction model; and obtaining a first character vector of each character according to the feature extraction model, wherein the first character vector is obtained according to the average value of the feature vectors of at least one character of the characters. By adopting the implementation mode, a specific acquisition mode of the first text vector is shown, and the prediction probability can be improved by deleting the normalized softmax layer and deleting the full connection layer connected with the softmax layer.
In one implementation, establishing a glyph distance matrix from a first text vector of any two texts includes: acquiring cosine similarity between first character vectors of any two characters; discretizing the first text vector according to the cosine similarity to obtain the font distance of any two texts, wherein the discretizing comprises: obtaining a first difference value between the similarity of 1 and the cosine, multiplying the first difference value by 100, and rounding; and establishing a font distance matrix according to the font distance of any two characters. By adopting the implementation mode, a specific construction mode of the font distance matrix is shown, and the similarity of two characters can be determined by determining the font distance between any two characters.
In one implementation, determining a first query pattern based on a glyph distance matrix and a metric Tree BK-Tree algorithm includes: determining at least one preset word as a node of BK-Tree; establishing an error correction tree according to the nodes and the node distance between any two nodes, wherein the node distance is determined according to the font distance between characters in any two preset words; a first query pattern is determined from the error correction tree. By adopting the implementation mode, the error correction Tree can be built through BK-Tree algorithm building, so that the matching result can be obtained rapidly by matching the search word to be corrected with the error correction Tree.
In one implementation, the determining the node distance includes: acquiring at least one group of first characters in any two nodes, wherein each group of first characters is a character with a position corresponding relation between any two nodes; inputting each group of first characters into a first font distance function to obtain a first distance between each group of first characters, wherein the first font distance function is determined according to a font distance matrix, and the first distance is the font distance between each group of first characters; and determining the node distance according to the sum of the first distances and the second distance, wherein the second distance is determined according to the product of the absolute value of a second difference value and a first preset distance, the second difference value is the difference value of the lengths of characters in any two nodes, and the first preset distance is used for assigning values to the characters which do not have a position corresponding relation between any two nodes. By adopting the implementation mode, a specific setting mode of the node distance in the error correction tree is shown. In this way, the node distance is determined based on the font distance on the basis of the traditional BK-Tree algorithm, and the method can be used for matching the font-nearest characters.
In one implementation, each node corresponds to at least one child node, and the first node in the error correction tree is the root node; establishing an error correction tree according to the nodes and the node distance between any two nodes, wherein the error correction tree comprises the following steps: inputting a newly added word into the error correction tree; acquiring a first node distance between the newly added word and the root node and a second node distance between the root node and the corresponding child node, wherein the first node distance is a node distance between the newly added word and a current matching node in the error correction tree, and the second node distance is a node distance between the current matching node and the corresponding child node; if the first node distance is equal to the second node distance, determining a child node corresponding to the current matching node as a next matching node, and acquiring a first node distance between the newly added word and the next matching node and a second node distance between the next matching node and the corresponding child node; repeatedly executing the previous step until the distance between the first node and the second node is not equal to the distance between the first node and the second node, and determining the newly added word as a new child node; and inserting new child nodes into the error correction tree, and establishing an updated error correction tree. By adopting the implementation mode, a specific mode of inserting the new child node into the error correction tree is shown. In this way, the embodiment of the application can update the error correction tree, so that the error correction tree can be used for matching more preset words.
In one implementation, querying the search word to be corrected based on the first query mode and/or the second query mode to obtain a search result corresponding to the search word to be corrected, including: acquiring a third node distance between the search word to be corrected and each node; pruning redundant nodes in the error correction tree according to a pruning mechanism to obtain a pruning result corresponding to the search word to be corrected, wherein the redundant nodes are nodes smaller than a first threshold and larger than a second threshold, the first threshold is a third difference value between a third node distance and a second preset distance, and the second threshold is a sum of the third node distance and the second preset distance; if the pruning result comprises a target node, determining a preset word corresponding to the target node as a first target search word, wherein the target node is the node with the shortest distance to the node of the search word to be corrected and is smaller than a second preset distance. By adopting the implementation mode, the obviously unmatched nodes can be deleted through a pruning mechanism of the BK-Tree algorithm, so that the matching efficiency of the search word to be corrected and the target node is improved.
In one implementation, querying the search word to be corrected based on the first query mode and/or the second query mode to obtain a search result corresponding to the search word to be corrected, including: if the search word to be corrected is queried based on the first query mode, a first target search word is obtained, and the first target search word is determined to be a search result; if the search word is queried based on the first query mode, the first target search word is not obtained, and then the search word is queried based on the second query mode, the second target search word is obtained, and the second target search word is determined to be a search result. By adopting the implementation mode, under the condition that the first query mode cannot be queried, the second query mode is replaced for query.
In one implementation, determining the second query pattern based on the glyph distance matrix and the dynamic programming algorithm includes: establishing a first circulation variable i and a second circulation variable j, wherein i represents the position of characters in a preset word, j represents the position of characters in a search word to be corrected, i is a positive integer greater than or equal to 0, and j is a positive integer greater than or equal to 0; establishing a first distance function dis 1 (i, j), wherein if the minimum value in i or j is equal to 0, dis 1 (i, j) is equal to the product of the maximum value of i or j and a third preset distance, if the minimum value of i or j is not equal to 0, dis 1 (i, j) equals dis 1 (i-1, j) and a third preset distance, dis 1 (i, j-1) and a third predetermined distance, dis 1 The minimum value between (i-1, j-1) and the sum of a second font distance function dis-Dict (s 1, i, s2, j), wherein the second font distance function is determined according to a font distance matrix, dis-Dict (s 1, i, s2, j) represents the font distance between the ith character in a preset word and the jth character in a search word to be corrected in the font distance matrix, and the third preset distance is used for initializing and assigning the minimum value in i or j to be equal to 0; a second query pattern is determined based on the first distance function. By adopting the implementation mode, the setting mode of the program algorithm of the second query mode is shown, so that when the lengths of the words to be corrected and the words of the second target search word are different, the correct search result can be obtained.
In one implementation, querying the search word to be corrected based on the first query mode and/or the second query mode to obtain a search result corresponding to the search word to be corrected, including: determining a first glyph distance dis when i equals m and j equals n according to a first distance function 1 (m, n), wherein m is the word length of a preset word, and n is the word length of a search word to be corrected; will dis 1 The difference value between the (m, n) and the redundant distance is determined as a first target font distance, wherein the redundant distance is the product of a fourth difference value and a fifth difference value, the fourth difference value is the absolute value of the difference value between the m and the n, the fifth difference value is the difference value between a third preset distance and a fourth preset distance, and the fourth preset distance is used for carrying out custom assignment on the search word to be corrected or the unmatched characters in the preset word; and determining a second target search word, wherein the second target search word is a preset word with the shortest first target font distance with the search word to be corrected. By adopting the implementation mode, a specific query mode of the second query mode is shown, and the second target search word corresponding to the search word to be corrected can be determined.
In one implementation, determining the second query pattern based on the glyph distance matrix and the dynamic programming algorithm includes: establishing a first circulation variable i and a second circulation variable j, wherein i represents that characters are in advance Setting the position in the word, wherein j represents the position of the word in the search word to be corrected, i is a positive integer greater than or equal to 0, and j is a positive integer greater than or equal to 0; establishing a second distance function dis 2 (i, j), wherein if the minimum value in i or j is equal to 0, dis 2 (i, j) is equal to the product of the maximum value of i or j and the fifth preset distance, if the minimum value of i or j is not equal to 0, dis (i, j) is equal to dis 2 Sum of (i-1, j) and fifth preset distance dis 2 The minimum value between (i-1, j-1) and the sum of a second font distance function dis-Dict (s 1, i, s2, j), wherein the second font distance function is determined according to a font distance matrix, the dis-Dict (s 1, i, s2, j) represents the font distance between the ith character in a preset word and the jth character in a search word to be corrected in the font distance matrix, and the fifth preset distance is used for initializing and assigning the minimum value in i or j to be equal to 0; a second query pattern is determined based on a second distance function. By adopting the implementation mode, the setting mode of the program algorithm of the second query mode is shown, so that when the length of the search word to be corrected is the same as that of the second target search word, a correct search result can be obtained.
In one implementation, querying the search word to be corrected based on the first query mode and/or the second query mode to obtain a search result corresponding to the search word to be corrected, including: determining a second glyph distance dis when i equals m and j equals n according to a second distance function 2 (m, n), wherein m is the word length of a preset word, and n is the word length of a search word to be corrected; will dis 2 The difference value between the (m, n) and the redundant distance is determined as a second target font distance, wherein the redundant distance is the product of a fourth difference value and a fifth difference value, the fourth difference value is the absolute value of the difference value between the m and the n, the fifth difference value is the difference value between a fifth preset distance and a sixth preset distance, and the sixth preset distance is used for carrying out custom assignment on characters which are not matched in a search word to be corrected or a preset word; and determining a second target search word, wherein the second target search word is a preset word with the shortest second target font distance from the search word to be corrected. With the implementation, a second query module is shownThe specific query mode of the formula can determine a second target search word corresponding to the search word to be corrected.
In one implementation, querying the search word to be corrected based on the first query mode and/or the second query mode to obtain a search result corresponding to the search word to be corrected, including: if dis 1 (i, j) or dis 2 (i, j) in the traversal of i equal to 0 to i equal to m, j equal to 0 to j equal to n,
Figure SMS_1
or->
Figure SMS_2
The current preset word is determined to not comprise the second target search word when the current preset word is larger than the seventh preset distance; traversing the next preset word until a search result corresponding to the search word to be corrected is obtained. By adopting the implementation mode, in the process of inquiring the second target search word through the second inquiry mode, the unmatched preset words are screened out by setting the seventh preset distance, so that the inquiry efficiency can be accelerated.
In a second aspect, the present application further provides an error correction apparatus, including: the training module is used for identifying at least one character of each character in a preset dictionary by utilizing the trained optical character recognition OCR model to obtain a first character vector of each character, the fonts of the at least one character are different, and the first character vector comprises the average value of the feature vectors of the at least one character of the characters; the matrix building module is used for building a font distance matrix according to the first character vectors of any two characters, wherein the font distance matrix comprises font distances between any two characters, and the font distances are obtained according to cosine similarity between the first character vectors of any two characters; the determining module is used for determining a first query mode based on the font distance matrix and the BK-Tree algorithm and determining a second query mode based on the font distance matrix and the dynamic programming algorithm, wherein the first query mode is used for querying a first target search word corresponding to a search word to be corrected, which is input by a user, the first target search word has the same text length as the search word to be corrected, and the second query mode is used for querying a second target search word corresponding to the search word to be corrected, and the second target search word has the same text length as the search word to be corrected or different text lengths from the search word to be corrected; the query module is used for querying the search word to be corrected based on the first query mode and/or the second query mode, and obtaining a search result corresponding to the search word to be corrected, wherein the search result comprises a first target search word or a second target search word.
According to the error correction device, the first query mode and the second query mode can be configured in the search engine or the application program to perform quick query or supplementary query, so that the search engine or the application program can correct search words with shape and near word errors input by a user, correct search results can be obtained under the condition that the search words have multiple words, wrong words or missed words, and the problem that the recall rate of the search results is low when the user searches the search words with shape and near word errors is solved.
In a third aspect, the present application further provides an electronic device, including: a processor and a memory for storing at least one instruction which when loaded and executed by the processor is adapted to carry out the method as in the first aspect and its implementation described above.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments will be briefly described below, and it will be obvious to those skilled in the art that other drawings can be obtained from these drawings without inventive effort.
Fig. 1 is a schematic hardware structure of an electronic device 100 according to an embodiment of the present application;
FIG. 2 is a software architecture block diagram of an electronic device 100 of an embodiment of the present application;
FIG. 3 is a first flowchart of an error correction method provided by an embodiment of the present application;
FIG. 4 is a schematic view of an OCR model training scenario provided by an embodiment of the present application;
FIG. 5 is a schematic diagram of a glyph distance matrix provided by embodiments of the present application;
FIG. 6 is a second flow chart of an error correction method provided by an embodiment of the present application;
FIG. 7 is a schematic diagram of an error correction tree according to an embodiment of the present application;
FIG. 8 is a third flow chart of an error correction method provided by an embodiment of the present application;
FIG. 9 is a schematic diagram of a scene of an error correction tree search process according to an embodiment of the present application;
FIG. 10 is a schematic diagram of a first query pattern query scenario provided in an embodiment of the present application;
FIG. 11 is a fourth flowchart of an error correction method provided by an embodiment of the present application;
FIG. 12 is a visual schematic of a dynamic programming algorithm shown in an embodiment of the present application;
FIG. 13 is a schematic diagram of a second query pattern query scenario provided in an embodiment of the present application;
FIG. 14 is a diagram of search results provided by an embodiment of the present application.
Detailed Description
The technical solutions of the embodiments of the present application will be clearly described below with reference to the drawings in the embodiments of the present application.
In the description of the present application, "/" means "or" unless otherwise indicated, for example, a/B may mean a or B. "and/or" herein is merely an association relationship describing an association object, and means that three relationships may exist, for example, a and/or B may mean: a exists alone, A and B exist together, and B exists alone. Furthermore, "at least one" means one or more, and "a plurality" means two or more. The terms "first," "second," and the like do not limit the number and order of execution, and the terms "first," "second," and the like do not necessarily differ.
In this application, the terms "exemplary" or "such as" are used to mean serving as an example, instance, or illustration. Any embodiment or design described herein as "exemplary" or "for example" should not be construed as preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "such as" is intended to present related concepts in a concrete fashion.
The terminology used in the description of the embodiments of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application, as will be described in detail with reference to the accompanying drawings.
Chinese input methods generally include pinyin input methods, handwriting input methods, five-stroke input methods, and the like. When a user inputs Chinese, an input error is liable to occur. In which, when the user uses the pinyin input method, a word sound error may occur, for example, input "hello (nih a o)", but output "you number (nih a o)". When a user uses the handwriting input method, a word-approaching error easily occurs, for example, input "express" but output "express".
The input method has certain error correction capability. For example, when a user uses a pinyin input method, he wants to input "fermentation", and the actual pinyin is (fa ji a o), but he perceives his pinyin as (fa xi a o) by subjective mistakes. After the user inputs (fa xi a o), the pinyin input method can correct the errors, and the 'fermentation' is displayed in the input box. For example, when a user uses a handwriting input method, the user wants to input "enter" and when writing "enter", the user writes as "person". After the user inputs "person", the handwriting input method can correct the error, and the "enter" is displayed in the input box.
However, the error correction capability of the input method is limited. Whether pinyin or handwriting, the search term may still be the wrong search term when entered into a search engine or application by input.
For example, when a user uses a pinyin input method in a search engine, he or she wishes to input "sell something" and actually inputs "buy something" in the search box. Because both words are correct, but the representative meanings are different, for the input method, the input method cannot recognize the specific use scene of the words, and cannot determine whether the words are correct search words, and for the user, the user actually inputs the search words with word sound errors.
By way of example, when a user uses a handwriting input method in a search engine, the following input scenario exists.
In the first scenario, the user subjectively inputs an error.
When the user wishes to input "car", the user writes "car" as "east" because he forgets "car" writing method, resulting in displaying "car east" in the search box.
In the second scenario, the user writes in disorder, which causes the input method to be recognized incorrectly or not.
When the user wishes to input "things", the input method recognizes "east" word as "car" because the user writes "horizontal" and "vertical" of "east" word in succession, resulting in "car west" being displayed in the search box.
In a third scenario, the input method corrects the user input words to high frequency words.
When the user hopes to input 'X take-out express', the user writes the 'send' word in a poor way, and the 'express' is a high-frequency word in the word stock of the input method, so the input method recognizes the 'express' as the 'express', and the 'X take-out express' is displayed in the search box.
Fourth scenario, user input shorthand.
When the user wishes to search for "agricultural banks", the user simply wishes to input shorthand for "farm" only, and because the input method identifies the "farm" as "clothing" due to the error in recognition of the "farm" word by the input method, the input method causes the "clothing" to be displayed in the search box.
In a fifth scenario, the input method performs error correction based on a conventional distance editing algorithm.
The traditional font error correction method is to encode by five strokes and calculate the distance of Chinese characters by using an edit distance algorithm. However, there is a difference between the five-stroke code and the actual font of the Chinese character, for example, the five-stroke code of the "fast" word is "nnw", the five-stroke code of the "rest" word is "rfw", and only one code is the same between the two words, but the fonts are often misidentified by the input method due to the similarity. That is, recognition errors of the input method generally occur between words of the shape close to each other, rather than between characters of similar five-stroke codes, and the conventional font error correction method has a great limitation.
In this way, in the usage scenario of the handwriting input method, the user actually inputs the search word having the shape-near word error.
When an erroneous search term occurs in the search box, the search engine or application typically performs further error correction on the erroneous search term by means of a configured error correction function.
However, due to the wide use of pinyin input methods, error correction functions in search engines or applications are typically based on word-tone errors generated by pinyin input methods, but cannot be based on shape-near word errors generated by handwriting input methods. That is, in the above example, the search engine or application can usually correct "buy (mLei) thing" as "sell (mLei) thing" only according to the usage scenario, and cannot correct "coat (y ī) line" as "farm (nLeg) line".
Thus, when a user inputs a search term having a word-in-word error in a search engine or an application, the user cannot obtain a correct search result.
In order to solve the above problems, the present application provides an error correction method.
The error correction method provided by the application can be applied to electronic equipment. The electronic device includes, but is not limited to, a mobile phone, a tablet computer, a personal computer, a workstation device, a large screen device (such as a smart screen, a smart television and the like), a wearable device (such as a smart bracelet, a smart watch) and a palm game machine, a household game machine, a virtual reality device, an augmented reality device, a mixed reality device and the like, a vehicle-mounted intelligent terminal and the like.
Fig. 1 is a schematic hardware structure of an electronic device according to an embodiment of the present application. As shown in fig. 1, the electronic device 100 may include a processor 110, a memory 120, a universal serial bus (universal serial bus, USB) interface 130, a charge management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, keys 190, a motor 191, a camera 192, a display 193, and a subscriber identity module (subscriber identification module, SIM) card interface 194, etc. The sensor module 180 may include a touch sensor 180A, a gyro sensor 180B, an air pressure sensor 180C, a geomagnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, and the like. Among them, the gyro sensor 180B, the air pressure sensor 180C, the geomagnetic sensor 180D, the acceleration sensor 180E, and the like can be used to detect a motion state of an electronic apparatus, and thus, may also be referred to as a motion sensor.
It is to be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, electronic device 100 may include more or fewer components than shown, or certain components may be combined, or certain components may be split, or different arrangements of components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.
The processor 110 may include one or more processing units, such as: the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), a controller, a video codec, a digital signal processor (digital signal processor, DSP), a baseband processor, and/or a neural network processor (neural-network processing unit, NPU), etc. Wherein the different processing units may be separate devices or may be integrated in one or more processors.
Memory 120 may be used to store computer-executable program code that includes instructions. The memory 120 may include a stored program area and a stored data area. The storage program area may store an application program (such as a sound playing function, an image playing function, etc.) required for at least one function of the operating system, etc. The storage data area may store data created during use of the electronic device 100 (e.g., audio data, phonebook, etc.), and so on. In addition, the memory 120 may include a high-speed random access memory, and may also include a nonvolatile memory, such as at least one magnetic disk storage device, a flash memory device, a universal flash memory (universal flash storage, UFS), and the like. The processor 110 performs various functional applications and data processing of the electronic device 100 by executing instructions stored in the memory 120 and/or instructions stored in a memory provided in the processor.
The USB interface 130 is an interface conforming to the USB standard specification, and may specifically be a Mini USB interface, a Micro USB interface, a USB Type C interface, or the like. The USB interface 130 may be used to connect a charger to charge the electronic device 100, and may also be used to transfer data between the electronic device 100 and a peripheral device. And can also be used for connecting with a headset, and playing audio through the headset. The interface may also be used to connect other electronic devices, such as AR devices, etc.
It should be understood that the interfacing relationship between the modules illustrated in the embodiments of the present application is only illustrative, and does not limit the structure of the electronic device 100. In other embodiments of the present application, the electronic device 100 may also use different interfacing manners, or a combination of multiple interfacing manners in the foregoing embodiments.
The charge management module 140 is configured to receive a charge input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charge management module 140 may receive a charging input of a wired charger through the USB interface 130. In some wireless charging embodiments, the charge management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100. The charging management module 140 may also supply power to the electronic device through the power management module 141 while charging the battery 142.
The power management module 141 is used for connecting the battery 142, and the charge management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and/or the charge management module 140 and provides power to the processor 110, the memory 120, the display 193, the camera 192, the wireless communication module 160, and the like. The power management module 141 may also be configured to monitor battery capacity, battery cycle number, battery health (leakage, impedance) and other parameters. In other embodiments, the power management module 141 may also be provided in the processor 110. In other embodiments, the power management module 141 and the charge management module 140 may be disposed in the same device.
The wireless communication function of the electronic device 100 may be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, a modem processor, a baseband processor, and the like.
The antennas 1 and 2 are used for transmitting and receiving electromagnetic wave signals. Each antenna in the electronic device 100 may be used to cover a single or multiple communication bands. Different antennas may also be multiplexed to improve the utilization of the antennas. For example: the antenna 1 may be multiplexed into a diversity antenna of a wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.
The mobile communication module 150 may provide a solution for wireless communication including 2G/3G/4G/5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (low noise amplifier, LNA), etc. The mobile communication module 150 may receive electromagnetic waves from the antenna 1, perform processes such as filtering, amplifying, and the like on the received electromagnetic waves, and transmit the processed electromagnetic waves to the modem processor for demodulation. The mobile communication module 150 can amplify the signal modulated by the modem processor, and convert the signal into electromagnetic waves through the antenna 1 to radiate. In some embodiments, at least some of the functional modules of the mobile communication module 150 may be disposed in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 may be provided in the same device as at least some of the modules of the processor 110.
The modem processor may include a modulator and a demodulator. The modulator is used for modulating the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used for demodulating the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low frequency baseband signal to the baseband processor for processing. The low frequency baseband signal is processed by the baseband processor and then transferred to the application processor. The application processor outputs sound signals through an audio device (not limited to the speaker 170A, the receiver 170B, etc.), or displays images or videos through the display screen 193. In some embodiments, the modem processor may be a stand-alone device. In other embodiments, the modem processor may be provided in the same device as the mobile communication module 150 or other functional module, independent of the processor 110.
The wireless communication module 160 may provide solutions for wireless communication including wireless local area network (wireless local area networks, WLAN) (e.g., wireless fidelity (wireless fidelity, wi-Fi) network), bluetooth (BT), global navigation satellite system (global navigation satellite system, GNSS), frequency modulation (frequency modulation, FM), near field wireless communication technology (near field communication, NFC), infrared technology (IR), etc., as applied to the electronic device 100. The wireless communication module 160 may be one or more devices that integrate at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the electromagnetic wave signals, filters the electromagnetic wave signals, and transmits the processed signals to the processor 110. The wireless communication module 160 may also receive a signal to be transmitted from the processor 110, frequency modulate it, amplify it, and convert it to electromagnetic waves for radiation via the antenna 2.
In some embodiments, antenna 1 and mobile communication module 150 of electronic device 100 are coupled, and antenna 2 and wireless communication module 160 are coupled, such that electronic device 100 may communicate with a network and other devices through wireless communication techniques. The wireless communication techniques may include the Global System for Mobile communications (global system for mobile communications, GSM), general packet radio service (general packet radio service, GPRS), code division multiple access (code division multiple access, CDMA), wideband code division multiple access (wideband code division multiple access, WCDMA), time division code division multiple access (time-division code division multiple access, TD-SCDMA), long term evolution (long term evolution, LTE), BT, GNSS, WLAN, NFC, FM, and/or IR techniques, among others. The GNSS may include a global satellite positioning system (global positioning system, GPS), a global navigation satellite system (global navigation satellite system, GLONASS), a beidou satellite navigation system (beidou navigation satellite system, BDS), a quasi zenith satellite system (quasi-zenith satellite system, QZSS) and/or a satellite based augmentation system (satellite based augmentation systems, SBAS).
The electronic device 100 implements display functions through a GPU, a display screen 193, an application processor, and the like. The GPU is a microprocessor for image processing, and is connected to the display 193 and an application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or change display information.
The display 193 is used to display images, videos, and the like. The display 193 includes a display panel. The display panel may employ a liquid crystal display (liquid crystal display, LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED) or an active-matrix organic light-emitting diode (matrix organic light emitting diode), a flexible light-emitting diode (flex), a mini, a Micro led, a Micro-OLED, a quantum dot light-emitting diode (quantum dot light emitting diodes, QLED), or the like. In some embodiments, electronic device 100 may include 1 or N display screens 193, N being a positive integer greater than 1.
The electronic device 100 may implement photographing functions through an ISP, a camera 192, a video codec, a GPU, a display screen 193, an application processor, and the like.
The ISP is used to process the data fed back by the camera 192. For example, when photographing, the shutter is opened, light is transmitted to the camera photosensitive element through the lens, the optical signal is converted into an electric signal, and the camera photosensitive element transmits the electric signal to the ISP for processing and is converted into an image visible to naked eyes. ISP can also optimize the noise, brightness and skin color of the image. The ISP can also optimize parameters such as exposure, color temperature and the like of a shooting scene. In some embodiments, the ISP may be located in the camera 192.
The camera 192 is used to capture still images or video. The object generates an optical image through the lens and projects the optical image onto the photosensitive element. The photosensitive element converts the optical signal into an electrical signal, which is then transferred to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, RYYB, YUV, or the like format. In some embodiments, the electronic device 100 may include 1 or N cameras 192, N being a positive integer greater than 1.
The electronic device 100 may implement audio functions through an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, an application processor, and the like. Such as music playing, recording, etc.
The touch sensor 180A, also referred to as a "touch device". The touch sensor 180A may be disposed on the display 193, and the touch sensor 180A and the display 193 form a touch screen, which is also referred to as a "touch screen". The touch sensor 180A is used to detect a touch operation acting thereon or thereabout. The touch sensor may communicate the detected touch operation to the application processor to determine the touch event type. Visual output related to the touch operation may be provided through the display 193. In other embodiments, the touch sensor 180A may also be disposed on a surface of the electronic device 100 at a location different from the location of the display 193.
The gyro sensor 180B may be used to determine a motion gesture of the electronic device 100. In some embodiments, the angular velocity of electronic device 100 about three axes (i.e., x, y, and z axes) may be determined by gyro sensor 180B. The gyro sensor 180B may be used for photographing anti-shake. For example, when the shutter is pressed, the gyro sensor 180B detects the shake angle of the electronic device 100, calculates the distance to be compensated by the lens module according to the angle, and makes the lens counteract the shake of the electronic device 100 through the reverse motion, so as to realize anti-shake. The gyro sensor 180B may also be used for navigating, somatosensory game scenes.
The air pressure sensor 180C is used to measure air pressure. In some embodiments, electronic device 100 calculates altitude from barometric pressure values measured by barometric pressure sensor 180C, aiding in positioning and navigation.
The geomagnetic sensor 180D includes a hall sensor. The electronic device 100 may detect the opening and closing of the flip cover using the geomagnetic sensor 180D. In some embodiments, when the electronic device 100 is a flip machine, the electronic device 100 may detect the opening and closing of the flip according to the geomagnetic sensor 180D. And then according to the detected opening and closing state of the leather sheath or the opening and closing state of the flip, the characteristics of automatic unlocking of the flip and the like are set.
The acceleration sensor 180E may detect the magnitude of acceleration of the electronic device 100 in various directions (typically three axes). The magnitude and direction of gravity may be detected when the electronic device 100 is stationary. The electronic equipment gesture recognition method can also be used for recognizing the gesture of the electronic equipment, and is applied to horizontal and vertical screen switching, pedometers and other applications.
A distance sensor 180F for measuring a distance. The electronic device 100 may measure the distance by infrared or laser. In some embodiments, the electronic device 100 may range using the distance sensor 180F to achieve quick focus.
The proximity light sensor 180G may include, for example, a light emitting diode and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The electronic device 100 emits infrared light outward through the light emitting diode. The electronic device 100 detects infrared reflected light from nearby objects using a photodiode. When sufficient reflected light is detected, it may be determined that there is an object in the vicinity of the electronic device 100. When insufficient reflected light is detected, the electronic device 100 may determine that there is no object in the vicinity of the electronic device 100. The electronic device 100 can detect that the user holds the electronic device 100 close to the ear by using the proximity light sensor 180G, so as to automatically extinguish the screen for the purpose of saving power. The proximity light sensor 180G may also be used in holster mode, pocket mode to automatically unlock and lock the screen.
The fingerprint sensor 180H is used to collect a fingerprint. The electronic device 100 may utilize the collected fingerprint feature to unlock the fingerprint, access the application lock, photograph the fingerprint, answer the incoming call, etc.
The temperature sensor 180J is for detecting temperature. In some embodiments, the electronic device 100 performs a temperature processing strategy using the temperature detected by the temperature sensor 180J. For example, when the temperature reported by temperature sensor 180J exceeds a threshold, electronic device 100 performs a reduction in the performance of a processor located in the vicinity of temperature sensor 180J in order to reduce power consumption to implement thermal protection. In other embodiments, when the temperature is below another threshold, the electronic device 100 heats the battery 142 to avoid the low temperature causing the electronic device 100 to be abnormally shut down. In other embodiments, when the temperature is below a further threshold, the electronic device 100 performs boosting of the output voltage of the battery 142 to avoid abnormal shutdown caused by low temperatures.
The keys 190 include a power-on key, a volume key, etc. The keys 190 may be mechanical keys. Or may be a touch key. The electronic device 100 may receive key inputs, generating key signal inputs related to user settings and function controls of the electronic device 100.
The motor 191 may generate a vibration cue. The motor 191 may be used for incoming call vibration alerting as well as for touch vibration feedback. For example, touch operations acting on different applications (e.g., photographing, audio playing, etc.) may correspond to different vibration feedback effects. The motor 191 may also correspond to different vibration feedback effects by touch operations applied to different areas of the display screen 193. Different application scenarios (such as time reminding, receiving information, alarm clock, game, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect may also support customization.
The SIM card interface 194 is used to connect to a SIM card. The SIM card may be inserted into the SIM card interface 194, or removed from the SIM card interface 194 to enable contact and separation with the electronic device 100. The electronic device 100 may support 1 or N SIM card interfaces, N being a positive integer greater than 1. The SIM card interface 194 may support a Nano SIM card, micro SIM card, etc. The same SIM card interface 194 may be used to insert multiple cards simultaneously. The types of the plurality of cards may be the same or different. The SIM card interface 194 may also be compatible with different types of SIM cards. The SIM card interface 194 may also be compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to realize functions such as communication and data communication. In some embodiments, the electronic device 100 employs esims, i.e.: an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.
The software system of the electronic device 100 may employ a layered architecture, an event driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture. In this embodiment, taking an Android system with a layered architecture as an example, a software structure of the electronic device 100 is illustrated.
Fig. 2 is a software configuration block diagram of the electronic device 100 according to the embodiment of the present application.
The layered architecture divides the software into several layers, each with distinct roles and branches. The layers communicate with each other through a software interface. In some embodiments, the Android system is divided into four layers, from top to bottom, an application layer, an application framework layer, an Zhuoyun row (Android run) and system libraries, and a kernel layer, respectively.
The application layer may include a series of application packages.
As shown in fig. 2, the application package may include battery management, camera, gallery, calendar, talk, map, navigation, music, video, short message, etc. applications.
The application framework layer provides an application program interface (application programming interface, API) and programming framework for application programs of the application layer. The application framework layer includes a number of predefined functions.
As shown in FIG. 2, the application framework layer may include a window manager, an input manager InputManager, a sensor manager SensorManager, a phone manager, a resource manager, a notification manager, and so forth.
The input manager may be used to monitor input events of the user, such as click events, swipe events, etc., performed by the user's finger on the display screen 193 of the electronic device 100. By listening for input events, the electronic device 100 can determine whether the electronic device is being used.
The sensor manager is used to monitor data returned by various sensors in the electronic device, such as motion sensor data, proximity sensor data, temperature sensor data, and the like. Using the data returned by the various sensors, the electronic device can determine whether it is jittered, whether the display 193 is occluded, etc.
Android run time includes a core library and virtual machines. Android run time is responsible for scheduling and management of the Android system.
The core library consists of two parts: one part is a function which needs to be called by java language, and the other part is a core library of android.
The application layer and the application framework layer run in a virtual machine. The virtual machine executes java files of the application program layer and the application program framework layer as binary files. The virtual machine is used for executing the functions of object life cycle management, stack management, thread management, security and exception management, garbage collection and the like.
The system library may include a plurality of functional modules. For example: surface manager (surface manager), media Libraries (Media Libraries), three-dimensional graphics processing Libraries (e.g., openGL ES), 2D graphics engines (e.g., SGL), etc.
The surface manager is used to manage the display subsystem and provides a fusion of 2D and 3D layers for multiple applications.
Media libraries support a variety of commonly used audio, video format playback and recording, still image files, and the like. The media library may support a variety of audio and video encoding formats, such as MPEG4, h.264, MP3, AAC, AMR, JPG, PNG, etc.
The three-dimensional graphic processing library is used for realizing three-dimensional graphic drawing, image rendering, synthesis, layer processing and the like.
The 2D graphics engine is a drawing engine for 2D drawing.
The kernel layer is a layer between hardware and software. The inner core layer at least comprises a display driver, a camera driver, an audio driver and a sensor driver.
The error correction method shown in the embodiment of the application can be configured in a search engine or an application program of an application program layer.
It is to be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device may include more or less components than illustrated, or certain components may be combined, or certain components may be split, or different arrangements of components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.
Fig. 3 is a first flowchart of an error correction method provided in an embodiment of the present application.
As shown in fig. 3, the error correction method includes steps S11 to S14.
Step S11, recognizing at least one character of each character in a preset dictionary by utilizing the trained optical character recognition (Optical Character Recognition, OCR) model to obtain a first character vector of each character, wherein the first character vector comprises an average value of feature vectors of at least one character of the character.
OCR is a technology for analyzing, identifying and processing an image file containing text data to acquire text and layout information. The method generally comprises the processes of image input, preprocessing, layout analysis, character cutting, character recognition, layout recovery, post-processing, verification and the like. The character font information of the characters cannot be well represented by adopting the traditional five-stroke coding, so that characters are recognized by adopting an OCR model, and the character font information of the characters can be well represented.
In some embodiments, step S01-step S02 are also included before step S11.
In step S01, a training data set is acquired, the training data set comprising at least one single-word image data for recording one character in each image. For example, each letter corresponds to a character of at least one handwriting, and a character of at least one print. Typically, the character of the handwriting more facilitates the OCR model learning the association between glyphs.
And step S02, inputting at least one single-word image data into the initial OCR model for training, and obtaining a trained OCR model.
In some embodiments, step S11 includes step S111-step S113.
And step S111, deleting the normalized softmax layer from the trained OCR model, and deleting at least one full-connection layer connected with the softmax layer to obtain a feature extraction model.
Typically, after the single-word image data is input into the initial OCR model, the initial OCR model converts the characters in the single-word image data into a composite vector, and inputs the composite vector into a fully-connected feed-forward network, where the feed-forward network includes at least one fully-connected layer, and after passing through the fully-connected layer, the single-word image data enters a softmax layer, where the softmax layer predicts which of the characters corresponds to.
The softmax layer is capable of normalizing the various classification results. Deleting the softmax layer from the trained OCR model and deleting at least one full-connection layer connected with the softmax layer can improve the feature extraction capability of the obtained feature extraction model on single-word image data, and further can improve the prediction probability of the feature extraction model on words. Therefore, by inputting characters into such a feature extraction model, it is possible to obtain a feature vector of high quality corresponding to characters, for example: and obtaining the one-dimensional feature vector of the text.
By way of example, the handwriting characters of the "medium" word and the print characters of the "medium" word are input into the feature extraction model, and the feature extraction model does not normalize the two characters, but predicts the two characters respectively, because the softmax layer is deleted in the feature extraction model, so that the feature extraction model can predict whether the handwriting characters or the print characters are the "medium" characters or the characters corresponding to the characters.
Step S112, a first character vector of each character is obtained according to the feature extraction model.
Wherein, for any one character, the first character vector is the average value of the feature vectors corresponding to the characters of each font of the character.
In a specific implementation, characters of at least one font may be input into the feature extraction model, for example: for a "middle" word, a "middle" handwritten character and a "middle" print character may be entered. Thus, the feature extraction model can obtain one-dimensional feature vectors corresponding to the characters of each font. And then, averaging the feature vectors corresponding to the characters of each font to obtain a first character vector corresponding to the character.
Fig. 4 is a schematic view of an OCR model training scene provided in an embodiment of the present application.
As shown in fig. 4, the first character vector corresponding to the "medium" word can be obtained by inputting the "medium" word into the OCR model. For example, the first text vector a= (x) corresponding to the "medium" word 1 ,y 1 ,z 1 ,……)。
Step S12, a font distance matrix is established according to the first character vectors of any two characters, wherein the font distance matrix comprises font distances between any two characters, and the font distances are obtained according to cosine similarity between the first character vectors of any two characters.
In some embodiments, step S12 includes steps S121-S123.
S121, obtaining cosine similarity between first character vectors of any two characters.
For example, any two words are "car", respectively. First text vector b= (x) corresponding to "steam" word 2 ,y 2 ,z 2 … …) and the first text vector corresponding to the "car" word is c= (x) 3 ,y 3 ,z 3 … …), the cosine similarity between two first text vectors is obtained, namely:
Figure SMS_3
wherein the cosine similarity sim has a value in the range of 0, 1.
S122, performing discretization processing on the first character vector according to the cosine similarity to obtain the font distance of any two characters, wherein the discretization processing comprises: and obtaining a first difference value between the 1 and the cosine similarity, multiplying the first difference value by 100, and then rounding.
In a specific implementation process of step S122, the calculation is performed according to the following formula.
Figure SMS_4
For example, sim is equal to 0.854, then 1-sim=0.146, multiplying 0.146 by 100 gives 14.6, and rounding 14.6 gives 14.
In the above embodiment, 100mod1 is a discretization granularity set by human, and at the discretization granularity of 100mod1, the discretization method is to multiply the two bits after the decimal point of 1-sim by 100, then to make rounding, and to discard the numerical value after the decimal point. In a specific implementation process, the discretization granularity can be changed, for example, the discretization granularity is set to 1000mod1, and at this time, the discretization mode is to round three bits after the decimal point of 1-sim is multiplied by 1000, and the numerical value after the decimal point is discarded. The embodiment of the application does not limit the granularity of discretization, and the rounding process in the discretization mode does not follow the rounding principle, for example, 14.6, and only 14 is reserved.
Thus, in the above embodiment, the grapheme distance between "steam" and "car" is 14.
S123, establishing a font distance matrix according to the font distances of any two characters.
Fig. 5 is a schematic diagram of a glyph distance matrix according to an embodiment of the present application.
As shown in fig. 5, after the font distance between any two characters is obtained, a font distance matrix between any two characters is established. For example, the font distance between "car" and "car" is 14, the font distance between "car" and "east" is 13, the font distance between "car" and "east" is 18, the font distance between "car" and "east" is 17, the font distance between "car" and "east" is 3, the font distance between "car" and "machine" is 18, the font distance between "car" and "east" is 11, the font distance between "car" and "east" is 13, and the font distance between "west" and "east" is 11 … …
Step S13, determining a first query mode based on a word shape distance matrix and a metric Tree (BK-Tree) algorithm, and determining a second query mode based on the word shape distance matrix and a dynamic programming algorithm, wherein the first query mode is used for querying a first target search word corresponding to a search word to be corrected, which is input by a user, the first target search word has the same text length with the search word to be corrected, and the second query mode is used for querying a second target search word corresponding to the search word to be corrected, and the second target search word has the same or different text length with the search word to be corrected.
The BK-Tree algorithm, also called a metric Tree algorithm, is an algorithm which firstly builds BK-Tree according to a dictionary, then searches in child nodes meeting inequality conditions in the Tree by utilizing triangle inequality of an edit (Levenshtein) distance, and further filters out a large number of invalid operations, and has higher searching efficiency.
According to the method and the device, on the basis of an original BK-Tree algorithm, the edit distance between original nodes is changed into the font distance, and a new error correction Tree is established, so that the new error correction Tree can be used for matching the shape near words. The specific implementation manner of the embodiment of the application is as follows.
Fig. 6 is a second flowchart of the error correction method provided in the embodiment of the present application.
As shown in fig. 6, in some embodiments, step S13 includes steps S131-S133.
S131, determining at least one preset word as a node of BK-Tree.
The preset words can be selected from a dictionary, a contact list and an application list. By way of example, the preset words may be "Zhang San", "Liqu", "XX express", "XX music", and the like.
S132, establishing an error correction tree according to the nodes and the node distance between any two nodes, wherein the node distance is determined according to the font distance between characters in any two preset words.
Fig. 7 is a schematic diagram of an error correction tree according to an embodiment of the present application.
As shown in FIG. 7, each node of the error correction tree is a preset word, wherein a first node in the error correction tree is a root node, each node corresponds to at least one child node, and a node distance is reserved between any two nodes.
By way of example, the root node is "locomotive", at least one child node corresponding to the root node is "car", "guangdong", … …, where the node distance between "locomotive" and "car" is 18, the node distance between "locomotive" and "guangdong" is 22, and the node distance between "locomotive" and another further child node is 30.
In some embodiments, the node distance determination includes steps S1321-S1323.
Step S1321, at least one set of first words in any two nodes is acquired, where each set of first words is a word having a position corresponding relationship between any two nodes.
For example, take the root node as "locomotive" and the child node as "Guangdong", the root node has the first word of "locomotive" and the second word of "locomotive". In the child node, "car" is the first word of "car", and "car" is the second word of "car".
Thus, there are two sets of first words between "locomotive" and "car". The first text is text with the same position relation between any two nodes. Wherein the first word of the first group is the first word "machine" of "locomotive" and the first word "car" of "car". The second group of first words is the second word "car" of "locomotive" and the first word "car" of "car".
In step S1322, each group of first characters is input into a first font distance function, so as to obtain a first distance between each group of first characters, where the first font distance function is determined according to a font distance matrix, and the first distance is a font distance between each group of first characters.
Wherein the first font distance function is
Figure SMS_5
m is the word length of the preset word, i is the position of the word in the preset word, s1 is the i-th word in the first preset word, and s2 is the i-th word in the second preset word.
For example, when i is equal to 1, in "locomotive" corresponding to "machine" word, in "car" corresponding to "car" word, the first distance is the glyph distance in the glyph distance matrix of "machine" word and "car" word in the above embodiment. By way of example, the first distance between the "machine" word and the "steam" word is 18.
When i is equal to 2, in the "locomotive", corresponding to the "car" word, in the "car", corresponding to the "car" word, the first distance is the font distance in the font distance matrix of the "car" word and the "car" word in the above embodiment. By way of example, the first distance between the "car" word and the "car" word is 0.
In this way, a first distance between each set of first words may be obtained.
Step S1323, determining a node distance according to the sum of the first distances and a second distance, wherein the second distance is determined according to the product of the absolute value of a second difference value and a first preset distance, the second difference value is the difference value of the lengths of characters in any two nodes, and the first preset distance is used for assigning values to characters which do not have a position corresponding relation between any two nodes.
By way of example, since the first distance between the "machine" word and the "steam" word is 18 and the font distance of the "car" word and the "car" word is 0 in the above-described embodiment, the sum of the first distances is 18+0 equal to 18.
However, the preset words are not words of two words, and there are words of more words.
For example, the first preset word is "agricultural bank", and the second preset word is "car", so that only the first two words in the first preset word and the second preset word have a position correspondence, that is, a first distance between "farm" and "car", and a first distance between "industry" and "car", and no words corresponding to two words of "bank" in the second preset word.
For this purpose, the "silver" word and the "line" word are assigned so that these words having no positional correspondence also have a font distance, and a comparison of the font distances can be made.
For this purpose, a first preset distance defaultDis is set to assign a value to each text that does not have a correspondence.
Illustratively, defaultDis is equal to 20.
It should be noted that, the defaultDis is generally customized to a larger value, so that the distinguishing capability of the preset word can be improved.
Step S1322 and step S1323 are specifically calculated by the following formulas.
Figure SMS_6
dis is the node distance, m is the word length of the first preset word, n is the word length of the second preset word, i is the position of the word in the preset word, s1 is the i-th word in the first preset word, and s2 is the i-th word in the second preset word.
That is, the second difference is m-n, and in the above embodiment, the second difference is a difference between the text length 4 of "agricultural bank" and the text length 2 of "car", and the second difference is equal to 4-2=2.
The second distance is
Figure SMS_7
By way of example, a first distance between "farm" and "car" is 17, a first distance between "business" and "car" is 18, and then a node distance of 17+18+40=75 can be obtained from the sum of the first distances and the second distance, so a node distance between "farm bank" and "car" is 75. Any two preset words with larger node distances are not approximate words, so that an agricultural bank and an automobile are not matched.
Based on the above embodiments, an error correction tree having a plurality of nodes can be established.
In some embodiments, step S132 includes steps S1324-S1328.
Step S1324, inputting a new word into the error correction tree.
Step S1325, obtaining a first node distance between the newly added word and the root node and a second node distance between the root node and the corresponding child node, where the first node distance is a node distance between the newly added word and a current matching node in the error correction tree, and the second node distance is a node distance between the current matching node and the corresponding child node.
By way of example, the newly added word "fly" is input into the error correction tree as shown in FIG. 7, and a first node distance between "fly" and the root node "locomotive" is obtained. The specific determination manner of the first node distance may refer to the determination manner of the node distance between the "locomotive" and the "automobile" in the above embodiment.
For example, the first node distance between "fly" and the root node "locomotive" is 18, and the second node distance between the root node "locomotive" and the corresponding child node "car" is 18, at which point step S1326 is performed. In step S1326, if the first node distance is equal to the second node distance, the child node corresponding to the current matching node is determined as the next matching node, and the first node distance between the newly added word and the next matching node and the second node distance between the next matching node and the corresponding child node are obtained.
Since the first node distance between the "flight" and the root node "locomotive" is 18 and the second node distance between the root node "locomotive" and the corresponding child node "car" is 18, the "car" is determined to be the sub-tree root node, and thus, the "car" is matched with the newly added word "flight" as the next matching node.
As described herein, each node forms a sub-tree with its corresponding child node, and the node from which the child node extends downward.
Step S1327, repeating the previous step until the first node distance is not equal to the second node distance, and determining the newly added word as the newly added child node.
Step S1328, inserting new child nodes into the error correction tree, and establishing an updated error correction tree.
According to the manner in the above embodiment, the error correction tree may be updated so that there are more preset words in the error correction tree for matching.
S133, determining a first query mode according to the error correction tree.
In this way, the search word to be corrected is matched with the nodes in the correction tree, so that whether the search word to be corrected is matched with the preset word in the correction tree or not can be quickly inquired as a first inquiry mode.
Step S14, searching the search word to be corrected based on the first search mode and/or the second search mode, and obtaining a search result corresponding to the search word to be corrected, wherein the search result comprises a first target search word or a second target search word.
Fig. 8 is a third flowchart of an error correction method provided in an embodiment of the present application.
Fig. 9 is a schematic diagram of an error correction tree search process according to an embodiment of the present application.
As shown in fig. 8 and 9, in some embodiments, step S14 includes steps S141-S143.
Step S141, obtaining a third node distance between the search word to be corrected and each node.
And inputting search words to be corrected in the established error correction tree.
By way of example, the search term to be corrected is "automobile east", and a third node distance between the search term to be corrected "automobile east" and the root node "locomotive" is obtained. The specific determination manner of the third node distance may refer to the determination manner of the node distance between the "locomotive" and the "automobile" in the above embodiment. Thus, a third node distance of 21 between "automobile east" and "locomotive" may be obtained.
Step S142, pruning is carried out on redundant nodes in the error correction tree according to a pruning mechanism, a pruning result corresponding to the search word to be corrected is obtained, the redundant nodes are nodes smaller than a first threshold value and larger than a second threshold value, the first threshold value is a third difference value between a third node distance and a second preset distance, and the second threshold value is a sum of the third node distance and the second preset distance.
Because there are multiple nodes in the error correction tree, if each node is matched, the matching time is too long, therefore, a certain threshold value is set for the node distance, the obviously unmatched nodes and subtrees thereof can be pruned, and only the nodes with higher matching degree are reserved.
In a specific implementation, nodes within the following range are acquired.
[max(0,dis-radius),dis+radius]。
Wherein the first threshold is max (0, dis-radius), the second threshold is dis+radius, dis is the third node distance, radius is the second preset distance, and radius is equal to 5, for example. Here, it should be noted that, in the embodiment of the present application, only the value of the second preset distance is illustrated by way of example, and the second preset distance may be set according to the actual situation.
By way of example, the third node distance between "automobile east" and "locomotive" is 21, the second node distance between "locomotive" and "car" is 18, the second node distance between "locomotive" and "Guangdong" is 22, and the second node distance between "locomotive" and "XX" is 30.
Since the node distance 30 between "locomotive" and "XX" exceeds the second threshold 21+5=26. Therefore, "XX" is used as an extra node for pruning, and a pruning result is obtained. Thus, the "XX" node and its subtrees are pruned.
The pruning result may or may not include the target node, for example, all the nodes are pruned.
Illustratively, the target node is "car", "guangdong".
In step S143, if the pruning result includes a target node, determining a preset word corresponding to the target node as a first target search word, where the target node is a node having a shortest distance from the node of the search word to be corrected and being smaller than a second preset distance.
By way of example, the node distance between "automobile" and "automobile" is 3, and the node distance between "automobile" and "Guangdong" is 17. An "automobile" is a node with the shortest node distance.
Here, the node with the shortest node distance is not necessarily the target node, and the screening needs to be further performed by the second preset distance, and since the node distance between "automobile" and "automobile" is 3 and less than the second preset distance 5, the "automobile" may be the target node of "automobile".
This is because if the node distance between the shortest node and the search term to be corrected is too large, for example, 17, it indicates that the preset term in the node has a larger font distance from the search term to be corrected, the font difference is too large, and the preset term in the node is not the first target search term.
For example, the "car" does not exist in the preset words, after the matching process is performed, it is determined that the node distance between the search word to be corrected and the shortest node, for example, the "guangdong", is 17, and the 17 is greater than the second preset distance 5, so that the "guangdong" is not the target node of the "car east".
Fig. 10 is a schematic diagram of a first query pattern query scenario provided in an embodiment of the present application.
As shown in fig. 10, when the user inputs "X national b bank", the search engine can perform a query according to the first query mode shown in the above embodiment, to obtain "X national industrial b bank".
In the first query mode, in the process of matching the search word to be corrected with the preset word in the error correction tree, only the first target search word with the node distance smaller than the second preset distance can be generally matched, so that the first query mode can only query the first target search word with the same word length as the search word to be corrected, and if no child node with the node distance smaller than the second preset distance exists in the remaining child nodes, the first query mode cannot obtain the search result.
According to the embodiment of the application, two query modes are established based on the font distance matrix, wherein the first query mode can be used for quick query, and the second query mode can be used for supplementary query so as to perform more comprehensive and accurate query. The first query mode queries at a slightly higher rate than the second query mode. Thus, the two query modes may be used in parallel, or the second query mode may be used when the first query mode does not result in a search.
In some embodiments, if the search term to be corrected is queried based on the first query pattern, a first target search term is obtained, and the first target search term is determined to be a search result; if the search word is queried based on the first query mode, the first target search word is not obtained, and then the search word is queried based on the second query mode, the second target search word is obtained, and the second target search word is determined to be a search result.
Fig. 11 is a fourth flowchart of an error correction method provided in an embodiment of the present application.
As shown in fig. 11, in some embodiments, step S13 includes steps S231-S233.
In step S231, a first cyclic variable i and a second cyclic variable j are established, where i represents a position of a word in a preset word, j represents a position of the word in a search word to be corrected, i is a positive integer greater than or equal to 0, and j is a positive integer greater than or equal to 0.
Step S232, establishing a first distance function dis 1 (i, j), wherein if the minimum value in i or j is equal to 0, dis 1 (i, j) is equal to the product of the maximum value of i or j and a third preset distance, if the minimum value of i or j is not equal to 0, dis 1 (i, j) equals dis 1 (i-1, j) and a third preset distance, dis 1 (i, j-1) and a third predetermined distance, dis 1 The minimum value between (i-1, j-1) and the sum of a second font distance function dis-Dict (s 1, i, s2, j), wherein the second font distance function is determined according to a font distance matrix, dis-Dict (s 1, i, s2, j) represents the font distance between the ith character in a preset word and the jth character in a search word to be corrected in the font distance matrix, and the third preset distance is used for initializing and assigning the minimum value in i or j to be equal to 0;
step S233, determining a second query mode according to the first distance function.
The dynamic programming (Dynamic Programming, DP) algorithm is a branch of operations research and is the process of solving the optimization of the decision process.
In some embodiments, the program implementation manner of the dynamic programming algorithm of the above step S231-step S233 is as follows:
Figure SMS_8
fig. 12 is a visual schematic of a dynamic programming algorithm shown in an embodiment of the present application.
As shown in fig. 12, the implementation of the above-described program is described by this matching matrix.
In some input scenarios, a user may enter shorthand words for a target search term. For example, a user may wish to enter a "worker". The "row b" is actually entered, and the correct search result should be "X national industrial bank". That is, the second query pattern requires obtaining a second target search term that is different from the word length of the search term to be corrected.
By way of example, taking the preset word as "X national commercial bank" and the search word to be corrected as "row b", the matching process of the preset word and the search word to be corrected is described.
Firstly, the dynamic programming algorithm needs to perform initialization assignment, which is specifically realized through the following procedures.
Figure SMS_9
Wherein defaultDis is a third preset distance. Illustratively, defaultDis is equal to 20.defaultDis is typically custom defined as a large distance to represent the glyph distance when the search term to be corrected does not match the preset term.
Based on the method, initializing assignment is carried out on the preset words and the search words to be corrected.
In some embodiments, in the matching matrix, when i=0, j=0, "/" is assigned a value of 0;
i=1, j=0, the first word "X" corresponding to the preset word, and "/" where the value is 20;
i=2, j=0, the second word "country" and "/" corresponding to the preset word, where the value is 40;
i=3, j=0, the third word "work" and "/" corresponding to the preset word, where the value is 60;
i= 4,j =0, the fourth word "quotient" and "/" corresponding to the preset word, where the value is 80;
When i= 5,j =0, the fifth word "silver" and "/" corresponding to the preset word, where 100 is assigned;
i= 6,j =0, the sixth word "row" and "/" corresponding to the preset word, where the value is 120;
i=0 and j=1, corresponding to "/" and the first word "b" of the search word to be corrected, where the value is 20;
i=0, j=2, which corresponds to "/" and the first word "row" of the search word to be error corrected, where a value of 40 is assigned.
When assigning values to other positions in the matching matrix, the dynamic programming algorithm specifically executes the following procedure.
Figure SMS_10
Illustratively, the assignments for "X" and "b" are performed when i=1, j=1, specifically: first obtain dis 1 Values of (i-1, j) +defaultdis, for example: dis (dis) 1 (0, 1) +20=20+20=40. Second, obtain dis 1 Values of (i, j-1) +defaultdis, for example: dis (dis) 1 (1, 0) +20=20+20=40. Finally, obtain
Figure SMS_11
For example: dis (dis) 1 Adding "X" and "b" to the glyph distance 20 in the glyph distance matrix to (0, 0) yields 0+20=20.
That is, when a value of a certain position is obtained, the final value of the position is actually determined according to the values of the upper, left and upper left directions of the position. When i=1 and j=1, the corresponding position in the matching matrix has the upper value of 40. The left-hand value is 40, the upper left-hand value is 20, and the minimum value in the three directions is 20, so dis 1 The value of (1, 1) was 20. The procedure is sequentially cycled to obtain a matching matrix as shown in fig. 12.
After the matching matrix is obtained, the dynamic programming algorithm performs matching based on two words closest to the matching matrix. In the above-mentioned matching matrix, the assignment of "b" and "i" is 45, and there is a shortest distance 5 in the whole matching matrix between 40 in the upper left corner thereof, so that the matching starts from 40-45, and the final minimum matching distance 85 is obtained according to the path a shown in fig. 12. The minimum matching distance represents the total font distance corresponding to the row B and the national commercial bank X.
In some embodiments, the dynamic programming algorithm further includes the following procedure.
Distance=dis(m,n)-|m-n|×(defaultDis-gapDis)。
m represents the word length of the preset word, and n represents the word length of the search word to be corrected.
Because the preset word and the word length of the search word to be corrected are different, the obtained minimum matching distance 85 in the matching matrix actually includes the defaultDis of the four words of "X", "country", "quotient" and "silver", which are not matched, so that the total font distance corresponding to "b" and "X country industrial and commercial bank" is too large, and the "b" and "X country industrial and commercial bank" cannot be successfully matched.
Therefore, after the defaultDis of the four words "X", "country", "quotient" and "silver" are deleted, the gapdi is used to perform custom assignment on the redundant words "X", "country", "quotient" and "silver" and the like.
Illustratively, gapdi is equal to 5.
In a specific implementation, if only two words of "b" and "i" are matched, the font distance between "b" and "i" should be 5, however, since the actual matching result is "b" and "X country commercial bank", the resulting distance is increased by "X", "country", "quotient" and "silver" default dis, and the total font distance is 5+20×4=85. Thus, the matching of the second line and the national commercial bank is unsuccessful due to the excessive value of 85. Therefore, defaultDis of redundant characters such as "X", "country", "quotient" and "silver" is deleted and replaced by gapdi with smaller value. Thus, the font distance for each word of "X", "country", "quotient" and "silver" is reduced from 20 to 5, and the total font distance is reduced from 85 to 25. Distance=25, therefore.
Thus, through the assignment of gapdi to the unmatched 'X', 'country', 'quotient' and 'silver', the redundant words can have smaller font distances, and even if the word lengths of the search word to be corrected and the preset word are different, the search word to be corrected and the preset word have smaller total font distances, which indicates that the search word to be corrected and the preset word have strong correlation.
It should be noted that, although the program algorithm has been written in the search engine or the reference program, the values of m and n are actually obtained after the user inputs the search word to be corrected. That is, the specific calculation process of Distance is implemented in step S14.
In some embodiments, step S14 includes steps S241-S244.
Step S241, determining a first font distance dis when i equals m and j equals n according to a first distance function 1 (m, n), wherein m is the word length of the preset word, and n is the word length of the search word to be corrected.
Step S242, dis 1 The difference value between the (m, n) and the redundant distance is determined as a first target font distance, wherein the redundant distance is the product of a fourth difference value and a fifth difference value, the fourth difference value is the absolute value of the difference value between the m and the n, the fifth difference value is the difference value between a third preset distance and a fourth preset distance, and the fourth preset distance is used for carrying out custom assignment on the search word to be corrected or the unmatched characters in the preset word.
Step S243, determining a second target search word, which is a preset word having the shortest first target font distance from the search word to be corrected.
Wherein defaultDis is a third preset distance and gapdi is a fourth preset distance.
After a plurality of first target font distances are acquired according to the above embodiment, a preset word having the shortest first target font distance with the search word to be corrected is determined as the second target search word.
For example, if Distance 1=25, distance 2=30, and Distance 3=35, the second target search word corresponding to Distance 1=25, that is, "X national commercial bank" is determined as the second target search word corresponding to "b".
Fig. 13 is a schematic diagram of a second query pattern query scenario provided in an embodiment of the present application.
As shown in fig. 13, in the second query mode, the situation that the word length of the search word to be corrected is greater than the preset word and the word length of the search word to be corrected is less than the preset word is considered, so that the correct search result can be obtained under multiple conditions of multiple words, word staggering, word missing and the like.
In some embodiments, step S13 includes steps S331-S333.
Step S331, a first circulation variable i and a second circulation variable j are established, wherein i represents the position of a word in a preset word, j represents the position of the word in a search word to be corrected, i is a positive integer greater than or equal to 0, and j is a positive integer greater than or equal to 0.
Step S332, establishing a second distance function dis 2 (i, j), wherein if the minimum value in i or j is equal to 0, dis 2 (i, j) is equal to the product of the maximum value of i or j and the fifth preset distance, if the minimum value of i or j is not equal to 0, dis 2 (i, j) equals dis 2 Sum of (i-1, j) and fifth preset distance dis 2 The minimum value between (i-1, j-1) and the sum of the font distance functions dis-Dict (s 1, i, s2, j) which are determined according to the font distance matrix, dis-Dict (s 1, i, s2, j) representing the font distance between the ith word in the preset word and the jth word in the search word to be corrected in the font distance matrix, the fifth preset distance being used for the minimum value in i or j being equal to 0Initializing assignment;
step S333, determining a second query pattern according to a second distance function.
In some embodiments, step S14 includes steps S341-S344.
Step S341, determining a second glyph distance dis when i equals m and j equals n according to a second distance function 2 (m, n), wherein m is the word length of the preset word, and n is the word length of the search word to be corrected.
Step S342, dis 2 The difference value between the (m, n) and the redundant distance is determined as a second target font distance, wherein the redundant distance is the product of a fourth difference value and a fifth difference value, the fourth difference value is the absolute value of the difference value between the m and the n, the fifth difference value is the difference value between a fifth preset distance and a sixth preset distance, and the sixth preset distance is used for carrying out custom assignment on the search word to be corrected or the unmatched characters in the preset word;
In step S343, a second target search word is determined, the second target search word being a preset word having the shortest second target glyph distance from the search word to be corrected.
This embodiment is a matching procedure performed without regard to the case where the search word to be corrected contains redundant text.
In some embodiments, the program implementation of the dynamic programming algorithm of step S341-step S344 is as follows:
Figure SMS_12
in some embodiments, the dynamic programming algorithm further includes the following procedure.
Distance=dis(m,n)-|m-n|×(defaultDis-gapDis)。
The specific implementation method in this embodiment may refer to the above embodiment. This embodiment is based on the above embodiment without consideration
Figure SMS_13
In other words, in this embodiment, when a value of a certain position is acquired, it is actuallyThe final value of the position is obtained according to the values of the upper direction and the left direction of the position. In this embodiment, only the situation that the search word to be corrected does not have multiple words is considered, and the program algorithm is simpler than that in steps S231-S233, and the specific implementation manner of this embodiment is not repeated in this embodiment.
In some embodiments, if dis 1 (i, j) or dis 2 (i, j) in the traversal of i equal to 0 to i equal to m, j equal to 0 to j equal to n,
Figure SMS_14
Or->
Figure SMS_15
And if the current preset word is larger than the seventh preset distance, determining that the current preset word does not comprise the second target search word, and traversing the next preset word until a search result corresponding to the search word to be corrected is obtained.
Wherein, the liquid crystal display device comprises a liquid crystal display device,
Figure SMS_16
for the minimum total font distance corresponding to the current jth word, i.e.)>
Figure SMS_17
The seventh preset Distance is a user-defined threshold.
If the minimum total font distance is larger than the seventh preset distance, the current preset word is larger than the to-be-corrected search word, and the word cannot be matched, so that the next matching can be exited in advance.
For example, in the above embodiment, the Distance is 25, and if the seventh preset Distance is 20, it may be determined that "X national commercial bank" is not the second target search term.
The specific value of the seventh preset distance is not limited in the embodiment of the present application.
In the specific implementation process, "line b" needs to be matched with all preset words, for example, a seventh preset distance is set to 5 first, if the second target search word is not matched in the range, the seventh preset distance is increased to 10, and the seventh preset distance is sequentially increased until a preset condition is reached or the second target search word is matched, and then the matching is stopped. The preset condition may be that the seventh preset distance reaches a larger value or the total matching time is too long, and a specific setting mode of the preset condition may be set according to actual conditions, which is not limited in the application.
FIG. 14 is a diagram of search results provided by an embodiment of the present application.
As shown in fig. 14, the method in the above embodiment searches for "X national clothing line" in the search engine, and the search engine can perform shape-near word error correction to obtain the correct search result of "X national agricultural bank".
According to the error correction method, the first query mode and the second query mode can be configured in the search engine or the application program to correct the search word with the shape near word error input by the user, the shape near word error of multiple characters, wrong characters, missing characters and other various conditions can be corrected, a correct search result is obtained, and the problem that the recall rate of the search result is low when the user searches the search word with the shape near word error is solved.
It will be appreciated that the electronic device, in order to achieve the above-described functions, includes corresponding hardware structures and/or software modules that perform the respective functions. Those of skill in the art will readily appreciate that one of the error correction method steps of the examples described in connection with the embodiments disclosed herein may be implemented as hardware or a combination of hardware and computer software. Whether a function is implemented as hardware or electronic device software driven hardware depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
The application also provides an error correction device, comprising: the training module is used for identifying at least one character of each character in a preset dictionary by utilizing the trained optical character recognition OCR model to obtain a first character vector of each character, the fonts of the at least one character are different, and the first character vector comprises the average value of the feature vectors of the at least one character of the characters; the matrix building module is used for building a font distance matrix according to the first character vectors of any two characters, wherein the font distance matrix comprises font distances between any two characters, and the font distances are obtained according to cosine similarity between the first character vectors of any two characters; the determining module is used for determining a first query mode based on the font distance matrix and the BK-Tree algorithm and determining a second query mode based on the font distance matrix and the dynamic programming algorithm, wherein the first query mode is used for querying a first target search word corresponding to a search word to be corrected, which is input by a user, the first target search word has the same text length as the search word to be corrected, and the second query mode is used for querying a second target search word corresponding to the search word to be corrected, and the second target search word has the same text length as the search word to be corrected or different text lengths from the search word to be corrected; the query module is used for querying the search word to be corrected based on the first query mode and/or the second query mode, and obtaining a search result corresponding to the search word to be corrected, wherein the search result comprises a first target search word or a second target search word.
Embodiments of the present application also provide a chip system including at least one processor and at least one interface circuit. The processors and interface circuits may be interconnected by wires. For example, the interface circuit may be used to receive signals from other devices (e.g., a memory of an electronic apparatus). For another example, the interface circuit may be used to send signals to other devices. The interface circuit may, for example, read instructions stored in the memory and send the instructions to the processor. The instructions, when executed by a processor, may cause an electronic device to perform the various steps of the embodiments described above. Of course, the chip system may also include other discrete devices, which are not specifically limited in this embodiment of the present application.
The embodiment of the application also provides a computer readable storage medium, which comprises computer instructions, when the computer instructions run on the electronic device, the electronic device is caused to execute the functions or steps executed by the mobile phone in the embodiment of the method.
The present application also provides a computer program product, which when run on a computer, causes the computer to perform the functions or steps performed by the mobile phone in the above-mentioned method embodiments.
It will be apparent to those skilled in the art from this description that, for convenience and brevity of description, only the above-described division of the functional modules is illustrated, and in practical application, the above-described functional allocation may be performed by different functional modules according to needs, i.e. the internal structure of the apparatus is divided into different functional modules to perform all or part of the functions described above.
In the several embodiments provided in this application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the modules or units is merely a logical functional division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another apparatus, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and the parts displayed as units may be one physical unit or a plurality of physical units, may be located in one place, or may be distributed in a plurality of different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in each embodiment of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a readable storage medium. Based on such understanding, the technical solution of the embodiments of the present application may be essentially or a part contributing to the prior art or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium, including several instructions for causing a device (may be a single-chip microcomputer, a chip or the like) or a processor (processor) to perform all or part of the steps of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read Only Memory (ROM), a random access memory (random access memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
The foregoing is merely a specific embodiment of the present application, but the scope of the present application is not limited thereto, and any changes or substitutions within the technical scope of the present disclosure should be covered in the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (16)

1. An error correction method, comprising:
identifying at least one character of each character in a preset dictionary by utilizing the trained optical character recognition OCR model to obtain a first character vector of each character, wherein the fonts of the at least one character are different, and the first character vector comprises the average value of the feature vectors of at least one character of the characters;
establishing a font distance matrix according to the first character vectors of any two characters, wherein the font distance matrix comprises font distances between any two characters, and the font distances are obtained according to cosine similarity between the first character vectors of any two characters;
determining a first query mode based on the font distance matrix and a metric Tree BK-Tree algorithm, and determining a second query mode based on the font distance matrix and a dynamic programming algorithm, wherein the first query mode is used for querying a first target search word corresponding to a search word to be corrected, which is input by a user, the first target search word has the same text length as the search word to be corrected, and the second query mode is used for querying a second target search word corresponding to the search word to be corrected, and the second target search word has the same text length as the search word to be corrected or different text length from the search word to be corrected;
And inquiring the search word to be corrected based on the first inquiry mode and/or the second inquiry mode to obtain a search result corresponding to the search word to be corrected, wherein the search result comprises the first target search word or the second target search word.
2. The method for correcting errors according to claim 1, wherein the step of recognizing at least one character of each character in a predetermined dictionary by using the trained OCR model, before obtaining a first character vector of each character, further comprises:
acquiring a training data set, wherein the training data set comprises at least one single-word image data, and the single-word image data is used for recording one character in each image;
and inputting the at least one single-word image data into an initial OCR model for training to obtain the trained OCR model.
3. The method of claim 2, wherein the identifying at least one character of each word using the trained OCR model to obtain a first word vector for each word comprises:
deleting a normalized softmax layer from the trained OCR model, and deleting at least one full-connection layer connected with the softmax layer to obtain a feature extraction model;
And obtaining the first character vector of each character according to the characteristic extraction model, wherein the first character vector is obtained according to the average value of the characteristic vector of at least one character of the characters.
4. The method of claim 1, wherein said creating a glyph distance matrix from said first text vector of any two of said texts comprises:
acquiring cosine similarity between the first character vectors of any two characters;
performing discretization processing on the first text vector according to the cosine similarity to obtain the font distance of any two texts, wherein the discretization processing comprises: obtaining a first difference value between the 1 and the cosine similarity, multiplying the first difference value by 100, and then rounding;
and establishing the font distance matrix according to the font distances of any two characters.
5. The error correction method of claim 1, wherein determining a first query pattern based on the glyph distance matrix and a metric Tree BK-Tree algorithm comprises:
determining at least one preset word as a node of BK-Tree;
establishing an error correction tree according to the nodes and the node distances of any two nodes, wherein the node distances are determined according to the font distances between characters in any two preset words;
And determining the first query mode according to the error correction tree.
6. The method of error correction according to claim 5, wherein the determining the node distance includes:
acquiring at least one group of first characters in any two nodes, wherein each group of first characters is a character with a position corresponding relation between any two nodes;
inputting each group of first characters into a first font distance function to obtain a first distance between each group of first characters, wherein the first font distance function is determined according to the font distance matrix, and the first distance is the font distance between each group of first characters;
the node distance is determined according to the sum of the first distances and a second distance, the second distance is determined according to the product of the absolute value of a second difference value and a first preset distance, the second difference value is the difference value of the lengths of characters in any two nodes, and the first preset distance is used for assigning values to characters which do not have a position corresponding relation between any two nodes.
7. The method of claim 6, wherein each of the nodes corresponds to at least one child node, and a first of the nodes in the error correction tree is a root node; the step of establishing an error correction tree according to the nodes and the node distance between any two nodes comprises the following steps:
Inputting a newly added word into the error correction tree;
acquiring a first node distance between the newly added word and the root node and a second node distance between the root node and the corresponding child node, wherein the first node distance is a node distance between the newly added word and a current matching node in the error correction tree, and the second node distance is a node distance between the current matching node and the corresponding child node;
if the first node distance is equal to the second node distance, determining the child node corresponding to the current matching node as a next matching node, and acquiring the first node distance between the newly added word and the next matching node and the second node distance between the next matching node and the corresponding child node;
repeatedly executing the previous step until the distance between the first node and the second node is not equal to the distance between the first node and the second node, and determining the newly added word as a new child node; and inserting the new child node into the error correction tree, and establishing the updated error correction tree.
8. The method according to claim 7, wherein the querying the search term to be corrected based on the first query pattern and/or the second query pattern to obtain the search result corresponding to the search term to be corrected includes:
Acquiring a third node distance between the search word to be corrected and each node;
pruning redundant nodes in the error correction tree according to a pruning mechanism to obtain a pruning result corresponding to the search word to be corrected, wherein the redundant nodes are nodes smaller than a first threshold and larger than a second threshold, the first threshold is a third difference value between the third node distance and a second preset distance, and the second threshold is a sum of the third node distance and the second preset distance;
and if the pruning result comprises a target node, determining the preset word corresponding to the target node as the first target search word, wherein the target node is the node with the shortest node distance of the search word to be corrected and smaller than the second preset distance.
9. The method according to claim 1, wherein the querying the search term to be corrected based on the first query pattern and/or the second query pattern to obtain the search result corresponding to the search term to be corrected includes:
if the search word to be corrected is queried based on the first query mode, obtaining the first target search word, and determining the first target search word as the search result;
And if the search word is queried based on the first query mode, the first target search word is not obtained, and then the search word is queried based on the second query mode, the second target search word is obtained, and the second target search word is determined to be the search result.
10. The method of error correction according to claim 1, wherein said determining a second query pattern based on the glyph distance matrix and dynamic programming algorithm comprises:
establishing a first circulation variable i and a second circulation variable j, wherein i represents the position of characters in a preset word, j represents the position of characters in the search word to be corrected, i is a positive integer greater than or equal to 0, and j is a positive integer greater than or equal to 0;
establishing a first distance function dis 1 (i, j), wherein if the minimum value in i or j is equal to 0, dis 1 (i, j) is equal to the product of the maximum value of i or j and a third preset distance, if the minimum value of i or j is not equal to 0, dis 1 (i, j) equals dis 1 (i-1, j) sum of the third preset distance dis 1 (i, j-1) sum of said third predetermined distance and dis 1 A minimum value between (i-1, j-1) and a second font distance function dis-dct (s 1, i, s2, j), wherein the second font distance function is determined according to the font distance matrix, dis-dct (s 1, i, s2, j) represents the font distance between the ith character in the preset word and the jth character in the search word to be corrected in the font distance matrix, and the third preset distance is used for initializing and assigning that the minimum value in i or j is equal to 0;
And determining the second query mode according to the first distance function.
11. The method according to claim 10, wherein the querying the search term to be corrected based on the first query pattern and/or the second query pattern to obtain the search result corresponding to the search term to be corrected includes:
determining a first font distance dis when i equals m and j equals n according to the first distance function 1 (m, n), wherein m is the word length of a preset word, and n is the word length of a search word to be corrected;
will dis 1 The difference between (m, n) and the redundant distance is determined as a first orderThe mark-shaped distance is the product of a fourth difference value and a fifth difference value, wherein the fourth difference value is the absolute value of the difference value of m and n, the fifth difference value is the difference value of the third preset distance and a fourth preset distance, and the fourth preset distance is used for carrying out custom assignment on the search word to be corrected or the unmatched words in the preset word;
and determining the second target search word, wherein the second target search word is a preset word with the shortest first target font distance with the search word to be corrected.
12. The method of error correction according to claim 1, wherein said determining a second query pattern based on the glyph distance matrix and dynamic programming algorithm comprises:
establishing a first circulation variable i and a second circulation variable j, wherein i represents the position of characters in a preset word, j represents the position of characters in the search word to be corrected, i is a positive integer greater than or equal to 0, and j is a positive integer greater than or equal to 0;
establishing a second distance function dis 2 (i, j), wherein if the minimum value in i or j is equal to 0, dis 2 (i, j) is equal to the product of the maximum value of i or j and the fifth preset distance, if the minimum value of i or j is not equal to 0, dis 2 (i, j) equals dis 2 (i-1, j) sum of the fifth preset distance and dis 2 A minimum value between (i-1, j-1) and a second font distance function dis-dct (s 1, i, s2, j), wherein the second font distance function is determined according to the font distance matrix, dis-dct (s 1, i, s2, j) represents the font distance between the ith character in the preset word and the jth character in the search word to be corrected in the font distance matrix, and the fifth preset distance is used for initializing and assigning that the minimum value in i or j is equal to 0;
And determining the second query mode according to the second distance function.
13. The method according to claim 12, wherein the querying the search term to be corrected based on the first query pattern and/or the second query pattern, to obtain the search result corresponding to the search term to be corrected, includes:
determining a second glyph distance dis when i equals m and j equals n according to the second distance function 2 (m, n), wherein m is the word length of a preset word, and n is the word length of a search word to be corrected;
will dis 2 The difference value between the (m, n) and the redundant distance is determined as a second target font distance, wherein the redundant distance is the product of a fourth difference value and a fifth difference value, the fourth difference value is the absolute value of the difference value between the m and the n, the fifth difference value is the difference value between the fifth preset distance and a sixth preset distance, and the sixth preset distance is used for carrying out custom assignment on the search word to be corrected or the unmatched characters in the preset word;
and determining the second target search word, wherein the second target search word is a preset word with the shortest second target font distance with the search word to be corrected.
14. The method for correcting errors according to claim 11 or 13, wherein the querying the search term to be corrected based on the first query pattern and/or the second query pattern to obtain the search result corresponding to the search term to be corrected includes:
if dis 1 (i, j) or dis 2 (i, j) in the traversal of i equal to 0 to i equal to m, j equal to 0 to j equal to n,
Figure QLYQS_1
or->
Figure QLYQS_2
Greater than a seventh preset distance, determining that the current preset term does not include the second target search term;
and traversing the next preset word until the search result corresponding to the search word to be corrected is obtained.
15. An error correction device, comprising:
the training module is used for identifying at least one character of each character in a preset dictionary by utilizing the trained optical character recognition OCR model to obtain a first character vector of each character, the fonts of the at least one character are different, and the first character vector comprises the average value of the feature vectors of at least one character of the characters;
the matrix building module is used for building a font distance matrix according to the first text vectors of any two texts, the font distance matrix comprises font distances between any two texts, and the font distances are obtained according to cosine similarity between the first text vectors of any two texts;
The determining module is used for determining a first query mode based on the font distance matrix and a metric Tree BK-Tree algorithm and determining a second query mode based on the font distance matrix and a dynamic programming algorithm, wherein the first query mode is used for querying a first target search word corresponding to a search word to be corrected, which is input by a user, the first target search word has the same text length as the search word to be corrected, and the second query mode is used for querying a second target search word corresponding to the search word to be corrected, and the second target search word has the same text length as the search word to be corrected or different text length from the search word to be corrected;
the query module is used for querying the search word to be corrected based on the first query mode and/or the second query mode to obtain a search result corresponding to the search word to be corrected, and the search result comprises the first target search word or the second target search word.
16. An electronic device, the electronic device comprising:
a processor and a memory for storing at least one instruction which, when loaded and executed by the processor, is to implement the method of any one of claims 1-14.
CN202310518790.1A 2023-05-10 2023-05-10 Error correction method and device and electronic equipment Active CN116227471B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310518790.1A CN116227471B (en) 2023-05-10 2023-05-10 Error correction method and device and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310518790.1A CN116227471B (en) 2023-05-10 2023-05-10 Error correction method and device and electronic equipment

Publications (2)

Publication Number Publication Date
CN116227471A true CN116227471A (en) 2023-06-06
CN116227471B CN116227471B (en) 2023-09-01

Family

ID=86570057

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310518790.1A Active CN116227471B (en) 2023-05-10 2023-05-10 Error correction method and device and electronic equipment

Country Status (1)

Country Link
CN (1) CN116227471B (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103927329A (en) * 2014-03-19 2014-07-16 北京奇虎科技有限公司 Instant search method and system
CN109545283A (en) * 2018-11-23 2019-03-29 西安交通大学 A kind of phylogenetic tree construction method based on Sequential Pattern Mining Algorithm
CN111814455A (en) * 2020-06-29 2020-10-23 平安国际智慧城市科技股份有限公司 Search term error correction pair construction method, terminal and storage medium
CN112613522A (en) * 2021-01-04 2021-04-06 重庆邮电大学 Method for correcting recognition result of medicine taking order based on fusion font information

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103927329A (en) * 2014-03-19 2014-07-16 北京奇虎科技有限公司 Instant search method and system
CN109545283A (en) * 2018-11-23 2019-03-29 西安交通大学 A kind of phylogenetic tree construction method based on Sequential Pattern Mining Algorithm
CN111814455A (en) * 2020-06-29 2020-10-23 平安国际智慧城市科技股份有限公司 Search term error correction pair construction method, terminal and storage medium
CN112613522A (en) * 2021-01-04 2021-04-06 重庆邮电大学 Method for correcting recognition result of medicine taking order based on fusion font information

Also Published As

Publication number Publication date
CN116227471B (en) 2023-09-01

Similar Documents

Publication Publication Date Title
CN107871117B (en) Apparatus and method for detecting object
CN111415388B (en) Visual positioning method and terminal
CN110059652B (en) Face image processing method, device and storage medium
CN108259746B (en) Image color detection method and mobile terminal
CN111209377B (en) Text processing method, device, equipment and medium based on deep learning
CN114119758A (en) Method for acquiring vehicle pose, electronic device and computer-readable storage medium
CN108038431A (en) Image processing method, device, computer equipment and computer-readable recording medium
CN110866254A (en) Vulnerability detection method and electronic equipment
CN107704514A (en) A kind of photo management method, device and computer-readable recording medium
CN113971271A (en) Fingerprint unlocking method and device, terminal and storage medium
CN115115679A (en) Image registration method and related equipment
US20210406524A1 (en) Method and device for identifying face, and computer-readable storage medium
CN116227471B (en) Error correction method and device and electronic equipment
CN113468929A (en) Motion state identification method and device, electronic equipment and storage medium
CN115437601B (en) Image ordering method, electronic device, program product and medium
CN111798019B (en) Intention prediction method, intention prediction device, storage medium and electronic equipment
CN111768765A (en) Language model generation method and electronic equipment
CN110163192B (en) Character recognition method, device and readable medium
CN113313966A (en) Pose determination method and related equipment
CN112560612B (en) System, method, computer device and storage medium for determining business algorithm
CN113192072B (en) Image segmentation method, device, equipment and storage medium
CN112232890B (en) Data processing method, device, equipment and storage medium
CN114140839B (en) Image transmission method, device, equipment and storage medium for face recognition
CN113591514B (en) Fingerprint living body detection method, fingerprint living body detection equipment and storage medium
CN112183208A (en) Identity recognition method and equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant