WO2021208727A1

WO2021208727A1 - Text error detection method and apparatus based on artificial intelligence, and computer device

Info

Publication number: WO2021208727A1
Application number: PCT/CN2021/083936
Authority: WO
Inventors: 回艳菲; 王健宗; 吴天博; 程宁
Original assignee: 平安科技（深圳）有限公司
Priority date: 2020-11-24
Filing date: 2021-03-30
Publication date: 2021-10-21
Also published as: CN112434131A; CN112434131B

Abstract

A text error detection method and apparatus based on artificial intelligence, and a computer device, which can be applied to an application scenario of text error detection in a smart city. The method comprises: according to model configuration information and an initialized detection model, performing construction to obtain multiple error detection models (S110); respectively training the multiple error detection models (S120); inputting text to be subjected to detection into the multiple trained error detection models, so as to acquire multiple pieces of model detection information (S130); screening the model detection information to obtain screened detection data that meets a condition (S140); and performing integration processing on the screened detection data to obtain a text checking result (S150). The method is based on intelligent decision-making technology, belongs to the field of artificial intelligence, and relates to blockchain technology. A text checking result can be uploaded into a blockchain. Multiple error detection models are constructed to respectively acquire multiple pieces of model detection information, and screening and integration processing are performed on the model detection information to obtain a text checking result, so that the efficiency and accuracy of performing error detection on Chinese text can be greatly improved.

Description

Artificial intelligence-based text error detection method, device and computer equipment

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on November 24, 2020, the application number is 202011329034.7, and the invention title is "artificial intelligence-based text error detection method, device, computer equipment", and the entire content of it is approved The reference is incorporated in this application.

Technical field

This application relates to the field of artificial intelligence technology, and in particular to an artificial intelligence-based text error detection method, device, and computer equipment.

Background technique

Chinese has always been regarded as one of the most difficult languages in the world to learn. In its long history of use, it has accumulated many differences from other languages. For example, unlike English, Chinese has neither singular or plural changes nor tense changes of verbs. The expression is more flexible, the grammatical structure is loose, there are more short sentences and fewer clauses. There are usually many ways to express the same meaning in Chinese. In the process of daily use, the company will recognize the voice information through Automatic Speech Recognition (ASR) to obtain the corresponding text information. The text information may contain text errors, and the company may also receive the user through the keyboard or touch screen and other devices. The input of near-phonetic characters is wrong (the pronunciation is the same but the input is wrong), and the text error will affect the reading comprehension, and even have an important impact on the company in the process of handling the business for the customer, causing huge losses to the company.

However, the inventor found that traditional technical methods use fixed template matching to detect errors in Chinese text. However, this technical method must build a huge matching template library during the implementation process, and match text errors through a large number of matching modules. Detection results in low detection efficiency; and because of the flexible expression of Chinese text, there will be diametrically opposite results when judging the same text in different contexts. Therefore, this technical method is difficult to obtain accurate error detection results. Therefore, the Chinese text error detection method in the prior art method has the problems of low detection efficiency and detection accuracy.

Summary of the invention

The embodiments of the present application provide an artificial intelligence-based text error detection method, device, computer equipment, and storage medium, aiming to solve the problem of low detection efficiency and detection accuracy of Chinese text errors in the prior art methods.

In the first aspect, an embodiment of the present application provides an artificial intelligence-based text error detection method, which includes:

Receiving model configuration information input by a user, and configuring parameter values in the initialized detection model according to the model configuration information to obtain multiple error detection models;

Training a plurality of said error detection models respectively according to a preset conversion dictionary and a pre-stored training corpus database to obtain a plurality of error detection models after training;

If the text to be detected input by the user is received, input the text to be detected into multiple error detection models to obtain corresponding multiple model detection information;

Screening and obtaining screening detection data satisfying preset screening conditions from the plurality of model detection information;

Performing integrated processing on the screening detection data to obtain a text detection result that matches the text to be detected.

In the second aspect, an embodiment of the present application provides an artificial intelligence-based text error detection device, which includes:

The detection model configuration unit is configured to receive model configuration information input by the user, and configure parameter values in the initialized detection model according to the model configuration information to obtain multiple error detection models;

The detection model training unit is configured to train multiple error detection models according to a preset conversion dictionary and a pre-stored training corpus database to obtain multiple error detection models after training;

The model checking information acquiring unit is configured to, if the to-be-detected text input by the user is received, input the to-be-detected text into multiple error detection models to obtain corresponding multiple model detection information;

The model checking information screening unit is used to screen a plurality of the model checking information to obtain screening and checking data that meets preset screening conditions;

The integrated processing unit is configured to perform integrated processing on the screening detection data to obtain a text detection result that matches the text to be detected.

In a third aspect, an embodiment of the present application provides a computer device, which includes a memory, a processor, and a computer program stored on the memory and running on the processor, and the processor executes the computer The program implements the artificial intelligence-based text error detection method described in the first aspect above.

In a fourth aspect, an embodiment of the present application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor executes the above-mentioned first On the one hand, the artificial intelligence-based text error detection method.

The embodiments of the application provide an artificial intelligence-based text error detection method, device, computer equipment, and storage medium. Build multiple error detection models according to the model configuration information and the initialized detection model; train multiple error detection models separately, input the text to be detected into the trained multiple error detection models to obtain multiple model detection information, and then check from the model The information is screened to obtain screening test data that meets the preset screening conditions; the screening test data is integrated to obtain the text detection result. Through the above method, multiple error detection models are constructed to obtain multiple model detection information corresponding to the text to be detected, and the model detection information is filtered and integrated to obtain the text detection result. Compared with the fixed template matching method, it can be greatly improved. Improve the efficiency and accuracy of error detection for Chinese text.

Description of the drawings

In order to explain the technical solutions of the embodiments of the present application more clearly, the following will briefly introduce the drawings used in the description of the embodiments. Obviously, the drawings in the following description are some embodiments of the present application. Ordinary technicians can obtain other drawings based on these drawings without creative work.

FIG. 1 is a schematic flowchart of an artificial intelligence-based text error detection method provided by an embodiment of the application;

FIG. 2 is a schematic diagram of a sub-flow of an artificial intelligence-based text error detection method provided by an embodiment of the application;

FIG. 3 is a schematic diagram of another sub-flow of the artificial intelligence-based text error detection method provided by an embodiment of the application;

FIG. 4 is a schematic diagram of another sub-flow of the artificial intelligence-based text error detection method provided by an embodiment of the application;

FIG. 5 is a schematic diagram of another sub-flow of the artificial intelligence-based text error detection method provided by an embodiment of the application;

FIG. 6 is a schematic diagram of another sub-process of the artificial intelligence-based text error detection method provided by an embodiment of the application;

FIG. 7 is a schematic diagram of another sub-flow of the artificial intelligence-based text error detection method provided by an embodiment of the application;

FIG. 8 is a schematic block diagram of an artificial intelligence-based text error detection device provided by an embodiment of the application;

FIG. 9 is a schematic block diagram of a computer device provided by an embodiment of the application.

Detailed ways

The technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of them. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of this application.

It should be understood that when used in this specification and appended claims, the terms "including" and "including" indicate the existence of the described features, wholes, steps, operations, elements and/or components, but do not exclude one or The existence or addition of multiple other features, wholes, steps, operations, elements, components, and/or collections thereof.

It should also be understood that the terms used in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in the specification of this application and the appended claims, unless the context clearly indicates other circumstances, the singular forms "a", "an" and "the" are intended to include plural forms.

It should be further understood that the term "and/or" used in the specification and appended claims of this application refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations .

Please refer to FIG. 1. FIG. 1 is a schematic flowchart of an artificial intelligence-based text error detection method provided by an embodiment of the present application. The artificial intelligence-based text error detection method is applied to a user terminal. The application software is executed, and the user terminal is a terminal device used for error detection of the text to be detected input by the user, such as a desktop computer, a notebook computer, a tablet computer, or a mobile phone. The user terminal can also be an enterprise server constructed by an enterprise. As shown in Figure 1, the method includes steps S110 to S150.

S110: Receive model configuration information input by the user, and configure parameter values in the initialized detection model according to the model configuration information to obtain multiple error detection models.

Specifically, the user can input model configuration information to configure the parameter values in the initialized detection model to obtain multiple error detection models. The initialized detection model includes the Long Short-Term Memory (LSTM, Long Short-Term Memory) weight layer And the state transition matrix, the model configuration information can include the number of model configurations, weight layer configuration information, and transition matrix configuration information. The state transition matrix in the initialized detection model is configured through the transition matrix configuration information, and the initialization is initialized through the weight layer configuration information. Configure the weight layer in the detection model, and create multiple error detection models according to the number of model configurations. Each error detection model includes a long and short-term memory network, a configured weight layer, and a configured state transition matrix.

Among them, the long and short-term memory network is used to calculate the input text information to obtain the memory network output information, the weight layer is used to weight the memory network output information to obtain the weighted memory network output information, and the state transition matrix is used to weight The output information of the subsequent memory network undergoes state transition processing to obtain model detection information, and the file detection result corresponding to the text information can be obtained by analyzing the model detection information.

S120: Training a plurality of said error detection models respectively according to a preset conversion dictionary and a pre-stored training corpus database to obtain a plurality of error detection models after training.

Before using multiple error detection models to perform error detection on Chinese text, multiple error detection models need to be trained separately. Specifically, multiple error detection models can be trained according to the conversion dictionary and training corpus database. Pre-stored loss function calculation formulas and gradient calculation formulas are also used in the training corpus. The training corpus database contains multiple training corpora. Each training sentence is composed of corpus information and target detection information. The corpus information is the corpus text information, and the target detection The information is the correct detection information corresponding to the corpus information, and the target detection information can be obtained by artificial judgment based on the corpus information and correspondingly added.

In one embodiment, as shown in FIG. 2, step S120 includes sub-steps S121, S122, and S123.

S121. Randomly allocate training corpus of the training corpus database according to the number of error detection models to obtain multiple training corpus sets having the same number.

Determine the number of error detection models, randomly allocate multiple training corpora in the training corpus database according to the number, and obtain multiple training corpus sets. The training corpus contained in one training corpus set can complete the training of an error detection model.

For example, the number of error detection models is 10, the number of training corpus in the training corpus database is 2000, and the training corpus is randomly allocated to obtain 10 training corpus sets, and each training corpus set contains 200 training corpora.

S122: Convert the corpus information in the multiple training corpus sets according to the conversion dictionary to obtain a piece of corpus information corresponding to each of the corpus information.

The corpus information in the plurality of training corpus sets is converted according to the conversion dictionary to obtain a piece of corpus information corresponding to each of the corpus information. Each character can be matched to a corresponding feature code in the conversion dictionary. The feature code is a 1×M-dimensional vector. The characters contained in each corpus information in the training corpus can be converted according to the conversion dictionary. The feature code corresponding to each character is combined to obtain the corresponding corpus code. The obtained corpus code represents the feature of the corpus information in a vector manner. The size of the corpus code is (N, M), which means that the corpus code is N lines The feature vector of column M, the length of the corpus code is N (such as N=39), and the feature code of the corpus information is filled as a value to obtain the corpus code. If the number of characters in the corpus information is greater than N, the first N characters are intercepted and converted accordingly N 1×M-dimensional vectors are formed. If the number of characters in the corpus information is less than N, the insufficient characters are filled with "0" as the feature code, and then correspondingly converted to obtain N 1×M-dimensional vectors.

S123: Perform iterative training on one of the error detection models corresponding to each of the training corpus sets according to the plurality of training corpus sets to obtain a trained error detection model corresponding to each of the training corpus sets. Wherein, each corpus code in the training corpus corresponds to a piece of preset target detection information.

The error detection model is iteratively trained according to the corpus codes in a training corpus set and the target detection information corresponding to each corpus code, and the error detection model is iteratively trained in combination with the pre-stored loss function calculation formula and gradient calculation formula until The corpus encoding and target detection information in the training corpus are all used for model training and then the training is stopped, and a trained error detection model can be obtained.

In one embodiment, as shown in FIG. 3, step S123 includes sub-steps S1231, S1232, S1233, and S1234.

S1231. Input a piece of corpus code in one of the training corpus sets into the error detection model to obtain training detection information corresponding to the corpus code.

Specifically, a corpus code is an N×M-dimensional vector. Calculating the memory network output information of a corpus code can include the following four steps: ①Calculate the output information of the forgetting gate: f(t)=σ(W _f × h(t_1)+U _f ×X(t)+b _f ), where f(t) is the parameter value of the forget gate, 0≤f(t)≤1; σ is the activation function calculation symbol, and σ can be specifically expressed as f (x)=(e ^x -e ^(-x) )/(e ^x + e ^(-x) ), then the calculation result of _{W f} ×h(t_1)+U _f ×X(t)+b _{f is taken as} x input activation function σ to calculate f(t); W _f , U _f and b _f are the parameter values of the formula in this cell; h(t_1) is the output gate information of the previous cell; X(t) is Input the 1×M-dimensional vector of the current cell into the corpus encoding. If the current cell is the first cell in the long-short-term memory network, h(t_1) is zero. ②Calculate the input gate information: i(t)=σ(W _i ×h(t_1)+U _i ×X(t)+b _i ); a(t)=tanh(W _a ×h(t-1)+ _{U a × X (t) +} b a), where i (t) is a parameter value input gate, 0≤i (t) ≤1; W i, U i, b i, W a, U a and b _a are It is the parameter value of the formula in this cell, a(t) is the calculated input gate vector value, and a(t) is a 1×M-dimensional vector. ③Update cell memory information: C(t)=C(t_1)⊙f(t)+i(t)⊙a(t), C is the accumulated cell memory information in each calculation process, C(t) is the current The cell memory information output by the cell, C(t_1) is the cell memory information output by the previous cell, ⊙ is the vector operator, and the calculation process of C(t_1)⊙f(t) is the calculation process of each in the vector C(t_1) The one-dimensional value is respectively multiplied by f(t), and the calculated vector dimension is the same as the dimension in the vector C(t_1). ④Calculate and output gate information: o(t)=σ(W _o ×h(t_1)+U _o ×X(t)+b _o ); h(t)=o(t)⊙tanh(C(t)) , O(t) is the output gate parameter value, 0≤o(t)≤1; W _o , U _o and b _o are the parameter values of the formula in the cell, h(t) is the output gate information of the cell, h(t) is a 1×M-dimensional vector. Each cell is calculated to obtain an output information h(t), and the output information of N cells can be combined to obtain a corpus-encoded memory network output information S, and a corpus-encoded memory network output information is an N×M-dimensional vector , The number of weight values contained in the weight layer is equal to M, and the output information of a corpus-encoded memory network is multiplied by the weight layer (each 1×M dimensional vector in the output information of the memory network is multiplied by the weight layer), The output information P of the memory network with additional weight value can be obtained, where P _ij is the emission score of the i-th character and the j-th error type label in the corpus information corresponding to the corpus encoding, i∈[1,N] and i is positive integer, j∈ [1, M] and j is a positive integer, M being the number of the wrong type of tag corpus corpus encoded information corresponding to the character corresponding to, for example, i = 1 and j = 1, P ₁₁ represents the i-th The character and the error type label of the replacement error correspond to a transmission score, j=2, then P ₁₂ indicates that the i-th character and the error type label of the insertion error correspond to a transmission score. The error detection model also includes a state transition matrix A, A is a matrix of size M×M, A _rt is the transition score of the r-th error type label to the t-th error type label, where r∈[1,M ], t∈[1,M]; if r=t, then A _rt =0. The output information P of the memory network and the state transition matrix A with the weight value added are the training detection information.

S1232: Calculate the loss value between the training detection information and the target detection information encoded by the corpus according to the pre-stored loss function calculation formula.

Specifically, the loss function calculation formula can be expressed by formula (1):

Among them, L is the calculated loss value, S(X, Y) is the score of the target detection information, S(X, Y') is the score of the training detection information, X is the input corpus code, and Y is the target detection information contains The error type label of Y'is the error type label contained in the training detection information, and Y _X is the label of all possible error types. Specifically, the score can be calculated using formula (2);

S1233: Calculate the updated value of the transition matrix in the error detection model according to the pre-stored gradient calculation formula, the loss value and the calculated value of the training detection information, and update the parameter value of the transition matrix.

According to the gradient calculation formula, the calculated loss value and the calculated value of the training detection information, the updated value of the transition matrix is calculated, and the parameter value in the transition matrix is updated through the updated value. This is the process of training the error detection model That is, gradient descent calculation.

Specifically, the gradient calculation formula can be expressed by formula (3):

in,

Is the calculated update value of a certain transition score, ω _t is the original parameter value of the transition score, and γ is the preset learning rate in the gradient calculation formula,

It is the partial derivative value of the transfer score based on the calculated value corresponding to the loss value and the transfer score (the difference between the emission scores of two adjacent error type tags in the calculated value of the training detection information).

S1234. Obtain the next piece of corpus coding information in the training corpus and input it into the error detection model and repeat the above steps until all corpus coding information included in the training corpus is used for training.

To update the transition matrix in the error detection model once is to train the error detection model once. The error detection model can be trained for multiple iterations according to the above process.

S130: If the to-be-detected text input by the user is received, input the to-be-detected text into multiple error detection models to obtain corresponding multiple model detection information.

If the text to be detected input by the user is received, the text to be detected can be input into each error detection model for calculation, and the model detection information of each error detection model is obtained correspondingly. Specifically, the text to be detected is input into an error detection Model, the output information of the model can be obtained. The output information of the model includes the output information of the memory network with the additional weight value and the state transition matrix. According to the state transition matrix, the output information of the memory network with the additional weight value is transferred to obtain the model detection information.

In one embodiment, as shown in FIG. 4, step S130 includes sub-steps S131 and S132.

S131. Convert the to-be-detected text into a corresponding text code according to the conversion dictionary; S132. Input the text code into a plurality of the error detection models for calculation, so as to obtain the output of each error detection model Model checking information.

Specifically, the text code corresponding to the text to be detected can be obtained through the above conversion dictionary. The text to be detected is a sentence of Chinese text information. The text code is an N×M-dimensional vector obtained by converting the text to be detected. The text codes are respectively Input multiple error detection models for calculation. The specific calculation process is the same as the calculation process for corpus encoding, so I will not repeat it here. The state transition matrix of an error detection model is correspondingly accumulated to the memory network output information of the error detection model with the additional weight value, and the state transition of the memory network output information with the additional weight value can be realized, and the corresponding model detection information can be obtained. The model detection information calculated by an error detection model is an N×M-dimensional vector, where N represents the total number of characters, and M represents the number of error type tags, which is used to represent the score value of each character in the text to be detected and each error type tag .

S140. Filtering a plurality of the model detection information to obtain screening detection data that meets a preset screening condition.

In the process of analyzing the obtained multiple model checking information, the data in the model checking information needs to be screened first. The screening checking data that meets the preset screening conditions can be filtered from the model checking information. Specifically, The screening conditions include detection data screening ratio and model screening ratio, and the screening detection data includes multiple sets of detection data information.

In one embodiment, as shown in FIG. 5, step S140 includes sub-steps S141, S142, and S143.

S141. Obtain a comprehensive detection score of each model detection information, and sort a plurality of model detection information according to the comprehensive detection score.

Calculate the comprehensive detection score of each model detection information, accumulate all the score values in a model detection information to obtain the comprehensive detection score of the model detection information, and sort the model detection information according to the comprehensive detection score.

S142. Cut the detection information of multiple models with the highest ranking according to the model screening ratio, and use them as multiple sets of candidate model detection information.

According to the model screening ratio, the detection information of multiple models with the highest ranking of comprehensive detection scores is intercepted, and multiple sets of candidate model detection information are obtained. For example, if the model screening ratio is 80%, and the model detection information is 10, the two model detection information of the inverse order of the comprehensive detection score are eliminated, and the remaining 8 candidate model detection information are obtained.

S143. Filter each group of candidate model detection information according to the detection data screening ratio, and obtain a plurality of detection data ranked higher in each group of candidate model detection information as a set of detection data information.

Each group of candidate model detection information is screened according to the detection data screening ratio, and multiple detection data ranked at the top are obtained as the detection data information of the group of candidate model detection information. For example, if a set of candidate model detection information contains N×M score values, and the detection data screening ratio is 40%, then the first 40% of the N×M score values will be obtained and retained, and the retained 40% will be retained. The value is used as the detection data information of the candidate model detection information.

S150. Perform integrated processing on the screening detection data to obtain a text detection result that matches the text to be detected.

The screening test data can be integrated and processed to obtain a text detection result that matches the text to be detected. Specifically, the screening detection data contains multiple sets of detection data information, the corresponding text error location and text error type can be determined according to each set of detection data information, and the text error location and text error type of multiple sets of detection data information can be integrated processing , Get the unified text error location and text error type as the text detection result of the text to be detected.

In addition, the obtained text detection results can be uploaded to the blockchain for storage. Specifically, the corresponding summary information may be obtained based on the text detection result. Specifically, the summary information is obtained by hashing the text detection result, for example, obtained by the sha256s algorithm. Uploading summary information to the blockchain can ensure its security and fairness and transparency to users. The user can download the summary information from the blockchain through the user terminal to verify whether the text detection result has been tampered with. The blockchain referred to in this example is a new application mode of computer technology such as distributed data storage, point-to-point transmission, consensus mechanism, and encryption algorithm. Blockchain, essentially a decentralized database, is a series of data blocks associated with cryptographic methods. Each data block contains a batch of network transaction information for verification. The validity of the information (anti-counterfeiting) and the generation of the next block. The blockchain can include the underlying platform of the blockchain, the platform product service layer, and the application service layer.

In an embodiment, as shown in FIG. 6, step S150 includes sub-steps S151, S152, S153, S154, S155, and S156.

S151: Determine the text error location and text error type of each group of detection data information according to each group of detection data information in the screening detection data.

Specifically, multiple score values of a character in a group of detection data information are obtained, and the error type label corresponding to the highest score value among the multiple score values of the character is obtained as the text error type of the group of detection data information. If a character does not contain a score value in the detection data information, it is considered that the character does not contain an error. According to the text error type of multiple characters, the text error position corresponding to the group of detection data information is determined, and all the characters that match the text error type are obtained. The position in the text to be detected is used as the text error position of the group of detected data information.

S152: Determine whether all the text error positions are the same.

Determine whether the text error positions of all groups of detected data information are the same. For example, the text error position of a certain group of detected data information is character 5 to character 9, and the text error position of another group of detected data information is character 6 to character 10. , The text error positions of the two groups of detected data information are not the same.

S153. If all the text error positions are not the same, judge whether all the text error positions contain a word segmentation error according to a preset judgment rule; S154, delete the text error positions that contain a word segmentation error and return to execution The step of judging whether all the text error positions are the same.

Specifically, a set of text error positions of the detected data information can be obtained, and it can be judged according to the judgment rule whether the text error position contains the segmented phrase, and if the text error position contains the segmented phrase, the text is determined to be obtained The error position contains a word segmentation error.

For example, the text error position of a certain group of detected data information is character 5 to character 9; get the 4th character, 5th character and 6th character in the text to be detected, and judge the 4th character and the 5th character according to the judgment rule Whether the probability of a character combination becoming a phrase is greater than the probability of combining the 5th character and the 6th character to become a phrase, if it is greater, the text error position contains the segmented phrase, if it is not greater, the same method is used to obtain the text to be detected In the 8th character, 9th character and 10th character, continue to determine whether it contains the segmented phrase. Delete the text error text containing the word segmentation error, and again judge whether the remaining text error positions are the same.

S155: If all the text error positions are the same, determine whether all the text error types are the same. S156: If all the text error types are the same, output the text error location and the text error type as the text detection result.

If the text error positions are all the same, determine whether the multiple text error types corresponding to each character in the text error position are all the same. If multiple text error types corresponding to each character in the text error position are the same, the currently obtained text error position and text error type can be output as the text detection result. If all the text error types are not the same, a prompt message that the text detection result cannot be obtained can be sent to prompt the user.

In one embodiment, as shown in FIG. 7, steps S1521 and S1522 are further included before step S152.

S1521, according to a preset grammar template, matches the text error position and text error type of each group of the detected data information; S1522, removes the text error position and text error type that match the grammar template.

Specifically, the text error position and text error type of each group of detected data information can also be matched according to the preset grammar template. If the text error position and text error type of a certain group of detected data information match any grammar template If it matches, the text error position and text error type of the group of test data information are eliminated.

Specifically, the preset grammar templates include some grammatical rules that cannot be detected by the error detection model. Hundreds of grammar templates can be pre-configured, and each grammar template can be judged in turn whether it corresponds to a set of text error positions and texts of the detection data information. The error type matches.

For example, in "playing happily", "happy" is an adjective, and "playing" is a verb. The two need to be connected with "地". If the text error position and text error type of a certain set of measured data information are judged to be "happy If there is a grammatical error in "Playing", the text error location and text error type of this group of test data information match the grammatical template "X to D" (where X represents an adjective, and D represents a verb). The text error location and text error type are eliminated.

The technical methods in this application can be applied to smart government affairs/smart city management/smart communities/smart security/smart logistics/smart healthcare/smart education/smart environmental protection/smart transportation and other application scenarios that include error detection of Chinese texts, thereby promoting The construction of smart cities.

In the artificial intelligence-based text error detection method provided by the embodiments of the present application, multiple error detection models are constructed according to the model configuration information and the initialized detection model; the multiple error detection models are trained separately, and the text to be detected is input The trained multiple error detection models acquire multiple model detection information, and filter the model detection information to obtain the screening detection data that meets the preset screening conditions; perform integrated processing on the screening detection data to obtain the text detection result. Through the above method, multiple error detection models are constructed to obtain multiple model detection information corresponding to the text to be detected, and the model detection information is filtered and integrated to obtain the text detection result. Compared with the fixed template matching method, it can be greatly improved. Improve the efficiency and accuracy of error detection for Chinese text.

The embodiment of the present application also provides an artificial intelligence-based text error detection device, which is used to execute any embodiment of the aforementioned artificial intelligence-based text error detection method. Specifically, please refer to FIG. 8, which is a schematic block diagram of a text error detection apparatus provided by an embodiment of the present application. The artificial intelligence-based text error detection device can be configured in a user terminal.

As shown in FIG. 8, the text error detection device 100 based on artificial intelligence includes a detection model configuration unit 110, a detection model training unit 120, a model detection information acquisition unit 130, a model detection information screening unit 140 and an integrated processing unit 150.

The detection model configuration unit 110 is configured to receive model configuration information input by a user, and configure parameter values in the initialized detection model according to the model configuration information to obtain multiple error detection models.

The detection model training unit 120 is configured to train multiple error detection models according to a preset conversion dictionary and a pre-stored training corpus database to obtain multiple error detection models after training.

In an embodiment, the detection model training unit 120 includes sub-units: a training corpus acquisition unit, a corpus code acquisition unit, and an iterative training unit.

The training corpus acquisition unit is used to randomly allocate the training corpus of the training corpus database according to the number of the error detection models to obtain multiple training corpus sets with the same number; the corpus code acquisition unit is used to The conversion dictionary converts the corpus information in the multiple training corpus sets to obtain a piece of corpus information corresponding to each of the corpus information; One of the error detection models corresponding to the training corpus is iteratively trained to obtain a trained error detection model corresponding to each of the training corpus sets, wherein each corpus code in the training corpus corresponds to A preset target detection information.

In an embodiment, the iterative training unit includes subunits: a training detection information acquisition unit, a loss value calculation unit, a transition matrix parameter update unit, and a repeat unit.

The training detection information acquisition unit is used to input a corpus code in the training corpus set into the error detection model to obtain training detection information corresponding to the corpus code; The function calculation formula calculates the loss value between the training detection information and the target detection information encoded by the corpus; the transition matrix parameter update unit is used to calculate the loss value according to the pre-stored gradient calculation formula, the loss value and the training detection information The calculated value is calculated to obtain the updated value of the transition matrix in the error detection model, and the parameter value of the transition matrix is updated; the repeat unit is used to obtain the next piece of corpus encoding information in the training corpus set and input the error detection model and Repeat the above steps until all the corpus coding information included in the training corpus set is used for training.

The model checking information acquiring unit 130 is configured to, if the to-be-detected text input by the user is received, input the to-be-detected text into a plurality of error detection models respectively to obtain a plurality of corresponding model detection information.

In an embodiment, the model detection information acquiring unit 130 includes sub-units: a text encoding acquiring unit and a text encoding calculating unit.

The text code acquisition unit is configured to convert the text to be detected into a corresponding text code according to the conversion dictionary; the text code calculation unit is configured to input the text code into multiple error detection models for calculation, respectively Obtain model detection information output by each of the error detection models.

The model checking information screening unit 140 is configured to obtain screening and checking data satisfying preset screening conditions from a plurality of the model checking information.

In an embodiment, the model detection information screening unit 140 includes subunits: a model detection information ranking unit, a model detection information intercepting unit, and a detection data screening unit.

The model checking information sorting unit is used to obtain the comprehensive detection score of each model checking information, and to sort a plurality of model checking information according to the comprehensive detection score; the model checking information intercepting unit is used to intercept the selection ratio according to the model The multiple model detection information ranked at the top are used as multiple sets of candidate model detection information; the detection data screening unit is used to screen each set of candidate model detection information according to the detection data screening ratio, and obtain each The plurality of detection data ranked higher in the group of candidate model detection information is used as a group of the detection data information.

The integrated processing unit 150 is configured to perform integrated processing on the screening detection data to obtain a text detection result that matches the text to be detected.

In an embodiment, the integrated processing unit 150 includes sub-units: an error location type determination unit, a text error location judgment unit, a word segmentation error judgment unit, a deletion unit, a text error type judgment unit, and a text detection result acquisition unit.

The error location type determining unit is used to determine the text error location and text error type of each group of detection data information according to each group of detection data information in the screening detection data; the text error location judgment unit is used to judge all the text errors Whether the positions are all the same; the word segmentation error judgment unit is used for determining whether all the text error positions contain word segmentation errors according to preset judgment rules if all the text error positions are not the same; the deletion unit is used for Delete the text error position containing the word segmentation error and return to execute the step of judging whether all the text error positions are the same; the text error type judgment unit is used for judging all the text error positions if all the text error positions are the same Whether the text error types are all the same; the text detection result obtaining unit is configured to output the text error location and the text error type as the text detection result if all the text error types are the same.

In an embodiment, the integrated processing unit 150 further includes sub-units: a grammar template matching unit and a rejection unit.

The grammar template matching unit is used to match the text error position and text error type of each group of the detected data information according to the preset grammar template; the elimination unit is used to match the text error position of the grammar template And text error types are eliminated.

The artificial intelligence-based text error detection device provided in the embodiments of the present application applies the above-mentioned artificial intelligence-based text error detection method to construct multiple error detection models according to the model configuration information and the initialized detection model; for multiple error detection models Train separately, input the text to be detected into multiple error detection models after training to obtain multiple model detection information, filter the model detection information to obtain the screening detection data that meets the preset screening conditions; perform integrated processing on the screening detection data to obtain the text Test results. Through the above method, multiple error detection models are constructed to obtain multiple model detection information corresponding to the text to be detected, and the model detection information is filtered and integrated to obtain the text detection result. Compared with the fixed template matching method, it can be greatly improved. Improve the efficiency and accuracy of error detection for Chinese text.

The above text error detection device can be implemented in the form of a computer program, and the computer program can be run on a computer device as shown in FIG. 9.

Please refer to FIG. 9, which is a schematic block diagram of a computer device according to an embodiment of the present application. The computer device may be a user terminal for executing an artificial intelligence-based text error detection method to perform error detection on Chinese text.

Referring to FIG. 9, the computer device 500 includes a processor 502, a memory, and a network interface 505 connected through a system bus 501, where the memory may include a non-volatile storage medium 503 and an internal memory 504.

The non-volatile storage medium 503 can store an operating system 5031 and a computer program 5032. When the computer program 5032 is executed, the processor 502 can execute a text error detection method based on artificial intelligence.

The processor 502 is used to provide calculation and control capabilities, and support the operation of the entire computer device 500.

The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503, and when the computer program 5032 is executed by the processor 502, the processor 502 can make the processor 502 execute a text error detection method based on artificial intelligence.

The network interface 505 is used for network communication, such as providing data information transmission. Those skilled in the art can understand that the structure shown in FIG. 9 is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device 500 to which the solution of the present application is applied. The specific computer device 500 may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

Wherein, the processor 502 is configured to run a computer program 5032 stored in a memory to implement the corresponding function in the above-mentioned artificial intelligence-based text error detection method.

Those skilled in the art can understand that the embodiment of the computer device shown in FIG. 9 does not constitute a limitation on the specific configuration of the computer device. In other embodiments, the computer device may include more or less components than those shown in the figure. Or some parts are combined, or different parts are arranged. For example, in some embodiments, the computer device may only include a memory and a processor. In such embodiments, the structures and functions of the memory and the processor are consistent with the embodiment shown in FIG. 9 and will not be repeated here.

It should be understood that, in this embodiment of the application, the processor 502 may be a central processing unit (Central Processing Unit, CPU), and the processor 502 may also be other general-purpose processors, digital signal processors (Digital Signal Processors, DSPs), Application Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor.

In another embodiment of the present application, a computer-readable storage medium is provided. The computer-readable storage medium may be a non-volatile computer-readable storage medium. The computer-readable storage medium stores a computer program, where the computer program implements the steps included in the above-mentioned artificial intelligence-based text error detection method when the computer program is executed by the processor.

Those skilled in the art can clearly understand that, for the convenience and conciseness of description, the specific working process of the above-described equipment, device, and unit can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here. A person of ordinary skill in the art may be aware that the units and algorithm steps of the examples described in the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of both, in order to clearly illustrate the hardware and software Interchangeability, in the above description, the composition and steps of each example have been generally described in accordance with the function. Whether these functions are executed by hardware or software depends on the specific application and design constraint conditions of the technical solution. Professionals and technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered beyond the scope of this application.

In the several embodiments provided in this application, it should be understood that the disclosed equipment, device, and method may be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods, or the units with the same function may be combined into one. Units, for example, multiple units or components can be combined or integrated into another system, or some features can be omitted or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may also be electrical, mechanical or other forms of connection.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments of the present application.

In addition, the functional units in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or software functional unit.

If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application is essentially or the part that contributes to the existing technology, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product can be stored in a computer. The read storage medium includes several instructions to enable a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned computer-readable storage medium includes: U disk, mobile hard disk, Read-Only Memory (ROM, Read-Only Memory), magnetic disk or optical disk and other media that can store program codes.

The above are only specific implementations of this application, but the protection scope of this application is not limited to this. Anyone familiar with the technical field can easily think of various equivalents within the technical scope disclosed in this application. Modifications or replacements, these modifications or replacements shall be covered within the protection scope of this application. Therefore, the protection scope of this application shall be subject to the protection scope of the claims.

Claims

An artificial intelligence-based text error detection method applied to a user terminal, wherein the method includes:

Receiving model configuration information input by a user, and configuring parameter values in the initialized detection model according to the model configuration information to obtain multiple error detection models;

Training a plurality of said error detection models respectively according to a preset conversion dictionary and a pre-stored training corpus database to obtain a plurality of error detection models after training;

If the text to be detected input by the user is received, input the text to be detected into multiple error detection models to obtain corresponding multiple model detection information;

Screening and obtaining screening detection data satisfying preset screening conditions from the plurality of model detection information;

Performing integrated processing on the screening detection data to obtain a text detection result that matches the text to be detected.
The method for text error detection based on artificial intelligence according to claim 1, wherein said training a plurality of said error detection models respectively according to a preset conversion dictionary and a pre-stored training corpus database comprises:

Randomly allocate the training corpus of the training corpus database according to the number of the error detection models to obtain multiple training corpus sets of the same number;

Converting the corpus information in the plurality of training corpus sets according to the conversion dictionary to obtain a piece of corpus information corresponding to each of the corpus information;

According to a plurality of the training corpus sets, iterative training is performed on one of the error detection models corresponding to each of the training corpus sets to obtain a trained error detection model corresponding to each of the training corpus sets, wherein, Each corpus code in the training corpus corresponds to a piece of preset target detection information.
The method for text error detection based on artificial intelligence according to claim 2, wherein the iterative training of one of the error detection models corresponding to each of the training corpus sets on the basis of a plurality of the training corpus sets respectively comprises :

Inputting a piece of corpus code in one of the training corpus sets into the error detection model to obtain training detection information corresponding to the corpus code;

Calculating a loss value between the training detection information and the target detection information encoded by the corpus according to a pre-stored loss function calculation formula;

Calculating the updated value of the transition matrix in the error detection model according to the pre-stored gradient calculation formula, the loss value and the calculated value of the training detection information, and updating the parameter value of the transition matrix;

Obtain the next piece of corpus coding information in the training corpus set and input it into the error detection model and repeat the above steps until all corpus coding information included in the training corpus set is used for training.
The method for text error detection based on artificial intelligence according to claim 1, wherein said inputting said text to be detected into a plurality of said error detection models respectively to obtain corresponding plurality of model detection information comprises:

Converting the to-be-detected text into a corresponding text encoding according to the conversion dictionary;

The text codes are respectively input to a plurality of the error detection models for calculation, so as to obtain the model detection information output by each of the error detection models.
The method for text error detection based on artificial intelligence according to claim 1, wherein the screening conditions include detection data screening ratios and model screening ratios, and the screening detection data includes multiple sets of detection data information. The screening and testing data that meets the preset screening conditions are screened in the model testing information, including:

Acquiring a comprehensive detection score of each model detection information, and sorting a plurality of model detection information according to the comprehensive detection score;

Intercepting a plurality of model detection information ranked at the top according to the model screening ratio, as multiple sets of candidate model detection information;

Each group of candidate model detection information is screened according to the detection data screening ratio, and a plurality of detection data ranked higher in each group of candidate model detection information is obtained as a set of detection data information.
The artificial intelligence-based text error detection method according to claim 1, wherein said performing integrated processing on said screening detection data to obtain a text detection result matching said text to be detected comprises:

Determine the text error location and text error type of each group of detection data information according to each group of detection data information in the screening detection data;

Determine whether all the text error positions are the same;

If all the text error positions are not the same, judge whether all the text error positions contain word segmentation errors according to a preset judgment rule;

Delete the text error position containing the word segmentation error and return to execute the step of judging whether all the text error positions are the same;

If all the text error positions are the same, determine whether all the text error types are the same;

If all the text error types are the same, the text error location and the text error type are output as the text detection result.
The method for text error detection based on artificial intelligence according to claim 1, wherein said determining whether all the text error positions are the same includes:

Match the text error position and text error type of each group of the detected data information according to the preset grammar template;

The text error location and text error type matching the grammar template are eliminated.
An artificial intelligence-based text error detection device, which includes:

The detection model configuration unit is configured to receive model configuration information input by the user, and configure parameter values in the initialized detection model according to the model configuration information to obtain multiple error detection models;

The detection model training unit is configured to train multiple error detection models according to a preset conversion dictionary and a pre-stored training corpus database to obtain multiple error detection models after training;

The model checking information acquiring unit is configured to, if the to-be-detected text input by the user is received, input the to-be-detected text into multiple error detection models to obtain corresponding multiple model detection information;

The model checking information screening unit is used to screen a plurality of the model checking information to obtain screening and checking data that meets preset screening conditions;

The integrated processing unit is configured to perform integrated processing on the screening detection data to obtain a text detection result that matches the text to be detected.
A computer device includes a memory, a processor, and a computer program that is stored on the memory and can run on the processor, wherein the processor implements the following steps when the computer program is executed:

Receiving model configuration information input by a user, and configuring parameter values in the initialized detection model according to the model configuration information to obtain multiple error detection models;

Training a plurality of said error detection models respectively according to a preset conversion dictionary and a pre-stored training corpus database to obtain a plurality of error detection models after training;

If the text to be detected input by the user is received, input the text to be detected into multiple error detection models to obtain corresponding multiple model detection information;

Screening and obtaining screening detection data satisfying preset screening conditions from the plurality of model detection information;

Performing integrated processing on the screening detection data to obtain a text detection result that matches the text to be detected.
The computer device according to claim 9, wherein the training a plurality of the error detection models respectively according to a preset conversion dictionary and a pre-stored training corpus database comprises:

Randomly allocate the training corpus of the training corpus database according to the number of the error detection models to obtain multiple training corpus sets of the same number;

Converting the corpus information in the plurality of training corpus sets according to the conversion dictionary to obtain a piece of corpus information corresponding to each of the corpus information;

According to a plurality of the training corpus sets, iterative training is performed on one of the error detection models corresponding to each of the training corpus sets to obtain a trained error detection model corresponding to each of the training corpus sets, wherein, Each corpus code in the training corpus corresponds to a piece of preset target detection information.
11. The computer device according to claim 10, wherein the iterative training of one of the error detection models corresponding to each of the training corpus sets on the basis of a plurality of the training corpus sets respectively comprises:

Inputting a piece of corpus code in one of the training corpus sets into the error detection model to obtain training detection information corresponding to the corpus code;

Calculating a loss value between the training detection information and the target detection information encoded by the corpus according to a pre-stored loss function calculation formula;

Calculating the updated value of the transition matrix in the error detection model according to the pre-stored gradient calculation formula, the loss value and the calculated value of the training detection information, and updating the parameter value of the transition matrix;

Obtain the next piece of corpus coding information in the training corpus set and input it into the error detection model and repeat the above steps until all corpus coding information included in the training corpus set is used for training.
9. The computer device according to claim 9, wherein said inputting said text to be detected into a plurality of said error detection models respectively to obtain corresponding plurality of model detection information comprises:

Converting the to-be-detected text into a corresponding text encoding according to the conversion dictionary;

The text codes are respectively input to a plurality of the error detection models for calculation, so as to obtain the model detection information output by each of the error detection models.
8. The computer device according to claim 9, wherein the screening conditions include a screening ratio of detection data and a screening ratio of a model, the screening detection data includes multiple sets of detection data information, and the screening is performed from a plurality of the model detection information Obtain the screening test data that meets the preset screening conditions, including:

Acquiring a comprehensive detection score of each model detection information, and sorting a plurality of model detection information according to the comprehensive detection score;

Intercepting a plurality of model detection information ranked at the top according to the model screening ratio, as multiple sets of candidate model detection information;

Each group of candidate model detection information is screened according to the detection data screening ratio, and a plurality of detection data ranked higher in each group of candidate model detection information is obtained as a set of detection data information.
9. The computer device according to claim 9, wherein the integrated processing of the screening detection data to obtain a text detection result that matches the text to be detected comprises:

Determine the text error location and text error type of each group of detection data information according to each group of detection data information in the screening detection data;

Determine whether all the text error positions are the same;

If all the text error positions are not the same, judge whether all the text error positions contain word segmentation errors according to a preset judgment rule;

Delete the text error position containing the word segmentation error and return to execute the step of judging whether all the text error positions are the same;

If all the text error positions are the same, determine whether all the text error types are the same;

If all the text error types are the same, the text error location and the text error type are output as the text detection result.
8. The computer device according to claim 9, wherein said determining whether all the text error positions are the same includes:

Match the text error position and text error type of each group of the detected data information according to the preset grammar template;

The text error location and text error type matching the grammar template are eliminated.
A computer-readable storage medium, wherein the computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to perform the following operations:

Receiving model configuration information input by a user, and configuring parameter values in the initialized detection model according to the model configuration information to obtain multiple error detection models;

Training a plurality of said error detection models respectively according to a preset conversion dictionary and a pre-stored training corpus database to obtain a plurality of error detection models after training;

If the text to be detected input by the user is received, input the text to be detected into multiple error detection models to obtain corresponding multiple model detection information;

Screening and obtaining screening detection data satisfying preset screening conditions from the plurality of model detection information;

Performing integrated processing on the screening detection data to obtain a text detection result that matches the text to be detected.
15. The computer-readable storage medium according to claim 16, wherein the training a plurality of the error detection models respectively according to a preset conversion dictionary and a pre-stored training corpus database comprises:

Randomly allocate the training corpus of the training corpus database according to the number of the error detection models to obtain multiple training corpus sets of the same number;

Converting the corpus information in the plurality of training corpus sets according to the conversion dictionary to obtain a piece of corpus information corresponding to each of the corpus information;

According to a plurality of the training corpus sets, iterative training is performed on one of the error detection models corresponding to each of the training corpus sets to obtain a trained error detection model corresponding to each of the training corpus sets, wherein, Each corpus code in the training corpus corresponds to a piece of preset target detection information.
18. The computer-readable storage medium according to claim 17, wherein the iterative training of one of the error detection models corresponding to each of the training corpus sets on the basis of a plurality of the training corpus sets respectively comprises:

Inputting a piece of corpus code in one of the training corpus sets into the error detection model to obtain training detection information corresponding to the corpus code;

Calculating a loss value between the training detection information and the target detection information encoded by the corpus according to a pre-stored loss function calculation formula;

Calculating the updated value of the transition matrix in the error detection model according to the pre-stored gradient calculation formula, the loss value and the calculated value of the training detection information, and updating the parameter value of the transition matrix;

Obtain the next piece of corpus coding information in the training corpus set and input it into the error detection model and repeat the above steps until all corpus coding information included in the training corpus set is used for training.
15. The computer-readable storage medium according to claim 16, wherein said inputting said text to be detected into a plurality of said error detection models respectively to obtain corresponding plurality of model detection information comprises:

Converting the to-be-detected text into a corresponding text encoding according to the conversion dictionary;

The text codes are respectively input to a plurality of the error detection models for calculation, so as to obtain the model detection information output by each of the error detection models.
The computer-readable storage medium according to claim 16, wherein the screening conditions include a screening ratio of detection data and a screening ratio of models, the screening detection data includes multiple sets of detection data information, and the detection from a plurality of the models The information is filtered to obtain the screening test data that meets the preset screening conditions, including:

Acquiring a comprehensive detection score of each model detection information, and sorting a plurality of model detection information according to the comprehensive detection score;

Intercepting a plurality of model detection information ranked at the top according to the model screening ratio, as multiple sets of candidate model detection information;

Each group of candidate model detection information is screened according to the detection data screening ratio, and a plurality of detection data ranked higher in each group of candidate model detection information is obtained as a set of detection data information.