CN111178042A - Data processing method and device and computer storage medium - Google Patents
Data processing method and device and computer storage medium Download PDFInfo
- Publication number
- CN111178042A CN111178042A CN201911409695.8A CN201911409695A CN111178042A CN 111178042 A CN111178042 A CN 111178042A CN 201911409695 A CN201911409695 A CN 201911409695A CN 111178042 A CN111178042 A CN 111178042A
- Authority
- CN
- China
- Prior art keywords
- text
- rule
- optimal
- alternative
- probability information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L13/00—Speech synthesis; Text to speech systems
- G10L13/08—Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination
- G10L13/10—Prosody rules derived from text; Stress or intonation
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The embodiment of the invention discloses a data processing method, a data processing device and a computer storage medium, wherein the method comprises the following steps: acquiring an initial text; performing text analysis on the initial text according to a target regularization system to obtain a standard text and a confidence coefficient corresponding to the standard text, wherein the target regularization system at least comprises an optimal rule, an alternative rule, optimal probability information corresponding to the optimal rule and alternative probability information corresponding to the alternative rule; and marking the standard text according to the confidence coefficient to obtain a marked text. According to the data processing method, the data processing device and the computer storage medium, provided by the embodiment of the invention, the editing system is optimized by calculating the confidence coefficient of text analysis and marking the text in different degrees according to the confidence coefficients of different texts, so that the user experience is improved, and the text editing efficiency is increased.
Description
Technical Field
The present invention relates to the field of information processing technologies, and in particular, to a data processing method and apparatus, and a computer storage medium.
Background
TTS (text to speech, speech synthesis) is a technology for converting characters into natural human language, and is widely applied to aspects of navigation broadcasting, online customer service of merchants, robot-only speech interaction and the like. The TTS system is mainly divided into a front end and a back end, wherein the front end mainly performs text analysis work, and a machine knows how to read the text. The TTS editing system is a TTS synthesis system which can be edited by a user, and the user can edit the pronunciation, the reading method, the pause and the like of words in the text so as to meet the requirements of the user. For example, TN (Text Normalization) editing, "13 years" can be read as "thirteen years", and the user can also edit the reading method to make it read as "year reading method" and read as "three years", or other reading methods. Text analysis plays a significant role in the synthesis of TTS. TN, i.e. text regularization, is an important step of text analysis, and is a process of converting irregular text, i.e. text with various numbers and symbols, into standard text, i.e. text without numbers and only with a few symbols representing pauses specified by the system. For example, a TN of "33 +12 equals 45" would result in "thirty-three plus twelve equals forty-five".
When the existing TTS editing system edits the text regularization, error-prone TNs and error-prone TNs cannot be distinguished, and the problems of disordered interfaces and low editing efficiency can be caused by the TTS editing system.
Disclosure of Invention
In order to effectively overcome the above-mentioned defects in the prior art, embodiments of the present invention creatively provide a data processing method, including: acquiring an initial text; performing text analysis on the initial text according to a target regularization system to obtain a standard text and a confidence coefficient corresponding to the standard text, wherein the target regularization system at least comprises an optimal rule, an alternative rule, optimal probability information corresponding to the optimal rule and alternative probability information corresponding to the alternative rule; and marking the standard text according to the confidence coefficient to obtain a marked text.
In an implementation manner, before performing text analysis on the initial text according to a target regularization system to obtain a standard text and a confidence corresponding to the standard text, the method includes: obtaining a sample text and an initial regularization system; modifying rule matching conditions in the initial regularization system to obtain an adjustment regularization system, wherein the adjustment regularization system is a system capable of matching the optimal rule and the alternative rule of each text; and performing confidence training on the adjustment regularization system according to the sample text to obtain a target regularization system.
In an implementation manner, the performing confidence training on the regularization adjustment system according to the sample text to obtain a target regularization system includes: obtaining an optimal rule and an alternative rule corresponding to the sample text according to the regulation regularization system; carrying out rule matching training on the adjustment regularization system according to the adjacent texts of the sample texts to obtain optimal probability information corresponding to the optimal rules of the sample texts and alternative probability information corresponding to the alternative rules of the sample texts, wherein the optimal probability information and the alternative probability information respectively correspond to the adjacent texts of the sample texts; and establishing a target regularization system according to the optimal rule and the alternative rule of the sample text, the optimal probability information of the optimal rule corresponding to the sample text and the alternative probability information of the alternative rule corresponding to the sample text.
In an implementation manner, the performing text analysis on the initial text according to the target regularization system to obtain a standard text and a confidence corresponding to the standard text includes: carrying out rule matching on the initial text according to the target regularization system to obtain an optimal rule and an alternative rule corresponding to the initial text; transferring the initial text according to the optimal rule of the initial text to obtain a standard text; acquiring optimal probability information of an optimal rule corresponding to the initial text and alternative probability information of an alternative rule corresponding to the initial text in the target regularization system according to the adjacent text of the initial text; and determining the confidence of the standard text according to the optimal probability information of the optimal rule of the initial text and the alternative probability information of the alternative rule corresponding to the initial text.
In an implementation manner, the labeling the standard text according to the confidence level to obtain a labeled text includes: judging whether the confidence coefficient accords with a first threshold value, and if so, performing first marking processing on the standard text; judging whether the confidence coefficient accords with a second threshold value, and if so, performing second marking processing on the standard text; judging whether the confidence coefficient accords with a third threshold value, and if so, performing third marking processing on the standard text; the first threshold, the second threshold and the third threshold are arranged from low to high according to the confidence degree, and the first mark, the second mark and the third mark are arranged from high to low according to the mark content suggestive degree.
Another aspect of an embodiment of the present invention provides a data processing apparatus, including: the initial text acquisition module is used for acquiring an initial text; the confidence coefficient analysis module is used for performing text analysis on the initial text according to a target regularization system to obtain a standard text and a confidence coefficient corresponding to the standard text, wherein the target regularization system at least comprises an optimal rule, an alternative rule, optimal probability information corresponding to the optimal rule and alternative probability information corresponding to the alternative rule; and the marking module is used for marking the standard text according to the confidence coefficient to obtain a marked text.
In one embodiment, the apparatus further comprises: the sample acquisition module is used for acquiring a sample text and an initial regularization system; the system modification module is used for modifying the rule matching conditions in the initial regularization system to obtain an adjustment regularization system, and the adjustment regularization system is a system capable of being matched with the optimal rule and the alternative rule of each text; and the system training module is used for carrying out confidence training on the adjustment regularization system according to the sample text to obtain a target regularization system.
In one embodiment, the system training module comprises: the rule obtaining unit is used for obtaining an optimal rule and an alternative rule corresponding to the sample text according to the regulation regularization system; the system training unit is used for carrying out rule matching training on the adjustment regularization system according to the adjacent texts of the sample texts to obtain optimal probability information corresponding to the optimal rules of the sample texts and alternative probability information corresponding to the alternative rules of the sample texts, wherein the optimal probability information and the alternative probability information respectively correspond to the adjacent texts of the sample texts; and the system establishing unit is used for establishing a target regularization system according to the optimal rule and the alternative rule of the sample text, the optimal probability information of the optimal rule corresponding to the sample text and the alternative probability information of the alternative rule corresponding to the sample text.
In one embodiment, the confidence analysis module comprises: the rule matching unit is used for carrying out rule matching on the initial text according to the target regularization system to obtain an optimal rule and an alternative rule corresponding to the initial text; the text transcription unit is used for transcribing the initial text according to the optimal rule of the initial text to obtain a standard text; a probability obtaining unit, configured to obtain, according to a neighboring text of the initial text, optimal probability information of an optimal rule corresponding to the initial text and alternative probability information of an alternative rule corresponding to the initial text in the target regularization system; and the confidence determining unit is used for determining the confidence of the standard text according to the optimal probability information of the optimal rule of the initial text and the alternative probability information of the alternative rule corresponding to the initial text.
In one embodiment, the marking module comprises: the first marking unit is used for judging whether the confidence coefficient accords with a first threshold value or not, and if so, performing first marking processing on the standard text; the second marking unit is used for judging whether the confidence coefficient accords with a second threshold value or not, and if so, performing second marking processing on the standard text; the third marking unit is used for judging whether the confidence coefficient accords with a third threshold value or not, and if so, performing third marking processing on the standard text; the first threshold, the second threshold and the third threshold are arranged from low to high according to the confidence degree, and the first mark, the second mark and the third mark are arranged from high to low according to the mark content suggestive degree.
Another aspect of the embodiments of the present invention provides a computer-readable storage medium, in which computer-executable instructions are stored, and when the instructions are executed, the computer-readable storage medium is configured to perform the data processing method described in any one of the above.
According to the data processing method, the data processing device and the computer storage medium, provided by the embodiment of the invention, the editing system is optimized by calculating the confidence coefficient of text analysis and marking the text in different degrees according to the confidence coefficients of different texts, so that the user experience is improved, and the text editing efficiency is increased.
Drawings
The above and other objects, features and advantages of exemplary embodiments of the present invention will become readily apparent from the following detailed description read in conjunction with the accompanying drawings. Several embodiments of the invention are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings and in which:
in the drawings, the same or corresponding reference numerals indicate the same or corresponding parts.
Fig. 1 is a schematic flow chart illustrating an implementation of a data processing method according to an embodiment of the present invention;
fig. 2 is a schematic flow chart of another implementation of a data processing method according to an embodiment of the present invention;
fig. 3 is a flowchart illustrating an implementation of a data processing method according to an embodiment of the present invention;
fig. 4 is a schematic flowchart illustrating another specific implementation of a data processing method according to an embodiment of the present invention;
FIG. 5 is a block diagram of a data processing apparatus according to an embodiment of the present invention;
fig. 6 is a block diagram of another embodiment of a data processing apparatus;
FIG. 7 is a block diagram of a system training module according to an embodiment of the present invention;
fig. 8 is a structural diagram of an embodiment of a confidence level analysis module according to an embodiment of the present invention.
Detailed Description
In order to make the objects, features and advantages of the present invention more obvious and understandable, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "a plurality" means two or more unless specifically defined otherwise.
The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present specification. Rather, they are merely examples of methods, apparatus or devices consistent with certain aspects of the specification, as detailed in the claims that follow.
Referring to fig. 1, an embodiment of the present invention provides a data processing method, including:
102, performing text analysis on the initial text according to a target regularization system to obtain a standard text and a confidence coefficient corresponding to the standard text, wherein the target regularization system at least comprises an optimal rule, an alternative rule, optimal probability information corresponding to the optimal rule and alternative probability information corresponding to the alternative rule;
and 103, marking the standard text according to the confidence coefficient to obtain a marked text.
In order to solve the problems that interface confusion and low editing efficiency are caused by the fact that error-prone TNs and error-prone TNs cannot be distinguished when a TTS editing system edits a text in the prior art, an initial text to be edited is obtained through a step 101, and then the initial text is analyzed through a step 102 according to a target regularization system, wherein the target regularization system comprises an optimal rule and an alternative rule for regularizing a large number of different texts, optimal probability information corresponding to the optimal rule and alternative probability information corresponding to the alternative rule. In the embodiment of the invention, the target regularization system is used for carrying out text analysis on the initial text, namely when the rules are matched, the first TN rule hit by the text is the optimal rule, wherein the text transcribed according to the optimal rule is the standard text; and then, continuing to match other subsequent TN rules until all the rules are matched once, wherein the other subsequent TN rules are the alternative rules. The optimal probability information refers to probability distribution information of each text which is analyzed to hit the optimal rule under different context conditions according to the context in the article paragraph where the text is located, namely after combination statistics of adjacent words before and after the text is combined. Similarly, the candidate probability information refers to probability distribution information of each text, which is obtained by analyzing the type of text under different context conditions according to the context in the article passage where the text is located, that is, after combining the statistics of the combinations of the adjacent words before and after. Because the candidate probability information of the candidate rule and the optimal probability information of the optimal rule for transcribing the standard text are already calculated according to the target regularization system, the reliability, namely the confidence coefficient, of the standard text can be easily calculated according to the optimal probability information and the candidate probability information. Finally, marking the standard text according to the confidence coefficient through step 103, specifically, adjusting the display degree of the standard text according to the confidence coefficient, for example, adjusting the text with high confidence coefficient by adopting a transparent or gray background, adjusting the text with low confidence coefficient which needs to be modified by adopting a red background and the like; the method can also be used for adjusting the text with low reliability in ways of thickening, amplifying or adding labels and the like so as to enable the attention of the user to be more concentrated, thereby reducing unnecessary TN information on an editing interface, further improving the experience of the user and increasing the editing efficiency.
Referring to fig. 2, in an implementation, before performing text analysis on an initial text according to a target regularization system to obtain a standard text and a confidence corresponding to the standard text, the method includes:
and 203, performing confidence training on the adjustment regularization system according to the sample text to obtain a target regularization system.
The initial regularization system obtained in the embodiment of the invention can be all internal systems or can be directly obtained from an external source, and is an existing TN system, in the existing TN system, regularization rules are arranged according to priorities, a section of text only hits one TN rule according to the priority of the TN rule, and then is immediately transcribed, so that other alternative rules cannot be hit, other possible reading methods of the section of text cannot be known, and the reliability of the transcribed text cannot be known everywhere. In the embodiment of the invention, the adjustment regularization system is obtained by modifying the rule matching conditions in the initial regularization system, so that when each text is matched according to the adjustment regularization system, the subsequent TN rules are continuously matched after the first TN rule is hit until all the rules are matched once, the optimal rule corresponding to the first hit rule and the alternative rules corresponding to the subsequent hit rules are obtained, and all the possible reading methods of the text are obtained. And then training and optimizing the regulation and regularization system through the sample text, calculating and analyzing probability distribution information of the type of text hitting the optimal rule and probability distribution information of the alternative rule under different context conditions, and finally obtaining a target regularization system for more conveniently calculating text confidence so as to improve text editing efficiency.
The alternative rule in the embodiment of the present invention may be a scheme obtained by matching according to an existing TN system, or may be a preset scheme defined in advance according to a text type, so as to better meet the requirements of different users in different application scenarios.
Referring to fig. 3, in an implementation, performing confidence training on the regularization system according to the sample text to obtain the target regularization system includes:
301, obtaining an optimal rule and an alternative rule corresponding to a sample text according to an adjustment regularization system;
According to the embodiment of the invention, firstly, the optimal rule and the alternative rule of the sample text are obtained through the step 301 according to the regulation and regularization system, then, the probability distribution condition of the sample text corresponding to the type to each rule under the condition of different contexts is obtained through the rule matching calculation of the regulation and regularization system according to the adjacent text of the sample text, namely the combination statistics of 4-5 words before and after the sample text, namely, the optimal probability information of the optimal rule corresponding to the sample text and the alternative probability information of the alternative rule corresponding to the sample text are obtained. Step 303, specifically, an association relationship is established between each obtained probability information and the corresponding rule and between the text and the text adjacent to the text, so as to finally establish a target regularization system which completes the optimal rule and the candidate rule including a large number of texts, and the optimal probability information corresponding to the optimal rule and the candidate probability information corresponding to the candidate rule.
Referring to fig. 4, in an implementation, performing text analysis on the initial text according to the target regularization system to obtain the standard text and the confidence corresponding to the standard text includes:
and step 404, determining the confidence of the standard text according to the optimal probability information of the optimal rule of the initial text and the alternative probability information of the alternative rule corresponding to the initial text.
According to the embodiment of the invention, firstly, an optimal rule and an alternative rule are obtained by carrying out rule matching on an initial text according to a target regularization system in step 401, wherein the optimal rule is used for transcribing the initial text in step 402 to obtain a standard text for displaying or subsequent editing on the standard text. Step 403 may specifically be that type matching is performed on the neighboring text of the initial text and the neighboring text corresponding to the probability information of each rule in the target regularization system, so as to obtain optimal probability information of the optimal rule corresponding to the initial text and candidate probability information of the candidate rule corresponding to the initial text; finally, in step 404, the confidence of the standard text is calculated according to the ratio of the known candidate probability information to the optimal probability information, and the confidence calculation training process can adopt various machine learning methods capable of realizing reliable calculation, so that the reliability degree of different texts can be accurately distinguished compared with the reliability degree of the traditional method, thereby being beneficial to reminding a user to concentrate on the text which needs to be edited more and improving the editing efficiency.
In an implementation manner, the labeling the standard text according to the confidence level, and obtaining a labeled text includes:
judging whether the confidence coefficient accords with a first threshold value, and if so, performing first marking processing on the standard text;
judging whether the confidence coefficient accords with a second threshold value, and if so, performing second marking processing on the standard text;
judging whether the confidence coefficient accords with a third threshold value, and if so, performing third marking processing on the standard text;
the first threshold, the second threshold and the third threshold are arranged from low to high according to the confidence degree, and the first mark, the second mark and the third mark are arranged from high to low according to the mark content suggestive degree.
In the embodiment of the present invention, after the confidence of each TN text is obtained through calculation, different labeling methods may be used to distinguish the TN text according to different levels of the confidence, for example, the confidence is divided into three threshold levels, where a first threshold, a second threshold, and a third threshold are arranged from low to high according to the confidence, and the labeling processing actions are also divided into three different degrees, where the first label, the second label, and the third label are arranged from high to low according to the indication degree of the label content. Specifically, for TN text with a first threshold value, that is, with low confidence, the first marking process with the highest content cue degree may be adopted, for example, red or other vivid colors are adopted to make the attention of the user more focused, or a marking method such as enlarging a font by two font sizes is adopted. For the TN text with the second threshold value, that is, with the medium confidence level, the second marking process with the medium content cue degree may be adopted, for example, the second marking process may adopt a light yellow color or other slightly vivid colors, or may adopt a marking method of enlarging a font size and the like. For the text with the third threshold value, that is, the text with high confidence level, the third marking process with low prompting level may be adopted, specifically, the third marking process is carried out by adopting gray or transparent color, so that the text interface is clear in primary and secondary, redundant TN information is reduced, the user experience is improved, and the editing efficiency is increased.
Referring to fig. 5, another embodiment of the present invention provides a data processing apparatus, including:
an initial text obtaining module 501, configured to obtain an initial text;
a confidence coefficient analysis module 502, configured to perform text analysis on the initial text according to a target regularization system to obtain a standard text and a confidence coefficient corresponding to the standard text, where the target regularization system at least includes an optimal rule and an alternative rule, and optimal probability information corresponding to the optimal rule and alternative probability information corresponding to the alternative rule;
and the marking module 503 is configured to perform marking processing on the standard text according to the confidence degree to obtain a marked text.
In order to solve the problems that interface confusion and editing efficiency are low due to the fact that error-prone TNs and error-prone TNs cannot be distinguished when a TTS editing system edits a text in the prior art, an initial text to be edited is obtained through an initial text obtaining module 501, and then the initial text is analyzed through a confidence coefficient analyzing module 502 according to a target regularization system, wherein the target regularization system comprises an optimal rule and an alternative rule for regularizing a large number of different texts, optimal probability information corresponding to the optimal rule and alternative probability information corresponding to the alternative rule. In the embodiment of the invention, the target regularization system is used for carrying out text analysis on the initial text, namely when the rules are matched, the first TN rule hit by the text is the optimal rule, wherein the text transcribed according to the optimal rule is the standard text; and then, continuing to match other subsequent TN rules until all the rules are matched once, wherein the other subsequent TN rules are the alternative rules. The optimal probability information refers to probability distribution information of each text which is analyzed to hit the optimal rule under different context conditions according to the context in the article paragraph where the text is located, namely after combination statistics of adjacent words before and after the text is combined. Similarly, the candidate probability information refers to probability distribution information of each text, which is obtained by analyzing the type of text under different context conditions according to the context in the article passage where the text is located, that is, after combining the statistics of the combinations of the adjacent words before and after. Because the candidate probability information of the candidate rule and the optimal probability information of the optimal rule for transcribing the standard text are already calculated according to the target regularization system, the reliability, namely the confidence coefficient, of the standard text can be easily calculated according to the optimal probability information and the candidate probability information. Finally, the marking module 503 is used for marking the standard text according to the confidence level, specifically, the display degree of the standard text can be adjusted according to the confidence level, for example, the text with high confidence level is adjusted by adopting a transparent or gray background, while the text with low confidence level is adjusted by adopting a red background and the like; the method can also be used for adjusting the text with low reliability in ways of thickening, amplifying or adding labels and the like so as to enable the attention of the user to be more concentrated, thereby reducing unnecessary TN information on an editing interface, further improving the experience of the user and increasing the editing efficiency.
Referring to fig. 6, in an implementation manner, the apparatus further includes:
a sample obtaining module 601, configured to obtain a sample text and an initial regularization system;
a system modification module 602, configured to modify a rule matching condition in the initial regularization system to obtain an adjusted regularization system, where the adjusted regularization system is a system capable of matching an optimal rule and an alternative rule of each text;
and the system training module 603 is configured to perform confidence training on the adjustment regularization system according to the sample text to obtain a target regularization system.
The initial regularization system obtained in the embodiment of the invention can be all internal systems or can be directly obtained from an external source, and is an existing TN system, in the existing TN system, regularization rules are arranged according to priorities, a section of text only hits one TN rule according to the priority of the TN rule, and then is immediately transcribed, so that other alternative rules cannot be hit, other possible reading methods of the section of text cannot be known, and the reliability of the transcribed text cannot be known everywhere. In the embodiment of the present invention, the system modification module 602 modifies the rule matching condition in the initial regularization system to obtain the adjusted regularization system, so that when each text is matched according to the adjusted regularization system, the subsequent TN rule is continuously matched after the first TN rule is hit until all the rules are matched once, thereby obtaining the optimal rule corresponding to the first hit rule and the alternative rule corresponding to the subsequent hit rule, and obtaining all possible reading methods of the text. Then, the system training module 603 performs confidence training optimization on the adjustment regularization system according to the sample text, and calculates and analyzes probability distribution information of the type of text hitting the optimal rule and probability distribution information of the alternative rule under different context conditions, so as to finally obtain a target regularization system for more conveniently calculating text confidence, thereby improving text editing efficiency.
The alternative rule in the embodiment of the present invention may be a scheme obtained by matching according to an existing TN system, or may be a preset scheme defined in advance according to a text type, so as to better meet the requirements of different users in different application scenarios.
Referring to fig. 7, in an implementation, the system training module 603 includes:
a rule obtaining unit 701, configured to obtain an optimal rule and an alternative rule corresponding to the sample text according to the regularization adjustment system;
the system training unit 702 is configured to perform rule matching training on the regularization system according to the neighboring texts of the sample text, so as to obtain optimal probability information corresponding to the optimal rule of the sample text and alternative probability information corresponding to the alternative rule of the sample text, where the optimal probability information and the alternative probability information correspond to the neighboring texts of the sample text, respectively;
the system establishing unit 703 is configured to establish a target regularization system according to the optimal rule and the candidate rule of the sample text, and the optimal probability information of the optimal rule corresponding to the sample text and the candidate probability information of the candidate rule corresponding to the sample text.
According to the embodiment of the invention, firstly, an optimal rule and an alternative rule of a sample text are obtained through a rule obtaining unit 701 according to an adjusting and regularizing system, then, a probability distribution condition of the sample text corresponding to the type to each rule under the condition of different contexts is obtained through rule matching calculation of the adjusting and regularizing system according to adjacent texts of the sample text, namely combination statistics of 4-5 words before and after the sample text, namely, optimal probability information of the optimal rule corresponding to the sample text and alternative probability information of the alternative rule corresponding to the sample text are obtained. The system establishing unit 703 may specifically establish an association relationship between each obtained probability information and a corresponding rule and between a text and a text adjacent to the text, and finally establish a target regularization system that completes an optimal rule and an alternative rule including a large number of texts, and optimal probability information corresponding to the optimal rule and alternative probability information corresponding to the alternative rule.
Referring to fig. 8, in an implementation, the confidence analysis module 502 includes:
a rule matching unit 801, configured to perform rule matching on the initial text according to a target regularization system to obtain an optimal rule and an alternative rule corresponding to the initial text;
the text transcription unit 802 is configured to transcribe the initial text according to the optimal rule of the initial text to obtain a standard text;
a probability obtaining unit 803, configured to obtain, according to an adjacent text of the initial text, optimal probability information of an optimal rule corresponding to the initial text and alternative probability information of an alternative rule corresponding to the initial text in the target regularization system;
a confidence determining unit 804, configured to determine the confidence of the standard text according to the optimal probability information of the optimal rule of the initial text and the candidate probability information of the candidate rule corresponding to the initial text.
According to the embodiment of the invention, the initial text is subjected to rule matching through the rule matching unit 801 according to the target regularization system to obtain the optimal rule and the alternative rule, wherein the optimal rule is used for the text transcription unit 802 to transcribe the initial text to obtain the standard text for display or subsequent editing on the text. The probability obtaining unit 803 may specifically be configured to perform type matching on the neighboring text of the initial text and the neighboring text corresponding to the probability information of each rule in the target regularization system, so as to obtain optimal probability information of the optimal rule corresponding to the initial text and alternative probability information of the alternative rule corresponding to the initial text; and finally, the confidence coefficient determining unit 804 calculates the confidence coefficient of the standard text according to the ratio of the known alternative probability information and the optimal probability information, and the confidence coefficient calculation training process can adopt various machine learning methods capable of realizing reliable calculation.
In one embodiment, the marking module 503 includes:
the first marking unit is used for judging whether the confidence coefficient accords with a first threshold value or not, and if so, performing first marking processing on the standard text;
the second marking unit is used for judging whether the confidence coefficient accords with a second threshold value or not, and if so, performing second marking processing on the standard text;
the third marking unit is used for judging whether the confidence coefficient accords with a third threshold value or not, and if so, performing third marking processing on the standard text;
the first threshold, the second threshold and the third threshold are arranged from low to high according to the confidence degree, and the first mark, the second mark and the third mark are arranged from high to low according to the mark content suggestive degree.
In the embodiment of the present invention, after the confidence of each TN text is obtained through calculation, different labeling methods may be used to distinguish the TN text according to different levels of the confidence, for example, the confidence is divided into three threshold levels, where a first threshold, a second threshold, and a third threshold are arranged from low to high according to the confidence, and the labeling processing actions are also divided into three different degrees, where the first label, the second label, and the third label are arranged from high to low according to the indication degree of the label content. Specifically, for the TN text with the first threshold value, that is, with low confidence, the first marking unit may adopt the first marking process with the highest content prompting degree, for example, adopt red or other vivid colors to make the attention of the user more concentrated, or adopt a marking method of enlarging a font by two font sizes, etc. For the TN text with the second threshold value, that is, with the medium confidence level, the second marking unit may adopt a second marking process with a medium content cue degree, for example, a light yellow color or other slightly vivid colors, or a marking method of enlarging a font size with a font, etc. For the text with the third threshold value, that is, the text with high confidence level, the third marking unit may adopt a third marking process with a low prompting degree, specifically, adopt a gray or transparent color for marking, and the like, so that the text interface is clear in primary and secondary, redundant TN information is reduced, user experience is improved, and editing efficiency is increased.
Another aspect of the embodiments of the present invention provides a computer-readable storage medium, in which computer-executable instructions are stored, and when the instructions are executed, the computer-readable storage medium is used for executing the data processing method of any one of the above.
Here, it should be noted that: the above description of the embodiments is similar to the above description of the method embodiments, and has similar beneficial effects to the method embodiments, and for technical details not disclosed in the embodiments of the present invention, please refer to the description of the method embodiments of the present invention for understanding, so that details are not repeated.
In the embodiment of the present invention, the implementation order among the steps may be replaced without affecting the implementation purpose.
The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and the changes or substitutions should be covered within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (11)
1. A data processing method, comprising:
acquiring an initial text;
performing text analysis on the initial text according to a target regularization system to obtain a standard text and a confidence coefficient corresponding to the standard text, wherein the target regularization system at least comprises an optimal rule, an alternative rule, optimal probability information corresponding to the optimal rule and alternative probability information corresponding to the alternative rule;
and marking the standard text according to the confidence coefficient to obtain a marked text.
2. The method of claim 1, prior to performing text analysis on the initial text according to a target regularization system to obtain a standard text and a confidence corresponding to the standard text, comprising:
obtaining a sample text and an initial regularization system;
modifying rule matching conditions in the initial regularization system to obtain an adjustment regularization system, wherein the adjustment regularization system is a system capable of matching the optimal rule and the alternative rule of each text;
and performing confidence training on the adjustment regularization system according to the sample text to obtain a target regularization system.
3. The method of claim 2, wherein the performing confidence training on the regularization system according to the sample text to obtain a target regularization system comprises:
obtaining an optimal rule and an alternative rule corresponding to the sample text according to the regulation regularization system;
carrying out rule matching training on the adjustment regularization system according to the adjacent texts of the sample texts to obtain optimal probability information corresponding to the optimal rules of the sample texts and alternative probability information corresponding to the alternative rules of the sample texts, wherein the optimal probability information and the alternative probability information respectively correspond to the adjacent texts of the sample texts;
and establishing a target regularization system according to the optimal rule and the alternative rule of the sample text, the optimal probability information of the optimal rule corresponding to the sample text and the alternative probability information of the alternative rule corresponding to the sample text.
4. The method of claim 3, wherein the performing text analysis on the initial text according to a target regularization system to obtain a standard text and a confidence corresponding to the standard text comprises:
carrying out rule matching on the initial text according to the target regularization system to obtain an optimal rule and an alternative rule corresponding to the initial text;
transferring the initial text according to the optimal rule of the initial text to obtain a standard text;
acquiring optimal probability information of an optimal rule corresponding to the initial text and alternative probability information of an alternative rule corresponding to the initial text in the target regularization system according to the adjacent text of the initial text;
and determining the confidence of the standard text according to the optimal probability information of the optimal rule of the initial text and the alternative probability information of the alternative rule corresponding to the initial text.
5. The method according to claim 1 or 2, wherein the labeling the standard text according to the confidence coefficient to obtain a labeled text comprises:
judging whether the confidence coefficient accords with a first threshold value, and if so, performing first marking processing on the standard text;
judging whether the confidence coefficient accords with a second threshold value, and if so, performing second marking processing on the standard text;
judging whether the confidence coefficient accords with a third threshold value, and if so, performing third marking processing on the standard text;
the first threshold, the second threshold and the third threshold are arranged from low to high according to the confidence degree, and the first mark, the second mark and the third mark are arranged from high to low according to the mark content suggestive degree.
6. A data processing apparatus, comprising:
the initial text acquisition module is used for acquiring an initial text;
the confidence coefficient analysis module is used for performing text analysis on the initial text according to a target regularization system to obtain a standard text and a confidence coefficient corresponding to the standard text, wherein the target regularization system at least comprises an optimal rule, an alternative rule, optimal probability information corresponding to the optimal rule and alternative probability information corresponding to the alternative rule;
and the marking module is used for marking the standard text according to the confidence coefficient to obtain a marked text.
7. The apparatus of claim 6, further comprising:
the sample acquisition module is used for acquiring a sample text and an initial regularization system;
the system modification module is used for modifying the rule matching conditions in the initial regularization system to obtain an adjustment regularization system, and the adjustment regularization system is a system capable of being matched with the optimal rule and the alternative rule of each text;
and the system training module is used for carrying out confidence training on the adjustment regularization system according to the sample text to obtain a target regularization system.
8. The apparatus of claim 7, wherein the system training module comprises:
the rule obtaining unit is used for obtaining an optimal rule and an alternative rule corresponding to the sample text according to the regulation regularization system;
the system training unit is used for carrying out rule matching training on the adjustment regularization system according to the adjacent texts of the sample texts to obtain optimal probability information corresponding to the optimal rules of the sample texts and alternative probability information corresponding to the alternative rules of the sample texts, wherein the optimal probability information and the alternative probability information respectively correspond to the adjacent texts of the sample texts;
and the system establishing unit is used for establishing a target regularization system according to the optimal rule and the alternative rule of the sample text, the optimal probability information of the optimal rule corresponding to the sample text and the alternative probability information of the alternative rule corresponding to the sample text.
9. The apparatus of claim 8, wherein the confidence analysis module comprises:
the rule matching unit is used for carrying out rule matching on the initial text according to the target regularization system to obtain an optimal rule and an alternative rule corresponding to the initial text;
the text transcription unit is used for transcribing the initial text according to the optimal rule of the initial text to obtain a standard text;
a probability obtaining unit, configured to obtain, according to a neighboring text of the initial text, optimal probability information of an optimal rule corresponding to the initial text and alternative probability information of an alternative rule corresponding to the initial text in the target regularization system;
and the confidence determining unit is used for determining the confidence of the standard text according to the optimal probability information of the optimal rule of the initial text and the alternative probability information of the alternative rule corresponding to the initial text.
10. The method of claim 6 or 7, wherein the tagging module comprises:
the first marking unit is used for judging whether the confidence coefficient accords with a first threshold value or not, and if so, performing first marking processing on the standard text;
the second marking unit is used for judging whether the confidence coefficient accords with a second threshold value or not, and if so, performing second marking processing on the standard text;
the third marking unit is used for judging whether the confidence coefficient accords with a third threshold value or not, and if so, performing third marking processing on the standard text;
the first threshold, the second threshold and the third threshold are arranged from low to high according to the confidence degree, and the first mark, the second mark and the third mark are arranged from high to low according to the mark content suggestive degree.
11. A computer-readable storage medium having stored therein computer-executable instructions for performing the data processing method of any one of claims 1-5 when the instructions are executed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911409695.8A CN111178042B (en) | 2019-12-31 | 2019-12-31 | Data processing method and device and computer storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911409695.8A CN111178042B (en) | 2019-12-31 | 2019-12-31 | Data processing method and device and computer storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111178042A true CN111178042A (en) | 2020-05-19 |
| CN111178042B CN111178042B (en) | 2023-04-28 |
Family
ID=70650625
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911409695.8A Active CN111178042B (en) | 2019-12-31 | 2019-12-31 | Data processing method and device and computer storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111178042B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114978794A (en) * | 2022-05-19 | 2022-08-30 | 北京有竹居网络技术有限公司 | Network access method, device, storage medium and electronic equipment |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100082348A1 (en) * | 2008-09-29 | 2010-04-01 | Apple Inc. | Systems and methods for text normalization for text to speech synthesis |
| CN105427855A (en) * | 2015-11-09 | 2016-03-23 | 上海语知义信息技术有限公司 | Voice broadcast system and voice broadcast method of intelligent software |
| CN106412053A (en) * | 2016-09-26 | 2017-02-15 | 广州华多网络科技有限公司 | Probability based service configuration method and system |
| US20170076725A1 (en) * | 2015-09-11 | 2017-03-16 | Microsoft Technology Licensing, Llc | Automatic speech recognition confidence classifier |
| CN107943856A (en) * | 2017-11-07 | 2018-04-20 | 南京邮电大学 | A kind of file classification method and system based on expansion marker samples |
| US20190278841A1 (en) * | 2018-03-12 | 2019-09-12 | Apple Inc. | Inverse text normalization for automatic speech recognition |
-
2019
- 2019-12-31 CN CN201911409695.8A patent/CN111178042B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100082348A1 (en) * | 2008-09-29 | 2010-04-01 | Apple Inc. | Systems and methods for text normalization for text to speech synthesis |
| US20170076725A1 (en) * | 2015-09-11 | 2017-03-16 | Microsoft Technology Licensing, Llc | Automatic speech recognition confidence classifier |
| CN105427855A (en) * | 2015-11-09 | 2016-03-23 | 上海语知义信息技术有限公司 | Voice broadcast system and voice broadcast method of intelligent software |
| CN106412053A (en) * | 2016-09-26 | 2017-02-15 | 广州华多网络科技有限公司 | Probability based service configuration method and system |
| CN107943856A (en) * | 2017-11-07 | 2018-04-20 | 南京邮电大学 | A kind of file classification method and system based on expansion marker samples |
| US20190278841A1 (en) * | 2018-03-12 | 2019-09-12 | Apple Inc. | Inverse text normalization for automatic speech recognition |
Non-Patent Citations (2)
| Title |
|---|
| CHIH-CHUNG KUO: "A Chinese Text-to-Speech System with Text Preprocessing and Confidence Measure for Practical Usage" * |
| 贾玉详，黄德智，刘武，俞士汶: "中文语音合成中的文本正则化研究" * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114978794A (en) * | 2022-05-19 | 2022-08-30 | 北京有竹居网络技术有限公司 | Network access method, device, storage medium and electronic equipment |
| CN114978794B (en) * | 2022-05-19 | 2023-06-23 | 北京有竹居网络技术有限公司 | Network access method, device, storage medium and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111178042B (en) | 2023-04-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| AU2016202974B2 (en) | Automatically creating a mapping between text data and audio data | |
| CN107678561A (en) | Phonetic entry error correction method and device based on artificial intelligence | |
| US10043519B2 (en) | Generation of text from an audio speech signal | |
| CN111681642B (en) | Speech recognition evaluation method, device, storage medium and equipment | |
| CN110119510B (en) | Relationship extraction method and device based on transfer dependency relationship and structure auxiliary word | |
| CN109460558B (en) | Effect judging method of voice translation system | |
| US20190362713A1 (en) | Dynamic extraction of contextually-coherent text blocks | |
| CN111883137A (en) | Text processing method and device based on voice recognition | |
| CN112016271A (en) | Language style conversion model training method, text processing method and device | |
| CN110853627B (en) | Method and system for voice annotation | |
| CN112037769A (en) | Training data generation method and device and computer readable storage medium | |
| CN111178042A (en) | Data processing method and device and computer storage medium | |
| CN114254658A (en) | Method, device, equipment and storage medium for generating translation evaluation training data | |
| CN109213970B (en) | Method and device for generating notes | |
| WO2019120248A1 (en) | Method for converting speech into characters, device, and system | |
| CN115116427B (en) | Labeling method, voice synthesis method, training method and training device | |
| CN115099222A (en) | Punctuation mark misuse detection and correction method, device, equipment and storage medium | |
| CN109389969B (en) | Corpus optimization method and apparatus | |
| CN114078470A (en) | Model processing method and device, and voice recognition method and device | |
| US20240144931A1 (en) | Systems and methods for gpt guided neural punctuation for conversational speech | |
| KR102400962B1 (en) | System for recognize musical note | |
| CN109817205B (en) | Text confirmation method and device based on semantic analysis and terminal equipment | |
| CN115440225B (en) | Intelligent voice processing method and system | |
| WO2023108459A1 (en) | Training and using a deep learning model for transcript topic segmentation | |
| KR101030777B1 (en) | Method and apparatus for producing script data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |