CN115062613A - Text processing method, electronic device and computer storage medium - Google Patents

Text processing method, electronic device and computer storage medium Download PDF

Info

Publication number
CN115062613A
CN115062613A CN202210816105.9A CN202210816105A CN115062613A CN 115062613 A CN115062613 A CN 115062613A CN 202210816105 A CN202210816105 A CN 202210816105A CN 115062613 A CN115062613 A CN 115062613A
Authority
CN
China
Prior art keywords
text
template
training sample
character
probability
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202210816105.9A
Other languages
Chinese (zh)
Other versions
CN115062613B (en
Inventor
段誉
康杨杨
孙常龙
刘晓钟
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Alibaba China Co Ltd
Original Assignee
Alibaba China Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alibaba China Co Ltd filed Critical Alibaba China Co Ltd
Priority to CN202210816105.9A priority Critical patent/CN115062613B/en
Publication of CN115062613A publication Critical patent/CN115062613A/en
Application granted granted Critical
Publication of CN115062613B publication Critical patent/CN115062613B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00Handling natural language data
    • G06F40/20Natural language analysis
    • G06F40/279Recognition of textual entities
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00Handling natural language data
    • G06F40/20Natural language analysis
    • G06F40/205Parsing
    • G06F40/216Parsing using statistical methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/049Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • General Engineering & Computer Science (AREA)
  • Biomedical Technology (AREA)
  • Evolutionary Computation (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Biophysics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Probability & Statistics with Applications (AREA)
  • Machine Translation (AREA)

Abstract

The embodiment of the application provides a text processing method, electronic equipment and a computer storage medium, which belong to the field of computer computing, and the text processing method comprises the following steps: obtaining a text training sample, wherein the text training sample comprises a template text and a variable text; performing text reconstruction on the text training sample by using a self-encoder which is trained in advance to obtain a corresponding reconstruction result; according to the difference between the reconstruction result and the text training sample, obtaining a first probability that each character in the text training sample belongs to the template text; and training a template recognition model for carrying out template text labeling according to the text training sample and the first probability. According to the scheme provided by the embodiment, the probability that each character of the text training sample belongs to the template text can be determined by utilizing the reconstruction characteristic of the self-encoder, and the template recognition model for labeling the template text is trained according to the probability, so that the method is not limited to a specific variable type, does not need to label the sample, and is high in universality.

Description

Text processing method, electronic device and computer storage medium
Technical Field
The embodiment of the application relates to the technical field of computers, in particular to a text processing method, electronic equipment and a computer storage medium.
Background
Short messages are used as a common information carrier, and are often used as a tool for batch notification by users, and the content of the short messages is generally generated according to a preset short message template. The short messages can be managed in batches based on the short message template, for example, the short messages are subjected to batch auditing, blackening, whitening and the like, and a large amount of redundant operations can be reduced.
However, since the short message templates are various, a large number of short messages do not store corresponding short message templates, which increases the management difficulty. For this purpose, the template corresponding to the short message needs to be identified through a neural network model at the training place.
At present, the training of a neural network model generally adopts a short message original text marked with a variable text as a training sample to perform supervision training on the neural network model. However, the types of variable texts which can be marked are limited, so that the neural network model at the training position has a long tail effect; in addition, the efficiency of adopting a supervision training mode is lower due to the fact that short message templates are continuously increased.
Disclosure of Invention
In view of the above, embodiments of the present application provide a text processing scheme to at least partially solve the above problems.
According to a first aspect of embodiments of the present application, there is provided a text processing method, including: obtaining a text training sample, wherein the text training sample comprises a template text and a variable text; performing text reconstruction on the text training sample by using a self-encoder which is trained in advance to obtain a corresponding reconstruction result; according to the difference between the reconstruction result and the text training sample, obtaining a first probability that each character in the text training sample belongs to the template text; and training a template recognition model for template text labeling according to the text training sample and the first probability.
According to a second aspect of the embodiments of the present application, there is provided a text processing method, including: obtaining a target text, wherein the target text comprises a template text and a variable text; inputting the target text into a template recognition model, wherein the template recognition model is obtained by training through the method; and carrying out template text labeling on the target text through the template recognition model, and determining the template text in the target text according to a labeling result.
According to a third aspect of embodiments of the present application, there is provided an electronic apparatus, including: the system comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete mutual communication through the communication bus; the memory is used for storing at least one executable instruction, and the executable instruction causes the processor to execute the corresponding operation of the method of the first aspect or the second aspect.
According to a fourth aspect of embodiments of the present application, there is provided a computer storage medium having stored thereon a computer program which, when executed by a processor, implements a method according to the first or second aspect.
According to the text processing scheme provided by the embodiment of the application, a text training sample is obtained, wherein the text training sample comprises a template text and a variable text; performing text reconstruction on the text training sample by using a self-encoder which is trained in advance to obtain a corresponding reconstruction result; according to the difference between the reconstruction result and the text training sample, obtaining a first probability that each character in the text training sample belongs to the template text; according to the scheme provided by the embodiment, the probability that each character of the text training sample belongs to the template text can be determined by utilizing the reconstruction characteristic of an autoencoder, and the template recognition model for labeling the template text is trained according to the probability, so that the method is not limited to a specific variable type, does not need to label the sample, only needs to collect the text training sample, and is high in universality.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the description below are only some embodiments described in the embodiments of the present application, and other drawings can be obtained by those skilled in the art according to these drawings.
FIG. 1 is a schematic diagram of an exemplary system to which a text processing method of an embodiment of the present application is applicable;
FIG. 2A is a flowchart illustrating steps of a method for processing text according to an embodiment of the present application;
FIG. 2B is a diagram illustrating an example of a scenario in the embodiment shown in FIG. 2A;
FIG. 3A is a flow chart illustrating steps of another method of text processing according to an embodiment of the present application;
FIG. 3B is a diagram illustrating an example of a scenario in the embodiment shown in FIG. 3A;
FIG. 4 is a flow chart illustrating steps of yet another method of text processing according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
In order to make those skilled in the art better understand the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, but not all embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application shall fall within the scope of the protection of the embodiments in the present application.
The following further describes specific implementations of embodiments of the present application with reference to the drawings of the embodiments of the present application.
Fig. 1 illustrates an exemplary system to which a text processing method according to an embodiment of the present application is applied. As shown in fig. 1, the system 100 may include a cloud server 102, a communication network 104, and/or one or more user devices 106, illustrated in fig. 1 as a plurality of user devices.
Cloud server 102 may be any suitable device for storing information, data, programs, and/or any other suitable type of content, including but not limited to distributed storage system devices, server clusters, computing cloud server clusters, and the like. In some embodiments, cloud server 102 may perform any suitable functions. For example, in some embodiments, the cloud server 102 may be configured to obtain, according to a difference between the reconstruction result and the text training sample, a first probability that each character in the text training sample belongs to the template text; and training a template recognition model for template text labeling according to the text training sample and the first probability. As an optional example, in some embodiments, the cloud service 102 may be used to pre-train the self-encoder.
In some embodiments, the communication network 104 may be any suitable combination of one or more wired and/or wireless networks. For example, the communication network 104 can include any one or more of the following: the network may include, but is not limited to, the internet, an intranet, a Wide Area Network (WAN), a Local Area Network (LAN), a wireless network, a Digital Subscriber Line (DSL) network, a frame relay network, an Asynchronous Transfer Mode (ATM) network, a Virtual Private Network (VPN), and/or any other suitable communication network. The user device 106 can be connected to the communication network 104 via one or more communication links (e.g., communication link 112), and the communication network 104 can be linked to the cloud server 102 via one or more communication links (e.g., communication link 114). The communication link may be any communication link suitable for communicating data between the user device 106 and the cloud service 102, such as a network link, a dial-up link, a wireless link, a hardwired link, any other suitable communication link, or any suitable combination of such links.
User devices 106 may include any user device or devices suitable for transmitting or receiving or presenting text training samples. In some embodiments, user devices 106 may comprise any suitable type of device. For example, in some embodiments, the user device 106 may include a mobile device, a tablet computer, a laptop computer, a desktop computer, a wearable computer, and/or any other suitable type of user device.
Based on the above system, the embodiment of the present application provides a text processing method, which is described below by using a plurality of embodiments.
Fig. 2A is a flowchart illustrating steps of a text processing method according to an embodiment of the present application, where the method includes:
s201, obtaining a text training sample, wherein the text training sample comprises a template text and a variable text.
In this embodiment, the template text may be a text that is not changed when the text training sample is generated, and the variable text may be a text that is changed when the text training sample is generated.
For example, if the text training sample is "this time verification code is 124", no other person is notified. "where" 124 "is variable text, which varies with each validation code provided, and the remainder is template text. The variable text may be, for example, the name of the person, the address, the amount of money, the contact address, and the pickup code.
It should be noted that the text in the text processing scheme provided in this embodiment may be any text formed according to the template, for example, a short message text, or a text sent by chat software, which is not limited in this embodiment.
S202, text reconstruction is carried out on the text training sample by using a self-encoder which is trained in advance, and a corresponding reconstruction result is obtained.
The self-encoder is a framework composed of an encoder and a decoder, and the training goal of the self-encoder is to reduce the reconstruction error as much as possible. For a method for obtaining a reconstruction result by performing text reconstruction using a pre-trained self-encoder, reference may be made to related technologies, which are not described herein again.
S203, according to the difference between the reconstruction result and the text training sample, obtaining a first probability that each character in the text training sample belongs to the template text;
in this embodiment, the self-encoder is trained in advance through the text training sample, and in the text training sample used for training the self-encoder, the self-encoder has a better reconstruction effect on the template text and a poorer reconstruction effect on the variable text because the template text has a higher repeatability and the variable text has a lower repeatability. Therefore, in this embodiment, a first probability that each character in the reconstruction result belongs to the template text may be obtained through a difference between the reconstruction result and the text training sample, where the reconstruction result has a smaller difference from the text training sample and a higher first probability that the character belongs to the template text, and conversely, the reconstruction result has a larger difference from the text training sample and a lower first probability that the character belongs to the template text.
And S204, training a template recognition model for template text labeling according to the text training sample and the first probability.
In this embodiment, since the first probability is a probability that the character belongs to the template text, the template recognition model can be trained through the text training sample and the first probability, so that the trained template recognition model can output the probability that the character belongs to the template text. For the specific training method, reference may be made to self-supervision or semi-supervision training methods, which are not described herein again.
Fig. 2B is a schematic diagram of a scene example in the embodiment shown in fig. 2A, as shown in the figure, the short message original text "pickup code 156 coming" may be used as a text training sample, and input to the self-encoder, the self-encoder may output a reconstruction result "pickup code 319 coming", and according to a difference between each character in the reconstruction result and the short message original text, a first probability that each character belongs to the template text may be determined.
Illustratively, the characters "pick-up code" and "come" have better reconstruction effect, the first probability of belonging to the template text is higher, and the first probability of belonging to the template text is lower due to the poor reconstruction effect of "156".
According to the text training sample and the first probability corresponding to each character in the text training sample, the template recognition model can be trained. For example, a text training sample may be input to the template recognition model as a sample, and the first probability corresponding to each character in the text training sample may be used as a supervision to perform supervision training on the template recognition model. The trained template recognition model can output the probability that each character in the short message belongs to the template text.
In the scheme provided by this embodiment, a text training sample is obtained, where the text training sample includes a template text and a variable text; performing text reconstruction on the text training sample by using a self-encoder which is trained in advance to obtain a corresponding reconstruction result; according to the difference between the reconstruction result and the text training sample, obtaining a first probability that each character in the text training sample belongs to the template text; according to the scheme provided by the embodiment, the probability that each character of the text training sample belongs to the template text can be determined by utilizing the reconstruction characteristic of an autoencoder, and the template recognition model for labeling the template text is trained according to the probability, so that the method is not limited to a specific variable type, does not need to label the sample, only needs to collect the text training sample, and is high in universality.
Fig. 3A is a flowchart illustrating steps of a text processing method according to an embodiment of the present application, where the method includes:
s301, obtaining a text training sample, wherein the text training sample comprises a template text and a variable text.
S302, pre-training the self-encoder by using the text training sample.
In this embodiment, a plurality of text training samples may be predetermined, and the text training samples are used to train the self-encoder first, and the specific training mode may refer to related technologies, which are not described herein again. The training process for the self-encoder may be a stage of training as shown in fig. 3B, in which the training is performed to reduce the reconstruction error, and the pre-collected text is used as the text training sample in fig. 3B.
As shown in fig. 3B, the self-encoder includes an encoder enroder, which may be a convolutional neural network model cnn, a long short term memory network model lstm, or a translation model transform, and a decoder, which may be a long short term memory network model lstm or a translation model transform. For specific implementation of the self-encoder, reference may be made to related technologies, which are not described herein again.
S303, reconstructing a plurality of characters included in the text training sample by using a pre-trained self-encoder to obtain a reconstruction result corresponding to each of the plurality of characters.
For example, as shown in fig. 3B, a self-encoder reconstructs a text of a short message as a text training sample, so as to obtain a reconstructed sample as a reconstruction result, and the reconstructed sample may be used to determine the first probability in the second stage training process.
S304, determining a first probability that each character in the text training sample belongs to the template text according to the difference between the reconstruction result corresponding to each character and the characters included in the text training sample.
Optionally, in this embodiment, step S304 includes: outputting the probability distribution of each character in the reconstruction result under the vocabulary through the self-encoder; determining difference information of each character in the reconstruction result and the character of the text training sample according to the probability distribution of each character in the reconstruction result under the vocabulary; and calculating to obtain a first probability that each character in the text training sample belongs to the template text according to the difference information. In this embodiment, the probability distribution is output by the self-encoder, and the fitting module shown in fig. 3B may convert the probability distribution into the first probability, but of course, in other implementations, the self-encoder may output the difference information or output the first probability, which are all within the protection scope of the present application.
Illustratively, given a short message, such as "pick-up code 156 coming", an output probability distribution of each character of the short message under the vocabulary, such as (0.01, 0.01, …, 0.8, 0.01), is obtained through an encoder and a decoder of the encoder, difference information of each character and the character of the text training sample can be directly calculated according to the probability distribution, and the first probability is determined according to the difference information. The specific method for obtaining the probability distribution can refer to the related art, and is not described herein again.
Optionally, in this embodiment, the determining, according to the probability distribution of each character in the reconstruction result under the vocabulary, difference information between each character in the reconstruction result and the character of the text training sample includes: at least calculating to obtain cross entropies corresponding to the characters in the reconstruction result according to the probability distribution of the characters in the reconstruction result under the vocabulary; the calculating, according to the difference information, a first probability that each character in the text training sample belongs to the template text includes: and carrying out normalization processing on the cross entropy to obtain a first probability that each character in the text training sample belongs to the template text.
In this embodiment, the cross entropy corresponding to each character can be calculated according to the output probability distribution of each character in the vocabulary, the cross entropy can be used to measure the reconstruction level of the character, the better the reconstruction effect is, the smaller the cross entropy is, and the specific method for obtaining the probability distribution of the character in the vocabulary can refer to the related art and is not described herein again.
Further, according to the output probability distribution of each character in the vocabulary, the entropy corresponding to each character can be calculated, the entropy can be used for measuring the uncertainty of the probability distribution, the uncertainty of the probability distribution of the template text is low, the entropy is also low, the uncertainty of the probability distribution of the variable text is high, and the entropy is also high.
After the cross entropy and the entropy corresponding to each character are obtained through calculation, normalization processing can be performed on the cross entropy and the entropy, and the normalization processing are performed to obtain a first probability corresponding to the character.
For example, given a short message, such as "pick-up code 156 coming", cross entropy and entropy corresponding to each character may be calculated according to probability distribution corresponding to each character in the short message, and the cross entropy and entropy are converted into a value between 0 and 1 by using a normalization method such as min-max, z-score, and the like, and then the two values obtained by normalizing the cross entropy and the entropy are linearly added to finally obtain a value between 0 and 1, which represents a first probability corresponding to the character.
S305, training a template recognition model for template text labeling according to the text training sample and the first probability.
The training process of the template recognition model can be two-stage training as shown in fig. 3B, and the template recognition model can be a patternrecognitionlayer in fig. 3B, which can be a multilayer perceptron mlp, a convolutional neural network model cnn, a long-short term memory network model lstm, or a translation model transformer.
Optionally, in this embodiment, the training, according to the text training sample and the first probability, a template recognition model for performing template text labeling includes: inputting the text training samples into the template recognition model, performing template text labeling on each character in the text training samples through the template recognition model, and outputting a second probability that the character is labeled as a template text; and training the template recognition model according to the difference between the first probability and the second probability, so that the template recognition model can be supervised and trained according to the reconstruction result of the self-encoder.
Referring to fig. 3B, when a short message is given, for example, the pickup code 156 comes quickly, the template recognition model may label template texts of each character of the short message, and output a second probability (0.8, 0.8, 0.9, 0.1, 0.1, 0.1, 0.9, 0.9) corresponding to each character, where the second probability may be a probability that the character is a template text determined by the template recognition model, and then the template recognition model may be supervised and trained according to a difference between the second probability and the first probability.
In the scheme provided by this embodiment, a text training sample is obtained, where the text training sample includes a template text and a variable text; performing text reconstruction on the text training sample by using a self-encoder which is trained in advance to obtain a corresponding reconstruction result; according to the difference between the reconstruction result and the text training sample, obtaining a first probability that each character in the text training sample belongs to the template text; according to the text training samples and the first probabilities, the template recognition model for template text labeling is trained, the scheme provided by the embodiment can determine the probability that each character of the text training samples belongs to the template text by utilizing the reconstruction characteristic of the self-encoder, and accordingly, the template recognition model for template text labeling is trained, the model is not limited to a specific variable type, does not need to label the samples, only needs to collect the text training samples, and is high in universality.
Fig. 4 is a flowchart illustrating steps of a text processing method according to an embodiment of the present application, where the method includes:
s401, obtaining a target text, wherein the target text comprises a template text and a variable text;
the text processing scheme provided in this embodiment may be any text formed according to the template, such as a short message text, or a text sent by chat software, which is not limited in this embodiment.
S402, inputting the target text into a template recognition model.
Wherein, the template recognition model is obtained by training through the method provided by the embodiment.
And S403, carrying out template text labeling on the target text through the template recognition model, and determining the template text in the target text according to a labeling result.
The scheme provided by the embodiment is particularly suitable for the short message platform, and as for the short message platform, a large number of channels of short messages use the short message template, and a large number of short messages do not have template information or the quality of the template information is very low, the short message platform can only manage the granularity of the original short message case, so that a large number of redundant operations are generated, the processing efficiency is low, and the cost is high.
The scheme provided by the embodiment can effectively identify the template used by the short message, is not limited by a short message sample, does not need to carry out template marking, can effectively improve the efficiency and reduce the cost, and has higher universality.
Referring to fig. 5, a schematic structural diagram of an electronic device according to an embodiment of the present application is shown, and the specific embodiment of the present application does not limit a specific implementation of the electronic device.
As shown in fig. 5, the electronic device may include: a processor (processor)502, a Communications Interface 504, a memory 506, and a communication bus 508.
Wherein:
the processor 502, communication interface 504, and memory 506 communicate with one another via a communication bus 508.
A communication interface 504 for communicating with other electronic devices or servers.
The processor 502 is configured to execute the program 510, and may specifically perform relevant steps in the foregoing text processing method embodiment.
In particular, program 510 may include program code that includes computer operating instructions.
The processor 502 may be a CPU, or an Application Specific Integrated Circuit (ASIC), or one or more Integrated circuits configured to implement embodiments of the present application. The intelligent device comprises one or more processors which can be the same type of processor, such as one or more CPUs; or may be different types of processors such as one or more CPUs and one or more ASICs.
And a memory 506 for storing a program 510. The memory 506 may comprise high-speed RAM memory, and may also include non-volatile memory (non-volatile memory), such as at least one disk memory.
The program 510 may be specifically configured to enable the processor 502 to execute operations corresponding to the text processing method described in any of the foregoing method embodiments.
The program 510 may specifically be configured to cause the processor 502 to perform the following steps: obtaining a text training sample, wherein the text training sample comprises a template text and a variable text; performing text reconstruction on the text training sample by using a self-encoder which is trained in advance to obtain a corresponding reconstruction result; according to the difference between the reconstruction result and the text training sample, obtaining a first probability that each character in the text training sample belongs to the template text; and training a template recognition model for template text labeling according to the text training sample and the first probability.
Optionally, in any embodiment of the present application, the performing text reconstruction on the text training sample by using a self-encoder that is trained in advance to obtain a corresponding reconstruction result includes:
reconstructing a plurality of characters included in the text training sample by using a pre-trained self-encoder to obtain reconstruction results corresponding to the characters respectively;
the obtaining, according to the difference between the reconstruction result and the text training sample, the probability that each character in the reconstruction result belongs to the template text includes:
and determining a first probability of each character in the text training sample belonging to the template text according to the difference between the reconstruction result corresponding to each character and the characters included in the text training sample.
Optionally, in any embodiment of the present application, the determining, according to a difference between a reconstruction result corresponding to each of the multiple characters and the multiple characters included in the text training sample, a first probability that each character in the text training sample belongs to the template text includes:
outputting the probability distribution of each character in the reconstruction result under the vocabulary through the self-encoder;
determining difference information of each character in the reconstruction result and the character of the text training sample according to the probability distribution of each character in the reconstruction result under the vocabulary;
and calculating to obtain a first probability that each character in the text training sample belongs to the template text according to the difference information.
Optionally, in any embodiment of the present application, the determining, according to the probability distribution of each character in the reconstruction result under the vocabulary, difference information between each character in the reconstruction result and the character of the text training sample includes:
at least calculating to obtain cross entropies corresponding to the characters in the reconstruction result according to the probability distribution of the characters in the reconstruction result under the vocabulary;
the calculating, according to the difference information, a first probability that each character in the text training sample belongs to the template text includes:
and carrying out normalization processing on the cross entropy to obtain a first probability that each character in the text training sample belongs to the template text.
Optionally, in any embodiment of the present application, the self-encoder performs pre-training using the text training samples.
Optionally, in any embodiment of the present application, the training, according to the text training sample and the first probability, a template recognition model for template text labeling includes:
inputting the text training samples into the template recognition model, performing template text labeling on each character in the text training samples through the template recognition model, and outputting a second probability that the character is labeled as a template text;
training the template recognition model according to a difference between the first probability and the second probability.
Optionally, in any embodiment of the present application, the template recognition model is a multilayer perceptron mlp, a convolutional neural network model cnn, a long-short term memory network model lstm, or a translation model transformer; or,
the self-encoder comprises an encoder and a decoder, wherein the encoder is a convolutional neural network model cnn, a long-short term memory network model lstm or a translation model transform, and the decoder is the long-short term memory network model lstm or the translation model transform.
Alternatively, the program 510 may specifically be configured to cause the processor 502 to perform the following steps: obtaining a target text, wherein the target text comprises a template text and a variable text; inputting the target text into a template recognition model, wherein the template recognition model is obtained by training through the method; and carrying out template text labeling on the target text through the template recognition model, and determining the template text in the target text according to a labeling result.
For specific implementation of each step in the program 510, reference may be made to corresponding steps and corresponding descriptions in units in the foregoing method embodiments, and corresponding beneficial effects are provided, which are not described herein again. It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described devices and modules may refer to the corresponding process descriptions in the foregoing method embodiments, and are not described herein again.
The embodiment of the present application further provides a computer program product, which includes a computer instruction, where the computer instruction instructs a computing device to execute an operation corresponding to any text processing method in the foregoing method embodiments.
It should be noted that, according to implementation needs, each component/step described in the embodiment of the present application may be divided into more components/steps, and two or more components/steps or partial operations of the components/steps may also be combined into a new component/step to achieve the purpose of the embodiment of the present application.
The above-described methods according to embodiments of the present application may be implemented in hardware, firmware, or as software or computer code storable in a recording medium such as a CD ROM, a RAM, a floppy disk, a hard disk, or a magneto-optical disk, or as computer code originally stored in a remote recording medium or a non-transitory machine-readable medium downloaded through a network and to be stored in a local recording medium, so that the methods described herein may be stored in such software processes on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware such as an ASIC or FPGA. It will be appreciated that a computer, processor, microprocessor controller, or programmable hardware includes memory components (e.g., RAM, ROM, flash memory, etc.) that can store or receive software or computer code that, when accessed and executed by a computer, processor, or hardware, implements the methods described herein. Further, when a general-purpose computer accesses code for implementing the methods illustrated herein, execution of the code transforms the general-purpose computer into a special-purpose computer for performing the methods illustrated herein.
Those of ordinary skill in the art will appreciate that the various illustrative elements and method steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the technical solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the embodiments of the present application.
The above embodiments are only used for illustrating the embodiments of the present application, and not for limiting the embodiments of the present application, and those skilled in the relevant art can make various changes and modifications without departing from the spirit and scope of the embodiments of the present application, so that all equivalent technical solutions also belong to the scope of the embodiments of the present application, and the scope of patent protection of the embodiments of the present application should be defined by the claims.

Claims (10)

1. A text processing method, comprising:
obtaining a text training sample, wherein the text training sample comprises a template text and a variable text;
performing text reconstruction on the text training sample by using a self-encoder which is trained in advance to obtain a corresponding reconstruction result;
according to the difference between the reconstruction result and the text training sample, obtaining a first probability that each character in the text training sample belongs to the template text;
and training a template recognition model for template text labeling according to the text training sample and the first probability.
2. The method of claim 1, wherein the performing text reconstruction on the text training samples by using a pre-trained self-encoder to obtain corresponding reconstruction results comprises:
reconstructing a plurality of characters included in the text training sample by using a pre-trained self-encoder to obtain reconstruction results corresponding to the characters respectively;
the obtaining the probability that each character in the reconstruction result belongs to the template text according to the difference between the reconstruction result and the text training sample comprises:
and determining a first probability that each character in the text training sample belongs to the template text according to the difference between the reconstruction result corresponding to each character and the characters included in the text training sample.
3. The method of claim 2, wherein the determining a first probability that each character in the text training sample belongs to the template text according to a difference between a reconstruction result corresponding to each of the characters and a plurality of characters included in the text training sample comprises:
outputting probability distribution of each character in the reconstruction result under a vocabulary through the self-encoder;
determining difference information of each character in the reconstruction result and the character of the text training sample according to the probability distribution of each character in the reconstruction result under the vocabulary;
and calculating to obtain a first probability that each character in the text training sample belongs to the template text according to the difference information.
4. The method of claim 3, wherein the determining difference information between each character in the reconstruction result and the character of the text training sample according to the probability distribution of each character in the reconstruction result under the vocabulary comprises:
at least calculating to obtain cross entropies corresponding to the characters in the reconstruction result according to the probability distribution of the characters in the reconstruction result under the vocabulary;
the calculating, according to the difference information, a first probability that each character in the text training sample belongs to the template text includes:
and carrying out normalization processing on the cross entropy to obtain a first probability that each character in the text training sample belongs to the template text.
5. The method of claim 1, wherein the self-encoder is pre-trained using the text training samples.
6. The method of claim 1, wherein training a template recognition model for template text labeling according to the text training samples and the first probability comprises:
inputting the text training samples into the template recognition model, performing template text labeling on each character in the text training samples through the template recognition model, and outputting a second probability that the character is labeled as a template text;
training the template recognition model according to a difference between the first probability and the second probability.
7. The method according to claim 1, wherein the template recognition model is a multilayer perceptron mlp, a convolutional neural network model cnn, a long-short term memory network model lstm, or a translation model transformer; or,
the self-encoder comprises an encoder and a decoder, wherein the encoder is a convolutional neural network model cnn, a long-short term memory network model lstm or a translation model transform, and the decoder is the long-short term memory network model lstm or the translation model transform.
8. A text processing method, comprising:
obtaining a target text, wherein the target text comprises a template text and a variable text;
inputting the target text into a template recognition model, wherein the template recognition model is obtained by training according to the method of any one of claims 1-8;
and carrying out template text labeling on the target text through the template recognition model, and determining the template text in the target text according to a labeling result.
9. An electronic device, comprising: the system comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete mutual communication through the communication bus;
the memory is used for storing at least one executable instruction, and the executable instruction causes the processor to execute the operation corresponding to the text processing method according to any one of claims 1-8.
10. A computer storage medium having stored thereon a computer program which, when executed by a processor, carries out the method of any one of claims 1 to 8.
CN202210816105.9A 2022-07-12 2022-07-12 Text processing method, electronic device and computer storage medium Active CN115062613B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210816105.9A CN115062613B (en) 2022-07-12 2022-07-12 Text processing method, electronic device and computer storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210816105.9A CN115062613B (en) 2022-07-12 2022-07-12 Text processing method, electronic device and computer storage medium

Publications (2)

Publication Number Publication Date
CN115062613A true CN115062613A (en) 2022-09-16
CN115062613B CN115062613B (en) 2024-09-06

Family

ID=83205366

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210816105.9A Active CN115062613B (en) 2022-07-12 2022-07-12 Text processing method, electronic device and computer storage medium

Country Status (1)

Country Link
CN (1) CN115062613B (en)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107493370A (en) * 2016-06-12 2017-12-19 阿里巴巴集团控股有限公司 Traffic Profile determines method, flow information recognition methods and device
WO2020143321A1 (en) * 2019-01-09 2020-07-16 平安科技(深圳)有限公司 Training sample data augmentation method based on variational autoencoder, storage medium and computer device
CN113362811A (en) * 2021-06-30 2021-09-07 北京有竹居网络技术有限公司 Model training method, speech recognition method, device, medium and equipment
CN113610151A (en) * 2021-08-05 2021-11-05 哈尔滨理工大学 System, method, equipment and storage medium for solving small sample image classification based on prototype network self-encoder network mechanism
CN114386480A (en) * 2021-12-09 2022-04-22 南方科技大学 Training method, application method, device and medium of video content description model
US20220130163A1 (en) * 2022-01-07 2022-04-28 Infrrd Inc System for template invariant information extraction
CN114692569A (en) * 2020-12-31 2022-07-01 阿里巴巴集团控股有限公司 Sentence generation method and device

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107493370A (en) * 2016-06-12 2017-12-19 阿里巴巴集团控股有限公司 Traffic Profile determines method, flow information recognition methods and device
WO2020143321A1 (en) * 2019-01-09 2020-07-16 平安科技(深圳)有限公司 Training sample data augmentation method based on variational autoencoder, storage medium and computer device
CN114692569A (en) * 2020-12-31 2022-07-01 阿里巴巴集团控股有限公司 Sentence generation method and device
CN113362811A (en) * 2021-06-30 2021-09-07 北京有竹居网络技术有限公司 Model training method, speech recognition method, device, medium and equipment
CN113610151A (en) * 2021-08-05 2021-11-05 哈尔滨理工大学 System, method, equipment and storage medium for solving small sample image classification based on prototype network self-encoder network mechanism
CN114386480A (en) * 2021-12-09 2022-04-22 南方科技大学 Training method, application method, device and medium of video content description model
US20220130163A1 (en) * 2022-01-07 2022-04-28 Infrrd Inc System for template invariant information extraction

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
段誉: "一种基于预训练模型的条件文本生成框架", 中国硕士学位论文全文数据库, 15 September 2019 (2019-09-15) *

Also Published As

Publication number Publication date
CN115062613B (en) 2024-09-06

Similar Documents

Publication Publication Date Title
US11741392B2 (en) Data sample label processing method and apparatus
CN109522304B (en) Abnormal object identification method and device and storage medium
WO2021189844A1 (en) Detection method and apparatus for multivariate kpi time series, and device and storage medium
CN108197652A (en) For generating the method and apparatus of information
CN111583911B (en) Speech recognition method, device, terminal and medium based on label smoothing
CN113240510B (en) Abnormal user prediction method, device, equipment and storage medium
CN111091278A (en) Edge detection model construction method and device for mechanical equipment anomaly detection
CN111368551B (en) Method and device for determining event main body
CN115496970A (en) Training method of image task model, image recognition method and related device
CN116628163A (en) Customer service processing method, customer service processing device, customer service processing equipment and storage medium
CN112948582B (en) Data processing method, device, equipment and readable medium
CN115062613B (en) Text processing method, electronic device and computer storage medium
CN111199454B (en) Real-time user conversion evaluation method and device and electronic equipment
CN115758211B (en) Text information classification method, apparatus, electronic device and storage medium
CN116127925B (en) Text data enhancement method and device based on destruction processing of text
CN111353039B (en) File category detection method and device
CN113259145A (en) End-to-end networking method of network slice, networking device and network slice equipment
CN116108810A (en) Text data enhancement method and device
CN115964999A (en) Model training and text generation method and device, electronic equipment and storage medium
CN114117051A (en) Training method of part-of-speech tagging model, part-of-speech tagging method and electronic equipment
CN115705464A (en) Information processing method, device and equipment
CN111310460B (en) Statement adjusting method and device
CN107609645B (en) Method and apparatus for training convolutional neural network
CN112036418A (en) Method and device for extracting user features
CN113066038A (en) Image evaluation method and device, electronic equipment and computer storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant