CN115905543A - Text processing method and device - Google Patents

Abstract

The application provides a text processing method and a device thereof, wherein the text processing method comprises the following steps: acquiring a text to be processed, and segmenting the text to be processed to obtain a plurality of text segments; inputting a plurality of text segments into a coding layer of a pre-trained label determination model for coding to obtain a first coding vector of the text segments; inputting the first coding vector into a sensor of a label determination model to determine a label, and obtaining a first label score of a corresponding text segment in a plurality of preset labels; and performing pooling treatment on the pooling layers of the first score input label determination models to obtain target labels of the texts to be treated, so as to accurately determine the labels of the long texts.

Description

Text processing method and device

Technical Field

The present application relates to the field of computer technologies, and in particular, to a text processing method and apparatus.

Background

Text classification is a process of determining the label of a text based on the content of the text, for example, the text is "the taste of this food is good" and the label of the text is "good comment".

The current scheme is to train a neural network model by using artificially labeled training data, and classify and process a short text by using the neural network model to obtain a label of the short text, but the neural network model cannot accurately determine the label of the long text.

Disclosure of Invention

Aspects of the present application provide a text processing method and apparatus thereof to achieve accurate determination of a label of a long text.

A first aspect of an embodiment of the present application provides a text processing method, including: acquiring a text to be processed, and segmenting the text to be processed to obtain a plurality of text segments; inputting a plurality of text segments into a coding layer of a pre-trained label determination model for coding to obtain a first coding vector of the text segments; inputting the first coding vector into a sensor of a label determination model to determine a label, and obtaining a first label score of a corresponding text segment in a plurality of preset labels; and performing pooling treatment on the pooling layers of the first score input label determination models to obtain target labels of the texts to be treated.

A second aspect of the embodiments of the present application provides a text processing method, which is applied to a terminal device, and the text processing method includes: acquiring a text to be processed; sending a text to be processed to a server; and receiving a target label sent by the server, wherein the target label is obtained after the server processes the target label according to the text processing method of the first aspect.

A third aspect of the embodiments of the present application provides a text processing method, which is applied to a text processing system, where the text processing system includes: the text processing method comprises the following steps: the terminal equipment acquires a text to be processed and sends the text to be processed to the server; the server cuts the text to be processed to obtain a plurality of text segments; inputting a plurality of text segments into a coding layer of a pre-trained label determination model for coding to obtain a first coding vector of the text segments; inputting the first coding vector into a sensor of a label determination model for label determination to obtain a first label score of the corresponding text segment in a plurality of preset labels; performing pooling treatment on the pooling layers of the plurality of first score input label determination models to obtain target labels of texts to be treated; and the terminal equipment receives the target label sent by the server.

A fourth aspect of the present embodiment provides a text processing apparatus, including:

the segmentation module is used for acquiring a text to be processed and segmenting the text to be processed to obtain a plurality of text segments;

the encoding module is used for inputting a plurality of text segments into an encoding layer of a pre-trained label determination model for encoding processing to obtain a first encoding vector of the text segment;

the first processing module is used for inputting the first coding vector into a sensor of the label determination model for label determination to obtain a first label score of the corresponding text segment in a plurality of preset labels;

and the second processing module is used for performing pooling processing on the pooling layers of the plurality of first score input label determination models to obtain target labels of the texts to be processed.

A fifth aspect of the embodiments of the present application provides a text processing system, including:

the system comprises a cloud server and terminal equipment, wherein a pre-trained label determination model is deployed on the cloud server;

the terminal equipment is used for acquiring the text to be processed and sending the text to be processed to the server;

the cloud server is used for segmenting the text to be processed to obtain a plurality of text segments; inputting a plurality of text segments into a coding layer of a pre-trained label determination model for coding to obtain a first coding vector of the text segments; inputting the first coding vector into a sensor of a label determination model to determine a label, and obtaining a first label score of a corresponding text segment in a plurality of preset labels; performing pooling treatment on the pooling layers of the plurality of first score input label determination models to obtain target labels of texts to be treated;

and the terminal equipment is used for receiving the target label sent by the server.

A sixth aspect of the embodiments of the present application provides an electronic device, including: a processor, a memory and a computer program stored on the memory and executable on the processor, the processor implementing the text processing method according to any one of the first to third aspects when executing the computer program.

A seventh aspect of embodiments of the present application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to implement the text processing method according to any one of the first to third aspects.

The method and the device are applied to a classification scene of a long text, and a plurality of text segments are obtained by acquiring the text to be processed and segmenting the text to be processed; inputting a plurality of text segments into a coding layer of a pre-trained label determination model for coding to obtain a first coding vector of the text segment; inputting the first coding vector into a sensor of a label determination model for label determination to obtain a first label score of the corresponding text segment in a plurality of preset labels; and performing pooling treatment on the pooling layers of the plurality of first score input label determination models to obtain target labels of the texts to be treated, so as to accurately determine the labels of the long texts.

Drawings

The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:

fig. 1 is a diagram of an application scenario provided in an exemplary embodiment of the present application;

FIG. 2 is a flowchart illustrating steps of a method for processing text according to an exemplary embodiment of the present application;

fig. 3 is a process diagram of a text processing method according to an exemplary embodiment of the present application;

FIG. 4 is a flowchart illustrating steps of another method for processing text in accordance with an exemplary embodiment of the present application;

FIG. 5 is a process diagram of another text processing method provided in an exemplary embodiment of the present application;

FIG. 6 is a flowchart illustrating steps of yet another method for processing text in accordance with an exemplary embodiment of the present application;

fig. 7 is a block diagram of a text processing apparatus according to an exemplary embodiment of the present application;

fig. 8 is a schematic structural diagram of an electronic device according to an exemplary embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions of the present application will be described in detail and completely with reference to the following specific embodiments of the present application and the accompanying drawings. It should be apparent that the described embodiments are only a few embodiments of the present application, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

Text classification is a basic task in natural language processing. The method generally adopted at present is to determine the label of the short text based on a neural network model of a transformer. Among them, due to the time complexity of the transform, when long text classification is performed using a transform-based Neural Network model, the limitation of text length is an unavoidable problem, and the existing method is usually to encode the long text first and then use an attention or Recurrent Neural Network (RNN) to obtain the embedding of the whole input text for the determination of the label. Where attention or recurrent neural networks do not enable accurate determination of the label.

Based on the above problems, a simple but effective classifier, called a perceptron, is proposed in the present application to replace the role of attention or RNN in the above method. Due to the mathematical characteristics of the sensor, the sensor can learn not only the labels of the entire text in the supervised training process, but also the labels of the text segments and their contribution to the entire text labels in an unsupervised manner. The perceptron is a general classifier which does not distinguish different encoders, the perceptron is superior to or obtains the effect equivalent to the current best-of-the-art model (SOTA) in classification accuracy and model efficiency, and the perceptron also shows the interpretability superior to the classification result (label) of long text.

In the present embodiment, the execution device of the text processing method is not limited. Alternatively, the text processing method may implement the entire text processing method by means of a cloud computing system. For example, text processing methods may be applied to a cloud server to run various models by virtue of resources on the cloud; compared with the application to the cloud, the text processing method can also be applied to server-side equipment such as a conventional server, a cloud server or a server array.

In addition, referring to fig. 1, an application scenario of the present application is illustrated. Inputting a long text to be processed into a label determination model, wherein the long text is composed of a segment text A, a segment text B, a segment text C and a segment text D, and the label of the long text and the segment text explaining the label, such as the segment text B, can be obtained through the label determination model. In the embodiment of the application, the provided tag determination model not only can accurately determine the tags of long texts, but also can accurately determine the tags of short texts.

Specifically, the terminal device is used for acquiring a text to be processed and sending the text to be processed to the server; the cloud server is used for segmenting the text to be processed to obtain a plurality of text segments; inputting a plurality of text segments into a coding layer of a pre-trained label determination model for coding to obtain a first coding vector of the text segments; inputting the first coding vector into a sensor of a label determination model to determine a label, and obtaining a first label score of a corresponding text segment in a plurality of preset labels; performing pooling treatment on the pooling layers of the plurality of first score input label determination models to obtain target labels of texts to be treated; and the terminal equipment is used for receiving the target label sent by the server.

In addition, the embodiment of the application can be applied to the fields of judicial arts, medical fields and the like, and the fields have higher requirements on the interpretability of the label. Fig. 1 is only an exemplary application scenario of the present application, and the present application may also be applied to other text classification scenarios, which are not limited herein.

The technical solutions provided by the embodiments of the present application are described in detail below with reference to the accompanying drawings.

Fig. 2 is a flowchart illustrating steps of a text processing method according to an exemplary embodiment of the present application. As shown in fig. 2, the text processing method specifically includes the following steps:

s201, obtaining a text to be processed, and segmenting the text to be processed to obtain a plurality of text segments.

In the embodiment of the present application, the text to be processed may be a long text or a short text. For example, in the judicial field, with litigation requests and factual justifications for a plaintiff and awarded answer text as pending text, determining target tags for the pending text may be more convenient for a judge to understand disputes that exist for the litigation.

In the embodiment of the present application, referring to fig. 3, for example, the text r to be processed is divided into text segments S1 to Sn by a text divider, where n is a positive integer.

Specifically, the text segmenter may perform segmentation on the text to be processed to obtain a plurality of text segments, including: segmenting a text to be processed by adopting at least one of the following modes to obtain a plurality of text segments: dividing the text to be processed by adopting a sliding window with a preset text size; segmenting the text to be processed based on punctuation marks in the text to be processed; and segmenting the text to be processed based on the text structure of the text to be processed.

In this embodiment of the present application, the preset text size may refer to the number of characters, and the sliding window is moved in sequence to obtain a plurality of text segments after segmentation. Segmenting the text to be processed based on punctuation marks in the text to be processed, wherein the punctuation marks can be preset punctuation marks such as semicolons; or period. ". In addition, the text structure is that the current dialog text of one party in the dialog scene is a text fragment, or a pair of dialog texts is a text fragment.

Illustratively, the pending text is "the original XX asks the court for litigation: 1. the paid commodity YY payment is 100 ten thousand yuan; 2. the subject is paid an overdue payment default (on a 100-ten-thousand basis, three-thousandths of a day). The reason is stated as follows: contractual provisions with the defendant, and so on. The quilt answers: the payment of the commodity YY is completed before 12 and 31 months in 2019, but the default money cannot be paid, and the reason is that. ". If the text is divided according to the sliding window and the size of the preset text is 20 characters, the text segment S1 obtained by the division is' the original report XX makes a litigation request to the court: 1. the reported payer ' S ' and the text segment S2 are ' article YY payment 100 ten thousand yuan; 2. the subject pays an overdue payment violation, "text segment S3" about deposit (on a base of 100 tens of thousands, by three thousandths of a day). Statement, "text segment S4 is" consisting of: contractual agreement with the defended subject, etc. The quilt answers are as follows: the reason why the good of the commodity YY and the text segment S5 are "money will be paid out before 31 days 12 and 12 in 2019 but the default sum of money is lost" and the text segment S7 is "payment cannot be made" are "to" is "to. If the text segment S1 is divided according to punctuation marks, such as semicolon and sentence number, the text segment S1 obtained by the division is' original report XX to make a litigation request to the court: 1. the paid commodity YY payment is 100 ten thousand yuan; "; text segment S2 is "2. The reported paid overdue payment default fund (based on 100 ten thousand, three thousandths of a day)"; the text passage S3 is "statement reason: contractual provisions with the defendant, etc.); text segment S4 is "dismissal: the payment of the commodity YY is completed before 31 days 12 months in 2019, but the default money cannot be paid, and the reason is- ". If the text is divided according to the text structure, the text segment S1 obtained by the division is 'original report XX' and makes a litigation request to the court: 1. the paid commodity YY payment is 100 ten thousand yuan; 2. the subject pays an overdue payment default (on a 100-ten-thousand basis, three thousandths of a day). The text passage S2 is "statement reason: contractual agreement with the defended, etc. ". Text segment S3 is "dismissal: the payment of the commodity YY is completed before 12 and 31 months in 2019, but the default money cannot be paid, and the reason is 'yes'.

In the embodiment of the application, the text to be processed of the long text is divided into the text segments of the plurality of short texts, so that the target label of the text to be processed can be determined conveniently.

S202, inputting a plurality of text segments into a coding layer of a pre-trained label determination model for coding, and obtaining a first coding vector of the text segments.

Referring to fig. 3, in an embodiment, the coding layer includes an encoder and a collection layer, and the second coded vector obtained by the collection layer may be determined as the first coded vector. In another embodiment, the encoding layer includes an encoder, a convergence layer, and an attention layer, wherein a vector output by the attention layer is a first encoded vector.

In the embodiment of the application, the text segment is correspondingly encoded to obtain a first encoding vector. For example, referring to FIG. 3, a text segment S1 is encoded by the encoding layer to obtain a first encoded vector S' 1. The text segment S2 is encoded by the encoding layer to obtain a first encoding vector S ″ 2. The text segment Sn is coded by a coding layer to obtain a first coding vector S' n.

S203, inputting the first coding vector into a sensor of the label determination model to determine the label, and obtaining a first label score of the corresponding text segment in a plurality of preset labels.

In the embodiment of the application, the sensor has a linear calculation formula for the label. Wherein, the linear calculation formula is the following formula (1):

Z＝W _i x+b _i formula (1)

Wherein Z represents a second score, W _i And b _i Denotes a parameter associated with the tag i and x denotes the first encoding vector. Wherein, W _i Either pre-trained parameters or the coded vector of label i. The second score may be determined as the first score.

For example, four tags a, B, C are preset. The first encoding vectors each have a corresponding first score (second score) for the tag. Exemplarily, referring to table 1, if there are 3 text segments, there are 3 first encoding vectors.

TABLE 1

In the embodiment of the present application, the second score Z may be directly used as the first score.

And S204, performing pooling treatment on the pooling layers of the plurality of first score input label determination models to obtain target labels of the texts to be treated.

Wherein the following steps are performed in the pooling layer: for a label in a preset plurality of labels, determining a third score of the plurality of text segments under the label according to a first score of the plurality of text segments, wherein the sum of the plurality of first scores is the third score, or a first score (such as the largest first score) in the plurality of first scores is the third score; and if the third score is greater than or equal to the score threshold value, determining that the label is the target label.

In the embodiment of the present application, the pooling layer may select a maximum pooling layer or a total pooling layer. For each first encoded vector, a third score under the label is determined. Specifically, the maximum pooling layer, the third score is determined using the following equation (2):

y _i ＝max{Z _i1 、Z _i2 、...、Z _in formula (2)

Illustratively, referring to Table 1, if n is 3 _A1 ＞Z _A2 ＞Z _A3 Then y is _A ＝Z _A1 。Z _B2 ＞Z _B3 ＞Z _B1 Then y is _B ＝Z _B2 。Z _C3 ＞Z _C1 ＞Z _C2 Then y is _C ＝Z _C3 。

Further, the pooling layer is summed, and the third score is determined using the following equation (3):

illustratively, referring to Table 1, if n is 3 _A ＝Z _A1 +Z _A2 +Z _A3 。y _B ＝Z _B1 +Z _B2 +Z _B3 。y _C ＝Z _C1 +Z _C2 +Z _C3 。

And further, determining a target label of the text to be processed according to the third score in the pooling layer. In particular, if the third fraction y _i If the score is more than 0, the target label of the text to be processed is label i, and if the third score y is greater than the first score, the target label of the text to be processed is label i _i And if the number is less than or equal to 0, the target label of the text to be processed is not the label i. Illustratively, if y _A ＞0，y _B And y _C If the target tags are all less than 0, the target tag of the text to be processed can be determined to be a.

In the embodiment of the present application, there may be a plurality of target tags of the text to be processed. The method and the device for marking the training texts can avoid the problem that the training texts need to be marked and trained in the related technology and the workload of marking is large. In addition, the target label of the long text can be accurately determined by adopting the sensor. In this context, we propose a simple but effective classifier, called perceptron, which is able to classify the text to be processed without length constraints.

In addition, the method and the device are applied to a long text classification scene, and a plurality of text segments are obtained by obtaining the text to be processed and segmenting the text to be processed; inputting a plurality of text segments into a coding layer of a pre-trained label determination model for coding to obtain a first coding vector of the text segments; inputting the first coding vector into a sensor of a label determination model to determine a label, and obtaining a first label score of a corresponding text segment in a plurality of preset labels; and performing pooling treatment on the pooling layers of the first score input label determination models to obtain target labels of the texts to be treated, so as to accurately determine the labels of the long texts.

Referring to fig. 4, a flowchart illustrating steps of another text processing method according to an exemplary embodiment of the present application is provided. As shown in fig. 4, the text processing method specifically includes the following steps:

s401, obtaining a text to be processed, and segmenting the text to be processed to obtain a plurality of text segments.

The specific implementation process of this step is referred to as S201, and is not described herein again.

S402, aiming at the text segment in the text segments, inputting the text segment into an encoder for encoding processing to obtain a character vector of characters in the text segment.

Wherein, referring to fig. 3, the coding layer includes: an encoder, an aggregation layer, and an attention layer. The encoding layer is composed of an embedding layer (embedding) and a plurality of converters, and the text segment can obtain a character vector of characters in the text segment through the encoder. For example, the text segment S1 passes through the encoder, resulting in a word vector S11 to a word vector S1m, where m represents the number of words in the text segment S1. The text segment S2 goes through the encoder to obtain a word vector S21 to a word vector S2p, where p represents the number of words in the text segment S2. And the text segment Sn passes through an encoder to obtain a text vector Sn1 to a text vector SnQ, wherein Q represents the number of characters in the text segment Sn.

And S403, inputting a plurality of character vectors belonging to the same text segment into a gathering layer for combination processing to obtain a second encoding vector of the text segment.

The collection layer combines a plurality of character vectors of the same text segment according to the sequence to obtain a second coding vector of the text segment. For example, in fig. 3, the second coded vector S '1 is obtained by combining the literal vector S11 to the literal vector S1m, the second coded vector S '2 is obtained by combining the literal vector S21 to the literal vector S2p, and the second coded vector S ' n is obtained by combining the literal vector Sn1 to the literal vector SnQ.

In the embodiments of the present application, one way is to use the second encoded vector as the first encoded vector of the input perceptron.

S404, inputting the plurality of second coding vectors into the attention layer to carry out correlation processing among different second coding vectors, and obtaining first coding vectors of the text segments.

Wherein, the attention layer is an optional layer in the coding layer and is composed of one or more transformers to establish the association between the text segments.

Illustratively, referring to fig. 3, each second encoded vector may be input to the attention layer, resulting in a plurality of first encoded vectors, where the first encoded vector S ″ 1 corresponds to the second encoded vector S'1, which is the encoded vector of the text segment S1. The first encoded vector S ″ 2 and the second encoded vector S'2 correspond to each other and are the encoded vectors of the text segment S2. The first encoding vector S ″ n and the second encoding vector S' n correspond to each other and are encoding vectors of the text segment Sn.

S405, inputting the first coding vector into a label prediction layer for label determination, and obtaining a second fraction of labels of the text segment in a plurality of preset labels.

Wherein, referring to fig. 3, the sensor comprises: a label prediction layer. Wherein, the second score is calculated as described in S203, and the obtained Z is the second score obtained by the label prediction layer.

S406, determining the second score as the first score.

Wherein the second score Z may be taken as the first score.

In an optional embodiment, referring to fig. 3, the sensor further includes a weight prediction layer, and the inputting the first coded vector into the sensor for tag determination to obtain a first tag score of the corresponding text segment in a preset plurality of tags includes: inputting the first coding vector into a weight prediction layer to perform weight determination to obtain the weight of a label of the text segment in a plurality of preset labels; and determining the product of the second score and the weight as the first score of the label of the text segment in the preset plurality of labels.

In the weight prediction layer, the labels all have corresponding weight calculation formulas, and the weight calculation formulas are as the following formula (4):

in the above-mentioned formula, the compound has the following structure,

indicating the weight of the kth first code vector K under the label i. σ is the activation function. />

And &>

Indicating the weight parameter corresponding to the label i. Exemplarily, with reference to table 2:

TABLE 2

In this embodiment, the product of the second score and the weight may be the first score. The first score obtained may be

In the embodiment of the present application, the sensor may be a standard sensor, i.e. a one-dimensional sensor, and then the formula (1) "Z = W" is adopted _i x+b _i "when determining the second score, x is the first code vector. Using formula (4)

And when the weight of the first code vector is determined, K is the first code vector.

In addition, the sensor comprises a multi-dimensional sensor, the first coding vector is input into the label prediction layer to determine the label, and the second fraction of the label of the text segment in the preset plurality of labels is obtained, and the method comprises the following steps: inputting the subvector into a label prediction layer for label determination aiming at each dimension subvector in the first coding vector to obtain a fourth fraction of labels of the subvector in a plurality of preset labels; and determining a fourth score as a second score of the text segment under the label aiming at the label and the first encoding vector.

In this embodiment, the multidimensional sensor is a multidimensional sensor carrying the largest pooling. The fourth fraction may also be determined using the formula (1) "Z = Wix + bi", where x is a sub-vector in the first encoded vector. Wherein S'1 is derived from

The u-dimensional vector component, where u isA positive integer. S'2 by>

The v-dimensional vector is composed, where v is a positive integer. S'2 is selected by>

The w-dimensional vector is composed, where v is a positive integer.

Exemplarily, referring to table 3, as illustrated by label a, the first encoded vectors S "1, S" 2, and S "3 are assumed to be three-dimensional vectors, and each include three subvectors. Wherein, a fourth fraction (e.g. the largest fourth fraction) of a plurality of fourth fractions corresponding to the sub-vector in the first encoding vector is the second fraction when the second fraction is the second fraction. In addition, the determination manner of the second scores corresponding to the tags B and C can refer to table 3, and is not described herein again.

TABLE 3

Illustratively, referring to FIG. 5, for each one-dimensional subvector, use is made of

Calculating the weight of each one-dimensional sub-vector, adding the weights of a plurality of sub-vectors of the same first coding vector, multiplying the weights by a normalization parameter sigma to obtain the weight of the first coding vector, multiplying the weight of the first coding vector by the second score of the first coding to obtain the first score of the first coding vector, adding the first scores of the first coding vectors to obtain the third score of the first coding vector under a label, and determining whether the text to be processed belongs to the label i or not through the third score.

S407, for the text segment, determining a label corresponding to a first score in the plurality of first scores as a segment label.

Referring to table 1, a first encoding vector corresponding to a text segment has a different first score (second score) under different labels, where a label corresponding to a first score (e.g., the highest first score) is determined to be a segment label of the text segment.

Illustratively, in table 1, the first encoding vector of the text segment S1 is S ″ 1, wherein the first score of the first encoding vector under the label a is Z _A1 . The first fraction of the first code vector under the label B is Z _B1 . The first fraction of the first code vector under the label C is Z _C1 . If Z is _A1 ＜Z _B1 ＜Z _C1 If the segment label of the text segment S1 is determined to be a, the segment label of the text segment S2 is determined to be B.

And S408, aiming at the fragment tags in the plurality of fragment tags, if the fragment tags are the same as the target tags, determining the text fragments belonging to the fragment tags as the target text fragments.

Wherein the target text segment is used for interpreting the target label. In this embodiment of the application, if the target tag is a, and the segment tag of the text segment S2 is also a, it may be determined that the text segment S2 is the target segment, and the reason that the target tag of the text to be processed is the tag a may be explained by using the text segment S2.

In the embodiment of the present application, there may be a plurality of target text segments. In addition, the application can learn key fragments by itself with good interpretability using a perceptron. Due to the mathematical nature of the perceptron, it can learn the labels of a text segment and its contribution to the entire text label in an unsupervised manner. As a general classifier which does not distinguish different encoders, the advantages of the sensor in the aspects of classification precision and model efficiency are proved through experiments. Unsupervised interpretable processes have brought about major breakthroughs for long text classification, and can better optimize the performance of the label determination model.

In addition, referring to fig. 6, a flowchart of steps of another text processing method provided in an exemplary embodiment of the present application is applied to a terminal device, and as shown in fig. 6, the text processing method specifically includes the following steps:

s601, acquiring a text to be processed.

Wherein, the text to be processed is input by a user or transmitted by other terminal equipment.

S602, sending the text to be processed to a server.

In the embodiment of the present application, the to-be-processed text is sent to the server, and the server may process the to-be-processed text according to the above embodiment to obtain the target tag and the target text fragment of the to-be-processed text. The target text segment is a part of text in the text to be processed. The target text segment can be used for explaining that the label of the text to be processed is the target label.

S603, receiving the target label sent by the server.

The target label is obtained after the server is processed according to the text processing method. In an embodiment of the present application, the target tag and the target text segment may be displayed to be provided to the user.

In an embodiment of the present application, there is further provided a text processing method applied to a text processing system, where the text processing system includes: the text processing method comprises the following steps: the terminal equipment acquires a text to be processed and sends the text to be processed to the server; the server cuts the text to be processed to obtain a plurality of text segments; inputting a plurality of text segments into a coding layer of a pre-trained label determination model for coding to obtain a first coding vector of the text segments; inputting the first coding vector into a sensor of a label determination model to determine a label, and obtaining a first label score of a corresponding text segment in a plurality of preset labels; performing pooling treatment on the pooling layers of the plurality of first score input label determination models to obtain target labels of texts to be treated; and the terminal equipment receives the target label sent by the server.

For the details of the present application, reference is made to the above embodiments, which are not repeated herein.

In the embodiment of the present application, in addition to providing a text processing method, there is also provided a text processing apparatus, as shown in fig. 7, the text processing apparatus 70 includes:

the segmentation module 71 is configured to obtain a text to be processed, and segment the text to be processed to obtain a plurality of text segments;

the encoding module 72 is configured to input a plurality of text segments into an encoding layer of a pre-trained label determination model to perform encoding processing, so as to obtain a first encoding vector of the text segment;

the first processing module 73 is configured to input the first coding vector into a sensor of the tag determination model to perform tag determination, so as to obtain a tag first score of the corresponding text segment in a plurality of preset tags;

and a second processing module 74, configured to perform pooling processing on the pooling layers of the multiple first score input label determination models to obtain target labels of the texts to be processed.

In an alternative embodiment, the coding layer comprises: encoder, convergence layer and attention layer, encoding module 62 is specifically configured to: inputting the text segments into an encoder for encoding aiming at the text segments in the text segments to obtain character vectors of characters in the text segments; inputting a plurality of character vectors belonging to the same text segment into a collection layer for combination processing to obtain a second coding vector of the text segment; and inputting the plurality of second coding vectors into the attention layer to perform association processing among different second coding vectors to obtain a first coding vector of the text segment.

In an alternative embodiment, the sensor comprises: a label prediction layer, the first processing module 73, is specifically configured to: inputting the first coding vector into a label prediction layer to determine labels, and obtaining a second fraction of labels of the text segment in a plurality of preset labels; determining the second score as the first score.

In an optional embodiment, the sensor further includes a weight prediction layer, and the first processing module 73 is specifically configured to: inputting the first coding vector into a weight prediction layer to perform weight determination to obtain the weight of a label of the text segment in a plurality of preset labels; and determining the product of the second score and the weight as the first score of the label of the text segment in the preset plurality of labels.

In an optional embodiment, the sensor includes a multi-dimensional sensor, and the first processing module 73 is specifically configured to, when a first coded vector is input to the tag prediction layer for tag determination and a second score of a text segment among a plurality of preset tags is obtained, input a sub-vector to the tag prediction layer for tag determination for the sub-vector in the first coded vector, and obtain a fourth score of the sub-vector among the plurality of preset tags; and determining a fourth score as a second score of the text segment under the label aiming at the label and the first encoding vector.

In an optional embodiment, the second processing module 64 is specifically configured to: performing the following steps in the pooling layer: determining a third score of the plurality of text segments under the label according to the first scores of the plurality of text segments aiming at the label in a plurality of preset labels, wherein the sum of the plurality of first scores is the third score, or one first score in the plurality of first scores is the third score; and if the third score is greater than or equal to the score threshold value, determining that the label is the target label.

In an alternative embodiment, the target text segment determination module (not shown): the method comprises the steps of determining a label corresponding to a first score in a plurality of first scores as a segment label aiming at a text segment; for a fragment tag in the plurality of fragment tags, if the fragment tag is the same as the target tag, determining that the text fragment belonging to the fragment tag is the target text fragment, and the target text fragment is used for explaining the target tag.

In an alternative embodiment, the segmentation module 71 is specifically configured to: segmenting a text to be processed by adopting at least one of the following modes to obtain a plurality of text segments: dividing the text to be processed by adopting a sliding window with a preset text size; segmenting the text to be processed based on punctuation marks in the text to be processed; and segmenting the text to be processed based on the text structure of the text to be processed.

Further, the present application provides a document processing apparatus (not shown) comprising:

the acquisition module is used for acquiring a text to be processed;

the sending module is used for sending the text to be processed to the server;

and the receiving module is used for receiving the target label sent by the server, wherein the target label is obtained after the server is processed according to the text processing method.

The text processing device provided by the embodiment of the application can improve the accuracy of determining the long text label. For the specific implementation process, reference is made to the above method embodiments, which are not described herein again.

In addition, in some of the flows described in the above embodiments and the drawings, a plurality of operations are included in a specific order, but it should be clearly understood that the operations may be executed out of order or in parallel as they appear in the present document, and only for distinguishing between the various operations, and the sequence number itself does not represent any execution order. Additionally, the flows may include more or fewer operations, and the operations may be performed sequentially or in parallel. It should be noted that, the descriptions of "first", "second", etc. in this document are used for distinguishing different messages, devices, modules, etc., and do not represent a sequential order, nor limit the types of "first" and "second" to be different.

Fig. 8 is a schematic structural diagram of an electronic device according to an exemplary embodiment of the present application. The electronic device is used for operating the text processing method, and the electronic device can be a cloud device. As shown in fig. 8, the electronic apparatus includes: a memory 84 and a processor 85.

The memory 84 is used for storing computer programs and may be configured to store other various data to support operations on the electronic device. The store 84 may be an Object Storage Service (OSS).

The memory 84 may be implemented by any type or combination of volatile or non-volatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.

A processor 85 coupled to the memory 84 for executing computer programs in the memory 84 for: acquiring a text to be processed, and segmenting the text to be processed to obtain a plurality of text segments; inputting a plurality of text segments into a coding layer of a pre-trained label determination model for coding to obtain a first coding vector of the text segment; inputting the first coding vector into a sensor of a label determination model to determine a label, and obtaining a first label score of a corresponding text segment in a plurality of preset labels; and performing pooling treatment on the pooling layers of the plurality of first score input label determination models to obtain target labels of the texts to be treated.

Further optionally, the coding layer comprises: the encoder, the gathering layer, and the attention layer, when the processor 85 inputs a plurality of text segments into the encoding layer of the pre-trained tag determination model to perform encoding processing, and obtains a first encoding vector of the text segment, the processor is specifically configured to: inputting the text segments into an encoder for encoding aiming at the text segments in the text segments to obtain character vectors of characters in the text segments; inputting a plurality of character vectors belonging to the same text segment into a gathering layer for combination processing to obtain a second coding vector of the text segment; and inputting the plurality of second coding vectors into the attention layer to perform association processing among different second coding vectors to obtain a first coding vector of the text segment.

Further optionally, the sensor comprises: in the label prediction layer, when the processor 85 inputs the first coded vector into the sensor for label determination to obtain a first label score of the corresponding text segment in a plurality of preset labels, the processor is specifically configured to: inputting the first coding vector into a label prediction layer to determine labels, and obtaining a second fraction of labels of the text segment in a plurality of preset labels; determining the second score as the first score.

Further optionally, the sensor further includes a weight prediction layer, and when the processor 85 inputs the first coded vector into the sensor for tag determination to obtain a first tag score of a corresponding text segment in a preset plurality of tags, the processor is further configured to: inputting the first coding vector into a weight prediction layer to perform weight determination to obtain the weight of a label of the text segment in a plurality of preset labels; and determining the product of the second score and the weight as the first score of the label of the text segment in the preset plurality of labels.

Further optionally, the sensor includes a multi-dimensional sensor, and the processor 85 is specifically configured to, when the first encoding vector is input to the tag prediction layer to perform tag determination, and a second score of a tag in the text segment among the plurality of tags is preset, perform: inputting the subvector into a label prediction layer for label determination aiming at the subvector in the first coding vector to obtain a fourth fraction of labels of the subvector in a plurality of preset labels; and determining a fourth score as a second score of the text segment under the label aiming at the label and the first encoding vector.

Further optionally, when the processor 85 inputs the plurality of first scores into the pooling layer for pooling processing to obtain the target tag of the text to be processed, the processor is specifically configured to: performing the following steps in the pooling layer: determining a third score of the plurality of text segments under the label according to the first scores of the plurality of text segments aiming at the label in a plurality of preset labels, wherein the sum of the plurality of first scores is the third score, or one first score in the plurality of first scores is the third score; and if the third score is greater than or equal to the score threshold value, determining that the label is the target label.

Further optionally, the processor 85 is further configured to determine, for a text segment, that a tag corresponding to a first score is a segment tag in the plurality of first scores; for a fragment tag in the plurality of fragment tags, if the fragment tag is the same as the target tag, determining that the text fragment belonging to the fragment tag is the target text fragment, and the target text fragment is used for explaining the target tag.

Further optionally, the processor 85 is configured to segment the text to be processed to obtain a plurality of text segments, and specifically configured to: the method comprises the following steps of segmenting a text to be processed by adopting at least one of the following modes to obtain a plurality of text segments: dividing the text to be processed by adopting a sliding window with a preset text size; segmenting the text to be processed based on punctuation marks in the text to be processed; and segmenting the text to be processed based on the text structure of the text to be processed.

In an alternative embodiment, the processor 85, coupled to the memory 84, is configured to execute the computer program in the memory 84 to further: acquiring a text to be processed; sending a text to be processed to a server; and receiving a target label sent by the server, wherein the target label is obtained after the server processes the target label according to any one of the text processing methods.

Further, as shown in fig. 8, the electronic device further includes: firewall 81, load balancer 82, communications component 86, power component 78, and other components. Only some of the components are schematically shown in fig. 8, and the electronic device is not meant to include only the components shown in fig. 8.

Accordingly, the present application also provides a computer readable storage medium storing a computer program, which when executed by a processor causes the processor to implement the steps of the above-mentioned method.

Accordingly, embodiments of the present application also provide a computer program product, which includes computer programs/instructions, when executed by a processor, cause the processor to implement the steps in the above-described illustrated method.

The communication component of fig. 8 described above is configured to facilitate communication between the device in which the communication component is located and other devices in a wired or wireless manner. The device where the communication component is located can access a wireless network based on a communication standard, such as WiFi, a mobile communication network such as 2G, 3G, 4G/LTE, 5G, or the like, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related text from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component further includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra Wideband (UWB) technology, bluetooth (BT) technology, and other technologies.

The power supply assembly of fig. 8 provides power to the various components of the device in which the power supply assembly is located. The power components may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device in which the power component is located.

As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable text processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable text processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable text processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable text processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

In a typical configuration, a computing device includes one or more processors (CPUs and/or GPUs), input/output interfaces, network interfaces, and memory.

The memory may include forms of volatile memory in a computer readable medium, random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.

Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement the text storage by any method or technology. The text may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store text that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.

It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrases "comprising a," "8230," "8230," or "comprising" does not exclude the presence of additional identical elements in the process, method, article, or apparatus comprising the element.

The above are merely examples of the present application and are not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims (13)

1. A method of text processing, comprising:

acquiring a text to be processed, and segmenting the text to be processed to obtain a plurality of text segments;

inputting the plurality of text segments into a coding layer of a pre-trained label determination model for coding to obtain a first coding vector of the text segment;

inputting the first coding vector into a sensor of the label determination model to determine a label, and obtaining a first label score of a corresponding text segment in a plurality of preset labels;

and inputting the plurality of first scores into a pooling layer of the label determination model for pooling to obtain a target label of the text to be processed.

2. The text processing method of claim 1, wherein the coding layer comprises: the method comprises an encoder, a gathering layer and an attention layer, wherein the encoding layer of a label determination model trained in advance is input into the text segments for encoding processing to obtain a first encoding vector of the text segment, and the method comprises the following steps:

inputting the text segments into the encoder to perform encoding processing aiming at the text segments in the text segments to obtain character vectors of characters in the text segments;

inputting a plurality of character vectors belonging to the same text segment into the collection layer for combination processing to obtain a second coding vector of the text segment;

and inputting a plurality of second coding vectors into the attention layer to carry out association processing among different second coding vectors to obtain a first coding vector of the text segment.

3. The method of claim 1, wherein the sensor comprises: the label prediction layer is used for inputting the first coding vector into the sensor for label determination to obtain a first label score of the corresponding text segment in a plurality of preset labels, and comprises the following steps:

inputting the first coding vector into the label prediction layer to determine labels, and obtaining a second fraction of labels of the text segments in a plurality of preset labels;

determining the second score to be the first score.

4. The method of claim 1, wherein the sensor comprises a label prediction layer and a weight prediction layer, and the inputting the first encoding vector into the sensor for label determination to obtain a first label fraction of a corresponding text segment among a plurality of labels in a preset manner comprises:

inputting the first coding vector into the label prediction layer to determine labels, and obtaining a second fraction of labels of the text segment in a plurality of preset labels;

inputting the first coding vector into the weight prediction layer to perform weight determination, so as to obtain the weight of the label of the text segment in a plurality of preset labels;

and determining the product of the second score and the weight as the first score of the label of the text segment in a preset plurality of labels.

5. The method as claimed in claim 3 or 4, wherein the perceptron includes a multidimensional perceptron, and the entering the first coded vector into the label prediction layer for label determination to obtain the second fraction of labels of the text segment in a predetermined plurality of labels comprises:

inputting the sub-vectors into the label prediction layer for label determination aiming at the sub-vectors in the first coding vector to obtain a fourth fraction of labels of the sub-vectors in a plurality of preset labels;

and determining a fourth score as a second score of the text segment under the label aiming at the label and the first encoding vector.

6. The method according to claim 3 or 4, wherein the inputting the plurality of first scores into the pooling layer for pooling to obtain the target tag of the text to be processed comprises:

performing the following steps in the pooling layer:

for a label in a preset plurality of labels, determining a third score of the plurality of text segments under the label according to a first score of the plurality of text segments, wherein the sum of the plurality of first scores is the third score, or a first score in the plurality of first scores is the third score;

if the third score is greater than or equal to a score threshold, determining that the tag is the target tag.

7. The text processing method according to any one of claims 1 to 4, further comprising:

determining a label corresponding to a first score in a plurality of first scores as a segment label aiming at the text segment;

for a fragment tag of the plurality of fragment tags, if the fragment tag is the same as the target tag, determining that a text fragment belonging to the fragment tag is a target text fragment, where the target text fragment is used for explaining the target tag.

8. The text processing method according to any one of claims 1 to 4, wherein the segmenting the text to be processed into a plurality of text segments comprises: segmenting the text to be processed by adopting at least one of the following modes to obtain a plurality of text segments:

adopting a sliding window with a preset text size to segment the text to be processed;

segmenting the text to be processed based on punctuation marks in the text to be processed;

and segmenting the text to be processed based on the text structure of the text to be processed.

9. A text processing method is applied to terminal equipment, and comprises the following steps:

acquiring a text to be processed;

sending the text to be processed to a server;

receiving a target label sent by the server, wherein the target label is obtained after the server processes the text according to any one of claims 1 to 8.

10. A text processing method is applied to a text processing system, and the text processing system comprises: the text processing method comprises the following steps:

the terminal equipment acquires a text to be processed and sends the text to be processed to the server;

the server cuts the text to be processed to obtain a plurality of text segments; inputting the plurality of text segments into a coding layer of a pre-trained label determination model for coding to obtain a first coding vector of the text segment; inputting the first coding vector into a sensor of the label determination model to determine a label, and obtaining a first label score of a corresponding text segment in a plurality of preset labels; inputting a plurality of first scores into a pooling layer of the label determination model for pooling to obtain a target label of the text to be processed;

and the terminal equipment receives the target label sent by the server.

11. A text processing apparatus, comprising:

the segmentation module is used for acquiring a text to be processed and segmenting the text to be processed to obtain a plurality of text segments;

the encoding module is used for inputting the plurality of text segments into an encoding layer of a pre-trained label determination model for encoding processing to obtain a first encoding vector of the text segment;

the first processing module is used for inputting the first coding vector into a sensor of the label determination model to determine a label to obtain a first label score of a corresponding text segment in a plurality of preset labels;

and the second processing module is used for inputting the plurality of first scores into the pooling layer of the label determination model for pooling processing to obtain the target label of the text to be processed.

12. A text processing system, comprising:

the system comprises a cloud server and terminal equipment, wherein a pre-trained label determination model is deployed on the cloud server;

the terminal equipment is used for acquiring a text to be processed and sending the text to be processed to the server;

the cloud server is used for segmenting the text to be processed to obtain a plurality of text segments; inputting the plurality of text segments into a coding layer of a pre-trained label determination model for coding to obtain a first coding vector of the text segment; inputting the first coding vector into a sensor of the label determination model to determine a label, and obtaining a first label score of a corresponding text segment in a plurality of preset labels; inputting a plurality of first scores into a pooling layer of the label determination model for pooling to obtain a target label of the text to be processed;

and the terminal equipment is used for receiving the target label sent by the server.

13. An electronic device, comprising: a processor, a memory and a computer program stored on the memory and executable on the processor, the processor implementing the text processing method according to any one of claims 1 to 9 when executing the computer program.

Images