CN112905787B

CN112905787B - Text information processing method, short message processing method, electronic device and readable medium

Info

Publication number: CN112905787B
Application number: CN202010116886.1A
Authority: CN
Inventors: 田帅; 鲁梦平; 师婷婷; 陈毅臻; 吴汉杰; 戴云峰
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2020-02-25
Filing date: 2020-02-25
Publication date: 2023-03-24
Anticipated expiration: 2040-02-25
Also published as: CN112905787A

Abstract

The application discloses a text information processing method, a short message processing method, electronic equipment and a readable medium, which relate to the technical field of computers, wherein text information comprises a first number of text units, and the method comprises the following steps: acquiring local feature vectors and global feature vectors of text units to be processed, wherein the text units to be processed are a second number of text units in the first number of text units, and the second number is smaller than or equal to the first number; obtaining at least one label of the text unit according to the local feature vector and the global feature vector; and extracting target content from the text information to be processed according to the label. Therefore, the characteristics of the text unit in the global and local areas can be considered in the determination of the label, so that the determination of the label is more accurate, and therefore, the target content extracted from the text information according to the label is more accurate, namely, the identification accuracy of the text information is higher.

Description

Text information processing method, short message processing method, electronic device and readable medium

Technical Field

The present application relates to the field of computer technologies, and in particular, to a text information processing method, a short message processing method, an electronic device, and a readable medium.

Background

Text data is the most common semi-structured data in computer science, many information in the real world need to be transmitted through texts, and communication between people can be realized through the communication of text information. Most of the existing recognition technologies for text data extract semantic features of the text data through a deep learning model according to contents of independent individuals such as each character or word in the text data, and recognition accuracy is not high.

Disclosure of Invention

The application provides a text information processing method, a short message processing method, an electronic device and a readable medium, so as to overcome the defects.

In a first aspect, an embodiment of the present application provides a text information processing method, where the text information includes a first number of text units, and the method includes: acquiring local feature vectors and global feature vectors of text units to be processed, wherein the text units to be processed are a second number of text units in the first number of text units, and the second number is smaller than or equal to the first number; acquiring a label of the text unit to be processed according to the local feature vector and the global feature vector; and extracting target content from the text information according to the label.

In a second aspect, an embodiment of the present application further provides a short message processing method, where the method includes: obtaining target content of the short message based on a local feature vector and a global feature vector corresponding to a text unit in the short message; and instructing the user terminal to execute target operation according to the target content.

In a third aspect, an embodiment of the present application further provides an electronic device, including: one or more processors; a memory; one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more programs configured to perform the above-described methods.

In a fourth aspect, the present application also provides a computer-readable storage medium, where a program code executable by a processor is stored, and when executed by the processor, the program code causes the processor to execute the above method.

The text information processing method, the short message processing method, the electronic device and the readable medium are applied to processing of text information, and the local feature vector and the global feature vector of the text unit in the text information are obtained. According to the method and the device, the labels of the text units can be obtained according to the local feature vectors and the global feature vectors, the local features and the global features of the text units can be referenced for determining the labels of the text units, the global and local features of the text units can be considered for determining the labels, and then the labels can be determined more accurately, therefore, the target content extracted from the text information according to the labels is more accurate, namely the identification accuracy of the text information is higher.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

FIG. 1 is a schematic diagram illustrating an operating environment of a text information processing method provided by an embodiment of the present application;

FIG. 2 is a flowchart illustrating a method of processing text information according to an embodiment of the present application;

FIG. 3 is a flowchart illustrating a method of processing text information according to another embodiment of the present application;

fig. 4 shows a flowchart of S330 in fig. 3;

FIG. 5 is a diagram illustrating an extraction model provided by an embodiment of the application;

FIG. 6 is a flowchart illustrating a method for obtaining a fusion vector according to an embodiment of the present disclosure;

FIG. 7 is a flowchart illustrating a short message entity content extraction model provided in an embodiment of the present application;

FIG. 8 is a flowchart illustrating a method of processing text messages according to another embodiment of the present application;

FIG. 9 illustrates a schematic view of a payment completion interface provided by embodiments of the present application;

FIG. 10 is a schematic diagram illustrating a billing reminder provided by an embodiment of the present application;

FIG. 11 is a schematic diagram illustrating a client login interface provided by an embodiment of the application;

FIG. 12 is a schematic diagram illustrating an interface filled with a verification code according to an embodiment of the present application;

fig. 13 is a schematic diagram illustrating travel reminding information provided by an embodiment of the application;

FIG. 14 is a block diagram of a text processing apparatus provided in an embodiment of the present application;

FIG. 15 is a block diagram of a text processing apparatus according to another embodiment of the present application;

FIG. 16 shows a block diagram of an electronic device provided by an embodiment of the application;

fig. 17 illustrates a storage unit provided in an embodiment of the present application and used for storing or carrying program codes for implementing a text information processing method according to an embodiment of the present application.

Detailed Description

In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application.

For better understanding of the solution of the embodiment of the present application, a brief description of a possible operating environment of the embodiment of the present application is provided below with reference to fig. 1.

Referring to fig. 1, fig. 1 illustrates an operating environment of a text information processing method according to an embodiment of the present application. As shown in fig. 1, the user terminal 100 and the server 200 are located in a wireless network or a wired network, and the user terminal 100 and the server 200 perform data interaction. The server 200 may be a single server, or a server cluster, or a local server, or a cloud server.

In some embodiments, server 200 has text processing capabilities. For example, an algorithm for text processing or an artificial intelligence model is deployed within the server 200. The server 200 receives a text to be recognized or a text recognition request sent from the user terminal 100 through an interactive interface, performs text processing in the manners of machine learning, deep learning, searching, reasoning, decision making and the like through a data storage of the server 200 and a data processing processor to obtain a text recognition result, and sends the text recognition result to the user terminal 100, or sends push content to the user terminal 100 according to the text recognition result, wherein the push content may be content indicating to be displayed by the user terminal 100 or information such as a control instruction controlling the user terminal 100 to execute a specified operation. The memory of the server 200 may be a generic term, and includes a data server for locally storing and storing historical data, and the data server may be deployed on the server 200, or on other network servers.

The server 200 may be a server cluster, that is, the server 200 may include a plurality of servers, and each server has different functions in the text processing process. For example, the server 200 may include a data processing server, an operation server, and a storage server, where the data processing server is configured to interact with a user terminal, acquire a to-be-processed text sent by the user terminal, and send the to-be-processed text to the operation server, the operation server processes the to-be-processed text, such as text classification, semantic recognition, or entity content recognition, to obtain a recognition result, and sends the recognition result to the data processing server, and the data processing server determines a push policy according to the recognition result, acquires, from the storage server, a push content corresponding to the push policy according to the push policy, and sends the push content to the user terminal. It should be noted that the storage server may not be used, and the data processing server may directly transmit the identification result to the user terminal, and the user terminal performs a specified operation according to the identification result.

As another embodiment, the user terminal 100 is installed with a client having a text processing capability, and the client may have a user operation interface or may be a service component without a user operation interface. For example, the client may be a broadcast receiver, and when the user terminal receives a short message, the system sends a broadcast: the method comprises the steps of "android, provider, telephony, SMS _ RECEIVED", knowing whether a new short message text exists through the broadcast, acquiring the content of the short message text through the broadcast, and executing the text information processing method provided by the embodiment of the application on the short message text. The server 200 may provide a push policy that the client sends the text recognition result to the server 200, and the server 200 determines the push content according to the text recognition result and pushes the push content to the client of the user terminal 100.

The current recognition of entity content in text mostly only considers the semantics of single words or characters in the text. For example, a deep learning method based on a bi-LSTM-CRF network is adopted to identify the content of the text entity, and the specific steps are as follows: firstly, inputting a text word by word or word by word into a bi-LSTM Network (bilateral LSTM Network), wherein the LSTM Network is a Short for a Long Short Memory Network model (Long Short Memory Network), then taking the hidden layer state of the bi-LSTM Network as the input of a Conditional Random Field (CRF) model, predicting the label of each word, finally driving the model by adopting a method of minimizing a negative log-likelihood loss function, updating parameters and obtaining an entity content recognition result.

Although the method has improved accuracy compared with manual template marking, the accuracy is still deficient.

The text information processing method provided by the embodiment of the application adopts the local context information and the global information to identify the text so as to extract the required content in the text and improve the accuracy of text identification.

For better understanding of the scheme of the embodiments of the present application, the following first explains the technical terms used in the embodiments of the present application.

Text refers to the original text of a written or printed work or a representation of the work, and may also be an edited or revised copy of the work. Text is understood to be a collection of words or characters that are encoded in some encoding format (e.g., ASCII or Unicode) and stored in a computer-readable format in a computer or similar computing device.

The text units, which may also be referred to as nodes, may be separate individuals after the text is segmented. A single word, punctuation, and consecutive digits and monetary numbers (with comma delimiters) are each a node. Suppose that the text message to be processed is a text message with the content of "your tail number xxxx card 9, 5, 10. "certain bank" then the plurality of text units are "you", "tail", "number", "xxxx", "card", "9", "month", "5", "day", "10", ": 25", "camp", "business", "net", "point", "branch", "go", "(", "card", "get", "1,000", "meta", ".", "", "some", "certain", "silver", "line", "jj".

The text information refers to information containing text, for example, the text information may be short messages and various network messages or other information that can be opened by text processing software. In a computer or similar computing device, textual information may refer to a string of characters.

The local feature vector may be local context information of a text unit, which is formed by feature vectors of N text units around a text unit as a center.

The global feature vector is global information formed by all text units of one text message. Specifically, the global feature vector is a vector representing a full-text semantic environment formed by all words in the text, and may represent full-text semantic information of the entire text, and specifically, the global feature vector may be obtained by word vectors corresponding to all words together.

The fused vector refers to a vector obtained after at least two vectors are fused. The fusion mode can be splicing, adding, weighting adding and the like.

A tag, refers to information for describing the type of text unit.

In order to overcome the above drawback, an embodiment of the present application provides a text information processing method, as shown in fig. 2, the method includes: s201 to S204. The execution subject of the method can be the server or the client, wherein the text processing client can be a service component installed in the user terminal or an application program with a user operation interface. As an implementation manner, the method of the embodiment of the present application is described by taking an example that an execution subject of the method is a server.

S201: and acquiring the local feature vector and the global feature vector of the text unit to be processed.

The text unit to be processed may be at least a part of all text units included in the text information. In some embodiments, the text information includes a first number of text units, the text units to be processed are a second number of text units of the first number of text units, and the second number is less than or equal to the first number.

In some embodiments, the number of text units included in the text information, i.e., the first number, may be a natural number greater than or equal to 1, and the second number is a natural number less than or equal to the first number. It should be noted that, if the first number is 1, the second number is also 1, that is, the second number is equal to the first number, and if the first number is multiple, the second number may be smaller than the first number, or may be equal to the first number, where multiple means at least two.

Therefore, the text unit to be processed may be all text units of all text units included in the text information, or may be a part of text units of all text units included in the text information, and specifically, please refer to the description of the following embodiments.

In this embodiment of the present application, for convenience of description, the text information processed in this embodiment of the present application may be referred to as text information to be processed, and the text information to be processed may be sent by the client to the server. As an embodiment, the client may send the text information to be processed to the server based on the obtaining request. The obtaining request may be sent by the server to the client, and is used to instruct the client to send the text content specified by the obtaining request to the server. For example, the text content specified by the acquisition request may be a text stored by the user terminal for a specified period of time, and the processed text information may be a text stored by the user terminal for a specified period of time.

For another example, the text content specified by the acquisition request may be text information received by the user terminal in a specified time period, where the received text information may be a short message text, or text data received by an application installed in the user terminal and sent by a user of another application, and may be referred to as an application text, and if the processed text information is a message text, the message text includes at least one of the short message text and the application text.

In this embodiment, the text information to be processed may be the above-mentioned message text. It should be noted that the text information to be processed may be information in a text format sent by another terminal or the client, or may be text information extracted by received information in a non-text format sent by another terminal or the client. Wherein the non-text formatted information may include at least one of speech and images. Wherein the image comprises a text image. For example, the information in the non-text format may be a voice or an image sent by a user of a social client at the opposite end, which is received by the social client in the user terminal, and the user terminal recognizes the voice to obtain text information, or recognizes a text image in the image to obtain text information.

For example, the text message may be a text message received by the user terminal, and the content of the text message is "your tail number xxxx card 9, 5, 10, 25 business outlet expenditure (card pickup) 1,000 yuan, and balance xxxx. [ some Bank ] is provided. The user terminal may receive a voice data, and analyze the voice to obtain the content, or the user terminal may receive a screenshot of a text of the content, analyze the image, and analyze a text image in the image to obtain the content.

At least one text unit may be included within the text information. As an embodiment, the text information may be processed by word segmentation to obtain a plurality of text units within the text information.

After a plurality of text units of the text information to be processed are obtained, the text units are vectorized to obtain feature vectors of the text units. The vectorization mode may be a word frequency statistical method or a One-hot encoding (One-hot encoding) method.

After the feature vectors of the text units of the text information to be processed are obtained, the feature vectors form a vector matrix corresponding to the text information to be processed, each vector of each text unit corresponds to a label in the vector matrix, and the label can indicate the position of the vector in the matrix. For example, reference character b ₃₃ Indicating that the vector is in the third row and column of the matrix.

As an embodiment, the local feature vector of each text unit may be determined by obtaining a position of the feature vector of the text unit in the vector matrix, and determining the local feature vector of the text unit according to N vectors near the position in the vector matrix. For example, the N vectors near the position in the vector matrix are used as the local feature vectors of the text unit.

As another implementation, the local feature vector of a text unit may be determined based on the relevance of the feature vector of the text unit to the feature vectors of nearby text units. For example, the local feature vector of the text unit can be obtained by using an attention model, and the detailed implementation can refer to the following embodiments.

In addition, the global feature vector may be determined based on all text units of the text information that are processed. All processed text units of the text information can be all text units in the text information or partial text units in the text information. As an implementation, the global feature vector may be obtained according to a Long Short Memory Network (LSTM), and the detailed implementation may refer to the following embodiments.

As an implementation manner, when acquiring text information to be processed, a client or a server determines whether there is an authority to process the text information, and if so, acquires a local feature vector and a global feature vector of at least one text unit.

In some embodiments, the client or the server obtains an application identifier of an application program generating the text information, searches whether the application program has the authority to process the text information generated by the application program of the application identifier, and if so, obtains the local feature vector and the global feature vector of at least one text unit. The text information generated by the application program can be the text information sent, received or generated by the application program.

In other embodiments, determining whether the text message has the right to process may be performed by obtaining a privacy type of the text message, wherein the privacy type includes private data and non-private data. And if the private type of the text information is the private data, determining that the text information has the authority to process the text information, and if the private type of the text information is the private data, determining that the text information does not have the authority to process the text information.

As an embodiment, the determining of the privacy type may be determining whether a specified keyword exists in the text message, if so, determining that the privacy type of the text message is private data, and if not, determining that the privacy type of the text message is non-private data. The specified keyword may be a private keyword, and may be preset by a user or set according to a requirement. For example, the secret key may be a mobile phone number or a password.

S202: and acquiring the label of the text unit to be processed according to the local feature vector and the global feature vector.

As an embodiment, an extraction model may be preset, and the function of the extraction model may be to determine the label of the text unit to be processed according to the local feature vector and the global feature vector of the text unit to be processed. The extraction model is trained through a sample marked with a label manually, and after the model training is finished, under the condition that a text unit of text information to be processed needs to be determined, the local feature vector and the global feature vector of the text unit to be processed are input into the extraction model after the training is finished, so that the label of the text unit can be obtained. The algorithm for obtaining the label of the text unit to be processed according to the local feature vector and the global feature vector of the text unit to be processed in the extraction model may be a Textrank algorithm or a conditional random field algorithm.

As one embodiment, the tag may be a biee tag. The biee tag may include four categories of tags, namely, a B-category tag, an I-category tag, an O-category tag, and an E-category tag. The determination of the four types of tags is associated with a specified content type that needs to be determined from the textual information.

In some embodiments, the content belonging to the specified content type is named entity content. For example, if the specified type is a financial type, the content related to the account balance, the card number, the account number, the amount of money, and the like in the text message may be entity content. The B-type label represents the beginning character of the entity content, the I-type label represents the middle character of the entity content, the E-type label represents the ending character of the entity content, and the O-type label represents the character unrelated to the entity content.

In addition, the class B tags, the class I tags and the class E tags may further comprise sub-tags, i.e., the class B tags may comprise at least one class B sub-tag, e.g., B-xxxx, where "xxxx" is a content type tag used to indicate the type of entity content. For example, "xxxx" is "Tail" indicating that the type of the entity content is a Tail number, and B-Tail indicates the beginning character of the entity content of the Tail number type.

Similarly, both class I and class E tags may include sub-tags, e.g., a class I sub-tag may be I-xxxx and a class E sub-tag may be E-xxxx. For example, I-Tail represents the middle character of the entity content of the Tail type, and E-Tail represents the end character of the entity content of the Tail type.

Typically, class B tags, class I tags, and class E tags, or class B tags and class E tags are contiguous. For example, the tag types corresponding to two characters of "flight" are BE in turn, that is, the entity content is composed of two characters, the first character "flight" of the two characters is a B-type tag representing the beginning character of the type of characters, and the second character "class" of the two characters is an E-type tag representing the ending character of the type of characters. For another example, the label types corresponding to three characters of "taxi" are BIE in sequence, the first character "out" of the three characters is a B-type label representing the starting character of the character, the third character "rent" of the three characters is an I-type label representing the middle character of the character, and the third character "car" of the three characters is an E-type label representing the ending character of the character.

Thus, in general, when the BE appears continuously, or the BIE, or B, a plurality of consecutive I and E appear continuously, the content type labels of the B-class label, the I-class label and the E-class label may BE identical. For example, labels corresponding to two characters of the flight are B-FLY and E-FLY in sequence, namely the content type labels of the B-type label and the E-type label are both FLY.

S203: and extracting target content from the text information according to the label.

After the tags of the text units to be processed are obtained, the types of the tags of the text units to be processed in the text information to be processed can be determined, so that the required entity content, namely the target content, can be searched according to the types. As an implementation mode, the appointed label corresponding to the required entity content is determined, the text unit corresponding to the appointed label is searched, and the target content is obtained according to the searched text unit.

The designated tag may be a corresponding relationship preset according to a preset corresponding relationship between a type of entity content and a plurality of tag types, as an implementation manner, a corresponding relationship is preset, and the corresponding relationship includes a plurality of types of entity content and a tag type corresponding to each type of entity content, where the tag type may be the above-mentioned B-type sub-tag, I-type sub-tag, and E-type sub-tag.

As an implementation manner, according to the corresponding relationship, a tag type corresponding to the type of the required entity content is determined as an assigned tag, and after a text unit corresponding to a tag matched with the assigned tag is found in the text information according to the assigned tag, the found text unit is integrated into the target content.

In some embodiments, if the vicinity of the class B label is both an O-class label or no character, the text unit corresponding to the class B character is determined to be a single character entity; if the type of the text unit before the text unit of the B-type label is determined to BE the O-type label or no character before the text unit of the B-type label according to the text direction, and the type of the text unit after the text unit of the B-type label is determined to BE the E-type label, two text units with continuous labels BE are found, and the two text units are judged to BE the double-character entity.

For example, the content of the text information is "i love to eat a stick", and a plurality of text units determined according to the character direction of the text information are "i", "love", "eat", "oil" and "stick" in sequence, that is, the text information is recorded in a sequence or an array a, and then a = ("i", "love", "eat", "oil" and "stick"), and each element in a is in sequence, a [1] = "i", a [2] = "love", a [3] = "eat", a [4] = "oil", and a [5] = "stick". Wherein, the order of a [1], a [2], a [3], a [4], a [5] in the sequence a matches the text direction, the feature vectors of a plurality of text units in the text information can be recorded in the above sequence or array, each text unit can correspond to an element, and the preceding text unit, the following text unit, and the adjacent text unit of the text unit can be determined according to the subscript of each element.

And if the type of the text unit before the text unit of the B-type label is determined to be the O-type label or no character is arranged before the text unit of the B-type label according to the text direction, and the types of two continuous text units after the text unit of the B-type label are the I-type label and the E-type label in sequence, judging that the three text units are marked as three-character entities.

Similarly, if the type of the text unit before the text unit of the B-type label is determined to be the O-type label or no character before the text unit of the B-type label according to the text direction, and the types of three continuous text units after the text unit of the B-type label are sequentially the I-type label, the I-type label and the E-type label, it is determined that the three text units are marked as four-character entities, and so on, and a five-character entity, a six-character entity, and so on can also be determined.

The target content is determined by the plurality of entities resulting from the integration. As an implementation manner, the policy for extracting the target content is determined according to the purpose or usage of the extracted content, that is, may be determined according to the requirement for using the target content, specifically, please refer to the following examples.

It should be noted that, if the extracting of the target content from the text information to be processed according to the tag may be performed by the processing client, the operation of obtaining the local feature vector and the global feature vector of the at least one text unit and obtaining the tag of the at least one text unit according to the local feature vector and the global feature vector may be performed by the server, the server sends the tag of the text unit to the processing client, and the processing client extracts the target content from the text information to be processed according to the tag.

Referring to fig. 3, fig. 3 illustrates a text information processing method according to an embodiment of the present application, where an execution subject of the method may be the server or the client. As an implementation manner, the method of the embodiment of the present application is described in the case that an execution subject of the method is a server. As shown in fig. 3, the method includes: s310 to S350.

S310: and acquiring the local feature vector and the global feature vector of the text unit to be processed.

S320: and acquiring a first weight corresponding to the local feature vector of the text unit to be processed and a second weight corresponding to the global feature vector.

As an implementation manner, in the embodiment of the present application, the extraction model may be used to determine the label of the text unit to be processed of the text information, that is, the local feature vector and the global feature vector of the text unit to be processed may be input into the extraction model, so as to obtain the label of the text unit to be processed.

In some embodiments, for convenience of calculation and improvement of the accuracy of tag identification, the local feature vector and the global feature vector may be fused into one vector, and a local feature group formed by the local feature vectors of the text units to be processed of the text information can be fused into one fused feature group. The feature set may be a matrix or an array formed by a plurality of feature vectors.

In both the splicing or adding manner and the weighted adding manner, the local feature vector and the global feature vector each correspond to a weight, and the weight is used to represent the specific gravity of the local feature vector and the global feature vector in the fused result, and the specific fusing manner may refer to the introduction of the following S330.

As an embodiment, the values of the first weight and the second weight may be set empirically or may be a default fixed value. In order to increase the adaptability and accuracy of the first weight and the second weight, the first weight corresponding to the local feature vector and the second weight corresponding to the global feature vector of the text unit can be determined according to the feature vector of the text unit.

As shown in fig. 4, S320 may include steps S321 and S322.

S321: and acquiring the feature vector of the text unit to be processed.

The above embodiments can be referred to for the manner of obtaining the feature vector, and details are not repeated here.

As an embodiment, the above extraction model is shown in FIG. 5, and the extraction model includes an embedding layer, a local feature layer, a global feature layer, a feature fusion layer, and a conditional random field layer, where s is ₁ 、s ₂ 、s ₃ 、s ₄ 、s ₅ Extracting feature vectors, y, of text units of a model for input ₁ 、y ₂ 、y ₃ 、y ₄ 、y ₅ Labels for the text units output for the extraction model. The role of the various layers within the extraction model will be described below in conjunction with a specific process flow.

In some embodiments, a vector obtained after vectorization of text units of a plurality of text information is denoted as a sequence S = (S) ₁ ,s ₂ ,…,s _n ) Where n is the number of text units, i.e., the second number described above. The number may be the number of all text units of the text information or the number of partial text units within the text information.

As an embodiment, the partial unit may be a remaining text unit after removing the first a text units and the last b text units from the text information. The values of a and b can be set empirically, for example, a is 3 and b is 1.

Considering that the first few characters of a text message may be a name, e.g. "hello", "good", "three text units may be removed, e.g. the last character of a text message may be a punctuation mark, e.g. a period. The last "may be. And removing. The removed text units may not necessarily be obtained feature vectors and may not necessarily be used for the determination of the first and second weights.

The vectorization may be to obtain the sequence S by the one-hot encoding method, and the vector obtained by the one-hot encoding method may be used as a feature vector of a text unit, but the vector obtained by the one-hot encoding method is too sparse, which increases the computational complexity and also fails to identify semantic features, so that the sequence S may be converted into an embedded vector.

As an embodiment, the sequence S = (S) ₁ ,s ₂ ,…,s _n ) The input embedding layer obtains a feature vector. Specifically, S = (S) ₁ ,s ₂ ,…,s _n ) Inputting an embedding layer, wherein the embedding layer is S = (S) ₁ ,s ₂ ,…,s _n ) And converting into a dense embedded vector, and recording the embedded vector of the text unit as a feature vector of the text unit. The resulting embedded vector is a dense vector and the semantic features of the character can be captured. For example, "bullfrog" and "frog" are closer together in the embedding vector space, and closer together the identifying character semantics are closer together. As an embodiment, the embedded vector is determined according to the following equation:

x _t ＝W _e [v _t ] (1)

wherein, the first and the second end of the pipe are connected with each other,

embedding a matrix for a word vector, m being the dimension of the embedded vector, D being the size of the dictionary, v _t Representing a node s _t Index in a dictionary, x _t For the t-th text unit s _t Embedded vector of W _e [v _t ]A representation matrix W _e V. of (b) _t Column, to be embedded in vector x _t As units of text s _t The feature vector of (2), then the feature vector of each text unit is X = (X) ₁ ,x ₂ ,…,x _n ) I.e. the output of the embedding layer is x ₁ ,x ₂ ,…,x _n . The dictionary is used for representing each text unit by a numerical value, the numerical value is used for representing the position of the text unit in the dictionary, for example, a sentence of words "interesting soul weighs two hundred jin", each word is coded according to the position of the dictionary, if the dictionary has 1000 words, the word is 'existed' and then the 100 th position of the dictionary, and the 'fun' is at the 107 th position of the dictionary, so that the feature vector of each text unit in the sentence can be obtained.

In addition, before the feature vectors of the text units to be processed are obtained, preprocessing operations can be performed on the samples to be processed, and the preprocessing operations include stop word filtering and dictionary generation, and the stop word filtering is a preprocessing method in text analysis. Its function is to filter noise (e.g., "yes", "o", etc.) in the word segmentation result. The dictionary generation mode can be that a large amount of text information is obtained, word segmentation processing is carried out on the text information, and pre-training is carried out on a word corpus after word segmentation to generate a word vector dictionary based on the algorithms such as word2 vec. The words are not repeated, word2vec can count the occurrence frequency of basic elements of words, words and punctuation in the text, and vector representation of the designated dimension corresponding to the words serving as basic constituent elements of the basic corpus is obtained through unsupervised training.

In this embodiment, the server or the client may obtain the local feature vector and the global feature vector of the text unit according to the feature vector of the text unit.

As shown in fig. 5, the feature vector of the text unit output by the embedding layer is input into the local feature layer to obtain a local feature vector, and the feature vector is input into the global feature layer to obtain a global feature vector, the flow direction of extracting the global feature vector is shown by a thick line in fig. 5, and the flow direction of extracting the local feature vector is shown by a thin line between the embedding layer and the local feature layer.

As an embodiment, the local feature layer may obtain the local feature vector by setting a window parameter w, which may be an optional super-parameter for indicating the size of the window, and the value may be an odd number, for example, w is 5. Local feature vectors are determined from the window parameters.

Specifically, the embodiment of obtaining the local feature vector of the text unit to be processed may be to obtain the feature vector of the text unit to be processed; determining a window vector of a text unit to be processed according to a window parameter and a feature vector which are acquired in advance; and acquiring a local feature vector of the text unit to be processed according to the window vector.

X can be selected in the above feature vector X by window parameters _t At least one feature vector in the vicinity, the selected feature vector may be taken as x _t The window vector of (2).

As an embodiment, each text unit corresponds to a window, and the size of the window is represented by the window parameter of the window, so that n text units can be divided into n windows. Suppose H _t The window vector of the t-th feature vector is defined as follows:

wherein t =1,2, \8230, n, wherein x _t Is the feature vector, x, of the t-th text unit _t+1 And (4) the feature vector of the t +1 th text unit, and so on, other x in parentheses of other formulas above represent other text units. As shown in the above formula, H _t The contained text sheetElement is x _t W feature vectors in the vicinity of the center.

In addition, it should be noted that if the X subscript in the above formula exceeds the subscript of the feature vector X, zero vector padding may be used. As shown in fig. 5 at s ₁ Left side of and s ₅ Are all padded with zero vectors. For example, a window vector with t of 2, w of 5, x2 is (0,x) ₁ ,x ₂ ,x ₃ ,x ₄ ) That is, if the calculation result of t- (w-1)/2 is 0, the subscript of X is in the range of [1, n ]]Then 0 has been out of the subscript of X, thus X ₀ Is filled with 0.

As an embodiment, the local feature vector of the text unit may be determined according to a self-attention mechanism according to a window vector of the text unit. Specifically, the local feature vector is determined according to:

wherein l _t Local feature vector for the t-th text unit, whose value is taken from the matrix L _t Line (w-1)/2, L _t Feature vector x for the t-th text unit _t In the corresponding window vector, a certain feature vector x _i (where i = t- (w-1)/2, \ 8230;, t + (w-1)/2) and the remaining w-1 eigenvectors, i.e., eigenvector x _i Similarity with other feature vectors. The softmax function is a probability normalization function, order

The expression of the softmax function is:

wherein Q _r,i Is the element of the r-th row, i-th column of the matrix Q.

By the above formulas (3), (4) and (5), the unit of text x can be represented _t For reference, after measuring the similarity between the text unit and each text unit in the corresponding window vector, normalizing the similarity, and reconstructing a text unit x according to the similarity _t The reconstructed feature vector is the local feature vector of the text unit.

Executing the above formulas (3) and (4) on all n window vectors to obtain the local feature vector of each text unit, and splicing the local feature vectors of all the text units to obtain the local feature matrix F of the text information _l ＝(l ₁ ,l ₂ ,…,l _n )。

As an implementation manner, in this embodiment of the present application, a global feature vector of a text unit may also be determined according to a feature vector of the text unit. For example, the LSTM network may be used to obtain a global feature vector for a text unit. The improvement of the traditional Text Recurrent Neural Network (Text RNN) in the LSTM Network model is that the last hidden element is not used as a classification, but the information of all hidden elements is used, so that the context information of each Text unit can be extracted more comprehensively.

The mathematical identification of the LSTM network is as follows:

c _t ＝f _t ⊙c _t-1 +i _t ⊙g _t (7)

h _t ＝o _t ⊙tanh(c _t ) (8)

wherein h is _t Global feature vector, x, for the t-th text unit _t Feature vector for the t-th text unit, M:

affine transformations formed for trainable parameters, in which，a＝m+h _lstm ，b＝4h _lstm ，h _lstm Is the number of hidden layer units of the LSTM network. The trainable parameters are parameters updated by gradient back propagation through a supervised learning algorithm. Affine transformation refers to converting input features into another feature space so that a model learns semantic features and context features of text.

Where σ is a probability normalization function, e.g., sigmoid. i.e. i _t And g _t For indicating the portion of the t-th input gate for saving into the cell state, where i _t G representing how much information the t-th input needs to be saved to the cell state _t New information representing the creation of the t-th input is added to the cell state. f. of _t For the input of the t-th forgetting gate, o _t Memory cell c representing the output of the t-th output gate at time t _t Last moment memory unit c for forgetting gate reconciliation _t-1 Current time information g reconciled with input gate _t Sum, at initialization, c ₀ And h ₀ Set to a zero vector.

For convenience of description, the above equations (6), (7) and (8) are denoted as LSTM (·).

Determining a global feature vector according to:

wherein ≧ is the stitching operation, i.e., the vector stitching operation, → represents forward propagation, and ← represents backward propagation. The global feature vector of each text unit is obtained through the formula, and the global feature matrix F of the text information can be obtained _g ＝(h ₁ ,h ₂ ,…,h _n )。

S322: and determining a first weight and a second weight of the text unit to be processed according to the feature vector.

As an embodiment, the specific gravity when the local feature vector and the global feature vector are fused, that is, the first weight and the second weight are determined according to the feature vector of the text unit, a value may be obtained according to the feature vector of the text unit, a sum of the first weight and the second weight determined according to the value is always equal to a fixed value, and the larger the value is, the smaller the first weight is, the larger the second weight is, the smaller the value is, the larger the first weight is, and the smaller the second weight is.

Of course, the smaller the value, the larger the first weight and the smaller the second weight, and the larger the value, the smaller the first weight and the larger the second weight. Thus, the magnitudes of the first weight and the second weight, i.e., the specific gravity when the local feature vector and the global feature vector are fused, are influenced by the feature vector.

In some embodiments, the value may be normalized to obtain a value smaller than 1 and larger than 0, and the first weight and the second weight may be obtained according to the value. Specifically, after the numerical value corresponding to the feature vector of the text unit is normalized, the numerical value is recorded as the score of the text unit, and then the first weight and the second weight of the text unit to be processed are determined according to the feature vector in the embodiment that the score of the text unit to be processed is determined according to the feature vector; and determining a first weight and a second weight of the text unit to be processed according to the scores.

Before fusing the vector features, the model calculates a score a at each time step _t And determining the weights of the local feature vector and the global feature vector corresponding to the current text unit. The time step may be an order of processing each text unit, for example, a processing cycle of processing each feature vector in the feature vector matrix X of the text unit, for example, each feature vector in the feature vector matrix X is processed according to the processing cycle T, and each cycle may be regarded as a time step.

As an implementation manner, in the embodiment of the present application, a gating mechanism is used to set weights for a local feature vector and a global feature vector, as shown in fig. 5, the gating mechanism obtains a feature vector of a text unit, and outputs a first weight and a second weight to a feature fusion layer. The gating mechanism may be a piece of program code or a functional component, and may set a first weight for a local feature vector of each text unit and a second weight for a global feature vector of the text unit according to a score of the feature vector of the text unit. Specifically, the score of the text unit to be processed may be determined according to the following equation:

a _t ＝σ(Wx _t +b) (12)

wherein, a _t Is the score of the t-th text unit, W is the weight matrix, x _t The feature vector of the t text unit, b is an offset value, and sigma is a probability normalization function. W is a linear transformation, data are mapped into another dimensional space, and through the nonlinear transformation of sigma and parameter updating under the supervision learning, W can be adjusted and learned in an adaptive mode. Obtaining the feature vector x _t Score a of _t Then, (1-a) is added _t ) As a first weight of the t-th text unit, a _t As a second weight of the t-th text unit, so when a _t When larger, the feature vector x _t Global feature vector h of _t With a larger weight of the feature vector x _t Local feature vector l of _t Is less weighted.

S330: and obtaining a fusion vector of the text unit to be processed according to the local feature vector, the global feature vector, the first weight and the second weight.

As shown in fig. 5, the feature fusion layer obtains a global feature vector input by the global feature layer, a local feature vector input by the local feature layer, and a first weight and a second weight input by the gating mechanism, and obtains a fusion vector of the text unit according to the local feature vector, the global feature vector, the first weight, and the second weight.

The first weight and the second weight respectively represent the proportion of the local feature vector and the global feature vector in the fusion vector, and the process of determining the fusion vector can be regarded as obtaining a first vector factor according to the first weight and the local feature vector, obtaining a second vector factor according to the second weight and the global feature vector, and obtaining the fusion vector according to the first vector factor and the second vector factor.

As an embodiment, a product of the first weight and the local feature vector may be used as the first vector factor, or the first vector factor may be obtained based on the product of the first weight and the local feature vector, for example, the product of the first weight and the local feature vector is obtained, and the first vector factor is obtained by adding a numerical value to the product or multiplying a numerical value by the product. Similarly, the product of the second weight and the global feature vector may also be used as the second vector factor.

As an embodiment, the sum of the first vector factor and the second vector factor may be taken as the fusion vector. Specifically, the fusion vector is determined according to the following formula:

r _t ＝a _t ⊙h _t-1 +(1-a _t )⊙l _t (13)

wherein, h is a dot-by-dot product operator _t Global feature vector for the t-th text unit, l _t Local feature vector for the t-th text unit, r _t Is a fused vector of t text units. The process of obtaining the fusion vector according to the formula (13) is shown in fig. 6.

Therefore, the ratio of the global feature vector and the local feature vector in the fusion vector can be adaptively adjusted according to the feature vector of the current text unit, for example, a _t If the size of the unit is larger, more global feature vectors are needed for the tth text unit, otherwise, more local feature vectors are needed.

S340: and determining the label of the text unit to be processed according to the fusion vector.

As an embodiment, the label of the text unit to be processed may be obtained according to a Conditional Random Field (CRF) model and a fusion vector of the text unit to be processed.

As shown in fig. 5, the fusion vector obtained in the above manner is sent to the conditional random field layer, and the conditional random field layer and the conditional random field algorithm obtain the label of the text unit through the fusion vector of the text unit. As an embodiment, the label of a text unit may be determined according to the following equation:

Y ^* ＝argmax _Y p(Y|S) (14)

wherein p (Y | S) is the conditional probability of the random field learned in advance, wherein S is the observation sequence, that is, the observation sequence S is converted into a fusion vector, and further a hidden state Y is obtained through the fusion vector, and Y is an output label. After the learning of the conditional probability is completed, the labels of a plurality of text units within the observation sequence S (i.e., the text information to be processed) can be obtained according to the above formula (14).

In the embodiment of the present application, the learning process of extracting the model is similar to the above-described process of obtaining the plurality of tags of the text information according to the extraction model. In particular, prior to learning the model, data organization operations need to be performed, i.e., to obtain the data sets required for model training. As shown in fig. 7, fig. 7 shows a short message entity extraction process provided in the embodiment of the present application, which can explain a learning process of the extraction model and a process of obtaining a content of a short message entity according to the extraction model.

Specifically, as shown in fig. 7, the learning process of the extraction model includes: s701 to S709. The process of obtaining the short message entity content according to the extraction model comprises the following steps: s701, S703 to S708, S710, and S711. The two different processes may be selected based on different requirements, i.e. model training or model application.

In this embodiment, the text information to be processed is a short message text, wherein for the collection of the short message text and the operations from S702 to S711, reference may be made to the preamble embodiment, which is not described herein again.

As an embodiment, in the learning process of the extraction model, S701 and S702 may be regarded as an operation of acquiring a data set, specifically, the data set may be acquired by acquiring a text sample data, the sample data includes a plurality of text samples, and a plurality of text units of the text samples are determined according to the above manner, so as to obtain a text sample S = (S) ₁ ,s ₂ ,…,s _n ) The specific process may refer to the foregoing process of obtaining the feature vector of the text unit. The manual sequential labeling operation is performed on the plurality of text units of each text sample, which may be referred to in the foregoing embodiments, for example, the plurality of text units of each text sample are labeled using a biee label, so that each text unit is labeled with a label.

Wherein n is the number of text units in the sample, Y is the label corresponding to the text sample, and Y = (Y) ₁ ,y ₂ ,…,y _n ) Wherein, y _i As units of text s _i The label of (a) is used,

the data set required for model training. Here, the subscript j of S is an index of a sample, and represents a data set Ω formed by 1 to Q text sample data.

After the text sample data is obtained, the learning process of extracting the model and the process of obtaining the short message entity content according to the extraction model can both pass through steps S704 to S708, determine the fusion vector of each text unit in the text sample data, and obtain the label of the text unit to be processed based on the fusion vector. As an embodiment, the fusion vector is input into the CRF model to obtain the maximum observed sequence S. In addition, the function of the CRF network is:

log(p(Y|S))＝g(S,Y)-log∑ _Y' exp(g(S,Y')) (15)

wherein the content of the first and second substances,

wherein the content of the first and second substances,

representing a node s _i Is given by the label y _i The probability of (c). />

Represents from y _i Transfer to y _i+1 The probability of (c). The transition probability isRefers to the probability of all values being transferred from the current state to the next state, e.g., the probability of adverb followed by verb, the probability of verb followed by noun, etc. When sequence prediction is carried out, the aim is to maximize the prediction probability of the whole sequence, rather than considering only a certain node singly, and the probability transition matrix between nodes is also a necessary parameter for the whole solution.

Bonding with

And &>

Can predict the node s _i The probability of the tag is maximized. For example, the probability is 0.6 when the current node takes a value of 1, and the probability is 0.4 when the current node takes a value of 2. The probability that the current node value 1 is transferred to any next node value is 0, which indicates that the current node value is not reasonable when 1.

As an embodiment, in the learning process of the extraction model, S709 needs to be executed to complete the learning of the extraction model, and specifically, the negative log-likelihood function corresponding to the formula (15) may be minimized through the ADAM gradient optimization algorithm and the back propagation algorithm, so as to update the model parameters until convergence. The converged model completes the learning of p (Y | S).

S350: and extracting target content from the text information according to the label.

Referring to fig. 7, in the process of obtaining the short message entity content according to the extraction model, S711 is required to be executed, that is, the user terminal is instructed to execute the target operation according to the target content, and the implementation of this step may refer to the subsequent embodiments.

Referring to fig. 8, fig. 8 shows a short message processing method provided in the embodiment of the present application, where an execution subject of the method may be the server or the client installed in the user terminal. As an implementation manner, the method of the embodiment of the present application is described in the case that the execution subject of the method is a client. As shown in fig. 8, the method includes: s801 to S802.

S801: and obtaining the target content of the short message based on the local feature vector and the global feature vector corresponding to the text unit in the short message.

Among them, the meaning of the text unit, the local feature vector, the global feature vector and the target content may refer to the foregoing embodiments, and the target content may be the entity content described above.

As an implementation manner, the user terminal can extract the received short message and send the short message to the server, and the server obtains the target content corresponding to the short message according to the foregoing embodiment. Specifically, the server obtains a short message to be processed sent by the user terminal, obtains a text unit to be processed corresponding to the short message, obtains a local feature vector and a global feature vector of the text unit to be processed, obtains a label of the text unit to be processed according to the local feature vector and the global feature vector, and extracts target content from text information according to the label. Specifically, the foregoing embodiments may be referred to for implementing obtaining the target content, and details are not repeated here.

S802: and instructing the user terminal to execute the target operation according to the target content.

After the target content is extracted from the text information to be processed according to the label, the user terminal can be instructed to execute the target operation according to the target content.

Wherein the aforementioned required entity content is related to the target operation. As an implementation manner, if the content required for executing the target operation is the required entity content, and the determination of the tag can also be determined according to the entity content required for the target operation, an implementation manner of obtaining the tag of the text unit to be processed according to the local feature vector and the global feature vector may be to obtain requirement information of the target operation, where the entity content required for the target operation is recorded in the requirement information, and obtain the tag of the text unit to be processed according to the requirement information, the local feature vector, and the global feature vector.

In some embodiments, the target operation may be an operation performed by a client within the user terminal. In the embodiment of the present application, the target operation may include instructing the user terminal to perform a bill reminding operation, a verification code filling operation, and a trip reminding operation.

It should be noted that, the execution subject for executing the target operation according to the target content may be a server, and the server sends the target content to the user terminal and instructs the user terminal to execute the target operation. The execution subject that executes the target operation according to the target content may be the above-described client that instructs the user terminal to execute the target operation.

As an embodiment, the target operation includes instructing the user terminal to perform a bill reminding operation, the target content includes consumption content, and the performing of the target operation according to the target content may be generating bill reminding information according to the consumption content; and displaying the bill reminding information on a screen of the user terminal.

The consumption content may include information such as time, card number, consumption amount, balance, etc. As an implementation manner, the consumption content setting may be a content composed of text units corresponding to types of tags that are consumption types or types related to consumption types, and the information such as the time, the card number, the consumption amount, the balance, and the like may be a content composed of text units of types of tags that are time, the card number, the consumption amount, the balance, and the like.

If the content is "your tail number xxxx card 9, 5, 10, 25 business outlet expenditure (card fetch) 1,000, balance xxxx. Text information of [ a certain bank ], the labels of the obtained text units are shown in the following table:

TABLE 1

You

Tail

Number (C)

xxxx

Card with a detachable cover

9

Moon cake

O

B-Tail

O

B-Month

O

5

Day(s)

10

：

25

Camp chair

Industry

B-Day

O

B-Hour

O

B-Minute

O

Net

Dot

Branch stand

Go out

(

Card with a detachable cover

Get

O

B-MeanEx

E-MeanEx

O

)

1,000

，

Surplus

Forehead (forehead)

xxxx.xx

Yuan

O

B-Amount

O

B-Balance

O

。

【

A certain

Silver (Ag)

Line of

】

O

B-PlatForm

I-PlatForm

E-PlatForm

O

As shown in table 1 above, according to the aforementioned manner of integrating text units into target content, the result after the integration of each tag in table 1 may be a single character entity: "xxxx" type is Tail (Tail number), "9" type is Month, "5" type is Day (type is date), "10" type is Hour (time), "25" type is Minute (Minute), "1,000" type is Amount, and "xxxx.xx" type is Balance; double-character entity: the "payout" type is MeanEx (pay), four character entity: the "certain bank" type is PlatForm (open bank).

The bill reminding information can be generated through the extracted consumption content, and can inform the user of the consumption amount or the remaining amount of a certain account.

For example, the target contents obtained in the above example are "xxxx", "9", "5", "10", "25", "1,000", "xxxx.xx", "payout", and "certain bank", where "9", "5", "10", "25" belong to the information of the time type, "payout" and "1,000" are the type of the amount of consumption, "xxxx.xx" is the type of the balance, "xxxx" and "certain bank" are the type of the card number.

Here, the type of the tag belonging to the consumption content may be preset, for example, if it is preset that MeanEx does not belong to the consumption amount type, the "expense" determined according to this example does not belong to the consumption amount type. In one embodiment, the reminder determined according to this example is "1,000 for XXXX consumption by a certain bank, and the balance xxxx.xx", that is, the card number, the consumption amount, the balance, and the like are selected as the consumption content, and the reminder is generated based on the consumption content.

As an embodiment, the displaying of the bill reminding information on the screen of the user terminal may be that the bill reminding information is displayed on a designated interface of the first target client of the user terminal.

In some embodiments, the first target client may be a payment client that the user is currently completing the payment, and the designated interface of the payment client may be the payment completion interface of the client. As shown in fig. 9, after the user completes payment of the current bill in the payment client, the payment completion interface is displayed, and a payment short message 901 corresponding to the current payment operation is acquired.

As shown in fig. 9, the short message is displayed in the top status bar through the short message reminding component, because the number of words of the content displayed in the top status bar is limited, the user cannot look up all the content of the short message through the short message reminding component, and if the content needs to be looked up, the short message needs to be switched to a short message client of the user terminal, the short message is clicked in the short message client to look up the complete content, or the short message reminding component is clicked in the interface shown in fig. 9, so that the interface can be switched to a look up interface of the payment short message in the short message client.

However, this may cause the payment client to be switched, interrupting the user's operation of the payment client. Through the bill reminding information provided by the embodiment of the application, the user can quickly read the content in the short message, and the payment client can be prevented from being switched to a background.

As shown in fig. 10, a bill reminder 1001 is displayed within the payment completion interface of the payment client. For example, the bill reminder information 1001 displays "bill reminder: xx "is the balance XXXX consumed 1,000 by a certain bank XXXX. That is to say, the server or the client executes the above method with the payment short message 901 as the text information to be processed to obtain the bill reminding information, and displays the bill reminding information in the payment completion interface, so that the user can obtain the content related to the bill of the short message even without entering the review interface of the short message, and the amount of money consumed this time and the balance in the account can be verified conveniently.

As another embodiment, the bill reminding information may not be limited to be displayed in the interface of the specific client. Specifically, the bill reminding information may be displayed on the screen immediately after the bill reminding information is acquired. Further, the bill reminding information can be displayed on the screen when the screen is lightened.

As another embodiment, the target operation includes instructing the user terminal to perform a verification code filling operation, the target content includes a verification code, and the target operation is performed according to the target content by sending the verification code to the second target client and instructing the second target client to input the verification code into the currently displayed verification code input area. The verification code may be obtained by extracting a tag of which the type is the verification code type from the text message.

The second target client may be a client that a user requests to obtain the verification code, or a client that a user requests to input the verification code. As shown in fig. 11, in the login interface of the second target client, the user needs to input a mobile phone number in the interface, and input the verification code received by the user terminal of the mobile phone number, and input the verification code in the verification code input area 1103 of the second target client, and after the verification passes, the user can successfully log in the second target client.

As an implementation manner, when the user clicks the verification code obtaining control 1102 on the interface of the second target client, the server corresponding to the second target client sends a verification code to the user terminal corresponding to the mobile phone number filled by the user, and the user terminal can receive a short message including the verification code sent by the server and is marked as the verification short message 1101, and the user can know the verification code only when needing to look up the content of the short message.

Therefore, the user also faces the inconvenience of referring to the content of the payment short message, that is, the user may switch between different applications and perform complicated operations when referring to the short message. Even if the user can see the verification code in the verification message 1101 through the message reminding component displayed on the top status bar, the user still needs to manually input the verification code into the verification code input area 1103, which is complicated to operate.

In the embodiment of the present application, after the server obtains the verification short message, the server obtains the verification code in the verification short message through the above method steps, and automatically inputs the currently displayed verification code input area 1103 of the second target client, as shown in fig. 12, the second target client can automatically input the verification code 1234 into the verification code input area 1103, so that the operations of the user can be reduced.

As another embodiment, the target content includes travel content, and the performing of the target operation according to the target content may be generating travel reminding information according to the travel content; and displaying the travel reminding information on a screen of the user terminal.

The travel content may include information of types such as a departure place, a destination, a departure time, a departure train number, and a seat, and the manner of acquiring the travel content may be to extract a text unit corresponding to a tag matched with the travel type from the text information, and obtain the travel content according to the text unit. The travel types comprise types of departure place, destination, departure time, departure train number, seats and the like.

As an implementation mode, when a user purchases tickets such as air tickets and high-speed rail tickets by using a travel APP, the server of the APP determines the mobile phone number of the user using the APP, and sends a travel short message to a user terminal corresponding to the mobile phone number. The travel short message comprises travel contents such as a departure place, a destination, departure time, a departure train number and seats.

In some embodiments, the user terminal may extract the travel short message through a third target client installed in advance, extract the travel content in the travel short message, and of course, may also send the travel short message to the server, and the server obtains the travel content in the travel short message. Therefore, the travel reminding information is generated according to the travel content. The trip reminding information is used for informing the user of the trip content and reminding the user of paying attention to the trip content, and the reminding mode can be a pop-up window or voice mode.

As an embodiment, it is considered that the user sets an alarm clock in order to avoid going late. The travel content includes a travel time, and the third target client is an alarm clock client.

In some embodiments, a server or a client obtains alarm information set by an alarm clock client in a user terminal, where the alarm information includes an alarm clock time and a corresponding alarm date, and if the alarm information belongs to a repeated alarm, a plurality of alarm dates corresponding to the alarm information can be determined according to the repeated rule. For example, if the alarm information is a repeating alarm whose repetition rule is one to friday every week, alarm dates corresponding to a plurality of pieces of alarm information after the time at which the travel time is currently acquired can be specified from the calendar.

In addition, if the alarm information does not belong to the repeat alarm, the alarm date corresponding to the alarm information can be obtained according to the current time. For example, if today is 1 month, 1 day, monday, and the alarm information is tuesday, 8 am, it can be determined that the alarm information is the first tuesday after 1 month, 1 day, i.e., 1 month, 2 days.

Wherein, this trip time includes trip moment and trip date, if set up a plurality of alarm information in the alarm client, then a plurality of alarm dates that every alarm information corresponds can both be confirmed, find the alarm date that matches with the trip date, as the target alarm date, wherein, the alarm date that matches with the trip date can be, belongs to the alarm date on same day with the trip date.

Determining alarm information of the alarm clock before the trip time as to-be-selected alarm information, determining target alarm information according to the to-be-selected alarm information, and then displaying the trip reminding information in a target alarm interface when the target alarm interface corresponding to the target alarm information is displayed. Therefore, when the user is reminded by the target alarm information, the trip reminding information can be received in the target alarm information interface, so that the trip content can be conveniently and timely known.

As shown in fig. 13, a travel reminder message 1301 is displayed in the alarm information interface, and as shown in fig. 13, the content of the travel reminder message 1301 is a travel reminder: day 20 of 10 months, 9 am, flights x, fly to city a. When being reminded by the alarm, the user can know the information such as the vehicle taken during traveling, the departure time of the vehicle, the destination and the like.

As an implementation manner, the implementation manner of determining the target alarm information according to the to-be-selected alarm information may be that, of the to-be-selected alarm information, alarm information with the earliest alarm clock time is used as the target alarm information.

In summary, the method for extracting the target content in the text information by combining the global feature vector and the local feature vector of the text information provided by the embodiment of the application has higher accuracy compared with the bi-LSTM-CRF model.

Specifically, the embodiment of the application performs experiments on two application scenarios, namely a billing bill and a verification code, to verify the effectiveness of the model. The experimental data set consists of 2640 bank payment short messages authorized by a user and 400 verification code short messages, and is divided into a training set, a verification set and a test set according to the proportion of 7. The experimental model comprises: the experimental results of the bi-LSTM-CRF model and the extraction model of the embodiment of the application are shown in tables 2 and 3, wherein the table 2 is the experimental result of the bill receiving and paying scene, and the table 3 is the experimental result of the short message verification code scene.

TABLE 2

Method	Accuracy (Acc.)	F1 value (F1 score)
			Text information processing method of embodiment of application	99.97	99.86
Method based on bi-LSTM-CRF	99.87	99.22

TABLE 3

Model (model)	Accuracy (Acc.)	F1 value (F1 score)
			Text information processing method of embodiment of application	95.19	82.89
Method based on bi-LSTM-CRF	94.87	80.27

As can be seen from tables 2 and 3, the extraction model provided in the embodiment of the present application is superior to the prior art in both evaluation indexes, thereby proving the effectiveness of the text information processing method in the embodiment of the present application. Especially, the verification code entity is identified, and the F1 value can reach more than 99%, so that the extraction model of the embodiment of the application can be competent for conveniently and quickly inputting the verification code. The F1 value (F1 Score) is an index used to measure the accuracy of the two-class model in statistics. The method simultaneously considers the accuracy rate and the recall rate of the classification model.

The extraction model of the embodiment of the application can extract the short message entity information end to end (for example, from a server to a user terminal, or from a text processing client in the user terminal to a client executing target operation), so that the consumption of labor cost is reduced, the accuracy of short message entity identification is improved, the quality of bottom layer data is guaranteed, and the experience of downstream tasks (namely, target operation execution) is better.

Referring to fig. 14, a block diagram of a text information processing apparatus 1400 according to an embodiment of the present application is shown. The text information comprises a first number of text units, and the apparatus may comprise: an acquisition unit 1410, a determination unit 1420, and an extraction unit 1430.

An obtaining unit 1410, configured to obtain a local feature vector and a global feature vector of a text unit to be processed, where the text unit to be processed is a second number of text units in the first number of text units, and the second number is smaller than or equal to the first number.

A determining unit 1420, configured to obtain a label of the text unit to be processed according to the local feature vector and the global feature vector.

The extracting unit 1430 is configured to extract the target content from the text information according to the tag.

Referring to fig. 15, a block diagram of a text information processing apparatus 1500 according to an embodiment of the present application is shown. The text information comprises a first number of text units, the apparatus may comprise: an acquisition unit 1510, a determination unit 1520, an extraction unit 1530 and a business unit 1540.

An obtaining unit 1510, configured to obtain a local feature vector and a global feature vector of a text unit to be processed, where the text unit to be processed is a second number of text units in the first number of text units, and the second number is smaller than or equal to the first number.

Further, the obtaining unit 1510 is further configured to obtain a feature vector of the text unit to be processed; determining a window vector of a text unit to be processed according to a window parameter and a feature vector which are acquired in advance; and acquiring a local feature vector of the text unit to be processed according to the window vector. Wherein the text information to be processed is a short message text.

The determining unit 1520 is configured to obtain a label of the text unit to be processed according to the local feature vector and the global feature vector.

Further, the determining unit 1520 is further configured to obtain a first weight corresponding to the local feature vector of the text unit to be processed and a second weight corresponding to the global feature vector; obtaining a fusion vector of the text unit to be processed according to the local feature vector, the global feature vector, the first weight and the second weight; and determining the label of the text unit to be processed according to the fusion vector.

Further, the determining unit 1520 is further configured to obtain a feature vector of the text unit to be processed; and determining a first weight and a second weight of the text unit to be processed according to the feature vector.

Further, the determining unit 1520 is further configured to determine, according to the feature vector, a score of the text unit to be processed; and determining a first weight and a second weight of the text unit to be processed according to the scores.

Further, the determining unit 1520 is further configured to determine a score of the text unit to be processed according to the following formula:

a _t ＝σ(Wx _t +b)，

wherein, a _t Is the score of the t-th text unit, W is the weight matrix, x _t B is a bias value and sigma is a probability normalization function; will be (1-a) _t ) As a first weight of the t-th text unit, a _t And the second weight is used as the second weight of the t text unit, wherein t is a natural number not greater than n, and n is the second number.

Further, the determining unit 1520 is further configured to determine a fusion vector according to the following formula:

r _t ＝a _t ⊙h _t-1 +(1-a _t )⊙l _t ，

wherein, h is a dot-by-dot product operator _t Global feature vector for the t-th text unit, l _t Local feature vector for the t-th text unit, r _t And the fusion vector is a fusion vector of t text units, wherein t is a natural number not greater than n, and n is the second number.

Further, the determining unit 1520 is further configured to obtain a label of the text unit to be processed according to the conditional random field model and the fusion vector of the text unit to be processed.

An extracting unit 1530, configured to extract the target content from the text information to be processed according to the tag.

The service unit 1540 is configured to instruct the user terminal to execute the target operation according to the target content.

Further, the business unit 1540 is further configured to generate bill reminding information according to the consumption content; and displaying the bill reminding information on a screen of the user terminal.

Further, the service unit 1540 is further configured to send the verification code to the client, and instruct the client to input the verification code into the currently displayed verification code input area.

Further, the business unit 1540 is further configured to generate travel prompting information according to the travel content; and displaying the travel reminding information on a screen of the user terminal.

It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described apparatuses and modules may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

In the several embodiments provided in the present application, the coupling between the modules may be electrical, mechanical or other type of coupling.

In addition, functional modules in the embodiments of the present application may be integrated into one processing module, or each of the modules may exist alone physically, or two or more modules are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode.

Referring to fig. 16, a block diagram of an electronic device according to an embodiment of the present disclosure is shown. The electronic device 10 may be a smart phone, a tablet computer, an electronic book, a computer, etc. capable of running an application. As an implementation manner, the electronic device 10 in the present application may be the server described above, and then the electronic device may execute the method embodiments of fig. 2 to fig. 7 described above. As another implementation, the electronic device may also be a user terminal, configured to execute the method embodiment in fig. 8.

The electronic device 10 in the present application may include one or more of the following components: a processor 110, a memory 120, and one or more applications, wherein the one or more applications may be stored in the memory 120 and configured to be executed by the one or more processors 110, the one or more programs configured to perform a method as described in the aforementioned method embodiments.

Processor 110 may include one or more processing cores. The processor 110 interfaces with various components throughout the electronic device 100 using various interfaces and circuitry to perform various functions of the electronic device 10 and process data by executing or executing instructions, programs, code sets, or instruction sets stored in the memory 120 and invoking data stored in the memory 120. Alternatively, the processor 110 may be implemented in hardware using at least one of Digital Signal Processing (DSP), field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor 110 may integrate one or more of a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), a modem, and the like. Wherein, the CPU mainly processes an operating system, a user interface, an application program and the like; the GPU is used for rendering and drawing display content; the modem is used to handle wireless communications. It is understood that the modem may not be integrated into the processor 110, but may be implemented by a communication chip.

The Memory 120 may include a Random Access Memory (RAM) or a Read-Only Memory (Read-Only Memory). The memory 120 may be used to store instructions, programs, code sets, or instruction sets. The memory 120 may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing various method embodiments described below, and the like. The storage data area may also store data created by the terminal in use, such as a phonebook, audio-video data, chat log data, and the like.

Referring to fig. 17, a block diagram of a computer-readable storage medium according to an embodiment of the present application is shown. The computer-readable medium 1700 has stored therein program code that can be invoked by a processor to perform the methods described in the method embodiments above.

The computer-readable storage medium 1700 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read only memory), an EPROM, a hard disk, or a ROM. Alternatively, the computer-readable storage medium 1700 includes a non-volatile computer-readable storage medium. The computer readable storage medium 1700 has storage space for program code 1710 for performing any of the method steps described above. The program code can be read from or written to one or more computer program products. Program code 1710 may be compressed, for example, in a suitable form.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, modifications may be made to the technical solutions described in the foregoing embodiments, or some technical features may be replaced with equivalents; such modifications and substitutions do not necessarily depart from the spirit and scope of the corresponding technical solutions in the embodiments of the present application.

Claims

1. A text information processing method is characterized in that the text information comprises a first number of text units, the text information is a short message text, and the method comprises the following steps:

acquiring local feature vectors and global feature vectors of text units to be processed, wherein the text units to be processed are a second number of text units in the first number of text units, and the second number is smaller than or equal to the first number;

acquiring the score of each text unit to be processed;

determining a first weight and a second weight of each text unit to be processed based on the score of the text unit to be processed, wherein the sum of the first weight and the second weight of each text unit to be processed is 1;

taking the product of the first weight of each text unit to be processed and the local feature vector of the text unit to be processed as a first vector factor of the text unit to be processed;

taking the product of the second weight of each text unit to be processed and the global feature vector of the text unit to be processed as a second vector factor of the text unit to be processed;

taking the sum of the first vector factor and the second vector factor of each text unit to be processed as a fusion vector of the text unit to be processed;

determining the label of the text unit to be processed according to the fusion vector;

extracting target content from the text information according to the label, wherein the target content comprises trip content, and the trip content comprises trip time and trip date;

generating travel reminding information according to the travel content;

acquiring all alarm information set by an alarm clock client in a user terminal, wherein each alarm information comprises an alarm clock time and an alarm date corresponding to the alarm clock time;

searching alarm information to be selected from all alarm information, wherein the alarm date of the alarm information to be selected is matched with the trip date, and the alarm clock time of the alarm information to be selected is before the trip time;

taking the alarm clock information to be selected which is the earliest in time as target alarm clock information;

and when the user is reminded through the target alarm information, displaying the travel reminding information in the target alarm interface corresponding to the target alarm information.

2. The method of claim 1, wherein obtaining the score of each of the text units to be processed, and determining the first weight and the second weight of each of the text units to be processed based on the score of the text unit to be processed comprises:

determining the score of the text unit to be processed according to:

，

wherein, a _t Is the score of the t-th text unit, W is the weight matrix, x _t B is a bias value and sigma is a probability normalization function;

will be (1-a) _t ) As a first weight of the t-th text unit, a _t And the second weight is used as the second weight of the t text unit, wherein t is a natural number not greater than n, and n is the second number.

3. The method as claimed in claim 2, wherein said determining a sum of the first vector factor and the second vector factor of each of the text units to be processed as a fused vector of the text unit to be processed comprises:

determining the fusion vector according to:

r _t =a _t ⊙h _t-1 +(1-a _t )⊙l _t ，

wherein, h is a dot-by-dot product operator _t Global feature vector for the t-th text unit, l _t Local feature vector for the t-th text unit, r _t Is a fusion vector of t text units, wherein t isA natural number not greater than n, n being the second number.

4. The method according to claim 1, wherein said obtaining the local feature vector of the text unit to be processed comprises:

acquiring a feature vector of the text unit to be processed;

determining a window vector of the text unit to be processed according to a window parameter and the feature vector which are acquired in advance;

and acquiring the local feature vector of the text unit to be processed according to the window vector.

5. A short message processing method is applied to a user terminal, and the method comprises the following steps:

the method according to any one of claims 1 to 4, obtaining target content of the short message based on a local feature vector and a global feature vector corresponding to a text unit in the short message, wherein the target content includes travel content, and the travel content includes a travel time and a travel date;

generating travel reminding information according to the travel content;

acquiring all alarm clock information set by an alarm clock client in a user terminal, wherein each alarm clock information comprises an alarm clock time and an alarm clock date corresponding to the alarm clock time;

6. An electronic device, comprising:

one or more processors;

a memory;

one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more programs configured to perform the method of claim 5.

7. A computer-readable medium, characterized in that the readable storage medium stores program code executable by a processor, the program code causing the processor to perform the method of any one of claims 1-4 when executed by the processor.