CN111898363A - Method and device for compressing long and difficult sentences of text, computer equipment and storage medium - Google Patents

Method and device for compressing long and difficult sentences of text, computer equipment and storage medium Download PDF

Info

Publication number
CN111898363A
CN111898363A CN202010733600.4A CN202010733600A CN111898363A CN 111898363 A CN111898363 A CN 111898363A CN 202010733600 A CN202010733600 A CN 202010733600A CN 111898363 A CN111898363 A CN 111898363A
Authority
CN
China
Prior art keywords
text information
type
text
neural network
network model
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010733600.4A
Other languages
Chinese (zh)
Other versions
CN111898363B (en
Inventor
李小娟
徐国强
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ping An Technology Shenzhen Co Ltd
Original Assignee
Ping An Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ping An Technology Shenzhen Co Ltd filed Critical Ping An Technology Shenzhen Co Ltd
Priority to CN202010733600.4A priority Critical patent/CN111898363B/en
Publication of CN111898363A publication Critical patent/CN111898363A/en
Priority to PCT/CN2021/097418 priority patent/WO2022022049A1/en
Application granted granted Critical
Publication of CN111898363B publication Critical patent/CN111898363B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/205—Parsing
    • G06F40/211—Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/33—Querying
    • G06F16/332—Query formulation
    • G06F16/3329—Natural language query formulation or dialogue systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/279—Recognition of textual entities
    • G06F40/289—Phrasal analysis, e.g. finite state techniques or chunking
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/30—Semantic analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

The invention relates to the technical field of artificial intelligence, and particularly discloses a method and a device for compressing long difficult sentences of texts, computer equipment and a computer readable storage medium, wherein the method comprises the following steps: acquiring text information to be processed; determining the type of the text information according to a first preset neural network model; and if the type of the text information is the core type, compressing the text information according to a second preset neural network model to obtain a core sentence of the text information, so that the problem of low semantic matching accuracy of long and short sentences due to overlong sentences during intention identification is solved.

Description

Method and device for compressing long and difficult sentences of text, computer equipment and storage medium
Technical Field
The present application relates to the field of artificial intelligence technologies, and in particular, to a method and an apparatus for compressing long difficult sentences of text, a computer device, and a computer-readable storage medium.
Background
The frequency of the task-based voice intelligent dialogue system in practical application is higher and higher, the input voice of a user is converted into a text through a voice recognizer (ASR), then a natural language understanding component (NLU) carries out semantic understanding, a Dialogue Manager (DM) keeps conversation history and state and manages circulation of conversation nodes, a Natural Language Generator (NLG) generates a dialogue text according to a dialogue strategy of the dialogue manager, and finally the dialogue is synthesized through a voice synthesizer (TTS) and output to the user.
At present, the dialect generated by NLG has the problems of being harsh and inaccurate, the conventional method is to manually arrange a batch of corpora, namely question and answer corpora, the system arranges and simplifies the problems possibly asked by the client into a question and answer library, when the input of the user is matched with a certain problem in the corpora, the answer corresponding to the problem is returned, and the problem which is intentionally identified is converted into the problem of text matching. However, in the actual process, it is found that when some overdue questions are asked in a conversation with a client, the answers of the user usually have serious spoken language, disordered logic and repeated sentences, and the core sentences in the user sentences cannot be accurately extracted.
Disclosure of Invention
The application mainly aims to provide a method, a device, a computer device and a computer readable storage medium for compressing long and difficult sentences of a text, and aims to solve the technical problems that in a dialogue with a client, when some overdue questions are inquired, answers of the user usually have serious spoken language, logic confusion and repeated sentences, and core sentences in user sentences cannot be accurately extracted.
In a first aspect, the present application provides a method for compressing a long hard sentence, including the following steps:
acquiring text information to be processed;
determining the type of the text information according to a first preset neural network model;
and if the type of the text information is the core type, compressing the text information according to a second preset neural network model to obtain a core sentence of the text information.
In a second aspect, the present application also provides a device for compressing a long hard sentence of text, including:
the acquisition module is used for acquiring text information to be processed;
the determining module is used for determining the type of the text information according to a first preset neural network model;
and the compression acquisition module is used for compressing the text information according to a second preset neural network model to acquire a core sentence of the text information if the type of the text information is the core type.
In a third aspect, the present application further provides a computer device, which includes a processor, a memory, and a computer program stored on the memory and executable by the processor, wherein the computer program, when executed by the processor, implements the steps of the method for compressing a long hard sentence.
In a fourth aspect, the present application further provides a computer-readable storage medium, having a computer program stored thereon, where the computer program, when executed by a processor, implements the steps of the method for compressing long hard sentences of text as described above.
The application provides a method and a device for compressing long and difficult sentences of a text, computer equipment and a computer readable storage medium, wherein the text information to be processed is obtained; determining the type of the text information according to a first preset neural network model; and if the type of the text information is the core type, compressing the text information according to a second preset neural network model to obtain a core sentence of the text information, so that the problem of low semantic matching accuracy of long and short sentences due to overlong sentences during intention identification is solved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a schematic flow chart of a method for compressing a long hard sentence in a text according to an embodiment of the present application;
FIG. 2 is a flow diagram illustrating sub-steps of a method for compressing a text long hard sentence in FIG. 1;
FIG. 3 is a flow diagram illustrating sub-steps of a method for compressing a text long hard sentence in FIG. 1;
fig. 4 is a schematic flowchart of another method for compressing a long hard sentence in a text according to an embodiment of the present application;
FIG. 5 is a schematic block diagram of a device for compressing a long hard sentence in a text according to an embodiment of the present application;
fig. 6 is a block diagram schematically illustrating a structure of a computer device according to an embodiment of the present application.
The implementation, functional features and advantages of the objectives of the present application will be further explained with reference to the accompanying drawings.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some, but not all, embodiments of the present application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The flow diagrams depicted in the figures are merely illustrative and do not necessarily include all of the elements and operations/steps, nor do they necessarily have to be performed in the order depicted. For example, some operations/steps may be decomposed, combined or partially combined, so that the actual execution sequence may be changed according to the actual situation.
The embodiment of the application provides a method and a device for compressing long and difficult sentences of a text, computer equipment and a computer readable storage medium. The method for compressing the long and difficult text sentences can be applied to terminal equipment, and the terminal equipment can be electronic equipment such as a mobile phone, a tablet computer, a notebook computer and a desktop computer.
Some embodiments of the present application will be described in detail below with reference to the accompanying drawings. The embodiments described below and the features of the embodiments can be combined with each other without conflict.
Referring to fig. 1, fig. 1 is a flowchart illustrating a method for compressing a long hard sentence in a text according to an embodiment of the present application.
As shown in fig. 1, the method for compressing a long hard sentence in text includes steps S101 to S103.
S101, acquiring text information to be processed;
the method comprises the steps of obtaining text information, and using the obtained text information as text information to be processed, wherein the text information comprises a plurality of sections of sentences, for example, "how to say with you". Do you don't want to say an hour because, i do not say i do not want to give you a happy feeling, do not want to open the mouth, give that what if you get a bill, what is doing soon, do not good? Call you to say it ". The acquisition mode comprises the step of converting voice information of a user into text information when the voice information is received.
S102, determining the type of the text information according to a first preset neural network model;
and determining the type of the text information according to the first preset neural network model, wherein the type comprises an irrelevant type and a core type. An example is that the text information is input to a first preset neural network model, the first preset neural network model identifies a keyword in the text information, and a sentence containing the keyword is determined to be of an irrelevant type. For example, when it is recognized that a sentence in the text information contains that what, it is determined that the sentence is of an irrelevant type. The first preset model is a neural network model obtained by training through identification sentences in advance.
In an embodiment, specifically referring to fig. 2, step S102 includes: substeps 1021 to substep S1025.
Substep S1021, inputting the text information into a first preset neural network model;
and after the text information is obtained, inputting the text information into a preset first neural network model. The text information includes a sentence, etc. For example, how you say, how i follow you say that this phone was given to the last owner who i am not really aware of, etc. And acquiring an input layer of the first preset neural network model, and inputting the acquired sentences, which are given by the previous unit, to the input layer of the first preset neural network model, so that the last owner who and does not know the sentences really, wherein the sentences are provided by the previous unit.
Substep S1022, converting the text information into a feature matrix through a word vector layer of the first preset neural network model;
when the obtained text information is what you say, the input words are not clearly heard and converted into texts in the characteristic matrix representation through an external pre-trained word vector layer with the dimension of b in a first preset neural network model, the words are firstly separated by characters to obtain 'I, No, hear, clear and clear', and the words are mapped to a b-dimensional space of the vector through a Chinese character vector of external pre-training to obtain the characteristic matrix of the text.
Step S1023, performing convolution operation on the feature matrix based on a convolution core of the first preset neural network model, extracting context feature information in the text information, and obtaining target feature vector information of the text information;
on the feature matrix of the text information obtained in the text, performing convolution operation through convolution kernels of different sizes of a first preset neural network model to extract different text context feature information, similar to different N in an N-gram language model, for example, a sentence "i am not clearly heard", where the contexts corresponding to different N include:
n is 2, I doesn't hear, hear clearly and clearly
n is 3, I can not hear, can not hear clearly, hear clearly
n is 4, i's not clearly listening to
And performing convolution operation on the feature matrix and convolution kernels with different sizes to obtain a feature vector corresponding to each convolution kernel convolution operation, and obtaining a target feature vector corresponding to the text information according to the feature vector corresponding to each convolution kernel convolution operation.
In an embodiment, obtaining the target feature vector information of the text information includes: multiplying the convolution kernel by elements at the corresponding positions of the characteristic matrix and summing to obtain corresponding first target characteristics; multiplying and summing the elements at the next position of the feature matrix through the convolution kernel to obtain a corresponding second target feature; and splicing the obtained first target feature and the second target feature to generate target feature vector information.
And performing convolution operation on the feature matrix and convolution kernels with different sizes respectively to obtain feature vectors (feature maps) corresponding to the convolution operation of each convolution kernel, wherein one convolution result corresponds to one feature map. For example, a convolution kernel with a size of 3 × 5 is convolved in the feature matrix, that is, the convolution kernel is multiplied by elements at corresponding positions in the feature matrix and summed, the convolution kernel is moved one step down after one calculation, the same operation is continued until the bottom end is reached, and the obtained sums are spliced together to form a feature map.
For example, 1 × 0+1 × 0+ (-1) × 0+0 × 0+ (-1) × 1+1 × 0+0 × 2+1 × 0+ (-1) × 0+1 × 0+1 × 1+1 × 1+ (-1) × 0 ═ 1, obtains the first target feature, obtains the element at the next position for calculation, obtains the second target feature 1 and the third target feature-4, respectively, for concatenation, obtaining the target feature vector information.
Step S1024, determining a type probability value of the text information output by the first preset neural network model according to the target feature vector information;
extracting the maximum value in each target feature vector information (feature map) through a one-dimensional maximum pooling (1-max Pooling) operation in a first preset neural network model, and then obtaining a final feature expression vector through a splicing (concat) operation. And classifying the feature expression vectors to obtain the type probability value of the text information. Thereby determining a probability value of the first preset neural network model outputting the text information.
In one embodiment, determining the type probability value that the first preset neural network model outputs the text information comprises: extracting the maximum value in the target feature vector information through the maximum pooling of the first neural network model to obtain a feature expression vector of the text information; and classifying the feature expression vectors to respectively obtain the probability value of the irrelevant type and the probability value of the core type.
The final feature expression vector is obtained through the maximum value in each target feature vector information (feature map) and then through the concatenation (concat) operation. Obtaining a probability formula in a first preset neural network model
Figure BDA0002604156880000051
Classifying the feature vector of the text information by the formula, wherein yiIs (1, 2), h is the metric, x is the type of input. Probability values for two types, namely the irrelevant class and the core class, are obtained.
And a substep S1025 of determining the type of the text information based on the type probability value.
When the probability value of the text information is acquired, the type of the text information is determined through the probability value. The types of text include an irrelevant type and a core type. For example, when the probability value is greater than the preset probability value, determining that the text type is an irrelevant type; and when the probability value is less than or equal to the preset probability value, determining the text type as the core type. Or when the probability value is greater than the preset probability value, determining that the text type is the core type; and when the probability value is less than or equal to the preset probability value, determining the text type as an irrelevant type.
In one embodiment, determining the type of the text information comprises: comparing the probability value of the irrelevant type with the probability value of the core type; if the probability value of the irrelevant type is greater than the probability value of the core type, determining that the type of the text information is the irrelevant type; and if the probability value of the irrelevant type is less than or equal to the probability value of the core type, determining that the type of the text information is the core type.
When the probability values of the irrelevant type and the core type are respectively obtained, comparing the obtained probability value of the irrelevant type with the obtained probability value of the core type, for example, if the obtained probability value of the irrelevant type is 0.6 and the obtained probability value of the core type is 0.4, determining that the type of the text information is the irrelevant type; and if the obtained probability value of the irrelevant type is 0.3 and the probability value of the core type is 0.7, determining that the type of the text information is the core type.
And S103, if the type of the text information is the core type, compressing the text information according to a second preset neural network model to obtain a core sentence of the text information.
And if the text information type is determined to be the core type, performing word segmentation on the text information through second preset model identification. And acquiring target participles with attributes such as subjects, predicates, objects and the like in the participles, and combining the target participles to acquire a core sentence of the text information. For example, the text information is "i say that i am who and me do not know who and me do the last owner given to this telephone in the previous unit", the second preset model divides "i am who and me do not know really who and me do the last owner given to this telephone in the previous unit", the text divided into the divided words is "i am, me, follow, your, say, me, this, telephone, be, before, unit, give, last owner, who and me do not know really", the main guest in this sentence is determined, and the combined core sentence is "i am who and me do not know what and the last owner of this telephone in the previous unit".
In an embodiment, specifically referring to fig. 3, step S103 includes: substeps 1031 to substep S1033.
Substep S1031, inputting the text information into a second preset neural network model, and performing word segmentation on the text information through the second preset neural network;
when the text information is determined to be the core type, the text information is input into a second preset neural network model, the second preset neural network model is obtained by training a large amount of text information to be labeled, words in the large amount of text information are labeled, and the words in the text information are labeled according to position and attribute relations among the words in the dependency syntax. Dependency syntax explains its syntax structure by analyzing the dependency relationship before the components in the language unit, proposing that the core verb in the sentence is the central component that governs the other components. But is not itself subject to any other constituent, all subject constituents being subject to a subject in some relationship. And segmenting the text information based on a segmentation word table or a segmentation word library in a second preset neural network.
Substep S1032, obtaining a weight matrix of each participle to determine an attribute relationship among the participles;
and acquiring a weight matrix of each participle through a hidden layer in the second preset neural network model. For example, after the weight matrix of each word is obtained, attribute vector feature information of each word is obtained based on the mapping of the weight matrix. And determining the attribute relation among the participles based on the attribute vector characteristic information of each participle. For example, a predicate relationship, a middle relationship, a left append relationship, a right append relationship, and a move guest relationship. Only one component in a sentence is independent; other components of the sentence are subordinate to a certain component; neither component can depend on two or more components; if component a is directly dependent on component B, and component C is located between a and B in the sentence, then component C is either dependent on a, or dependent on B, or dependent on some component between a and B; the other components on the left and right sides of the central component are not related to each other.
And a substep S1033 of obtaining a core sentence of the text information output by the second preset neural network model based on the attribute relationship among the participles.
Obtaining the main-predicate relationship, the centering relationship, the left additional relationship, the right additional relationship and the action-guest relationship of each participle, and determining whether the main-predicate relationship, the centering relationship, the left additional relationship, the right additional relationship and the action-guest relationship exist among the participles or not; and combining the participles with the main-predicate relationship, the middle relationship, the left additional relationship, the right additional relationship and the action-guest relationship to determine a core sentence to be output. And outputting the core sentence to be output through a second preset neural network model, and taking the core sentence to be output as the core sentence of the text information.
In the embodiment of the invention, the text information to be processed is obtained, and the type of the text information is determined according to the first preset neural network model. And if the type of the text information is determined to be the core type, compressing the text information according to a second preset neural network model to obtain a core sentence of the text information. The method can solve the problem that the semantic matching accuracy of long and short sentences is low due to too long sentences during intention recognition; meanwhile, by extracting the key information, redundant information of sentences can be removed, and the key information is reserved, so that the intention identification accuracy is improved.
Referring to fig. 4, fig. 4 is a schematic flow chart of another data acquisition method according to an embodiment of the present disclosure.
As shown in fig. 4, the data acquisition method includes steps S201 to S205.
Step S201, acquiring text information to be processed;
the method comprises the steps of obtaining text information, and using the obtained text information as text information to be processed, wherein the text information comprises a plurality of sections of sentences, for example, "how to say with you". Do you don't want to say an hour because, i do not say i do not want to give you a happy feeling, do not want to open the mouth, give that what if you get a bill, what is doing soon, do not good? Call you to say it ". The acquisition mode comprises the step of converting voice information of a user into text information when the voice information is received.
Step S202, determining the type of the text information according to a first preset neural network model;
and determining the type of the text information according to the first preset neural network model, wherein the type comprises an irrelevant type and a core type. An example is that the text information is input to a first preset neural network model, the first preset neural network model identifies a keyword in the text information, and a sentence containing the keyword is determined to be of an irrelevant type. For example, when it is recognized that a sentence in the text information contains that what, it is determined that the sentence is of an irrelevant type. The first preset model is a neural network model obtained by training through identification sentences in advance.
And step S203, if the type of the text information is the core type, compressing the text information according to a second preset neural network model to obtain a core sentence of the text information.
And if the text information type is determined to be the core type, performing word segmentation on the text information through second preset model identification. And acquiring target participles with attributes such as subjects, predicates, objects and the like in the participles, and combining the target participles to acquire a core sentence of the text information. For example, the text information is "i say that i am who and me do not know who and me do the last owner given to this telephone in the previous unit", the second preset model divides "i am who and me do not know really who and me do the last owner given to this telephone in the previous unit", the text divided into the divided words is "i am, me, follow, your, say, me, this, telephone, be, before, unit, give, last owner, who and me do not know really", the main guest in this sentence is determined, and the combined core sentence is "i am who and me do not know what and the last owner of this telephone in the previous unit".
Step S204, matching the core sentence with a preset question-answer library to obtain a question-answer text matched with the core sentence;
and when the core sentence of the text information is acquired, matching the core sentence with a preset question-answer library. The preset question-answer library comprises a text, wherein the question-answer text comprises questions and corresponding conversational information. And matching the core sentence with a preset question-answer library to obtain a question-answer text matched with the core sentence. And if a plurality of question and answer texts are obtained, obtaining the frequency of each question and answer text matched with the core sentence, comparing the frequency of each question and answer text matched with the core sentence, and obtaining the question and answer text corresponding to the maximum frequency.
And S205, sending the dialect information in the question and answer text to the user based on the question and answer text.
And when a question and answer text matched with the to-be-core sentence is acquired, transmitting the dialect information in the question and answer text to the user. The sending mode comprises sending the dialect information to the user in the form of characters or pictures, and converting the dialect information into voice information to be sent to the user in the form of voice information.
In the embodiment of the invention, the text information to be processed is obtained, the type of the text information is determined according to a first preset neural network model, and if the type of the text information is determined to be the core type, the text information is compressed according to a second preset neural network model to obtain the core sentence of the text information. The method can solve the problem that the semantic matching accuracy of long and short sentences is low due to too long sentences during intention recognition; meanwhile, by extracting the key information, redundant information of sentences can be removed, the key information is reserved, so that the intention recognition accuracy is improved, and corresponding voice information is quickly sent to a user.
Referring to fig. 5, fig. 5 is a schematic block diagram of a device for compressing long hard sentences in text according to an embodiment of the present application.
As shown in fig. 5, the apparatus 400 for compressing a long hard sentence includes: an obtaining module 401, a first determining module 402, and a compression obtaining module 403.
An obtaining module 401, configured to obtain text information to be processed;
a determining module 402, configured to determine a type of the text information according to a first preset neural network model;
a compression obtaining module 403, configured to, if the type of the text information is a core type, compress the text information according to a second preset neural network model, and obtain a core sentence of the text information.
Wherein the determining module 402 is further specifically configured to: inputting the text information into a first preset neural network model; converting the text information into a feature matrix through a word vector layer of the first preset neural network model; performing convolution operation on the feature matrix based on the convolution core of the first preset neural network model, and extracting context feature information in the text information to obtain target feature vector information of the text information; determining a type probability value of the text information output by the first preset neural network model according to the target feature vector information; determining the type of the text information based on the type probability value.
Wherein the determining module 402 is further specifically configured to: multiplying the convolution kernel by elements at the corresponding positions of the characteristic matrix and summing to obtain corresponding first target characteristics; multiplying and summing the elements at the next position of the feature matrix through the convolution kernel to obtain a corresponding second target feature; and splicing the obtained first target feature and the second target feature to generate target feature vector information.
Wherein the determining module 402 is further specifically configured to: extracting the maximum value in the target feature vector information through the maximum pooling of the first neural network model to obtain a feature expression vector of the text information; and classifying the feature expression vectors to respectively obtain the probability value of the irrelevant type and the probability value of the core type.
Wherein the determining module 402 is further specifically configured to: comparing the probability value of the irrelevant type with the probability value of the core type; if the probability value of the irrelevant type is greater than the probability value of the core type, determining that the type of the text information is the irrelevant type; and if the probability value of the irrelevant type is less than or equal to the probability value of the core type, determining that the type of the text information is the core type.
The compression obtaining module 403 is further specifically configured to: inputting the text information into a second preset neural network model, and segmenting the text information through the second preset neural network; acquiring a weight matrix of each participle to determine an attribute relationship among the participles; and acquiring a core sentence of the text information output by the second preset neural network model based on the attribute relation among the participles.
The device for compressing the long and difficult sentences of the text is specifically further used for: matching the core sentence with a preset question-answer library to obtain a question-answer text matched with the core sentence; and sending the dialect information in the question and answer text to the user based on the question and answer text.
It should be noted that, as will be clear to those skilled in the art, for convenience and brevity of description, the specific working processes of the above-described apparatus and each module and unit may refer to the corresponding processes in the foregoing embodiments of the text long hard sentence compression method, and are not described herein again.
The apparatus provided by the above embodiments may be implemented in the form of a computer program, which can be run on a computer device as shown in fig. 6.
Referring to fig. 6, fig. 6 is a schematic block diagram illustrating a structure of a computer device according to an embodiment of the present disclosure. The computer device may be a terminal.
As shown in fig. 6, the computer device includes a processor, a memory, and a network interface connected by a system bus, wherein the memory may include a nonvolatile storage medium and an internal memory.
The non-volatile storage medium may store an operating system and a computer program. The computer program includes program instructions that, when executed, cause a processor to perform any one of a number of methods for compressing long difficult sentences of text.
The processor is used for providing calculation and control capability and supporting the operation of the whole computer equipment.
The internal memory provides an environment for the execution of a computer program on a non-volatile storage medium, which when executed by a processor causes the processor to perform any method of compressing long text phrases.
The network interface is used for network communication, such as sending assigned tasks and the like. Those skilled in the art will appreciate that the architecture shown in fig. 6 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
It should be understood that the Processor may be a Central Processing Unit (CPU), and the Processor may be other general purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components, etc. Wherein a general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
Wherein, in one embodiment, the processor is configured to execute a computer program stored in the memory to implement the steps of:
acquiring text information to be processed; determining the type of the text information according to a first preset neural network model; and if the type of the text information is the core type, compressing the text information according to a second preset neural network model to obtain a core sentence of the text information.
In one embodiment, the processor, when said determining the type of the text information is implemented according to a first preset model, is configured to implement:
inputting the text information into a first preset neural network model; converting the text information into a feature matrix through a word vector layer of the first preset neural network model; performing convolution operation on the feature matrix based on the convolution core of the first preset neural network model, and extracting context feature information in the text information to obtain target feature vector information of the text information; determining a type probability value of the text information output by the first preset neural network model according to the target feature vector information; determining the type of the text information based on the type probability value.
In one embodiment, when obtaining the target feature vector information implementation of the text information, the processor is configured to implement:
multiplying the convolution kernel by elements at the corresponding positions of the characteristic matrix and summing to obtain corresponding first target characteristics; multiplying and summing the elements at the next position of the feature matrix through the convolution kernel to obtain a corresponding second target feature; and splicing the obtained first target feature and the second target feature to generate target feature vector information.
In one embodiment, when determining that the type probability value of the text output by the first preset neural network model is realized according to the target feature vector information, the processor is configured to realize:
extracting the maximum value in the target feature vector information through the maximum pooling of the first neural network model to obtain a feature expression vector of the text information; and classifying the feature expression vectors to respectively obtain the probability value of the irrelevant type and the probability value of the core type.
In one embodiment, the processor, when determining that the type of the text information is implemented, is configured to implement:
comparing the probability value of the irrelevant type with the probability value of the core type; if the probability value of the irrelevant type is greater than the probability value of the core type, determining that the type of the text information is the irrelevant type; and if the probability value of the irrelevant type is less than or equal to the probability value of the core type, determining that the type of the text information is the core type.
In an embodiment, when the processor compresses the text information according to the second preset neural network model to obtain a core sentence of the text information, the processor is configured to:
inputting the text information into a second preset neural network model, and segmenting the text information through the second preset neural network; acquiring a weight matrix of each participle to determine an attribute relationship among the participles; and acquiring a core sentence of the text information output by the second preset neural network model based on the attribute relation among the participles.
In one embodiment, when the processor is implemented after compressing the text information according to a second preset neural network model and acquiring a core sentence of the text information, the processor is configured to implement:
matching the core sentence with a preset question-answer library to obtain a question-answer text matched with the core sentence; and sending the dialect information in the question and answer text to the user based on the question and answer text.
Embodiments of the present application further provide a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, where the computer program includes program instructions, and when the program instructions are executed, a method implemented by the computer program instructions may refer to various embodiments of a method for compressing a long hard sentence according to the present application.
The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiment, for example, a hard disk or a memory of the computer device. The computer readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like provided on the computer device.
Further, the computer-readable storage medium may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function, and the like; the storage data area may store data created according to the use of the blockchain node, and the like.
The block chain is a novel application mode of computer technologies such as preset text storage, point-to-point transmission, a consensus mechanism, an encryption algorithm and the like. A block chain (Blockchain), which is essentially a decentralized database, is a series of data blocks associated by using a cryptographic method, and each data block contains information of a batch of network transactions, so as to verify the validity (anti-counterfeiting) of the information and generate a next block. The blockchain may include a blockchain underlying platform, a platform product service layer, an application service layer, and the like.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The above-mentioned serial numbers of the embodiments of the present application are merely for description and do not represent the merits of the embodiments. While the invention has been described with reference to specific embodiments, the scope of the invention is not limited thereto, and those skilled in the art can easily conceive various equivalent modifications or substitutions within the technical scope of the invention. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (10)

1. A method for compressing a long hard sentence of a text is characterized by comprising the following steps:
acquiring text information to be processed;
determining the type of the text information according to a first preset neural network model;
and if the type of the text information is the core type, compressing the text information according to a second preset neural network model to obtain a core sentence of the text information.
2. The method for compressing a long hard sentence in text according to claim 1, wherein the determining the type of the text information according to the first preset model comprises:
inputting the text information into a first preset neural network model;
converting the text information into a feature matrix through a word vector layer of the first preset neural network model;
performing convolution operation on the feature matrix based on the convolution core of the first preset neural network model, and extracting context feature information in the text information to obtain target feature vector information of the text information;
determining a type probability value of the text information output by the first preset neural network model according to the target feature vector information;
determining the type of the text information based on the type probability value.
3. The method for compressing a long hard sentence in text according to claim 2, wherein the obtaining of the target feature vector information of the text information comprises:
multiplying the convolution kernel by elements at the corresponding positions of the characteristic matrix and summing to obtain corresponding first target characteristics;
multiplying and summing the elements at the next position of the feature matrix through the convolution kernel to obtain a corresponding second target feature;
and splicing the obtained first target feature and the second target feature to generate target feature vector information.
4. The method for compressing long hard sentences of text according to claim 2, wherein the determining the type probability value of the first preset neural network model outputting the text according to the target feature vector information comprises:
extracting the maximum value in the target feature vector information through the maximum pooling of the first neural network model to obtain a feature expression vector of the text information;
and classifying the feature expression vectors to respectively obtain the probability value of the irrelevant type and the probability value of the core type.
5. The method of compressing a long hard sentence of text as claimed in claim 4, wherein said determining the type of the text information comprises:
comparing the probability value of the irrelevant type with the probability value of the core type;
if the probability value of the irrelevant type is greater than the probability value of the core type, determining that the type of the text information is the irrelevant type;
and if the probability value of the irrelevant type is less than or equal to the probability value of the core type, determining that the type of the text information is the core type.
6. The method for compressing long hard sentences of text according to claim 1, wherein the compressing the text information according to a second preset neural network model to obtain core sentences of the text information comprises:
inputting the text information into a second preset neural network model, and segmenting the text information through the second preset neural network;
acquiring a weight matrix of each participle to determine an attribute relationship among the participles;
and acquiring a core sentence of the text information output by the second preset neural network model based on the attribute relation among the participles.
7. The method for compressing long hard sentences of text according to claim 1, wherein after the compressing the text information according to the second preset neural network model and obtaining the core sentence of the text information, the method further comprises:
matching the core sentence with a preset question-answer library to obtain a question-answer text matched with the core sentence;
and sending the dialect information in the question and answer text to the user based on the question and answer text.
8. A device for compressing a long hard sentence in a text, comprising:
the acquisition module is used for acquiring text information to be processed;
the determining module is used for determining the type of the text information according to a first preset neural network model;
and the compression acquisition module is used for compressing the text information according to a second preset neural network model to acquire a core sentence of the text information if the type of the text information is the core type.
9. A computer arrangement comprising a processor, a memory, and a computer program stored on the memory and executable by the processor, wherein the computer program, when executed by the processor, implements the steps of the method of compressing a text passage according to any one of claims 1 to 7.
10. A computer-readable storage medium, having a computer program stored thereon, wherein the computer program, when executed by a processor, performs the steps of the method for compressing long hard sentences of text according to any one of claims 1 to 7.
CN202010733600.4A 2020-07-27 2020-07-27 Compression method, device, computer equipment and storage medium for long and difficult text sentence Active CN111898363B (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202010733600.4A CN111898363B (en) 2020-07-27 2020-07-27 Compression method, device, computer equipment and storage medium for long and difficult text sentence
PCT/CN2021/097418 WO2022022049A1 (en) 2020-07-27 2021-05-31 Long difficult text sentence compression method and apparatus, computer device, and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010733600.4A CN111898363B (en) 2020-07-27 2020-07-27 Compression method, device, computer equipment and storage medium for long and difficult text sentence

Publications (2)

Publication Number Publication Date
CN111898363A true CN111898363A (en) 2020-11-06
CN111898363B CN111898363B (en) 2023-07-28

Family

ID=73190182

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010733600.4A Active CN111898363B (en) 2020-07-27 2020-07-27 Compression method, device, computer equipment and storage medium for long and difficult text sentence

Country Status (2)

Country Link
CN (1) CN111898363B (en)
WO (1) WO2022022049A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022022049A1 (en) * 2020-07-27 2022-02-03 平安科技（深圳）有限公司 Long difficult text sentence compression method and apparatus, computer device, and storage medium
CN117033393A (en) * 2023-10-08 2023-11-10 四川酷赛科技有限公司 Information storage management system based on artificial intelligence

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080109425A1 (en) * 2006-11-02 2008-05-08 Microsoft Corporation Document summarization by maximizing informative content words
US10229111B1 (en) * 2016-02-03 2019-03-12 Google Llc Sentence compression using recurrent neural networks
CN110019758A (en) * 2019-04-11 2019-07-16 北京百度网讯科技有限公司 A kind of key element extracting method, device and electronic equipment
CN110110332A (en) * 2019-05-06 2019-08-09 中国联合网络通信集团有限公司 Text snippet generation method and equipment
CN111444703A (en) * 2020-03-04 2020-07-24 中国平安人寿保险股份有限公司 Statement compression method, device, equipment and computer readable storage medium

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105096942A (en) * 2014-05-21 2015-11-25 清华大学 Semantic analysis method and semantic analysis device
CN106528694B (en) * 2016-10-31 2019-12-06 百度在线网络技术（北京）有限公司 semantic judgment processing method and device based on artificial intelligence
KR102071582B1 (en) * 2017-05-16 2020-01-30 삼성전자주식회사 Method and apparatus for classifying a class to which a sentence belongs by using deep neural network
CN111414471B (en) * 2020-03-20 2023-07-28 北京百度网讯科技有限公司 Method and device for outputting information
CN111898363B (en) * 2020-07-27 2023-07-28 平安科技（深圳）有限公司 Compression method, device, computer equipment and storage medium for long and difficult text sentence

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080109425A1 (en) * 2006-11-02 2008-05-08 Microsoft Corporation Document summarization by maximizing informative content words
US10229111B1 (en) * 2016-02-03 2019-03-12 Google Llc Sentence compression using recurrent neural networks
CN110019758A (en) * 2019-04-11 2019-07-16 北京百度网讯科技有限公司 A kind of key element extracting method, device and electronic equipment
CN110110332A (en) * 2019-05-06 2019-08-09 中国联合网络通信集团有限公司 Text snippet generation method and equipment
CN111444703A (en) * 2020-03-04 2020-07-24 中国平安人寿保险股份有限公司 Statement compression method, device, equipment and computer readable storage medium

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
景秀丽: "Hedge Trimmer句子压缩技术的算法实现及改进", 沈阳师范大学学报（自然科学版）, vol. 30, no. 4, pages 519 - 524 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022022049A1 (en) * 2020-07-27 2022-02-03 平安科技（深圳）有限公司 Long difficult text sentence compression method and apparatus, computer device, and storage medium
CN117033393A (en) * 2023-10-08 2023-11-10 四川酷赛科技有限公司 Information storage management system based on artificial intelligence
CN117033393B (en) * 2023-10-08 2023-12-12 四川酷赛科技有限公司 Information storage management system based on artificial intelligence

Also Published As

Publication number Publication date
WO2022022049A1 (en) 2022-02-03
CN111898363B (en) 2023-07-28

Similar Documents

Publication Publication Date Title
CN111695352A (en) Grading method and device based on semantic analysis, terminal equipment and storage medium
CN108763535B (en) Information acquisition method and device
CN109461437B (en) Verification content generation method and related device for lip language identification
CN112233698B (en) Character emotion recognition method, device, terminal equipment and storage medium
CN108447471A (en) Audio recognition method and speech recognition equipment
CN112650854B (en) Intelligent reply method and device based on multiple knowledge graphs and computer equipment
CN111274797A (en) Intention recognition method, device and equipment for terminal and storage medium
WO2022252636A1 (en) Artificial intelligence-based answer generation method and apparatus, device, and storage medium
CN110597971B (en) Automatic question answering device and method based on neural network and readable storage medium
US20220261545A1 (en) Systems and methods for producing a semantic representation of a document
CN112417855A (en) Text intention recognition method and device and related equipment
CN111223476B (en) Method and device for extracting voice feature vector, computer equipment and storage medium
CN113094478B (en) Expression reply method, device, equipment and storage medium
CN110335608B (en) Voiceprint verification method, voiceprint verification device, voiceprint verification equipment and storage medium
WO2022022049A1 (en) Long difficult text sentence compression method and apparatus, computer device, and storage medium
CN112579733A (en) Rule matching method, rule matching device, storage medium and electronic equipment
CN112632248A (en) Question answering method, device, computer equipment and storage medium
CN113255328A (en) Language model training method and application method
CN112183106A (en) Semantic understanding method and device based on phoneme association and deep learning
CN112818096A (en) Dialog generating method and device
CN111401069A (en) Intention recognition method and intention recognition device for conversation text and terminal
CN114970470B (en) Method and device for processing file information, electronic equipment and computer readable medium
CN113299277A (en) Voice semantic recognition method and system
CN111522937B (en) Speaking recommendation method and device and electronic equipment
CN113987202A (en) Knowledge graph-based interactive telephone calling method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant