CN116911315B - Optimization method, response method and system of natural language processing model - Google Patents

Optimization method, response method and system of natural language processing model Download PDF

Info

Publication number
CN116911315B
CN116911315B CN202311180783.1A CN202311180783A CN116911315B CN 116911315 B CN116911315 B CN 116911315B CN 202311180783 A CN202311180783 A CN 202311180783A CN 116911315 B CN116911315 B CN 116911315B
Authority
CN
China
Prior art keywords
template
description
corpus
model
intention
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202311180783.1A
Other languages
Chinese (zh)
Other versions
CN116911315A (en
Inventor
李犇
张�杰
于皓
罗华刚
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Zhongguancun Kejin Technology Co Ltd
Original Assignee
Beijing Zhongguancun Kejin Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Zhongguancun Kejin Technology Co Ltd filed Critical Beijing Zhongguancun Kejin Technology Co Ltd
Priority to CN202311180783.1A priority Critical patent/CN116911315B/en
Publication of CN116911315A publication Critical patent/CN116911315A/en
Application granted granted Critical
Publication of CN116911315B publication Critical patent/CN116911315B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/30—Semantic analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/33—Querying
    • G06F16/332—Query formulation
    • G06F16/3329—Natural language query formulation or dialogue systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/35—Clustering; Classification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/205—Parsing
    • G06F40/216—Parsing using statistical methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G06N3/0455—Auto-encoder networks; Encoder-decoder networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/01—Customer relationship services
    • G06Q30/015—Providing customer assistance, e.g. assisting a customer within a business location or via helpdesk
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/02—Marketing; Price estimation or determination; Fundraising
    • G06Q30/0281—Customer communication at a business location, e.g. providing product or service information, consulting
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Computational Linguistics (AREA)
  • Business, Economics & Management (AREA)
  • Mathematical Physics (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • Development Economics (AREA)
  • Strategic Management (AREA)
  • Finance (AREA)
  • Accounting & Taxation (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Economics (AREA)
  • Software Systems (AREA)
  • General Business, Economics & Management (AREA)
  • Databases & Information Systems (AREA)
  • Evolutionary Computation (AREA)
  • Biophysics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Marketing (AREA)
  • Human Computer Interaction (AREA)
  • Probability & Statistics with Applications (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Game Theory and Decision Science (AREA)
  • Machine Translation (AREA)

Abstract

The invention provides an optimization method, a response method and a response system of a natural language processing model, and belongs to the technical field of artificial intelligence. The method comprises the following steps: training a general NLP model based on corpus data of the target field to obtain a field NLP model; determining an intention description symbol corresponding to statement description in the corpus data; generating a prompt template corpus through a native prompt template based on the sentence description and the intention description symbol, wherein the native prompt template is used for describing a replacement relationship, and the replacement relationship is a relationship of replacing the sentence description with the intention description symbol; training the domain NLP model based on the prompt template corpus and the native prompt template to obtain an optimized domain NLP model. The invention can be used for providing session message service of machine customer service and clients in the intelligent customer service field.

Description

Optimization method, response method and system of natural language processing model
Technical Field
The invention relates to the technical field of artificial intelligence, in particular to an optimization method of a natural language processing model, an intelligent customer service response method, a conversation intention classification method, a natural language processing system, electronic equipment and a machine-readable storage medium.
Background
In intelligent customer service systems, natural language understanding technology (Natural Language Understanding, NLU) is used to identify the user's intent to talk, which is a key factor in determining system performance and user experience. Therefore, more and more related enterprises and scientific researchers are invested in researching how to utilize a pre-training language model to try to identify the conversation intention of the user, so as to improve the performance and user experience of the intelligent customer service system.
Currently, for testing, intelligent customer service systems can perform simple sentences and dialogs containing high-frequency words with customers through trained machine models (such as NLP models, natural language processing models, natural Language Processing model), but the trained machine models are difficult to identify the intention of customers based on customer dialogs containing low-frequency words and complex sentences, so that online application is difficult to realize dialog responses conforming to the intention of customer dialogs.
Disclosure of Invention
The invention aims to provide an optimization method, a response method and a system of a natural language processing model, which avoid that conversation performance cannot meet intelligent customer service requirements caused by the fact that an NLP model is difficult to identify the intention of a customer conversation, further realize effective response of the customer conversation, and have the characteristic of realizing conversation response aiming at low-frequency words and complex sentences.
In order to achieve the above object, the present specification adopts the following scheme:
in a first aspect, an embodiment of the present invention provides a method for optimizing a natural language processing model, where the method includes:
training a general NLP model based on corpus data of the target field to obtain a field NLP model;
determining an intention description symbol corresponding to statement description in the corpus data;
generating a prompt template corpus through a native prompt template based on the sentence description and the intention description symbol, wherein the native prompt template is used for describing a replacement relationship, and the replacement relationship is a relationship of replacing the sentence description with the intention description symbol;
training the domain NLP model based on the prompt template corpus and the native prompt template to obtain an optimized domain NLP model. The corpus data comprises historical session messages, wherein the historical session messages can be provided by an intelligent customer service system in the target field; the database records in the intelligent customer service system comprise corresponding relation records of statement descriptions and intention description symbols in corpus data; the intent descriptor is determined by querying the database record.
In a second aspect, an embodiment of the present invention provides a response method for intelligent customer service, where the response method includes:
Acquiring statement description;
determining a returned statement description based on the statement description and an optimization domain NLP model; wherein,
the optimization field NLP model is obtained through the optimization method of the natural language processing model.
In a third aspect, an embodiment of the present invention provides a natural language processing system, including:
the training module is used for training the general NLP model based on the corpus data of the target field to obtain a field NLP model;
the determining module is used for determining an intention descriptor corresponding to the statement description in the corpus data;
the generation module is used for generating a prompt template corpus through a native prompt template based on the statement description and the intention descriptor, wherein the native prompt template is used for describing a replacement relation, and the replacement relation is a relation for replacing the statement description with the intention descriptor;
and the optimization module is used for training the domain NLP model based on the prompt template corpus and the native prompt template so as to obtain an optimized domain NLP model.
In a fourth aspect, an embodiment of the present invention provides an electronic device, including:
at least one processor;
A memory coupled to the at least one processor;
wherein the memory stores instructions executable by the at least one processor, the at least one processor implementing the aforementioned methods by executing the memory-stored instructions.
In a fifth aspect, embodiments of the present invention provide a machine-readable storage medium storing machine instructions that, when executed on a machine, cause the machine to perform the aforementioned method.
According to the invention, the corpus data of the target field is adopted, so that the general NLP model can be converted into the field NLP model in the target field instead of directly using the general corpus, and the general NLP model with anisotropy (anotropy) is obtained, thereby avoiding the situation that the general NLP model is adopted for recognition and understanding, and the meaning in the target field cannot be met due to the understanding difference caused by the different corpus fields. Then, the corresponding relation between the sentence descriptions and the intention description symbols in the domain corpus data is determined, and the relation information (the sentence descriptions can comprise low-frequency words and/or complex sentences) for forming the intention descriptions and the sentence descriptions in the target domain between the sentence descriptions in the corpus data of the target domain and forming the corpus for adjusting the model learning can be formed. And associating corresponding intention description symbols with sentence descriptions, and forming association information of the intention descriptions and the sentence descriptions in the target field and forming corpus for model learning adjustment through a native prompt template, so that a new intention recognition model is not constructed. And adjusting the field NLP model in the training process by using the prompt template corpus and the native prompt template to obtain an optimized field NLP model.
In the invention, the optimization field NLP model can realize the dialogue response of the customer dialogue containing low-frequency words and complex sentences, and meet the dialogue performance requirements of the intelligent customer service system which meets the customer intention and is applied to the target field, for example, in the automobile field, the customer in the dialogue message of the intelligent customer service system says: "my car pushed" the context of the statement description may be "my car pushed", the keyword text may be "pushed", the context of the voucher statement description alone and the keyword text are insufficient for the machine to understand the customer's intent to talk, the template is used for associating the whole sentence' My car is pushed back 'with strong vehicle acceleration performance (intention description), and the intention description which is recognized by the machine customer service and meets the customer's intention of conversation should be 'good vehicle acceleration performance' instead of the safe conversation intention of collision. Meanwhile, the invention does not need to train a newly constructed model, and further does not need to introduce any corpus data (including templates) except the corpus data in the target field, and the optimized model can avoid the adjustment of the disagreement between the customer intention identified in the session message and the actual intention of the customer due to the introduction of the templates.
Additional features and advantages of embodiments of the invention will be set forth in the detailed description which follows.
Drawings
The accompanying drawings are included to provide a further understanding of embodiments of the invention, and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain, without limitation, the embodiments of the invention. In the drawings:
FIG. 1 is a schematic diagram of the main steps of the method according to the embodiment of the present invention;
FIG. 2 is a schematic diagram of an exemplary corpus processing flow according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of an exemplary model optimization scenario of an application server according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a model usage scenario of an exemplary application server according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of a model usage scenario of an exemplary application server according to an embodiment of the present invention;
fig. 6 is a schematic diagram of an exemplary electronic device module according to an embodiment of the present invention.
Detailed Description
For the purposes of clarity, technical solutions and advantages of the present specification, the following detailed description of the embodiments of the present invention will be given with reference to the accompanying drawings. It should be understood that the detailed description and specific examples, while indicating and illustrating the invention, are not intended to limit the invention.
As mentioned above, the development of artificial intelligence technology has been advanced, and natural language processing technology has also been advanced rapidly, and with the development of an algorithm architecture such as a transducer, a pre-training language model (Pretraining Language Model, PLM) based on the architecture shows excellent performance on a plurality of natural language processing tasks, and is increasingly applied to various natural language processing tasks.
Most intelligent customer service systems mainly have a mode of combining a rule model, a similarity retrieval and an intention recognition model in the aspect of intention recognition, wherein the rule model is based on a regular expression and keywords, and the intention recognition is carried out by utilizing a character configuration algorithm; according to the similarity retrieval method, the intention type and the corresponding typical session thereof are summarized according to service experts, and the intention type of the most similar sentences is identified by utilizing a sentence similarity matching mode; the intention recognition model is based on a pre-training language model (PLM), an embedded vector (embedded) of the client conversation corpus is generated, then a linear model is used for training an intention classification model based on the embedded vectors and the intention types of corresponding labels, and the intention classification model is scored to finish the classification of the intention. However, the above method can only perform the customer session of simple and high frequency words, and it is difficult to solve the problems of customer session recognition and response including low frequency words and complex sentences.
In view of this, the present disclosure provides an optimization scheme of a natural language processing model, and trains a general NLP model using corpus data of a target domain to obtain the domain NLP model, so as to avoid adverse effects of anisotropy of the general NLP model on recognition results, and make each word frequency word or sentence be understood by the model as meaning of a target domain requirement in the target domain. Then, sentence descriptions in the corpus data are found, corresponding relations between the sentence descriptions and the intention description symbols exist, sentence representations of different words and grammar exist between the sentence descriptions, and the corresponding relations can provide associated information of the sentence descriptions and the intention descriptions, wherein the sentence descriptions comprise word frequencies and simple/complex sentences. Generating template corpus with the related information by means of the original prompting template, and finally training a field NLP model through the template corpus and the original prompting template, wherein at the moment, compared with training a general NLP model, the field NLP model is subjected to tiny adjustment (fine adjustment) at the moment, the obtained optimization field NLP model can realize client session recognition and response containing low-frequency words and complex sentences, and meanwhile, a new machine model does not need to be built and trained, and further, additional corpus (containing the template) does not need to be used.
It should be noted that, with respect to the target domain, the score value of each word involved in the corpus data can be determined through the configured word index and the score value table, and based on the score value and the score threshold value specified in the target domain, the word is determined to be a low-frequency word or a high-frequency word; the low-frequency word or the high-frequency word can be determined simply through the statistical value of the occurrence times of each word in the corpus data and the specified statistical threshold value in the target field. Similarly, whether the sentence description related in the corpus data is a complex sentence or a simple sentence can be determined through a configured sentence index and a scoring numerical table, for example, the scoring value of the sentence description related in the corpus data is determined based on indexes such as the number of qualifiers, the number of clauses, the number of words, the number of new words in the field, the existence of specified grammar attributes and the like of the sentence description, and the sentence description related in the corpus data is determined to be a complex sentence or a simple sentence based on the scoring value and a specified sentence scoring threshold in the target field. It should be appreciated that the methods provided herein may be performed by a device having computing and instruction processing capabilities, such as a server or electronic device.
In a first aspect, referring to fig. 1, an embodiment of the present invention provides an optimization method for a natural language processing model, which may be applied to an NLP server, where the NLP server may belong to or communicate with an intelligent customer service system to complete model training or use a model for recognition. The optimization method may include:
S1) the NLP server can train a general NLP model based on corpus data of the target field to obtain a field NLP model.
In an embodiment of the present invention, a natural language processing model (NLP model) may be deployed in a physical server or a server instance or a container instance, which may be referred to as an NLP server, where the hardware of the instance may be a resource instance with computing and instruction processing functions, which is formed by processor resources and memory resources in a server cluster.
In some possible implementations, in the optimization method, the corpus data can include historical conversation messages, which can be provided by an intelligent customer service system in the target domain. The historical session message may include a session message between a robot customer service in the intelligent customer service system and a terminal device used by the customer, and the historical session message may also include a session message between a human customer service in the intelligent customer service system and a terminal device used by the customer. The session message may include a plurality of statement descriptions, the statement descriptions may include a keyword text of a commodity term or a component term or a product integration term in the target field and a context in which the keyword text is located, where the keyword text includes "gearbox", "SUV", "interior", "engine oil", "carrier body", "turbo charging", "differential", "suspension", "off-road vehicle", etc. in the automotive field, and the session message may further include a keyword text of a product service in the target field, where the keyword text includes "replace air conditioning filter", "paint repair", "sheet metal repair", etc. in a maintenance service in the automotive field. In other possible implementations, the corpus data may further include website forum question-answer data and user comment data of the target field, where the website forum question-answer data and the user comment data may include term descriptions or non-term descriptions, where the term descriptions may include terms such as commodities, parts and the like of the target field and keywords text of popular words and net red words having correspondence with the terms, and a context in which the keywords text is located, where the keywords text includes "bare car price" (only sales price of the vehicle itself), "top cover" (damage of a cylinder cover of an engine cylinder due to damage), "push back feel" (acceleration performance of the vehicle) of the automobile field, and the like.
In some possible implementations, the NLP server may be in communication with the intelligent customer service system for returning processing result data in response to the sending data of the intelligent customer service system. Wherein, in some possible applications, the intelligent customer service system may be a server implementation different from the NLP server, and the intelligent customer service system may be configured with an interface program, where the interface program may receive customer consultation data sent by a customer service program such as a website program, a mobile device application program, a micro service program, and the like, where the customer service program may generate a session corresponding to a customer identifier, and function for receiving text data and/or voice data input by a customer, where the customer consultation data may carry information such as a customer statement description (which may be text data or converted from voice data), and a customer object identifier. The intelligent customer service system may send the customer sentence descriptions to the NLP server, receive the processing result data returned by the NLP server, the processing result data may include sentence descriptions associated with intent of the customer sentence descriptions, voice data (TTS conversion service implementation) converted by the sentence descriptions, image (text-to-image) converted by the sentence descriptions, etc. data, the intelligent customer service system may return the processing result data to the customer service program through the interface program, the customer service program may present the processing result data in the foregoing session, for example, display the returned sentence descriptions, play the voice data converted by the sentence descriptions, and display the image converted by the sentence descriptions, wherein the TTS conversion service and the TTI conversion service may be a lookup service, a service for calling a package interface, a service using a generated machine model, or the like.
In other possible applications, the intelligent customer service system may also include a user input unit, the aforementioned NLP server, and a presentation unit, both of which may be implemented by an application deployed on a server or computer or mobile electronic device. The user input unit may include an interface program that may receive client consultation data transmitted by a customer service program such as a website program, a mobile device application program, a micro service program, etc., and a communication program that may obtain the client consultation data from the session. The communication program in the user input unit may send the client sentence descriptions to the NLP server. The presentation unit may include a user interface program and a communication program, the communication program may receive the processing result data returned by the NLP server, and the user interface program in the presentation unit may instruct a customer service program for presenting the processing result data in a session. The communication programs of the user input unit and the presentation unit may share one communication program or employ separate communication programs.
In an embodiment of the present invention, the optimized NLP model may be a generic NLP model, i.e., a Pretrained Language Model (PLM) that is generic to each domain, with anisotropic text processing capabilities, such as model BERT (Bidirectional Encoder Representations from Transformers). However, in the intelligent customer service system of different industries, there are many industry-specific domain information in the session, and when the domain information is processed, the performance of the model is reduced. Therefore, in some possible implementation manners, the corpus data of the target field collected by the user can be adopted, and the corpus data can be communicated with the NLP server through the optimization server and used for transmitting the corpus data to the NLP server, so as to train a general NLP model deployed in the NLP server; in other possible implementations, the NLP server may be configured with an optimization program for model training, where the optimization program may receive corpus data, control instructions, training parameters, etc. input by the user and transmit the corpus data, control instructions, training parameters, etc. to the NLP model, where the optimization program in the NLP server may also present response information to the user, where the foregoing optimization server and the NLP server may be considered to be the same server, so that the NLP server may complete training of the NLP model in response to the interactive operation of the user (model training technician, server operation maintainer, etc.).
In some possible applications, question-answer text data and user comment data can be obtained from a portal/industry website focusing on the target field, historical session data can be obtained from a manual customer service system of the target field, and the like, and the data can be used as corpus data of the target field. For example, the model training is performed by adopting question-answer text data and user comment data of an automobile portal website/industry website in the automobile field, and corpus data such as historical conversation data of an artificial client system of an online automobile store, and after the training is completed, a field NLP model facing the automobile field can be obtained. In some possible automotive field applications, step S1) may comprise:
s101) the NLP server can determine a statement description set serving as a training sample based on corpus data in the automobile field;
s102) the NLP server can train a general NLP model (BERT model) through training samples based on the task of the mask language model (Masked Language Model, MLM), and corpus or sample processing in the training process can be automated through machine script.
After the training of the general NLP model is completed, a domain (pre-training) NLP model in the automotive domain can be obtained and can be denoted as a BERT domain model.
In an embodiment of the present invention, the foregoing optimization method may further include:
s2) the NLP server may determine an intent descriptor corresponding to the sentence description in the corpus data.
Since the general NLP model has anisotropy, for example, the distribution of the word vectors of the BERT model is tapered in the representation space, and is not uniformly distributed, the vector distribution of the word vectors may be affected by word frequency. The high-frequency words are distributed more densely and closer to the origin of the representation space, and the low-frequency words are distributed more sparsely and farther from the origin of the representation space; moreover, the distribution of high-frequency words in the corpus data of the target field and the distribution of high-frequency words in the general corpus have certain difference, so that an intention recognition model trained by sentence embedding vectors (emmbedding) is generated by using a pre-training language model, and the performance of the model is reduced when conversation corpus of the specific field is processed. Therefore, the sentence descriptions and the intention descriptions can be associated according to different intentions among the sentence descriptions in the corpus data of the target field, so that the field NLP model is adjusted (trained and realized) through the associated information, and adverse effects of the corresponding intention descriptions of the model understanding sentence descriptions caused by different word frequency are avoided.
In some possible implementations, the intent description may be text describing the actual demand and the plan that the behavior is desired to implement. The intention description can be determined according to a keyword text of a sentence description in a conversation message in the corpus data and a context in the sentence description where the keyword text is positioned, and the label. The intent description may be written to a database server, which may belong to an intelligent customer service system, along with the corresponding statement description for querying during training and use. The database server may be queried for information associated with the statement description that matches the intent description, the statement description in the corpus data may be recorded in the database server, and the intent description associated with the statement description. The intention description can be manually input text, and can be different texts representing the meaning of the statement description in different target fields aiming at the same statement description based on the actual use conditions such as the target fields, the definition of customer service requirements and the like; in the same target area, there may be a plurality of sentence descriptions associated with the same intention description to form association information between the sentence descriptions and the intention description. In the same target area, the number of classifications of the intention descriptions is very limited (different intention descriptions may belong to the same classification), for example, 10 classifications, 20 classifications, etc. of the automotive area, a limited, customizable number of classifications of before-market service classification, after-market service classification, parameters of the consulting automobile, maintenance classification, trial driving automobile classification, intention to purchase automobile classification, intention to not purchase automobile classification, etc. For example, in the automotive field, the sentence descriptions illustrated in the corpus data may include text of "how the pushing back of the vehicle is," the corresponding intent descriptions may be labeled as text of "required vehicle acceleration performance information," may be labeled as belonging to the category of counseling before-market services, and may be written into the database.
Prior to training the domain NLP model, step S2) may include:
s201) the NLP server may obtain the intent description of the sentence description in the corpus data from the database server.
In some possible implementations, the NLP server may send query instructions to the database server regarding the statement descriptions in the corpus data, wherein the database server may return intent descriptions in response to the query instructions. The NLP server can present the intention description in a list mode through a window interface of an optimization program, the optimization program can receive user-defined intention description input by a user aiming at statement description, the optimization program can update the list, so that adjustment of the current user aiming at the intention description is facilitated, the optimization program can write the intention description with adjustment into a database server, and the intention description corresponding to the statement description recorded in the database server is updated into the current user-adjusted intention description.
S202) the NLP server may generate or specify an intent descriptor for the intent description.
In some possible implementations, the intent descriptor is a machine-identifiable symbol, the intent descriptor may include a machine-identifiable symbol corresponding to one of the intent descriptions or a machine-identifiable symbol corresponding to one of the intent description classifications, and the intent descriptor may be written to a database server in the aforementioned intelligent customer service system along with the corresponding intent description (the database record in the intelligent customer service system has a correspondence record), so that the correspondence of the intent descriptor to the statement description may be determined by querying the intelligent customer service system. Thus, in terms of usage, the machine customer service in the intelligent customer service system may use the trained model recognition intent descriptor to determine the actual intent of the customer dialogue, e.g., the machine customer service understands how the "push back feel" is "the desired vehicle acceleration performance information" in the customer dialogue, rather than the crash safety aspect information, and perform a data analysis procedure, may determine the intent descriptor of the customer identification based on the session message in the intelligent customer service system and the trained model, e.g., from the session message between the machine customer service and the customer's terminal device, to analyze whether the customer is buying intent. The NLP server may query, according to a manually configured symbol list (in a database), an intention description symbol corresponding to the specified intention description, for example, a custom symbol [ STS ] corresponding to the purchase intention description, so as to provide association information between the statement description and the intention description in training data of the domain NLP model, and simultaneously establish association information between the statement description and the intention description classification. The NLP server may also generate or specify intent descriptors based on a configured symbol sequence or string generation algorithm. In some possible applications, the symbol list may be organized according to classifications of intent descriptions, the intent descriptions are composed of strings and numerical numbers, the same string and distinct numerical numbers may be configured for multiple intent descriptions under the same classification of intent descriptions, and the multiple intent descriptions under different classifications may be configured with different strings and numerical numbers. For example, sentence description 1 relates to a description related to counseling automobile power, corresponding to a classification of counseling automobile parameters, an intention description symbol [ VPA {1} ], sentence description 2 relates to a description related to counseling automobile interior, corresponding to a classification of counseling automobile parameters, an intention description symbol [ VPA {2} ], sentence description 3 relates to a description related to counseling automobile maintenance, corresponding to a classification of counseling maintenance, an intention description symbol [ VCA {1} ] can be specified, whereby it is also possible to provide association information of sentence descriptions and intention descriptions in training data of a domain NLP model, and simultaneously establish association information of sentence descriptions and intention description classifications. In other possible applications, the number of classifications of intent descriptions is less than the specified number, the same string and the numerical number of the distinction may be used as the intent descriptor to represent all classifications, or the string may be used as the intent descriptor to represent one of the classifications of two intent descriptions when only two classifications of intent descriptions are made.
In the embodiment of the invention, in order to enable the model to learn the related information, a native prompt template can be adopted, and sentence descriptions in the intention description symbol and the corpus data are associatively filled into the native prompt template to generate a prompt template corpus. The foregoing optimization method may further include:
s3) the NLP server can generate a prompt template corpus through a native prompt template based on the sentence description and the intention descriptor, wherein the native prompt template is used for describing a replacement relation, and the replacement relation is a relation of replacing the sentence description with the intention descriptor.
In some possible implementations, the native hint template (promt_raw) may include: a placer and a template description of the substitution relationship, wherein the placer is used for indicating word order of filling sentence description and word order of filling intention description symbol in the template description. After the native prompt template is filled with sentence descriptions and intention descriptor, generating prompt template corpus, wherein the prompt template corpus is a new sentence description, the number of words of the new sentence descriptions is larger than that of the sentence descriptions filled in the native prompt template, and high-low word frequency words and simple/complex sentences existing between the sentence descriptions in the corpus data of the target field can be directionally associated with the corresponding intention descriptor. When the new sentence description is used for model training, semantic information of the relation in the prompt template corpus is replaced, so that the model learns association information between the filled sentence description and the filled intention description symbol, a brand new model is not required to be constructed and used, and meanwhile, the generated template corpus can be automatically filled with the sentence description and the intention description symbol with the determined corresponding relation by a machine script based on a position symbol in the original prompt template, and the method has the advantage of corpus processing efficiency.
In some possible applications, a plurality of the foregoing native alert templates may be configured, where each template description may be different, and word meaning and position (may not be calculated) of the template description may be the same, and grammar attributes of the native alert templates may be used to remove the template itself as corpus during training of the model using the template description between at least two native alert templates, so that the model will not introduce corpus data other than the corpus data of the target domain (including the native alert template itself), and please combine fig. 2, the corpus data filled in the native alert template and the corpus data of the training general NLP model are both the corpus data of the foregoing target domain. Illustratively, 3 native hint templates may be configured;
the first native hint template may be:
the phrase "{ $L1}" means { $L2};
the second native hint template may be:
sentence: "{ $L1}" means { $L2};
the third native hint template may be:
the clients say that: "{ $L1}" represents { $L2}.
In the first native hint template, the template description is the meaning of this sentence "; the locality indicator { $l1} may indicate that the sentence description [ X ] in the corpus data (x=1, 2,3 … …, where the sentence description [ X ] may represent a client sentence in the corpus data that is described as a sentence in an embodiment of the present invention) is passed as a parameter, and the NLP server may place the sentence description [ X ] between double quotations in the template description; the placer { $l2} may indicate that the intent descriptor [ STS ] ([ STS may be a specified or generated symbol) is passed as a parameter, and the NLP server may place [ STS ] after "meaning" in the template description; the alert template corpus generated by the first native alert template may be:
The term "statement description [1]" means [ STS ].
In the second native hint template, the template description is a sentence: "" means that the placer { $l1} can indicate that the statement description [ X ] in the corpus data is passed as a parameter, and the NLP server can place the statement description [ X ] between double quotations in the template description; the placer { $l2} may indicate that the intent descriptor [ STS ] is passed as a parameter, and the NLP server may place [ STS ] after "meaning" in the template description; the alert template corpus generated by the second native alert template may be:
sentence: "statement description [1]" means [ STS ].
In the third native hint template, the template description is the customer's statement: "" represents that the place holder { $L1} can indicate the sentence description [ X ] in the corpus data to be transmitted as a parameter, and the NLP server can place the sentence description [ X ] between double quotation marks in the template description; the placer { $l2} may indicate that the intent descriptor [ STS ] is passed as a parameter, and the NLP server may place [ STS ] after "representation" in the template description; the alert template corpus generated by the third native alert template may be:
The clients say that: "statement description [1]" stands for [ STS ].
In the embodiment of the invention, after the prompt template corpus is obtained, positive and negative sample construction can be performed in the optimization method. The method of constructing positive and negative samples may include:
a1 Forming a positive sample based on a first prompt template corpus, wherein the first prompt template corpus comprises at least two prompt template corpuses, template descriptions among the prompt template corpuses in the first prompt template corpus are different, and sentence descriptions are the same;
a2 Based on a second prompt template corpus, forming a negative sample, wherein the second prompt template corpus comprises at least two prompt template corpuses, and sentence descriptions among the prompt template corpuses in the second prompt template corpus are different.
In some possible implementation manners, the NLP server can initialize two empty sets through a machine script, respectively serve as a first prompt template corpus and a second prompt template corpus, and can judge any two prompt template corpuses aiming at the first prompt template corpus to determine whether the template descriptions are the same and whether the sentence descriptions are the same; if the template descriptions are different and the sentence descriptions are the same, at this time, the two prompting template corpora are used as a positive sample in the first prompting template corpus, and a positive sample mark 1 can be placed at the same time. Similarly, the NLP server may determine, for the second alert template corpus, whether the template descriptions are the same and the sentence descriptions are the same, and if it is determined that the sentence descriptions are not the same, the two alert template corpora are used as a negative sample in the second alert template corpus, and at the same time, a negative sample flag 0 may be set. The samples obtained by the machine script processing can be input into the optimization program, so that the NLP server trains the deployed NLP model.
Illustratively, the positive samples may be:
the phrase "sentence description [1]" means [ STS ], sentence: "statement description [1]" means [ STS ],1.
Illustratively, the negative samples may be:
the phrase "statement description [1]" means [ STS ], the client says: "statement description [2]" stands for [ STS ],0.
Illustratively, the negative samples may also be:
the phrase "phrase description [1]" means [ STS ], and the phrase "phrase description [2]" means [ STS ],0.
In the embodiment of the invention, the sample formed by the prompt template corpus and the native prompt template can be used as training data together to train the field NLP model, and the influence of the native prompt template is removed, wherein, compared with the training of the general NLP model, the training of the model at this time can be called fine tuning of the model. The foregoing optimization method may further include:
s4) the NLP server can train the domain NLP model based on the prompt template corpus and the native prompt template to obtain an optimized domain NLP model.
In some possible implementations, the NLP server may perform model vector extraction such that the effect of the native hint template may be removed using a first hidden vector that inputs positive and negative samples into the domain NLP model and that outputs a sign of the intention descriptor STS, and a second hidden vector of the domain NLP model that inputs the native hint template into the domain NLP model. In some possible applications, step S4) may comprise:
S401) the NLP server may determine a word offset of the sentence description between the hinting template corpus and the native hinting template.
The NLP server can determine the number of words in the positive and negative samples of the statement description [ X ] currently input to the field NLP model, determine the number of words taking the corresponding native alert template itself as the statement description, and then, by differencing the two word numbers, obtain a word offset, for example, the statement description [ X ] in the alert template corpus has 5 words, then, with respect to the corresponding native alert template itself, the word offset is 5 (no intention descriptor can be considered and no word number can be counted), so that the method can be used for the position alignment of the hidden vector representing the statement description in the bert_domain. Step S4) may further include:
s402) the NLP server may adjust the position coding of the native hint template as input to the domain NLP model based on the word offset.
The domain NLP model may be a domain BERT model bert_domain, and when any statement description is input to bert_domain, a position code (i.e., a position id, a position number, or a position mapping value, such as a position_id) of the any statement description is input at the same time. For positive or negative samples, two prompt template corpus in each sample can be input to the BERT domain, respectively. After the prompt template corpus currently input into BERT_domain is input, the intent descriptor [ STS ] is output at BERT_domain ]When the token (token) is marked, the output hidden vector h of BERT_domain can be extracted [STS] (outputting a vector of hidden states) as a hidden vector h for model representation of a prompt template corpus s Can be marked as h s = h [STS] . In order to eliminate the influence of the templates, the position codes when the native prompt templates corresponding to the prompt template corpus input at present are input to the BERT_domain can be adjusted according to the word offset, so that hidden vectors in the BERT_domain, which represent the description of the native prompt templates and the prompt template corpus input at present, are aligned in position in the BERT_domain. For example, the word offset is 5, the position code when the native prompt template corresponding to the currently input prompt template corpus is input to BERT_domain can be increased by 5, then the native prompt template corresponding to the currently input prompt template corpus is input to BERT_domain, and the output hidden vector h of BERT_domain can be extracted p (outputting a vector of hidden states) as a model representing a hidden vector h of the native hint template itself p . Step S4) may further include:
s403) inputting positive and negative samples, the native prompt template and the adjusted position codes into the field NLP model, and performing model training by using a contrast learning method, wherein the positive and negative samples are constructed based on the prompt template corpus.
In the model training (fine tuning at this time), the output hidden vector h can be extracted from the domain NLP model for the prompt template corpus currently input in the positive and negative samples and the native prompt template corresponding to the prompt template corpus [STS] (which may be the first hidden vector described above) and an output hidden vector h p (which may be the aforementioned second hidden vector) and calculate and obtain the statement description i in the prompt template corpus currently input]Vector h of (i=1, 2,3 … …) i = h s – h p Vector h i As a hidden vector after removal of the template effect. Model training using a contrast learning method, the trained objective loss functionCan be selected as follows:
in this function, the function is used to determine,is statement description [ i ] in two different prompt corpus templates represented by field NLP model]Is embedded vector (embedding),>is a temperature super parameter, and N is a batch size (batch size). It should be noted that, the output result of the BERT model may be configurable, and the output result may include the tag of the intention descriptor and the tuple data such as the vector of the hidden state, or may determine the intention descriptor corresponding to the tag when in use. The universal language model is finely tuned through the domain corpus, then a promtt corpus pair of the domain corpus is constructed through a plurality of promts, the vector space of the domain pre-training model is optimized through a contrast learning method, the anisotropy of the pre-training language model is reduced, and finally the domain promt pre-training language model is obtained. After training is completed, the domain NLP model learns between the intention descriptor and the statement description in the corpus data The related information is not influenced by the additional corpus data and the additional corpus data comprising the template, and the fine tuning realizes the optimization field NLP model, so that when the model is applied to the intelligent customer service system, the machine customer service in the intelligent customer service system can provide session service conforming to the customer session intention.
An exemplary model optimization scenario applied to a server is provided in an embodiment of the present invention, please refer to fig. 3, where an NLP server may be deployed with an optimization program and a model file, where the model file may include a general BERT model, and the optimization program may provide information such as training data and parameters to the model file in response to an instruction sent by a training terminal, where the training terminal may be a terminal device with computing and instruction processing capabilities, and may communicate with the NLP server, such as a computer, a mobile electronic device, etc., so that a model training technician may perform an instruction of a processing operation on the NLP server through the training terminal. The NLP server may communicate with a database server, which may be the same server as the NLP server or a different server, may be configured with a template database, a corpus database, an intention descriptor database to enable obtaining corpus data, native prompt templates, and intention descriptors, wherein the NLP server may be deployed with a data collection machine script by which the NLP server may obtain the targeted domain forum question-answer data, web site user comment data, and historical session messages from a communication network and write the same into the corpus database, the historical session messages may be obtained from a customer information database (which may be deployed in the intelligent customer service system) that may include customer identifications and session messages between the customer terminal and the intelligent customer service system by means of a customer terminal, which may also be a terminal device with computing and instruction processing capabilities, and may communicate with the intelligent customer service system, such as a computer, mobile electronic device, etc.
In the above model optimization scenario, the NLP server may receive an instruction sent by a model training technician through a training terminal, and the NLP server may implement: acquiring corpus data of the target field from a corpus database, inputting the corpus data of the target field into a model file of a general NLP model, and training the general NLP model to obtain a field NLP model; the NLP server may perform an optimization procedure for implementing: querying in an intention descriptor database, and determining an intention descriptor corresponding to statement description in corpus data; the NLP server may perform an optimization procedure for implementing: acquiring a native prompt template from a template database, and generating a prompt template corpus through the native prompt template based on the statement description and the intention description symbol; the NLP server may perform an optimization procedure for implementing: and taking the prompt template corpus and the native prompt template as training data, and performing fine adjustment on the domain NLP model to obtain an optimized domain NLP model. The NLP server may also receive a statement description of the test sent by the model training technician through the training terminal and return a corresponding intent descriptor in response.
According to the embodiment of the invention, the corpus data and the prompt template of the target field are utilized to train the pre-training language model, the field NLP model is realized, then the native prompt template is utilized to obtain sentence representation, meanwhile, the influence of the template is eliminated, the optimized field NLP model of contrast learning is utilized, the anisotropy of the vector space of the field NLP model is reduced, the problem of anisotropy of the general pre-training language model and the problem of intention description recognition caused by the difference between the word frequency distribution of the general pre-training language model and the field corpus are avoided, so that the performance of the field pre-training language model is improved, the optimized and fine-tuned field NLP model has better performance on the downstream task in the intelligent customer service system, the performance of intention recognition of the machine model on the basis of the field corpus is improved, the dialogue response performance of the machine customer service in the intelligent customer service system is improved, and statement description is generated/selected based on the intention description symbol, and the response of customer dialogue in the conversation message is performed.
In a second aspect, in use, the embodiment of the present invention further provides a response method of intelligent customer service under the same inventive concept as the previous embodiment, where the response method may include:
B1 Acquiring statement descriptions;
b2 Determining a returned statement description based on the statement description and an optimization domain NLP model; wherein,
the optimization field NLP model is obtained through the optimization method of the natural language processing model.
In the embodiment of the invention, the response method can be executed by an intelligent customer service system, and the NLP model in the optimization field can be deployed on the NLP server in a model file mode and can be executed by the NLP server. In some possible implementations, the intelligent customer service system may be configured with an interface program that may receive customer consultation data sent by a customer service program, such as a website program, a mobile device application program, a micro-service program, etc., where the customer service program may generate a session corresponding to a customer identification, and function to receive text data and/or voice data entered by a customer, where the customer consultation data may carry information, such as a customer statement description and a customer object identification. The intelligent customer service system can communicate with the NLP server, and can send statement descriptions in customer consultation data to the NLP server, wherein in some possible applications, the NLP server can input the statement descriptions into an optimization field NLP model, obtain intention description symbols through the optimization field NLP model, and can look up a table to specify the statement descriptions corresponding to the intention description symbols as returned statement descriptions; in other possible applications, the NLP server may be deployed with a generative model pre-trained based on the intent descriptor and the statement description, the NLP server may also obtain the output intent descriptor by optimizing the domain NLP model, and obtain the returned statement description by the generative model in step B2) using the intent descriptor and the statement description obtained in step B1), and if the target domain is an automotive domain, the returned statement description is an answer of the intelligent customer service system to the customer consultation of the automotive domain.
An exemplary model usage scenario applied to a server is provided in the embodiments of the present invention, please refer to fig. 4, in which an nlp server may execute a session processing application for implementing: acquiring a statement description R in a real-time conversation message from an intelligent customer service system, the statement description R being, for example, "how does the car feel push back? The method comprises the steps of determining a returned statement description S based on a statement description R and an optimization field NLP model, wherein the statement description S is added into a session message with a client terminal in real time by an intelligent client system, the statement description S is strong in pushing back feeling, 0-100 Km/h is only required to be accelerated for 2-3 seconds, a session processing application program can determine an intention descriptor (classification of the intention description) of the statement description R through the optimization field NLP model, query a vehicle information database or a search engine by using the intention description of the intention descriptor corresponding to the intention descriptor requiring vehicle acceleration information, determine acceleration information of a vehicle queried by the client, and obtain the statement description S by using a semantic answer template (such as a template for pushing back feeling strength level and specific numerical acceleration information description) and the vehicle acceleration information.
In terms of use, the embodiment of the present invention also provides a method for classifying a session intention under the same inventive concept as the previous embodiment, which may include:
C1 A session message of the terminal device is obtained, the session message comprises a client identifier and a statement description, and the client identifier can be the identifier of the terminal device;
c2 Determining an intent classification corresponding to the customer identity based on the statement description and an optimization domain NLP model; wherein,
the optimization field NLP model is obtained through the optimization method of the natural language processing model.
In the embodiment of the invention, the conversation intention classification method can be executed by an intelligent customer service system, an optimization field NLP model can be deployed on an NLP server in a model file mode and can be executed by the NLP server, and terminal equipment can be provided with a customer service program which is communicated with the intelligent customer service system. The intelligent customer service system may acquire a session message of a terminal device (may be a terminal device used by a client side) and send a statement description to the NLP server, wherein the NLP server executes an optimization domain NLP model to determine an intention descriptor, and the intention descriptor may correspond to a category of the intention description one by one, for example, a target domain is an automotive domain, and if an intention descriptor ([ STS ]) of "having a purchase intention" is obtained, an intention category corresponding to a client identifier is determined, and the intention category may be a category having a purchase intention, and the NLP server may return the intention descriptor of "having a purchase intention" to the intelligent customer service system, and the intelligent customer service system may record the client identifier and the intention descriptor of "having a purchase intention" through a session consulted by the client, thereby determining a purchase intention of the client. In some possible implementations, the session message in the embodiment of the present invention may include the statement description in step B1) and the statement description in step B2) of the foregoing embodiments of the response method, and may also include the statement description of the dialogue between the client and the human customer service.
An exemplary model usage scenario applied to a server is provided in an embodiment of the present invention, please refer to fig. 5, in which an nlp server may execute a data analysis application program for implementing: acquiring session information of terminal equipment from client information data; based on the sentence description in the session message and the optimization domain NLP model, determining an intention classification corresponding to the client identifier VIP, namely, an intention descriptor [ STS ] with purchase intention, the NLP server may return the client identifier VIP and the intention descriptor [ STS ] to the intelligent customer service system, and the intelligent customer service system may record and update the intention classification of the client identifier deployed in a client information database of the intelligent customer service system itself, for example, marking the intention classification of the client identifier VIP as the intention descriptor [ STS ] with purchase intention.
In a third aspect, an embodiment of the present invention further provides a natural language processing system under the same inventive concept as the previous embodiment, which may be applied to an NLP server or an intelligent customer service system (the NLP server belongs to the intelligent customer service system), and the natural language processing system may include:
the training module is used for training the general NLP model based on the corpus data of the target field to obtain a field NLP model;
The determining module is used for determining an intention descriptor corresponding to the statement description in the corpus data;
the generation module is used for generating a prompt template corpus through a native prompt template based on the statement description and the intention descriptor, wherein the native prompt template is used for describing a replacement relation, and the replacement relation is a relation for replacing the statement description with the intention descriptor;
and the optimization module is used for training the domain NLP model based on the prompt template corpus and the native prompt template so as to obtain an optimized domain NLP model.
In a training aspect, specifically, wherein a generic NLP model is trained based on a mask language model task.
Specifically, the determining the intention descriptor corresponding to the sentence description in the corpus data includes:
acquiring intention description of sentence description in the corpus data;
an intent descriptor is generated or specified for the intent description.
Specifically, the native hint template includes: a placer and a template description of the substitution relationship, wherein the placer is used for indicating word order of filling sentence description and word order of filling intention description symbol in the template description.
Specifically, the template descriptions are different between at least two native alert templates used to generate the alert template corpus.
Specifically, training the domain NLP model based on the prompt template corpus and the native prompt template, including:
determining word offset of sentence descriptions between the prompt template corpus and the native prompt template;
based on the word offset, adjusting a position code of the native hint template when input as the domain NLP model;
and inputting positive and negative samples, the native prompt template and the adjusted position codes into the field NLP model, and performing model training by using a contrast learning method, wherein the positive and negative samples are constructed based on the prompt template corpus.
Specifically, the method for constructing the positive and negative samples comprises the following steps:
forming a positive sample based on a first prompt template corpus, wherein the first prompt template corpus comprises at least two prompt template corpuses, template descriptions among the prompt template corpuses in the first prompt template corpus are different, and sentence descriptions are the same;
and forming a negative sample based on a second prompt template corpus, wherein the second prompt template corpus comprises at least two prompt template corpuses, and sentence descriptions among the prompt template corpuses in the second prompt template corpus are different.
In use, the natural language processing system may be used to perform:
acquiring statement description;
determining a returned statement description based on the statement description and an optimization domain NLP model; wherein,
the optimization field NLP model is obtained by the optimization method of the natural language processing model.
In use, the natural language processing system may be used to perform:
acquiring a session message of the terminal equipment, wherein the session message comprises a client identifier and a statement description;
determining an intention classification corresponding to the client identifier based on the statement description and an optimization domain NLP model; wherein,
the optimization field NLP model is obtained through the optimization method of the natural language processing model.
In a fourth aspect, an embodiment of the present invention further provides an electronic device under the same inventive concept as the previous embodiment, including: at least one processor; a memory coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the at least one processor implementing the methods of the previous embodiments by executing the instructions stored by the memory. Referring to fig. 6, an exemplary electronic device is provided, and the internal structure of the electronic device may be a server, an industrial personal computer, a terminal device, a microcontroller, etc. as shown in fig. 6. The electronic device comprises a processor A01, a network interface A02 and a memory which are connected through a bus. Wherein the processor a01 of the electronic device is adapted to provide computing, instruction processing and control capabilities. The storage of the electronic device includes a memory a03 and a nonvolatile storage medium a04. The nonvolatile storage medium a04 stores an operating system B01 and a computer program B02. The memory a03 provides an environment for the operation of the operating system B01 and the computer program B02 on the nonvolatile storage medium a04. The network interface a02 of the electronic device is used for communication with a network. The computer program B02, when executed by the processor a01, implements the method in the foregoing embodiments.
In a fifth aspect, embodiments of the present invention also provide a machine-readable storage medium having stored thereon machine instructions which, when executed on a machine, cause the machine to perform the method of the previous embodiments.
The information collection, analysis, use, transmission, storage and other aspects related in the present specification should be used for legal and reasonable applications according to the rules of law, not shared, leaked or sold outside the legal applications and other aspects, and are under regulatory control.
The foregoing details of the optional implementation of the embodiment of the present invention have been described in detail with reference to the accompanying drawings, but the embodiment of the present invention is not limited to the specific details of the foregoing implementation, and various simple modifications may be made to the technical solution of the embodiment of the present invention within the scope of the technical concept of the embodiment of the present invention, and these simple modifications all fall within the protection scope of the embodiment of the present invention.
In addition, the specific features described in the above embodiments may be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, various possible combinations of embodiments of the present invention are not described in detail.
Those skilled in the art will appreciate that all or part of the steps in implementing the methods of the embodiments described above may be implemented by a program stored in a storage medium, including instructions for causing a single-chip microcomputer, chip or processor (processor) to perform all or part of the steps of the methods described in the embodiments of the present application. While the aforementioned storage medium may be non-transitory, the storage medium may include: a U-disk, a hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a Flash Memory (Flash Memory), a magnetic Memory, an optical Memory, or other various media capable of storing program codes.
In addition, any combination of various embodiments of the present invention may be performed, so long as the concept of the embodiments of the present invention is not violated, and the disclosure of the embodiments of the present invention should also be considered.

Claims (12)

1. A method for optimizing a natural language processing model, the method comprising:
training a general NLP model based on corpus data of the target field to obtain a field NLP model;
determining an intention description symbol corresponding to statement description in the corpus data;
Generating a prompt template corpus through a native prompt template based on the sentence description and the intention description symbol, wherein the native prompt template is used for describing a replacement relationship, and the replacement relationship is a relationship of replacing the sentence description with the intention description symbol;
training the domain NLP model based on the prompt template corpus and the native prompt template to obtain an optimized domain NLP model; wherein,
calculating to obtain a vector of statement description in the currently input prompt template corpus by using a first hidden vector and a second hidden vector extracted from the field NLP model;
the first hidden vector is a first hidden vector which is input into a domain NLP model by positive and negative samples and is extracted from the domain NLP model when a mark of an intention descriptor is output;
the second hidden vector is extracted from the domain NLP model when the native prompt template is input to the domain NLP model;
constructing template descriptions among prompt template corpus of positive samples in the positive and negative samples, wherein the template descriptions are different and the sentence descriptions are the same;
and constructing statement descriptions among the prompt template corpus of the negative sample in the positive and negative samples.
2. The method of claim 1, wherein the generic NLP model is trained based on a task of masking the language model.
3. The method of optimizing a natural language processing model according to claim 1, wherein the determining an intention descriptor corresponding to a sentence description in the corpus data includes:
acquiring intention description of sentence description in the corpus data;
an intent descriptor is generated or specified for the intent description.
4. The method of optimizing a natural language processing model of claim 1, wherein the native hint template comprises: a placer and a template description of the substitution relationship, wherein the placer is used for indicating word order of filling sentence description and word order of filling intention description symbol in the template description.
5. The method of claim 4, wherein the template descriptions are different between at least two native hinting templates used to generate the hinting template corpus.
6. The method of optimizing a natural language processing model according to any one of claims 1 to 5, wherein the training the domain NLP model based on the prompt template corpus and the native prompt template comprises:
Determining word offset of sentence descriptions between the prompt template corpus and the native prompt template;
based on the word offset, adjusting a position code of the native hint template when input as the domain NLP model;
and inputting the positive and negative samples, the original prompt template and the adjusted position codes into the field NLP model, and performing model training by using a contrast learning method.
7. The method of optimizing a natural language processing model of claim 6, wherein the method of constructing the positive and negative samples comprises:
forming a positive sample based on a first prompt template corpus, wherein the first prompt template corpus comprises at least two prompt template corpuses, template descriptions among the prompt template corpuses in the first prompt template corpus are different, and sentence descriptions are the same;
and forming a negative sample based on a second prompt template corpus, wherein the second prompt template corpus comprises at least two prompt template corpuses, and sentence descriptions among the prompt template corpuses in the second prompt template corpus are different.
8. The intelligent customer service response method is characterized by comprising the following steps of:
Acquiring statement description;
determining a returned statement description based on the statement description and an optimization domain NLP model; wherein,
the optimization domain NLP model is obtained by the optimization method of the natural language processing model of any one of claims 1 to 7.
9. A method of classifying conversational intents, the method comprising:
acquiring a session message of a terminal device, wherein the session message comprises a client identifier and a statement description;
determining an intention classification corresponding to the client identifier based on the statement description and an optimization domain NLP model; wherein,
the optimization domain NLP model is obtained by the optimization method of the natural language processing model of any one of claims 1 to 7.
10. A natural language processing system, the natural language processing system comprising:
the training module is used for training the general NLP model based on the corpus data of the target field to obtain a field NLP model;
the determining module is used for determining an intention descriptor corresponding to the statement description in the corpus data;
the generation module is used for generating a prompt template corpus through a native prompt template based on the statement description and the intention descriptor, wherein the native prompt template is used for describing a replacement relation, and the replacement relation is a relation for replacing the statement description with the intention descriptor;
The optimization module is used for training the field NLP model based on the prompt template corpus and the native prompt template so as to obtain an optimized field NLP model; wherein,
calculating to obtain a vector of statement description in the currently input prompt template corpus by using a first hidden vector and a second hidden vector extracted from the field NLP model; the first hidden vector is a first hidden vector which is input into a domain NLP model by positive and negative samples and is extracted from the domain NLP model when a mark of an intention descriptor is output; the second hidden vector is extracted from the domain NLP model when the native prompt template is input to the domain NLP model; constructing template descriptions among prompt template corpus of positive samples in the positive and negative samples, wherein the template descriptions are different and the sentence descriptions are the same; and constructing statement descriptions among the prompt template corpus of the negative sample in the positive and negative samples.
11. An electronic device, comprising:
at least one processor;
a memory coupled to the at least one processor;
wherein the memory stores instructions executable by the at least one processor, the at least one processor implementing the method of any one of claims 1 to 9 by executing the instructions stored by the memory.
12. A machine readable storage medium storing machine instructions which, when run on a machine, cause the machine to perform the method of any one of claims 1 to 9.
CN202311180783.1A 2023-09-13 2023-09-13 Optimization method, response method and system of natural language processing model Active CN116911315B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311180783.1A CN116911315B (en) 2023-09-13 2023-09-13 Optimization method, response method and system of natural language processing model

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311180783.1A CN116911315B (en) 2023-09-13 2023-09-13 Optimization method, response method and system of natural language processing model

Publications (2)

Publication Number Publication Date
CN116911315A CN116911315A (en) 2023-10-20
CN116911315B true CN116911315B (en) 2024-01-30

Family

ID=88355090

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311180783.1A Active CN116911315B (en) 2023-09-13 2023-09-13 Optimization method, response method and system of natural language processing model

Country Status (1)

Country Link
CN (1) CN116911315B (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113672718A (en) * 2021-09-02 2021-11-19 杭州一知智能科技有限公司 Dialog intention recognition method and system based on feature matching and field self-adaption
CN114757176A (en) * 2022-05-24 2022-07-15 上海弘玑信息技术有限公司 Method for obtaining target intention recognition model and intention recognition method
CN115292470A (en) * 2022-09-30 2022-11-04 中邮消费金融有限公司 Semantic matching method and system for intelligent customer service of petty loan
CN116595169A (en) * 2023-04-18 2023-08-15 北京信息科技大学 Question-answer intention classification method for coal mine production field based on prompt learning

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10643483B2 (en) * 2013-07-19 2020-05-05 PEAR Sports LLC Physical activity coaching platform with dynamically changing workout content

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113672718A (en) * 2021-09-02 2021-11-19 杭州一知智能科技有限公司 Dialog intention recognition method and system based on feature matching and field self-adaption
CN114757176A (en) * 2022-05-24 2022-07-15 上海弘玑信息技术有限公司 Method for obtaining target intention recognition model and intention recognition method
CN115292470A (en) * 2022-09-30 2022-11-04 中邮消费金融有限公司 Semantic matching method and system for intelligent customer service of petty loan
CN116595169A (en) * 2023-04-18 2023-08-15 北京信息科技大学 Question-answer intention classification method for coal mine production field based on prompt learning

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
基于BERT的任务导向对话系统自然语言理解的改进模型与调优方法;周奇安;李舟军;;中文信息学报(第05期);全文 *

Also Published As

Publication number Publication date
CN116911315A (en) 2023-10-20

Similar Documents

Publication Publication Date Title
CN112100383B (en) Meta-knowledge fine tuning method and platform for multitask language model
CN111708869B (en) Processing method and device for man-machine conversation
CN111191016A (en) Multi-turn conversation processing method and device and computing equipment
CN109933792A (en) Viewpoint type problem based on multi-layer biaxially oriented LSTM and verifying model reads understanding method
CN112860871B (en) Natural language understanding model training method, natural language understanding method and device
CN114676255A (en) Text processing method, device, equipment, storage medium and computer program product
CN112597298A (en) Deep learning text classification method fusing knowledge maps
CN117370378A (en) Method, device, equipment and medium for converting natural language into database statement
CN115129862A (en) Statement entity processing method and device, computer equipment and storage medium
CN112487813B (en) Named entity recognition method and system, electronic equipment and storage medium
CN116911315B (en) Optimization method, response method and system of natural language processing model
CN117009478A (en) Algorithm fusion method based on software knowledge graph question-answer question-sentence analysis process
CN115952260A (en) Question and answer method and device in automobile field, electronic equipment and storage medium
CN116595169A (en) Question-answer intention classification method for coal mine production field based on prompt learning
CN111401069A (en) Intention recognition method and intention recognition device for conversation text and terminal
CN115098665A (en) Method, device and equipment for expanding session data
CN116911314B (en) Training method of intention recognition model, conversation intention recognition method and system
CN110851572A (en) Session labeling method and device, storage medium and electronic equipment
CN111382265B (en) Searching method, device, equipment and medium
CN115238903B (en) Model compression method, system, electronic device and storage medium
CN114548103B (en) Named entity recognition model training method and named entity recognition method
CN111444308B (en) Method and system for simulating user to speak
CN117874207A (en) Reply sentence generation method and device, electronic equipment and storage medium
CN113987209B (en) Natural language processing method, device, computing equipment and storage medium based on knowledge-guided prefix fine adjustment
CN117668562B (en) Training and using method, device, equipment and medium of text classification model

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant