WO2021169745A1 - 基于语句前后关系预测的用户意图识别方法及装置 - Google Patents

基于语句前后关系预测的用户意图识别方法及装置 Download PDF

Info

Publication number
WO2021169745A1
WO2021169745A1 PCT/CN2021/074788 CN2021074788W WO2021169745A1 WO 2021169745 A1 WO2021169745 A1 WO 2021169745A1 CN 2021074788 W CN2021074788 W CN 2021074788W WO 2021169745 A1 WO2021169745 A1 WO 2021169745A1
Authority
WO
WIPO (PCT)
Prior art keywords
sentence
user
model
embedding vector
sentences
Prior art date
Application number
PCT/CN2021/074788
Other languages
English (en)
French (fr)
Inventor
高洋洋
Original Assignee
升智信息科技(南京)有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 升智信息科技(南京)有限公司 filed Critical 升智信息科技(南京)有限公司
Priority to US17/802,109 priority Critical patent/US20230080671A1/en
Publication of WO2021169745A1 publication Critical patent/WO2021169745A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/33—Querying
    • G06F16/332—Query formulation
    • G06F16/3329—Natural language query formulation or dialogue systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/30—Semantic analysis
    • G06F40/35—Discourse or dialogue representation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/33—Querying
    • G06F16/3331—Query processing
    • G06F16/334—Query execution
    • G06F16/3343—Query execution using phonetics
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/279—Recognition of textual entities
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Definitions

  • the present invention relates to the technical field of speech signal processing, and in particular to a method, device, computer equipment and storage medium for recognizing user intentions based on sentence context prediction.
  • intelligent dialogue robots have entered people's daily lives. These dialogue robots need to have a natural dialogue with the user, understand the semantics of the user's speech, and accurately recognize the user's intention, so as to interact with the user more efficiently and truthfully.
  • the intelligent dialogue robot system whether the user's intention recognition accurately determines whether the dialogue system can generate a reasonable response is the most important manifestation of whether the dialogue system is intelligent.
  • user semantic intention recognition has methods such as keywords, regular expressions, rule templates, traditional machine learning such as support vector machines, and deep learning based on the current booming development.
  • intent recognition method based on text similarity to solve the problem of intent recognition errors caused by errors in converting speech to text.
  • the text similarity calculation method used includes an algorithm based on character edit distance.
  • the algorithm based on the similarity of the word vectors obtained by deep learning; there is also a proposal to train the intent recognition deep learning model by combining the feature vectors of the characters and the pinyin, and convert the data sets in all fields into the word sequence and the corresponding pinyin sequence, and input the first
  • a deep learning network is trained to obtain a language model, and then the coding layer parameter matrix of the language model is initialized and updated, and then the word sequence and pinyin sequence are input into the second deep learning network coding, and then weighted into the second deep learning network training intention Recognition model and so on.
  • traditional user intention recognition schemes often have the problem of low accuracy.
  • the purpose of the present invention is to provide a user intention recognition method, device, computer equipment and storage medium based on the prediction of the context of sentences, which can improve the accuracy of user intention recognition.
  • the present invention provides a user intention recognition method based on sentence context prediction, including:
  • the sample data includes the first sentence, the second sentence, and the sentence attribute characteristics and position relationship of the first sentence and the second sentence;
  • S20 Input each sample data into the pre-training language model for pre-training, and when the recognition accuracy rate of the pre-training language model for the sample data reaches the first set accuracy rate, determine the initial model according to the current operating parameters of the pre-training language model;
  • S40 Use the intention recognition model to determine the next sentence of the sentence input by the user, and determine the user's intention according to the determined next sentence.
  • setting multiple sample data includes:
  • Each group of sentences includes a first sentence and a second sentence; the word embedding vector represents the content of the corresponding word; the identification embedding vector represents the corresponding word belongs to the first sentence or the second sentence; the position embedding vector represents the corresponding word The position in the sentence.
  • using the intention recognition model to determine the next sentence of the sentence input by the user includes:
  • the intention recognition model inputs multiple candidate sentences and the probability value of each candidate sentence, and determine the candidate sentence with the largest probability value as the user The sentence after the sentence entered.
  • a user intention recognition device based on sentence context prediction including:
  • the setting module is used to set a plurality of sample data;
  • the sample data includes the first sentence, the second sentence, and the sentence attribute characteristics and position relationship of the first sentence and the second sentence;
  • the pre-training module is used to input each sample data into the pre-training language model for pre-training, and when the pre-training language model's recognition accuracy rate for the sample data reaches the first set accuracy rate, according to the current operating parameters of the pre-training language model Determine the initial model;
  • the fine-tuning module is used to input test sentences into the initial model, and fine-tune the initial model with the next sentence of the predicted test sentence as the only target.
  • the prediction accuracy rate of the initial model reaches the second set accuracy rate, according to The current operating parameters of the initial model determine the intention recognition model;
  • the determination module is used to determine the next sentence of the sentence input by the user using the intention recognition model, and determine the user's intention according to the determined next sentence.
  • the setting module is further used for:
  • Each group of sentences includes a first sentence and a second sentence; the word embedding vector represents the content of the corresponding word; the identification embedding vector represents the corresponding word belongs to the first sentence or the second sentence; the position embedding vector represents the corresponding word The position in the sentence.
  • the determining module is further configured to:
  • the intention recognition model inputs multiple candidate sentences and the probability value of each candidate sentence, and determine the candidate sentence with the largest probability value as the user The sentence after the sentence entered.
  • a computer device comprising a memory, a processor, and a computer program stored on the memory and capable of running on the processor.
  • the processor executes the computer program, a user based on sentence context prediction in any of the above embodiments is implemented The steps of the intent identification method.
  • a computer-readable storage medium has a computer program stored thereon, and when the computer program is executed by a processor, the steps of the user intention recognition method based on sentence context prediction in any of the above embodiments are realized.
  • the present invention discloses the following technical effects:
  • the present invention proposes a user intention recognition method, device, computer equipment and storage medium based on sentence context prediction.
  • each sample data is input into a pre-training language model for pre-training, and in the pre-training language
  • the initial model is determined according to the current operating parameters of the pre-trained language model
  • the test sentence is input to the initial model
  • the next sentence of the predicted test sentence is used as the only target for the initial
  • the model is fine-tuned.
  • the intention recognition model is determined according to the current operating parameters of the initial model, and the intention recognition model is used to determine the next sentence of the sentence input by the user.
  • the latter sentence determines the user's intention, and the determined user's intention has a high degree of accuracy.
  • Fig. 1 is a flowchart of a user intention recognition method based on sentence context prediction according to an embodiment of the present invention
  • Figure 2 is a schematic diagram of a sentence composition process in an embodiment of the present invention.
  • FIG. 3 is a schematic diagram of a model and training target during fine-tuning according to an embodiment of the present invention
  • FIG. 4 is a schematic structural diagram of a user intention recognition device based on sentence context prediction according to an embodiment of the present invention
  • Fig. 5 is a schematic diagram of a computer device according to an embodiment of the present invention.
  • the purpose of the present invention is to provide a user intention recognition method, device, computer equipment and storage medium based on the prediction of the context of sentences, which can improve the accuracy of user intention recognition.
  • the user intention recognition method based on sentence context prediction provided in this application can be applied to related user intention recognition terminals (such as robots that need to communicate with users, etc.).
  • the above-mentioned user intention recognition terminal can set multiple sample data, input each sample data into the pre-training language model for pre-training, and when the pre-training language model's recognition accuracy rate for the sample data reaches the first set accuracy rate, according to the pre-training
  • the current operating parameters of the language model determine the initial model, input test sentences to the initial model, and fine-tune the initial model with the next sentence of the predicted test sentence as the only goal, and the prediction accuracy rate of the initial model reaches the second setting
  • the intention recognition model is determined according to the current operating parameters of the initial model
  • the intention recognition model is used to determine the next sentence of the sentence input by the user
  • the user intention is determined according to the determined next sentence to improve the determined user intention Accuracy.
  • the user intention recognition terminal can be, but is not limited to, various personal computers and notebook computers and other intelligent processing devices.
  • a user intention recognition method based on sentence context prediction is provided. Taking the method applied to a user intention recognition terminal as an example for description, the method includes the following steps:
  • the sample data includes the first sentence, the second sentence, and the sentence attribute characteristics and position relationship of the first sentence and the second sentence.
  • the above-mentioned sentence attribute features include the words included in the corresponding sentence and the position of each word.
  • setting multiple sample data includes:
  • Each group of sentences includes a first sentence and a second sentence; the word embedding vector represents the content of the corresponding word; the identification embedding vector represents the corresponding word belongs to the first sentence or the second sentence; the position embedding vector represents the corresponding word The position in the sentence.
  • each of the above groups of sentences includes a first sentence and a second sentence
  • the first sentence may be the previous sentence of the corresponding group of sentences
  • the second sentence may be the next sentence of the corresponding sentence.
  • sample data is used as the input of the subsequent pre-training language model, wherein the first label of each sequence may always be the corresponding classification label.
  • the final output hidden state corresponding to this flag is used to indicate whether the second sentence is the next sentence of the first sentence.
  • the first and second sentences can be encapsulated together to form a single sequence as a group of sentences.
  • sentences can be distinguished in two ways.
  • the first way is to use special symbols ([SEP]) to separate them.
  • the second way is to add the learned identification embedding vector to each word to indicate whether it belongs to sentence A (first sentence) or sentence B (second sentence).
  • For each word its input is obtained by adding the word embedding vector of the word itself, the identification embedding vector (E A , E B ) and the position embedding vector (E 0 , E 1 , E 2 ). Refer to Figure 2 for the specific process.
  • S20 Input each sample data into the pre-training language model for pre-training, and when the recognition accuracy rate of the pre-training language model for the sample data reaches a first set accuracy rate, determine the initial model according to the current operating parameters of the pre-training language model.
  • the above-mentioned first setting accuracy rate can be set according to the accuracy of user identification, for example, set to 98% or the like.
  • pre-training uses a large-scale monolingual corpus that has nothing to do with the dialogue system for training.
  • the corresponding model uses two tasks as targets for pre-training.
  • the first task is to shield the language model. A certain percentage of words are randomly shielded during input, and then these shielded words are predicted at the output of the model. This way, a two-way deep network can be constructed.
  • the second task is to predict whether the second sentence is the next sentence. When selecting two sentences for each pre-training sample, there is a 50% probability that the second sentence is the actual next sentence following the first sentence, and a 50% probability is a random sentence from the corpus.
  • the above-mentioned second setting accuracy rate can be set according to the accuracy of user recognition, for example, set to 98% or the like.
  • the sentence configured by the dialogue system is used to fine-tune the pre-trained model.
  • the shielding language model is no longer used as the training target, but only to predict the next sentence as the only target, so the input of the model no longer shields any words.
  • the sample in the fine-tuning phase takes the sentence expected by the user as the first sentence, and the sentence of the next node configured in the dialogue system as the second sentence to generate the positive samples in the task training set; take the sentence expected by the user as the second sentence In the first sentence, the sentences of other nodes configured in the speech art are used as the second sentence to generate negative samples in the task training set.
  • S40 Use the intention recognition model to determine the next sentence of the sentence input by the user, and determine the user's intention according to the determined next sentence.
  • using the intention recognition model to determine the next sentence of the sentence input by the user includes:
  • the intention recognition model inputs multiple candidate sentences and the probability value of each candidate sentence, and determine the candidate sentence with the largest probability value as the user The sentence after the sentence entered.
  • the input of the model also no longer shields any words.
  • the above-mentioned user intention recognition method based on sentence context prediction by setting multiple sample data, each sample data is input into the pre-training language model for pre-training, and the recognition accuracy of the sample data in the pre-training language model reaches the first setting
  • For accuracy determine the initial model based on the current operating parameters of the pre-trained language model, input test sentences to the initial model, and fine-tune the initial model with the next sentence predicting the test sentence as the only goal.
  • the prediction accuracy of the initial model reaches the first 2.
  • determine the intention recognition model based on the current operating parameters of the initial model use the intention recognition model to determine the next sentence of the sentence input by the user, and determine the user's intention based on the determined next sentence.
  • the determined user intention has Higher accuracy.
  • language model pre-training is very effective in improving many natural language processing tasks. These tasks include sentence-level tasks and word-level tasks, such as natural language inference to predict the relationship between sentences, named entity recognition, and knowledge questions and answers.
  • Transformer-based Bidirectional Encoding Representation (BERT) is a recently proposed pre-training language model.
  • the pre-training model can efficiently extract text information and apply it to various natural language processing tasks. Its appearance refreshes the most of 11 natural language processing tasks. Excellent performance record.
  • BERT proposes the task of predicting the next sentence from any monolingual corpus, that is, to determine whether two sentences should be a continuous sentence relationship.
  • take the sentence actually spoken by the user as the first sentence, and all the branch sentences in the current node as the second sentence execute the prediction method of the model, and get each sentence as the sentence spoken by the user. The probability of the next sentence. Take the branch of the sentence with the highest probability as the matching intent, and return the sentence with the highest probability as the reply.
  • FIG. 4 is a schematic structural diagram of a user intention recognition apparatus based on sentence context prediction according to an embodiment, including:
  • the setting module 10 is used to set a plurality of sample data; the sample data includes the first sentence, the second sentence, and the sentence attribute characteristics and position relationship of the first sentence and the second sentence;
  • the pre-training module 20 is used for inputting each sample data into the pre-training language model for pre-training, and when the pre-training language model's recognition accuracy rate for the sample data reaches the first set accuracy rate, according to the current running of the pre-training language model Parameters determine the initial model;
  • the fine-tuning module 30 is used to input test sentences into the initial model, and fine-tune the initial model with the next sentence of the predicted test sentence as the only target.
  • the prediction accuracy rate of the initial model reaches the second set accuracy rate, Determine the intention recognition model according to the current operating parameters of the initial model;
  • the determining module 40 is configured to determine the next sentence of the sentence input by the user using the intention recognition model, and determine the user's intention according to the determined next sentence.
  • the setting module is further used for:
  • Each group of sentences includes a first sentence and a second sentence; the word embedding vector represents the content of the corresponding word; the identification embedding vector represents the corresponding word belongs to the first sentence or the second sentence; the position embedding vector represents the corresponding word The position in the sentence.
  • the determining module is further configured to:
  • the intention recognition model inputs multiple candidate sentences and the probability value of each candidate sentence, and determine the candidate sentence with the largest probability value as the user The sentence after the sentence entered.
  • the various modules in the device for recognizing user intentions based on the prediction of the context of sentences can be implemented in whole or in part by software, hardware, and a combination thereof.
  • the above-mentioned modules may be embedded in the form of hardware or independent of the processor in the computer equipment, or may be stored in the memory of the computer equipment in the form of software, so that the processor can call and execute the operations corresponding to the above-mentioned modules.
  • a computer device is provided.
  • the computer device may be a terminal, and its internal structure diagram may be as shown in FIG. 5.
  • the computer equipment includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus.
  • the processor of the computer device is used to provide calculation and control capabilities.
  • the memory of the computer device includes a non-volatile storage medium and an internal memory.
  • the non-volatile storage medium stores an operating system and a computer program.
  • the internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium.
  • the network interface of the computer device is used to communicate with an external terminal through a network connection.
  • the display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen
  • the input device of the computer equipment can be a touch layer covered on the display screen, or it can be a button, a trackball or a touchpad set on the housing of the computer equipment , It can also be an external keyboard, touchpad, or mouse.
  • FIG. 5 is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied.
  • the specific computer device may Including more or fewer parts than shown in the figure, or combining some parts, or having a different arrangement of parts.
  • a computer device is further provided.
  • the computer device includes a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the The program implements any user intention recognition method based on sentence context prediction as in any of the foregoing embodiments.
  • the program can be stored in a non-volatile computer readable storage.
  • the program can be stored in the storage medium of a computer system and executed by at least one processor in the computer system to implement the user intention recognition method including the above-mentioned sentence context prediction The flow of the embodiment.
  • the storage medium can be a magnetic disk, an optical disc, a read-only memory (Read-Only Memory, ROM), or a random access memory (Random Access Memory, RAM), etc.
  • a computer storage medium a computer readable storage medium, on which a computer program is stored, wherein the program is executed by the processor to implement any sentence-based User intention recognition method based on context prediction.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Human Computer Interaction (AREA)
  • Acoustics & Sound (AREA)
  • Machine Translation (AREA)

Abstract

一种基于语句前后关系预测的用户意图识别方法、装置、计算机设备和存储介质，通过设置多个样本数据：所述样本数据包括第一条语句，第二条语句，以及第一条语句和第二条语句的语句属性特征和位置关系（S10），将各个样本数据输入预训练语言模型进行预训练，并在预训练语言模型针对样本数据的识别准确率达到第一设定准确率时，根据预训练语言模型当前的运行参数确定初始模型（S20），向初始模型输入测试语句，以预测测试语句的下一条语句作为唯一目标对初始模型进行微调，在初始模型的预测准确率达到第二设定准确率时，根据初始模型当前的运行参数确定意图识别模型（S30），采用意图识别模型确定用户输入的语句的后一句语句，根据所确定的后一句语句确定用户意图（S40），所确定的用户意图具有较高的准确性。

Description

基于语句前后关系预测的用户意图识别方法及装置
本申请要求于2020年2月25日提交中国专利局、申请号为202010116553.9、发明名称为“基于语句前后关系预测的用户意图识别方法及装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本发明涉及语音信号处理技术领域，特别是涉及一种基于语句前后关系预测的用户意图识别方法、装置、计算机设备和存储介质。
背景技术
随着人工智能的发展，智能对话机器人已进入人们的日常生活。这些对话机器人需要与用户自然对话，理解用户说话的语义，准确识别用户的意图，从而更高效、真实地与用户交互。在智能对话机器人系统中，用户意图识别是否准确决定对话系统是否能够生成合理的回复，是对话系统是否智能最重要的体现。
目前用户语义意图识别有基于关键字、基于正则表达式、基于规则模板、基于传统机器学习如支持向量机以及基于当前蓬勃发展的深度学习等方法。例如，有方案提出一种基于文本相似度的意图识别方法，解决通过语音转为文本存在错误而导致的意图识别错误的问题，其用到的文本相似度的计算方法包括基于字符编辑距离的算法和基于深度学习得到的词向量相似性的算法；还有方案提出通过组合字和拼音的特征向量训练意图识别深度学习模型，将所有领域的数据集转为字序列和相应的拼音序列，输入第一深度学习网络训练得到语言模型，然后将该语言模型的编码层参数矩阵初始化和更新，再将字序列和拼音序列输入到第二深度学习网络编码后，加权投入到第二深度学习网络训练意图识别模型等等。然而，传统的用户意图识别方案往往存在准确率低的问题。
发明内容
基于此，本发明的目的是提供一种基于语句前后关系预测的用户意图识别方法、装置、计算机设备和存储介质，能够提高用户意图识别的准确率。
为实现上述目的，本发明提供了一种基于语句前后关系预测的用户意图识别方法，包括：
S10，设置多个样本数据；所述样本数据包括第一条语句，第二条语句，以及第一条语句和第二条语句的语句属性特征和位置关系；
S20，将各个样本数据输入预训练语言模型进行预训练，并在预训练语言模型针对样本数据的识别准确率达到第一设定准确率时，根据预训练语言模型当前的运行参数确定初始模型；
S30，向所述初始模型输入测试语句，以预测测试语句的下一条语句作为唯一目标对所述初始模型进行微调，在初始模型的预测准确率达到第二设定准确率时，根据初始模型当前的运行参数确定意图识别模型；
S40，采用所述意图识别模型确定用户输入的语句的后一句语句，根据所确定的后一句语句确定用户意图。
在一个实施例中，设置多个样本数据包括：
获取多组语句，设置各组语句中各个词的单词嵌入向量、标识嵌入向量和位置嵌入向量，根据各组语句以及各自语句分别对应的单词嵌入向量、标识嵌入向量和位置嵌入向量确定样本数据；其中各组语句均包括第一语句和第二语句；所述单词嵌入向量表征相应单词的内容；所述标识嵌入向量表征相应单词属于第一语句或者第二语句；所述位置嵌入向量表征相应单词在所在语句中的位置。
在一个实施例中，采用所述意图识别模型确定用户输入的语句的后一句语句包括：
读取用户输入的语句，将用户输入的语句输入所述意图识别模型，所述意图识别模型输入多个待选语句以及各个待选语句的概率值，将概率值最大的待选语句确定为用户输入的语句的后一句语句。
一种基于语句前后关系预测的用户意图识别装置，包括：
设置模块，用于设置多个样本数据；所述样本数据包括第一条语句，第二条语句，以及第一条语句和第二条语句的语句属性特征和位置关系；
预训练模块，用于将各个样本数据输入预训练语言模型进行预训练，并在预训练语言模型针对样本数据的识别准确率达到第一设定准确率时，根据预训练语言模型当前的运行参数确定初始模型；
微调模块，用于向所述初始模型输入测试语句，以预测测试语句的下一条语句作为唯一目标对所述初始模型进行微调，在初始模型的预测准确率达到第二设定准确率时，根据初始模型当前的运行参数确定意图识别模型；
确定模块，用于采用所述意图识别模型确定用户输入的语句的后一句语句，根据所确定的后一句语句确定用户意图。
在一个实施例中，所述设置模块进一步用于：
获取多组语句，设置各组语句中各个词的单词嵌入向量、标识嵌入向量和位置嵌入向量，根据各组语句以及各自语句分别对应的单词嵌入向量、标识嵌入向量和位置嵌入向量确定样本数据；其中各组语句均包括第一语句和第二语句；所述单词嵌入向量表征相应单词的内容；所述标识嵌入向量表征相应单词属于第一语句或者第二语句；所述位置嵌入向量表征相应单词在所在语句中的位置。
在一个实施例中，所述确定模块进一步用于：
读取用户输入的语句，将用户输入的语句输入所述意图识别模型，所述意图识别模型输入多个待选语句以及各个待选语句的概率值，将概率值最大的待选语句确定为用户输入的语句的后一句语句。
一种计算机设备，包括存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现上述任一实施例的基于语句前后关系预测的用户意图识别方法的步骤。
一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时实现上述任一实施例的基于语句前后关系预测的用户意图识别方法的步骤。
根据本发明提供的具体实施例，本发明公开了以下技术效果：
本发明提出了一种基于语句前后关系预测的用户意图识别方法、装置、计算机设备和存储介质，通过设置多个样本数据，将各个样本数据输入预训练语言模型进行预训练，并在预训练语言模型针对样本数据的识别准确率达到第一设定准确率时，根据预训练语言模型当前的运行参数确定初始模型，向初始模型输入测试语句，以预测测试语句的下一条语句作为唯一目标对初始模型进行微调，在初始模型的预测准确率达到第二设定准 确率时，根据初始模型当前的运行参数确定意图识别模型，采用意图识别模型确定用户输入的语句的后一句语句，根据所确定的后一句语句确定用户意图，所确定的用户意图具有较高的准确性。
说明书附图
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。
图1为本发明实施例基于语句前后关系预测的用户意图识别方法流程图；
图2为本发明实施例语句构成过程示意图；
图3为本发明实施例微调时的模型与训练目标示意图；
图4为本发明实施例基于语句前后关系预测的用户意图识别装置结构示意图；
图5为本发明实施例计算机设备示意图。
具体实施方式
下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。
本发明的目的是提供一种基于语句前后关系预测的用户意图识别方法、装置、计算机设备和存储介质，能够提高用户意图识别的准确率。
为使本发明的上述目的、特征和优点能够更加明显易懂，下面结合附图和具体实施方式对本发明作进一步详细的说明。
本申请提供的基于语句前后关系预测的用户意图识别方法，可以应用于相关用户意图识别终端(如需要与用户交流的机器人等)。上述用户意图识别终端可以设置多个样本数据，将各个样本数据输入预训练语言模型进行预训练，并在预训练语言模型针对样本数据的识别准确率达到第一设定准确率时，根据预训练语言模型当前的运行参数确定初始模型，向所述初 始模型输入测试语句，以预测测试语句的下一条语句作为唯一目标对所述初始模型进行微调，在初始模型的预测准确率达到第二设定准确率时，根据初始模型当前的运行参数确定意图识别模型，采用所述意图识别模型确定用户输入的语句的后一句语句，根据所确定的后一句语句确定用户意图，以提高所确定的用户意图的准确性。其中，用户意图识别终端可以但不限于是各种个人计算机和笔记本电脑等智能处理设备。
在一个实施例中，如图1所示，提供了一种基于语句前后关系预测的用户意图识别方法，以该方法应用于用户意图识别终端为例进行说明，包括以下步骤：
S10，设置多个样本数据；所述样本数据包括第一条语句，第二条语句，以及第一条语句和第二条语句的语句属性特征和位置关系。
上述语句属性特征包括相应语句所包括的单词，以及各个单词的位置等内容。
在一个实施例中，设置多个样本数据包括：
获取多组语句，设置各组语句中各个词的单词嵌入向量、标识嵌入向量和位置嵌入向量，根据各组语句以及各自语句分别对应的单词嵌入向量、标识嵌入向量和位置嵌入向量确定样本数据；其中各组语句均包括第一语句和第二语句；所述单词嵌入向量表征相应单词的内容；所述标识嵌入向量表征相应单词属于第一语句或者第二语句；所述位置嵌入向量表征相应单词在所在语句中的位置。
具体地，上述各组语句均包括第一语句和第二语句，第一语句可以为相应组语句的前一条语句，第二语句可以为相应语句的后一条语句。
进一步地，上述样本数据作为后续预训练语言模型的输入，其中每个序列的第一个标记可以始终是相应分类标记。与此标记对应的最终输出隐藏状态用作表示第二条语句是否为第一条语句的下一语句。第一条和第二条语句可以封装在一起形成单个序列，作为一组语句。
在一个示例中，可以通过两种方式区分语句。第一种方式是使用特殊符号([SEP])将它们分开。第二种方式是向每个单词添加学习到的标识嵌入向量，以指示它是属于语句A(第一语句)还是语句B(第二语句)。对于每一个单词，它的输入由单词本身的单词嵌入向量、标识嵌入向量(E A、 E B)和位置嵌入向量(E 0、E 1、E 2...)相加得到。具体过程可以参考图2所示。
S20，将各个样本数据输入预训练语言模型进行预训练，并在预训练语言模型针对样本数据的识别准确率达到第一设定准确率时，根据预训练语言模型当前的运行参数确定初始模型。
上述第一设定准确率可以依据用户识别的精度进行设置，如设置为98％等值。
具体地，预训练使用与对话系统无关的大规模单语语料库进行训练。相应模型(预训练语言模型)使用两种任务作为目标进行预训练。第一个任务是屏蔽语言模型。在输入时随机屏蔽一定比例的单词，然后在模型的输出预测这些屏蔽掉的单词，这种方式可以构建双向的深度网络。第二个任务是预测第二条语句是否为下一个语句。在为每个预训练样本选择两个语句时，百分之五十的概率第二条语句是跟随第一条语句的实际下一个语句，百分之五十的概率是来自语料库的随机语句。
S30，向所述初始模型输入测试语句，以预测测试语句的下一条语句作为唯一目标对所述初始模型进行微调，在初始模型的预测准确率达到第二设定准确率时，根据初始模型当前的运行参数确定意图识别模型。
上述第二设定准确率可以依据用户识别的精度进行设置，如设置为98％等值。
具体地，预训练完成后，用对话系统配置的语句对预训练后的模型进行微调。在微调阶段，不再以屏蔽语言模型作为训练目标，而仅仅以预测下一条语句作为唯一目标，因此模型的输入不再屏蔽任何单词。微调阶段的样本以期望用户所说的语句作为第一条语句，以对话系统中配置的下一节点的语句作为第二条语句，生成任务训练集中的正样本；以期望用户所说的语句作为第一条语句，以话术中配置的其它节点的语句作为第二条语句，生成任务训练集中的负样本。
可选地，微调时的模型与训练目标如图3所示。
S40，采用所述意图识别模型确定用户输入的语句的后一句语句，根据所确定的后一句语句确定用户意图。
在一个实施例中，采用所述意图识别模型确定用户输入的语句的后一句语句包括：
读取用户输入的语句，将用户输入的语句输入所述意图识别模型，所述意图识别模型输入多个待选语句以及各个待选语句的概率值，将概率值最大的待选语句确定为用户输入的语句的后一句语句。
在实际人机对话过程中，以用户真实所说的语句作为第一条语句，当前节点中所有分支的语句作为第二条语句，执行相应模型(意图识别模型)的预测方法，得到每一条语句作为用户所说语句的下一条语句的概率。取概率最高的语句所在的分支作为匹配的意图，并返回概率最高的语句作为回复。
进一步地，在预测阶段，模型的输入同样不再屏蔽任何单词。
上述基于语句前后关系预测的用户意图识别方法，通过设置多个样本数据，将各个样本数据输入预训练语言模型进行预训练，并在预训练语言模型针对样本数据的识别准确率达到第一设定准确率时，根据预训练语言模型当前的运行参数确定初始模型，向初始模型输入测试语句，以预测测试语句的下一条语句作为唯一目标对初始模型进行微调，在初始模型的预测准确率达到第二设定准确率时，根据初始模型当前的运行参数确定意图识别模型，采用意图识别模型确定用户输入的语句的后一句语句，根据所确定的后一句语句确定用户意图，所确定的用户意图具有较高的准确性。
在一个实施例中，对于上述基于语句前后关系预测的用户意图识别方法的应用过程中，语言模型预训练对于改进许多自然语言处理任务效果非常显著。这些任务包括语句级别的任务以及单词级别的任务，如预测语句之间关系的自然语言推理、命名实体识别和知识问答等。基于Transformer的双向编码表征(BERT)是最近提出的预训练语言模型，该预训练模型能高效抽取文本信息并应用于各种自然语言处理任务，它的出现刷新了11项自然语言处理任务的最优性能记录。为了训练能够理解语句关系的模型，BERT提出可以从任何单语语料库训练预测下一条语句的任务，即判断两条语句是否应该为连续的前后语句关系。在为每个预训练样本选择两个语句时，百分之五十的概率第二条语句是跟随第一条语句的实际下一条语句，百分之五十的概率是来自语料库的随机语句，即第二条语句实际上不是第一条语句的下一条语句。在训练深度神经网络的双向表征时，为了不让每个单词影响注意力机制，BERT随机屏蔽一定比例的输入单词，然 后预测被屏蔽的单词。本发明以两条语句是否应该为连续的前后语句关系作为意图识别的一种判断依据，提高意图识别的准确率。具体来说，以期望用户所说的语句作为第一条语句，以对话系统中配置的下一节点的语句作为第二条语句，生成任务训练集中的正样本；以期望用户所说的语句作为第一条语句，以话术中配置的其它节点的语句作为第二条语句，生成任务训练集中的负样本。正负样本生成以后，在此数据上继续训练微调BERT预训练模型，直至模型的损失值收敛。在实际人机对话过程中，以用户真实所说的语句作为第一条语句，当前节点中所有分支的语句作为第二条语句，执行模型的预测方法，得到每一条语句作为用户所说语句的下一条语句的概率。取概率最高的语句所在的分支作为匹配的意图，并返回概率最高的语句作为回复。
参考图4所示，图4为一个实施例的基于语句前后关系预测的用户意图识别装置结构示意图，包括：
设置模块10，用于设置多个样本数据；所述样本数据包括第一条语句，第二条语句，以及第一条语句和第二条语句的语句属性特征和位置关系；
预训练模块20，用于将各个样本数据输入预训练语言模型进行预训练，并在预训练语言模型针对样本数据的识别准确率达到第一设定准确率时，根据预训练语言模型当前的运行参数确定初始模型；
微调模块30，用于向所述初始模型输入测试语句，以预测测试语句的下一条语句作为唯一目标对所述初始模型进行微调，在初始模型的预测准确率达到第二设定准确率时，根据初始模型当前的运行参数确定意图识别模型；
确定模块40，用于采用所述意图识别模型确定用户输入的语句的后一句语句，根据所确定的后一句语句确定用户意图。
在一个实施例中，所述设置模块进一步用于：
获取多组语句，设置各组语句中各个词的单词嵌入向量、标识嵌入向量和位置嵌入向量，根据各组语句以及各自语句分别对应的单词嵌入向量、标识嵌入向量和位置嵌入向量确定样本数据；其中各组语句均包括第一语句和第二语句；所述单词嵌入向量表征相应单词的内容；所述标识嵌 入向量表征相应单词属于第一语句或者第二语句；所述位置嵌入向量表征相应单词在所在语句中的位置。
在一个实施例中，所述确定模块进一步用于：
读取用户输入的语句，将用户输入的语句输入所述意图识别模型，所述意图识别模型输入多个待选语句以及各个待选语句的概率值，将概率值最大的待选语句确定为用户输入的语句的后一句语句。
关于基于语句前后关系预测的用户意图识别装置的具体限定可以参见上文中对于基于语句前后关系预测的用户意图识别方法的限定，在此不再赘述。上述基于语句前后关系预测的用户意图识别装置中的各个模块可全部或部分通过软件、硬件及其组合来实现。上述各模块可以硬件形式内嵌于或独立于计算机设备中的处理器中，也可以以软件形式存储于计算机设备中的存储器中，以便于处理器调用执行以上各个模块对应的操作。
在一个实施例中，提供了一种计算机设备，该计算机设备可以是终端，其内部结构图可以如图5所示。该计算机设备包括通过系统总线连接的处理器、存储器、网络接口、显示屏和输入装置。其中，该计算机设备的处理器用于提供计算和控制能力。该计算机设备的存储器包括非易失性存储介质、内存储器。该非易失性存储介质存储有操作系统和计算机程序。该内存储器为非易失性存储介质中的操作系统和计算机程序的运行提供环境。该计算机设备的网络接口用于与外部的终端通过网络连接通信。该计算机程序被处理器执行时以实现一种基于语句前后关系预测的用户意图识别方法。该计算机设备的显示屏可以是液晶显示屏或者电子墨水显示屏，该计算机设备的输入装置可以是显示屏上覆盖的触摸层，也可以是计算机设备外壳上设置的按键、轨迹球或触控板，还可以是外接的键盘、触控板或鼠标等。
本领域技术人员可以理解，图5中示出的结构，仅仅是与本申请方案相关的部分结构的框图，并不构成对本申请方案所应用于其上的计算机设备的限定，具体的计算机设备可以包括比图中所示更多或更少的部件，或者组合某些部件，或者具有不同的部件布置。
基于如上所述的示例，在一个实施例中还提供一种计算机设备，该计算机设备包括存储器、处理器及存储在存储器上并可在处理器上运行的计 算机程序，其中，处理器执行所述程序时实现如上述各实施例中的任意一种基于语句前后关系预测的用户意图识别方法。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通过计算机程序来指令相关的硬件来完成，所述的程序可存储于一非易失性的计算机可读取存储介质中，如本发明实施例中，该程序可存储于计算机系统的存储介质中，并被该计算机系统中的至少一个处理器执行，以实现包括如上述基于语句前后关系预测的用户意图识别方法的实施例的流程。其中，所述的存储介质可为磁碟、光盘、只读存储记忆体(Read-Only Memory，ROM)或随机存储记忆体(Random Access Memory，RAM)等。
据此，在一个实施例中还提供一种计算机存储介质计算机可读存储介质，其上存储有计算机程序，其中，该程序被处理器执行时实现如上述各实施例中的任意一种基于语句前后关系预测的用户意图识别方法。
本文中应用了具体个例对本发明的原理及实施方式进行了阐述，以上实施例的说明只是用于帮助理解本发明的方法及其核心思想；同时，对于本领域的一般技术人员，依据本发明的思想，在具体实施方式及应用范围上均会有改变之处。综上所述，本说明书内容不应理解为对本发明的限制。

Claims (8)

  1. 一种基于语句前后关系预测的用户意图识别方法，其特征在于，包括如下步骤：
    S10，设置多个样本数据；所述样本数据包括第一条语句，第二条语句，以及第一条语句和第二条语句的语句属性特征和位置关系；
    S20，将各个样本数据输入预训练语言模型进行预训练，并在预训练语言模型针对样本数据的识别准确率达到第一设定准确率时，根据预训练语言模型当前的运行参数确定初始模型；
    S30，向所述初始模型输入测试语句，以预测测试语句的下一条语句作为唯一目标对所述初始模型进行微调，在初始模型的预测准确率达到第二设定准确率时，根据初始模型当前的运行参数确定意图识别模型；
    S40，采用所述意图识别模型确定用户输入的语句的后一句语句，根据所确定的后一句语句确定用户意图。
  2. 根据权利要求1所述的基于语句前后关系预测的用户意图识别方法，其特征在于，设置多个样本数据包括：
    获取多组语句，设置各组语句中各个词的单词嵌入向量、标识嵌入向量和位置嵌入向量，根据各组语句以及各自语句分别对应的单词嵌入向量、标识嵌入向量和位置嵌入向量确定样本数据；其中各组语句均包括第一语句和第二语句；所述单词嵌入向量表征相应单词的内容；所述标识嵌入向量表征相应单词属于第一语句或者第二语句；所述位置嵌入向量表征相应单词在所在语句中的位置。
  3. 根据权利要求2所述的基于语句前后关系预测的用户意图识别方法，其特征在于，采用所述意图识别模型确定用户输入的语句的后一句语句包括：
    读取用户输入的语句，将用户输入的语句输入所述意图识别模型，所述意图识别模型输入多个待选语句以及各个待选语句的概率值，将概率值最大的待选语句确定为用户输入的语句的后一句语句。
  4. 一种基于语句前后关系预测的用户意图识别装置，其特征在于，包括：
    设置模块，用于设置多个样本数据；所述样本数据包括第一条语句，第二条语句，以及第一条语句和第二条语句的语句属性特征和位置关系；
    预训练模块，用于将各个样本数据输入预训练语言模型进行预训练，并在预训练语言模型针对样本数据的识别准确率达到第一设定准确率时，根据预训练语言模型当前的运行参数确定初始模型；
    微调模块，用于向所述初始模型输入测试语句，以预测测试语句的下一条语句作为唯一目标对所述初始模型进行微调，在初始模型的预测准确率达到第二设定准确率时，根据初始模型当前的运行参数确定意图识别模型；
    确定模块，用于采用所述意图识别模型确定用户输入的语句的后一句语句，根据所确定的后一句语句确定用户意图。
  5. 根据权利要求4所述的基于语句前后关系预测的用户意图识别装置，其特征在于，所述设置模块进一步用于：
    获取多组语句，设置各组语句中各个词的单词嵌入向量、标识嵌入向量和位置嵌入向量，根据各组语句以及各自语句分别对应的单词嵌入向量、标识嵌入向量和位置嵌入向量确定样本数据；其中各组语句均包括第一语句和第二语句；所述单词嵌入向量表征相应单词的内容；所述标识嵌入向量表征相应单词属于第一语句或者第二语句；所述位置嵌入向量表征相应单词在所在语句中的位置。
  6. 根据权利要求4所述的基于语句前后关系预测的用户意图识别装置，其特征在于，所述确定模块进一步用于：
    读取用户输入的语句，将用户输入的语句输入所述意图识别模型，所述意图识别模型输入多个待选语句以及各个待选语句的概率值，将概率值最大的待选语句确定为用户输入的语句的后一句语句。
  7. 一种计算机设备，包括存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，其特征在于，所述处理器执行所述计算机程序时实现权利要求1至3中任一项所述方法的步骤。
  8. 一种计算机可读存储介质，其上存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现权利要求1至3中任一项所述方法的步骤。
PCT/CN2021/074788 2020-02-25 2021-02-02 基于语句前后关系预测的用户意图识别方法及装置 WO2021169745A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/802,109 US20230080671A1 (en) 2020-02-25 2021-02-02 User intention recognition method and apparatus based on statement context relationship prediction

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010116553.9 2020-02-25
CN202010116553.9A CN111563144B (zh) 2020-02-25 2020-02-25 基于语句前后关系预测的用户意图识别方法及装置

Publications (1)

Publication Number Publication Date
WO2021169745A1 true WO2021169745A1 (zh) 2021-09-02

Family

ID=72071365

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/074788 WO2021169745A1 (zh) 2020-02-25 2021-02-02 基于语句前后关系预测的用户意图识别方法及装置

Country Status (3)

Country Link
US (1) US20230080671A1 (zh)
CN (1) CN111563144B (zh)
WO (1) WO2021169745A1 (zh)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114021572A (zh) * 2022-01-05 2022-02-08 苏州浪潮智能科技有限公司 一种自然语言处理方法、装置、设备及可读存储介质
CN114238566A (zh) * 2021-12-10 2022-03-25 零犀(北京)科技有限公司 一种语音或文本数据的数据增强方法及装置
WO2023130687A1 (zh) * 2022-01-05 2023-07-13 苏州浪潮智能科技有限公司 一种自然语言处理方法、装置、设备及可读存储介质
WO2023165111A1 (zh) * 2022-03-01 2023-09-07 达而观信息科技（上海）有限公司 客服热线中用户意图轨迹识别的方法及系统

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111563144B (zh) * 2020-02-25 2023-10-20 升智信息科技(南京)有限公司 基于语句前后关系预测的用户意图识别方法及装置
CN113076080B (zh) * 2021-04-21 2022-05-17 百度在线网络技术（北京）有限公司 模型训练方法和装置、意图识别方法和装置
US20220374604A1 (en) * 2021-05-18 2022-11-24 International Business Machines Corporation Natural language bias detection in conversational system environments
CN114398903B (zh) * 2022-01-21 2023-06-20 平安科技（深圳）有限公司 意图识别方法、装置、电子设备及存储介质
CN116911314B (zh) * 2023-09-13 2023-12-19 北京中关村科金技术有限公司 意图识别模型的训练方法、会话意图识别方法及系统

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090182549A1 (en) * 2006-10-10 2009-07-16 Konstantin Anisimovich Deep Model Statistics Method for Machine Translation
CN109597993A (zh) * 2018-11-30 2019-04-09 深圳前海微众银行股份有限公司 语句分析处理方法、装置、设备以及计算机可读存储介质
CN110287283A (zh) * 2019-05-22 2019-09-27 中国平安财产保险股份有限公司 意图模型训练方法、意图识别方法、装置、设备及介质
CN110795552A (zh) * 2019-10-22 2020-02-14 腾讯科技（深圳）有限公司 一种训练样本生成方法、装置、电子设备及存储介质
CN111563144A (zh) * 2020-02-25 2020-08-21 升智信息科技(南京)有限公司 基于语句前后关系预测的用户意图识别方法及装置

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108829894B (zh) * 2018-06-29 2021-11-12 北京百度网讯科技有限公司 口语词识别和语义识别方法及其装置
CN109635947B (zh) * 2018-12-14 2020-11-03 安徽省泰岳祥升软件有限公司 基于答案采样的机器阅读理解模型训练方法及装置
CN110516055A (zh) * 2019-08-16 2019-11-29 西北工业大学 一种结合bert的用于教学任务的跨平台智能问答实现方法
CN110674639B (zh) * 2019-09-24 2022-12-09 识因智能科技有限公司 一种基于预训练模型的自然语言理解方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090182549A1 (en) * 2006-10-10 2009-07-16 Konstantin Anisimovich Deep Model Statistics Method for Machine Translation
CN109597993A (zh) * 2018-11-30 2019-04-09 深圳前海微众银行股份有限公司 语句分析处理方法、装置、设备以及计算机可读存储介质
CN110287283A (zh) * 2019-05-22 2019-09-27 中国平安财产保险股份有限公司 意图模型训练方法、意图识别方法、装置、设备及介质
CN110795552A (zh) * 2019-10-22 2020-02-14 腾讯科技（深圳）有限公司 一种训练样本生成方法、装置、电子设备及存储介质
CN111563144A (zh) * 2020-02-25 2020-08-21 升智信息科技(南京)有限公司 基于语句前后关系预测的用户意图识别方法及装置

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114238566A (zh) * 2021-12-10 2022-03-25 零犀(北京)科技有限公司 一种语音或文本数据的数据增强方法及装置
CN114021572A (zh) * 2022-01-05 2022-02-08 苏州浪潮智能科技有限公司 一种自然语言处理方法、装置、设备及可读存储介质
CN114021572B (zh) * 2022-01-05 2022-03-22 苏州浪潮智能科技有限公司 一种自然语言处理方法、装置、设备及可读存储介质
WO2023130687A1 (zh) * 2022-01-05 2023-07-13 苏州浪潮智能科技有限公司 一种自然语言处理方法、装置、设备及可读存储介质
WO2023165111A1 (zh) * 2022-03-01 2023-09-07 达而观信息科技（上海）有限公司 客服热线中用户意图轨迹识别的方法及系统

Also Published As

Publication number Publication date
US20230080671A1 (en) 2023-03-16
CN111563144A (zh) 2020-08-21
CN111563144B (zh) 2023-10-20

Similar Documents

Publication Publication Date Title
WO2021169745A1 (zh) 基于语句前后关系预测的用户意图识别方法及装置
US11663404B2 (en) Text recognition method, electronic device, and storage medium
JP7250052B2 (ja) 自然言語及び知識グラフに基づく表現学習方法及び装置
US11967312B2 (en) Method and apparatus for training semantic understanding model, electronic device, and storage medium
CN110442718B (zh) 语句处理方法、装置及服务器和存储介质
WO2022100045A1 (zh) 分类模型的训练方法、样本分类方法、装置和设备
JP2021096812A (ja) セマンティック表現モデルの処理方法、装置、電子機器、及び記憶媒体
CN111062217B (zh) 语言信息的处理方法、装置、存储介质及电子设备
US20210192364A1 (en) Representation learning method and device based on natural language and knowledge graph
EP3852000A1 (en) Method and apparatus for processing semantic description of text entity, device and storage medium
WO2020244475A1 (zh) 用于语言序列标注的方法、装置、存储介质及计算设备
US20220044081A1 (en) Method for recognizing dialogue intention, electronic device and storage medium
US20230023789A1 (en) Method for identifying noise samples, electronic device, and storage medium
US11947920B2 (en) Man-machine dialogue method and system, computer device and medium
JP2022087814A (ja) 多言語モデルの訓練方法、装置、電子デバイス及び可読記憶媒体
Stoyanchev et al. Action state update approach to dialogue management
Ma et al. A two-stage selective fusion framework for joint intent detection and slot filling
Zhang et al. Chinese named-entity recognition via self-attention mechanism and position-aware influence propagation embedding
US11562150B2 (en) Language generation method and apparatus, electronic device and storage medium
CN113869324A (zh) 一种基于多模态融合的视频常识性知识推理实现方法
CHAURASIA et al. Conversational AI Unleashed: A Comprehensive Review of NLP-Powered Chatbot Platforms
WO2023116572A1 (zh) 一种词句生成方法及相关设备
Malviya et al. HDRS: Hindi dialogue restaurant search corpus for dialogue state tracking in task-oriented environment
Zhou et al. An image captioning model based on bidirectional depth residuals and its application
EP4272109A1 (en) Cross-lingual apparatus and method

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21760406

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21760406

Country of ref document: EP

Kind code of ref document: A1