WO2020113918A1 - 基于语义解析的语句合理性判断方法、装置、计算机设备 - Google Patents

基于语义解析的语句合理性判断方法、装置、计算机设备 Download PDF

Info

Publication number
WO2020113918A1
WO2020113918A1 PCT/CN2019/089107 CN2019089107W WO2020113918A1 WO 2020113918 A1 WO2020113918 A1 WO 2020113918A1 CN 2019089107 W CN2019089107 W CN 2019089107W WO 2020113918 A1 WO2020113918 A1 WO 2020113918A1
Authority
WO
WIPO (PCT)
Prior art keywords
sentence
judged
reasonable
preset
ngram model
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2019/089107
Other languages
English (en)
French (fr)
Inventor
韩铃
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ping An Technology Shenzhen Co Ltd
Original Assignee
Ping An Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ping An Technology Shenzhen Co Ltd filed Critical Ping An Technology Shenzhen Co Ltd
Publication of WO2020113918A1 publication Critical patent/WO2020113918A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/279—Recognition of textual entities
    • G06F40/289—Phrasal analysis, e.g. finite state techniques or chunking
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/30—Semantic analysis

Definitions

  • the present application relates to the technical field of semantic parsing, and in particular to a method, device, and computer equipment for sentence rationality judgment based on semantic parsing.
  • this application proposes a method, device and computer equipment for judging the reasonableness of sentences based on semantic parsing, which aims to solve the problem that in the existing speech training, students cannot judge without the presence of a professional teacher The question of the rationality of the statement I said.
  • a sentence rationality judgment method based on semantic analysis includes:
  • the present application also provides a sentence rationality judgment device based on semantic analysis.
  • the device includes:
  • the establishment module is used to establish an ngram model, wherein the ngram model is used to calculate the context occurrence probability of each word segmentation;
  • the first obtaining module is used to obtain the sentence to be judged input by the user
  • the word segmentation module is used to perform word segmentation processing on the sentence to be judged to obtain multiple word segments corresponding to the sentence to be judged;
  • a loading module configured to load a plurality of word segments corresponding to the sentence to be judged into the ngram model
  • a second obtaining module configured to obtain the context occurrence probability of each participle among the plurality of participles corresponding to the sentence to be judged output by the ngram model
  • a calculation module configured to multiply the context occurrence probability of each participle among the plurality of participles corresponding to the sentence to be judged, and output the confusion of the sentence to be judged;
  • the judgment module is used to compare the confusion degree of the sentence to be judged with a preset confusion degree threshold, and judge the rationality of the sentence to be judged according to the comparison result.
  • the present application also provides a computer device, including a memory and a processor, where the memory stores a computer program, and when the processor executes the computer program, the steps of any one of the above methods are implemented.
  • the present application also provides a computer non-volatile readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method described in any one of the above are implemented.
  • the ngram model calculates the multiple word segments, and calculates the context occurrence probability of each word segment in the multiple word segments.
  • the product of the context occurrence probability of each participle in the word segmentation output the confusion degree of the sentence to be judged, according to the confusion degree of the sentence to be judged, judge the rationality of the sentence to be judged, so that the user can judge whether the sentence he said is reasonable, It can quickly improve one's own speech skills, aiming to solve the problem that in the existing speech skills training, students cannot judge the rationality of their statements without the presence of professional teachers.
  • FIG. 1 is a flowchart of a method for judging the rationality of sentences based on semantic analysis provided by an embodiment of the present application
  • FIG. 2 is a functional block diagram of a device for judging the reasonableness of sentences based on semantic analysis provided by an embodiment of the present application
  • FIG. 3 is a schematic block diagram of a structure of a computer device provided by an embodiment of the present application.
  • an embodiment of the present application provides a method for judging the rationality of sentences based on semantic analysis.
  • the method includes the following steps:
  • Step S101 Establish an ngram model, where the ngram model is used to calculate the context occurrence probability of each word segmentation.
  • the ngram model carries the context information and probability of each word segmentation, after loading the word segmentation of the sentence into the ngram model, the ngram model calculates the context of each word segmentation Occurrence probability. After the calculation is completed, the context occurrence probability of each participle is output.
  • the training process of the ngram model includes:
  • the ngram model is judged to be a qualified model.
  • the corpus obtained by crawling contains a lot of noise, so it needs to be filtered, such as removing meaningless symbols, removing numbers, removing English, and removing meaningless punctuation marks.
  • Chinese word segmentation is performed. Chinese word segmentation has a higher accuracy rate for general language, but the result of word segmentation in a specific field is poorer, because the corpus of training word segmentation contains less domain-specific corpus, so the domain entity is corrected next;
  • the word segmentation results are trained on the ngram model.
  • the model training tool used is kenlm, and the evaluation index is perplexity. According to the training result, it is judged whether the preset goal is reached. If the preset goal is reached, the ngram model is judged as a qualified model, and the ngram model can be used.
  • Step S102 Acquire a sentence to be judged input by the user.
  • the sentence to be judged input by the user is acquired.
  • the input method of the sentence to be judged may be voice input.
  • Step S103 Perform word segmentation processing on the sentence to be judged to obtain multiple word segments corresponding to the sentence to be judged.
  • word segmentation processing is performed on the sentence to be judged, and after completing the word segmentation processing, a plurality of word segments corresponding to the sentence to be judged are obtained.
  • it has a word segmentation function, and directly performs word segmentation processing on the sentence to be judged, so as to obtain multiple word segments corresponding to the sentence to be judged.
  • the sentence to be judged is sent to the word segmentation model, and the sentence to be judged is subjected to word segmentation processing by the word segmentation model.
  • the word segmentation model receives the sentence corresponding to the sentence to be judged. Multiple participles to obtain multiple participles corresponding to the sentence to be judged.
  • Step S104 Load multiple word segments corresponding to the sentence to be judged into the ngram model.
  • the ngram model After obtaining the multiple word segments corresponding to the sentence to be judged, load the multiple word segments corresponding to the sentence to be judged into the ngram model. If no dependency value is sent to the ngram model, the ngram model calculates the to-be-judgment according to its default dependency value The context occurrence probability of each participle among the multiple participles corresponding to the sentence. If the dependent value is sent to the ngram model, the ngram model calculates the contextual appearance probability of each participle among the multiple participles corresponding to the sentence to be judged according to the received dependent value.
  • Step S105 Acquire the context occurrence probability of each participle among the plural participles corresponding to the sentence to be judged output by the ngram model.
  • the context occurrence probability of each participle among the plurality of participles corresponding to the sentence to be judged output by the ngram model is obtained.
  • step S104 it includes:
  • the preset dependency value is a word segment when calculating the context occurrence probability of the word segmentation Depends on the number of preceding participles.
  • the preset dependency value is the number of words that a participle depends on in front of the word segmentation when calculating the context occurrence probability of the word segmentation.
  • the preset dependency value is 1, and the word segmentation depends on the previous word segment in calculating the context occurrence probability of the word segmentation.
  • step S105 it includes:
  • the ngram model After the ngram model receives the preset dependent value and multiple word segments corresponding to the sentence to be judged, the ngram model calculates the context occurrence probability of each word segment in the multiple word segments corresponding to the sentence to be judged according to the preset dependent value. , Receiving the context occurrence probability of each participle among the multiple participles corresponding to the sentence to be judged output by the ngram model.
  • the step of loading a plurality of participles corresponding to the sentence to be judged into the ngram model and sending a preset dependent value to the ngram model includes:
  • the preset dependent value is set to the first value
  • the preset dependent value is set to a second value, where the second value is less than the first value ;
  • the number of word segments in the multiple word segments corresponding to the sentence to be judged is identified, and set to different values according to the different number of word segments in the multiple word segments corresponding to the sentence to be judged
  • the number of participles in the plurality of participles corresponding to the sentence to be judged is compared with a preset number.
  • the preset number is 10. If the number of participles in the multiple participles corresponding to the sentence to be judged is greater than the preset number, the preset dependent value is set to the first value.
  • the first value is 2, if the If the number of participles in multiple participles corresponding to the sentence is less than or equal to the preset number, the preset dependent value is set to the second value, where the second value is less than the first value, in this embodiment, the second value is 1 .
  • the preset dependency value After finishing setting the preset dependency value, multiple word segments corresponding to the sentence to be judged are loaded into the ngram model, and at the same time the preset dependency value is sent to the ngram model.
  • the step of loading a plurality of participles corresponding to the sentence to be judged into the ngram model and sending a preset dependent value to the ngram model includes:
  • the user After obtaining a plurality of participles corresponding to the sentence to be judged, the user is prompted to input a preset dependency value, and after the user completes the input, the preset dependency value input by the user is received. Multiple participles corresponding to the sentence of are loaded into the ngram model, and the preset dependent values are sent to the ngram model.
  • Step S106 Multiply the context occurrence probability of each participle among the plural participles corresponding to the sentence to be judged, and output the confusion degree of the sentence to be judged.
  • the context occurrence probability of each participle in the plural participles corresponding to the sentence to be judged is multiplied, and the confusion degree of the sentence to be judged is output ,
  • the confusion of the sentence to be judged is inversely proportional to the probability of the product of the context occurrence probability of each participle in the multiple participles corresponding to the sentence to be judged, the smaller the confusion of the sentence, the greater the probability of the sentence, the more reasonable the sentence .
  • Step S107 Compare the confusion degree of the sentence to be judged with a preset confusion degree threshold, and judge the rationality of the sentence to be judged according to the comparison result.
  • the confusion degree of the sentence to be judged is compared with a preset confusion degree threshold.
  • the preset confusion degree threshold is a confusion degree threshold input by the user, according to the comparison result To judge the rationality of the sentence to be judged.
  • step S107 it includes:
  • the confusion level of the sentence to be judged is compared with the preset confusion degree threshold, if the confusion degree of the sentence to be judged is less than the preset confusion degree threshold, the sentence to be judged is judged as Reasonable sentences, if the confusion of the sentence to be judged is greater than or equal to the preset confusion threshold, the sentence to be judged is judged to be unreasonable, the unreasonable sentence needs to be corrected, and the sentence to be judged is unreasonable When the statement is made, the user is warned to make corrections.
  • the preset perplexity threshold value is not a perplexity threshold value input by the user, but is obtained through calculation. Specifically, in step S107, it includes:
  • the user selects and enters multiple reasonable sentences.
  • the multiple reasonable sentences are 200 reasonable sentences.
  • the user receives multiple reasonable sentences, and for each reasonable sentence among the multiple reasonable sentences Perform word segmentation processing to obtain multiple word segments corresponding to each reasonable sentence in multiple reasonable sentences, that is to say, each reasonable sentence needs to be subjected to word segmentation processing, corresponding to each reasonable sentence in the multiple reasonable sentences
  • the multiple participles of each are loaded into the ngram model, that is to say, the multiple participles corresponding to each reasonable sentence need to be loaded into the ngram model.
  • the ngram model calculates the context occurrence of each participle in the multiple participles corresponding to each reasonable sentence Probability, then the ngram model outputs the context occurrence probability of each participle in multiple participles corresponding to each reasonable sentence, and obtains the context of each participle in multiple participles corresponding to each reasonable sentence in the multiple reasonable sentences output by the ngram model Probability of occurrence, multiply the contextual occurrence probability of each participle in multiple participles corresponding to each reasonable sentence in multiple reasonable sentences, and output the perplexity of each reasonable sentence in multiple reasonable sentences, according to The perplexity of each reasonable statement in a reasonable statement, calculate the average perplexity of multiple reasonable statements, and obtain the standard perplexity.
  • the standard perplexity is 60% is used as the preset confusion level threshold, thereby obtaining the preset confusion level threshold, comparing the confusion level of the sentence to be judged with the preset confusion level threshold, and judging the rationality of the sentence to be judged according to the comparison result.
  • the sentence to be judged by the user is subjected to word segmentation processing to obtain multiple word segments, and the multiple word segments are loaded into the ngram model.
  • the ngram model calculates the multiple word segments and calculates the context occurrence of each word segment in the multiple word segments.
  • Probability based on the product of the context occurrence probability of each participle in multiple participles, output the confusion degree of the sentence to be judged, according to the confusion degree of the sentence to be judged, judge the rationality of the sentence to be judged, so that the user can judge what he said Whether the sentence is reasonable, can quickly improve their own speech, aiming to solve the problem that in the existing speech training, without the presence of a professional teacher, students cannot judge the rationality of the sentence they say.
  • an embodiment of the present application proposes a text accuracy calculation device 1 based on semantic analysis.
  • the device 1 includes an establishment module 11, a first acquisition module 12, a word segmentation module 13, a loading module 14, and a second acquisition module 15. Calculation module 16 and judgment module 17.
  • the building module 11 is used to build an ngram model, wherein the ngram model is used to calculate the context occurrence probability of each word segmentation.
  • the device 1 includes:
  • the first crawling module is used to crawl the corpus in the field
  • the first filtering module is used to filter the corpus to obtain the filtered corpus
  • the first Chinese word segmentation module is used to perform Chinese word segmentation on the filtered corpus to obtain the segmented corpus;
  • the first correction module is used to correct the domain entity of the word segmentation corpus to obtain the corrected corpus;
  • the first training module is used to train the corrected corpus through the kenlm training tool
  • the first judgment module is used to judge whether the preset goal is reached according to the training result
  • the second judgment module is used to judge that the ngram model is a qualified model if the preset goal is reached.
  • the first obtaining module 12 is used to obtain the sentence to be judged input by the user.
  • the word segmentation module 13 is configured to perform word segmentation processing on the sentence to be judged to obtain multiple word segments corresponding to the sentence to be judged.
  • the loading module 14 is configured to load multiple word segments corresponding to the sentence to be judged into the ngram model.
  • the second obtaining module 15 is configured to obtain the context occurrence probability of each participle among the plurality of participles corresponding to the sentence to be judged output by the ngram model.
  • the loading module 14 includes:
  • the first sub-loading module is used to load multiple word segments corresponding to the sentence to be judged into the ngram model, and at the same time send a preset dependency value to the ngram model, wherein the preset dependency value is in When calculating the context occurrence probability of a word segmentation, a word segmentation depends on the number of previous word segments.
  • the second acquisition module 15 includes:
  • a first sub-calculation module configured to calculate the context occurrence probability of each participle among the plurality of participles corresponding to the sentence to be determined according to the preset dependent value through the ngram model;
  • the first sub-receiving module is configured to receive a context occurrence probability of each participle among a plurality of participles corresponding to the sentence to be judged output by the ngram model.
  • the first sub-loading module includes:
  • a first sub-recognition module used to identify the number of participles in multiple participles corresponding to the sentence to be judged
  • a first sub-judgment module which is used to compare the number of word segments in the plurality of word segments corresponding to the sentence to be judged with a preset number
  • a first sub-setting module configured to set the preset dependent value to the first value if the number of word segments in the plurality of word segments corresponding to the sentence to be judged is greater than the preset number
  • a second sub-setting module configured to set the preset dependent value to the second value if the number of word segments in the plurality of word segments corresponding to the sentence to be judged is less than or equal to the preset number, wherein the The second value is less than the first value;
  • the second sub-loading module is used to load multiple word segments corresponding to the sentence to be judged into the ngram model, and at the same time send the preset dependent value to the ngram model.
  • the first sub-loading module includes:
  • the first sub-prompt module is used to prompt the user to input the preset dependent value
  • the third sub-loading module is used to load a plurality of participles corresponding to the sentence to be judged into the ngram model when receiving the preset dependent value input by the user, and at the same time send the preset dependent value Give the ngram model.
  • the calculation module 16 is configured to multiply the context occurrence probability of each participle among the plurality of participles corresponding to the sentence to be judged, and output the confusion of the sentence to be judged.
  • the judging module 17 is configured to compare the confusion degree of the sentence to be judged with a preset confusion degree threshold, and judge the rationality of the sentence to be judged according to the comparison result.
  • the judgment module 17 includes:
  • the first sub-comparison module is used to compare the confusion degree of the sentence to be judged with a preset confusion degree threshold
  • a second sub-judgment module configured to judge that the sentence to be judged is a reasonable sentence if the confusion of the sentence to be judged is less than the preset confusion level threshold
  • a third sub-judgment module configured to judge that the sentence to be judged is an unreasonable sentence if the confusion of the sentence to be judged is greater than or equal to the preset confusion level threshold;
  • the first sub-reminder module is used to warn the user when the sentence to be judged is an unreasonable sentence.
  • the preset perplexity threshold is not a perplexity threshold input by the user, but is calculated.
  • the judgment module 17 includes:
  • the second sub-receiving module is used to receive multiple reasonable sentences input by the user;
  • the second sub-word segmentation module is used to perform word segmentation processing on each reasonable sentence in the plurality of reasonable sentences to obtain multiple word segments corresponding to each reasonable sentence in the plurality of reasonable sentences;
  • a third sub-loading module which is used to load multiple word segments corresponding to each reasonable sentence in the multiple reasonable sentences into the ngram model
  • the second sub-acquisition module is used to obtain the context occurrence probability of each participle among the multiple participles corresponding to each reasonable sentence among the multiple reasonable sentences output by the ngram model;
  • the second sub-calculation module is used to separately calculate the product of the context occurrence probability of each participle in the multiple participles corresponding to each reasonable sentence in the multiple reasonable sentences, and output each of the multiple reasonable sentences Reasonable statement confusion;
  • the third sub-calculation module is used to calculate the average perplexity of the multiple reasonable statements according to the perplexity of each of the multiple reasonable statements to obtain the standard perplexity;
  • a first sub-threshold setting module configured to use 60% of the standard perplexity as a preset perplexity threshold
  • the fourth sub-judgment module is used to compare the confusion degree of the sentence to be judged with the preset confusion degree threshold, and judge the rationality of the sentence to be judged according to the comparison result.
  • an embodiment of the present application further provides a computer device.
  • the computer device may be a server, and its internal structure may be as shown in FIG.
  • the computer device includes a processor, memory, network interface, and database connected by a system bus. Among them, the processor designed by the computer is used to provide computing and control capabilities.
  • the memory of the computer device includes a non-volatile storage medium and an internal memory.
  • the non-volatile storage medium stores an operating system, a computer program, and a database.
  • the memory device provides an environment for operating systems and computer programs in non-volatile storage media.
  • the database of the computer device is used to store data such as a model of sentence rationality judgment method based on semantic analysis.
  • the network interface of the computer device is used to communicate with external terminals through a network connection. When the computer program is executed by the processor, a method for judging the rationality of sentences based on semantic analysis is realized.
  • An embodiment of the present application also provides a computer non-volatile readable storage medium on which a computer program is stored.
  • a computer program is executed by a processor, a method for judging the reasonableness of a sentence based on semantic analysis is implemented.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Machine Translation (AREA)

Abstract

本申请涉及一种基于语义解析的语句合理性判断方法、装置、计算机设备。对待判断的语句进行分词处理，将待判断的语句对应的多个分词载入ngram模型；获取多个分词中各分词的上下文出现概率；将上下文出现概率进行乘积计算，输出待判断的语句的困惑度；将困惑度与预设困惑度阈值进行比较，判断待判断的语句的合理性。

Description

基于语义解析的语句合理性判断方法、装置、计算机设备
本申请要求于2018年12月06日提交中国专利局、申请号为2018114898189，申请名称为“基于语义解析的语句合理性判断方法、装置、计算机设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及语义解析技术领域，特别涉及一种基于语义解析的语句合理性判断方法、装置、计算机设备。
 背景技术
在现有的话术培训中，需要专业老师对学员所说语句进行判断，判断其是否合理，这需要在有专业老师在场的情况下，学员才能知道自己所说的语句是否合理，在没有专业老师在场的情况下，学员并不知道自己所说的语句是否合理，为此，在现有的话术培训中，在没有专业老师在场的情况下，学员不能判断自己所说语句的合理性。
 技术问题
针对现有技术不足，本申请提出一种基于语义解析的语句合理性判断方法、装置、计算机设备，旨在解决在现有的话术培训中，在没有专业老师在场的情况下，学员不能判断自己所说语句的合理性的问题。
 技术解决方案
本申请提出的技术方案是：
一种基于语义解析的语句合理性判断方法，所述方法包括：
建立ngram模型，其中所述ngram模型用于计算各分词的上下文出现概率；
获取用户输入的待判断的语句；
对所述待判断的语句进行分词处理，获得所述待判断的语句对应的多个分词；
将所述待判断的语句对应的多个分词载入所述ngram模型；
获取所述ngram模型输出的所述待判断的语句对应的多个分词中各分词的上下文出现概率；
将所述待判断的语句对应的多个分词中各分词的上下文出现概率进行乘积计算，输出所述待判断的语句的困惑度；
将所述待判断的语句的困惑度与预设困惑度阈值进行比较，根据比较结果，判断所述待判断的语句的合理性。
本申请还提供一种基于语义解析的语句合理性判断装置，所述装置包括：
建立模块，用于建立ngram模型，其中所述ngram模型用于计算各分词的上下文出现概率；
第一获取模块，用于获取用户输入的待判断的语句；
分词模块，用于对所述待判断的语句进行分词处理，获得所述待判断的语句对应的多个分词；
载入模块，用于将所述待判断的语句对应的多个分词载入所述ngram模型；
第二获取模块，用于获取所述ngram模型输出的所述待判断的语句对应的多个分词中各分词的上下文出现概率；
计算模块，用于将所述待判断的语句对应的多个分词中各分词的上下文出现概率进行乘积计算，输出所述待判断的语句的困惑度；
判断模块，用于将所述待判断的语句的困惑度与预设困惑度阈值进行比较，根据比较结果，判断所述待判断的语句的合理性。
本申请还提供一种计算机设备，包括存储器和处理器，所述存储器存储有计算机程序，所述处理器执行所述计算机程序时实现上述任一项所述的方法的步骤。
本申请还提供一种计算机非易失性可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时实现上述任一项所述的方法的步骤。
 有益效果
将用户输入的待判断的语句进行分词处理，得到多个分词，将多个分词载入ngram模型，ngram模型对多个分词进行计算，计算多个分词中各分词的上下文出现概率，根据多个分词中各分词的上下文出现概率的乘积，输出待判断的语句的困惑度，根据待判断的语句的困惑度，判断待判断的语句的合理性，从而用户可以判断自己所说的语句是否合理，可快速改进自己的话术，旨在解决在现有的话术培训中，在没有专业老师在场的情况下，学员不能判断自己所说语句的合理性的问题。
 附图说明
图1是应用本申请实施例提供的基于语义解析的语句合理性判断方法的流程图；
图2是应用本申请实施例提供的基于语义解析的语句合理性判断装置的功能模块图；
图3是应用本申请实施例提供的计算机设备的结构示意框图。
本申请的最佳实施方式
如图1所示，本申请实施例提出一种基于语义解析的语句合理性判断方法，所述方法包括以下步骤：
步骤S101、建立ngram模型，其中所述ngram模型用于计算各分词的上下文出现概率。
获得训练语料的分词，利用训练语料的分词和贝叶斯算法建立ngram模型，ngram模型承载每个分词的上下文信息和概率，在将语句的分词载入ngram模型之后，ngram模型计算各分词的上下文出现概率，在完成计算之后，输出各分词的上下文出现概率。
在一些实施例中，对ngram模型的训练过程包括：
爬取领域内的语料；
对语料进行过滤，获得过滤后的语料；
对过滤后的语料进行中文分词，获得分词后的语料；
对分词后的语料进行领域实体纠正，获得纠正后的语料；
通过kenlm训练工具对纠正后的语料进行训练；
根据训练结果判断是否达到预设目标；
若达到预设目标，则判断ngram模型为合格模型。
首先，爬取领域内的语料，爬取得到的语料包含了很多噪声，因此需要对其进行过滤，比如去除无意义符号、去除数字、去除英文以及去除无意义标点符号等。接着进行中文分词，中文分词对于通用的语言准确率较高，但是在特定领域内的分词结果效果较差，原因在于训练分词的语料包含特定领域语料较少，因此接下来进行领域实体的纠正；最后，将分词结果进行ngram模型的训练，采用的模型训练工具为kenlm，评价指标为困惑度Perplexity。根据训练结果判断是否达到预设目标，若达到预设目标，则判断ngram模型为合格模型，可以使用ngram模型。
步骤S102、获取用户输入的待判断的语句。
在用户输入待判断的语句之后，获取用户输入的待判断的语句。待判断的语句的输入方式可以是语音输入。
步骤S103、对所述待判断的语句进行分词处理，获得所述待判断的语句对应的多个分词。
在获取用户输入的待判断的语句之后，对待判断的语句进行分词处理，在完成分词处理之后，获得待判断的语句对应的多个分词。在本实施例中，自身具备分词功能，直接对待判断的语句进行分词处理，从而获得待判断的语句对应的多个分词。在一些实施例中，将待判断的语句发送给分词模型，通过分词模型对待判断的语句进行分词处理，在分词模型完成对待判断的语句的分词处理之后，接收分词模型发送待判断的语句对应的多个分词，获得待判断的语句对应的多个分词。
步骤S104、将所述待判断的语句对应的多个分词载入所述ngram模型。
在获得待判断的语句对应的多个分词之后，将待判断的语句对应的多个分词载入ngram模型，若没有发送依赖数值给ngram模型，则ngram模型按照自身默认的依赖数值计算待判断的语句对应的多个分词中各分词的上下文出现概率，若发送依赖数值给ngram模型，则ngram模型按照接收到的依赖数值计算待判断的语句对应的多个分词中各分词的上下文出现概率。
步骤S105、获取所述ngram模型输出的所述待判断的语句对应的多个分词中各分词的上下文出现概率。
在ngram模型完成计算待判断的语句对应的多个分词中各分词的上下文出现概率之后，获取所述ngram模型输出的待判断的语句对应的多个分词中各分词的上下文出现概率。
在本实施例中，在步骤S104中，包括：
将所述待判断的语句对应的多个分词载入所述ngram模型，同时将预设依赖数值发送给所述ngram模型，其中所述预设依赖数值是在计算分词的上下文出现概率时一个分词依赖前面分词的个数。
预设依赖数值是在计算分词的上下文出现概率时一个分词依赖前面分词的个数，例如，预设依赖数值是1，在计算分词的上下文出现概率时一个分词依赖前一个分词。在获得待判断的语句对应的多个分词之后，将待判断的语句对应的多个分词载入ngram模型，同时，将预设依赖数值发送给ngram模型，ngram模型按照接收到的预设依赖数值计算待判断的语句对应的多个分词中各分词的上下文出现概率。
相应地，在步骤S105中，包括：
通过所述ngram模型根据所述预设依赖数值计算所述待判断的语句对应的多个分词中各分词的上下文出现概率；
接收所述ngram模型输出的所述待判断的语句对应的多个分词中各分词的上下文出现概率。
在ngram模型接收到预设依赖数值、待判断的语句对应的多个分词之后，ngram模型根据预设依赖数值计算待判断的语句对应的多个分词中各分词的上下文出现概率，在完成计算之后，接收ngram模型输出的待判断的语句对应的多个分词中各分词的上下文出现概率。
在本实施例中，在所述将所述待判断的语句对应的多个分词载入所述ngram模型，同时将预设依赖数值发送给所述ngram模型的步骤中，包括：
识别所述待判断的语句对应的多个分词中分词的数量；
将所述待判断的语句对应的多个分词中分词的数量与预设数量进行比较；
若所述待判断的语句对应的多个分词中分词的数量大于所述预设数量，则将预设依赖数值设定为第一数值；
若所述待判断的语句对应的多个分词中分词的数量小于或者等于所述预设数量，则将预设依赖数值设定为第二数值，其中所述第二数值小于所述第一数值；
将所述待判断的语句对应的多个分词载入所述ngram模型，同时将所述预设依赖数值发送给所述ngram模型。
在获得待判断的语句对应的多个分词之后，识别待判断的语句对应的多个分词中分词的数量，根据待判断的语句对应的多个分词中分词的数量不同，设定为不同的数值，在识别到待判断的语句对应的多个分词中分词的数量之后，将待判断的语句对应的多个分词中分词的数量与预设数量进行比较，在本实施例中，预设数量为10，若待判断的语句对应的多个分词中分词的数量大于预设数量，则将预设依赖数值设定为第一数值，在本实施例中，第一数值为2，若待判断的语句对应的多个分词中分词的数量小于或者等于预设数量，则将预设依赖数值设定为第二数值，其中第二数值小于第一数值，在本实施例中，第二数值为1。在完成设定预设依赖数值之后，将待判断的语句对应的多个分词载入ngram模型，同时将预设依赖数值发送给所述ngram模型。
在一些实施例中，在所述将所述待判断的语句对应的多个分词载入所述ngram模型，同时将预设依赖数值发送给所述ngram模型的步骤中，包括：
提示用户输入预设依赖数值；
当接收到用户输入所述预设依赖数值时，将所述待判断的语句对应的多个分词载入所述ngram模型，同时将所述预设依赖数值发送给所述ngram模型。
在获得待判断的语句对应的多个分词之后，提示用户输入预设依赖数值，在用户完成输入之后，接收用户输入的预设依赖数值，当接收到用户输入预设依赖数值时，将待判断的语句对应的多个分词载入ngram模型，同时将预设依赖数值发送给ngram模型。
步骤S106、将所述待判断的语句对应的多个分词中各分词的上下文出现概率进行乘积计算，输出所述待判断的语句的困惑度。
在获取待判断的语句对应的多个分词中各分词的上下文出现概率之后，将将待判断的语句对应的多个分词中各分词的上下文出现概率进行乘积计算，输出待判断的语句的困惑度，待判断的语句的困惑度与待判断的语句对应的多个分词中各分词的上下文出现概率进行乘积计算后的概率成反比，语句的困惑度越小，语句的概率越大，语句越合理。
步骤S107、将所述待判断的语句的困惑度与预设困惑度阈值进行比较，根据比较结果，判断所述待判断的语句的合理性。
在获得待判断的语句的困惑度之后，将待判断的语句的困惑度与预设困惑度阈值进行比较，在本实施例中，预设困惑度阈值为用户输入一个困惑度阈值，根据比较结果，判断待判断的语句的合理性。
具体地，在步骤S107中，包括：
将所述待判断的语句的困惑度与预设困惑度阈值进行比较；
若所述待判断的语句的困惑度小于所述预设困惑度阈值，则判断所述待判断的语句为合理的语句；
若所述待判断的语句的困惑度大于或者等于所述预设困惑度阈值，则判断所述待判断的语句为不合理的语句；
当所述待判断的语句为不合理的语句时，对用户进行警告提醒。
在获得待判断的语句的困惑度之后，将待判断的语句的困惑度与预设困惑度阈值进行比较，若待判断的语句的困惑度小于预设困惑度阈值，则判断待判断的语句为合理的语句，若待判断的语句的困惑度大于或者等于预设困惑度阈值，则判断待判断的语句为不合理的语句，不合理的语句需要进行改正，当待判断的语句为不合理的语句时，对用户进行警告提醒，以便用户进行改正。
在一些实施例中，预设困惑度阈值并非是用户输入一个困惑度阈值，而是通过计算所得，具体地，在步骤S107中，包括：
接收用户输入多条合理的语句；
对所述多条合理的语句中每一条合理的语句进行分词处理，获得所述多条合理的语句中每一条合理的语句对应的多个分词；
将所述多条合理的语句中每一条合理的语句对应的多个分词分别载入所述ngram模型；
获取所述ngram模型输出的所述多条合理的语句中每一条合理的语句对应的多个分词中各分词的上下文出现概率；
将所述多条合理的语句中每一条合理的语句对应的多个分词中各分词的上下文出现概率分别进行乘积计算，输出所述多条合理的语句中每一条合理的语句的困惑度；
根据所述多条合理的语句中每一条合理的语句的困惑度，计算所述多条合理的语句的平均困惑度，获得标准困惑度；
将所述标准困惑度的60%作为预设困惑度阈值；
将所述待判断的语句的困惑度与所述预设困惑度阈值进行比较，根据比较结果，判断所述待判断的语句的合理性。
用户筛选出多个合理的语句并输入，在本实施例中，多个合理的语句为200条合理的语句，接收用户输入多条合理的语句，对多条合理的语句中每一条合理的语句进行分词处理，获得多条合理的语句中每一条合理的语句对应的多个分词，也就是说，每一条合理的语句都需要进行分词处理，将多条合理的语句中每一条合理的语句对应的多个分词分别载入ngram模型，也就是说，每一条合理的语句对应的多个分词都需要载入ngram模型，ngram模型计算每一条合理的语句对应的多个分词中各分词的上下文出现概率，然后ngram模型输出每一条合理的语句对应的多个分词中各分词的上下文出现概率，获取ngram模型输出的多条合理的语句中每一条合理的语句对应的多个分词中各分词的上下文出现概率，将多条合理的语句中每一条合理的语句对应的多个分词中各分词的上下文出现概率分别进行乘积计算，输出多条合理的语句中每一条合理的语句的困惑度，根据多条合理的语句中每一条合理的语句的困惑度，计算多条合理的语句的平均困惑度，获得标准困惑度，在得到标准困惑度之后，为了进一步提高语句的合理性，将标准困惑度的60%作为预设困惑度阈值，从而得到预设困惑度阈值，将待判断的语句的困惑度与预设困惑度阈值进行比较，根据比较结果，判断待判断的语句的合理性。
综上所述，将用户输入的待判断的语句进行分词处理，得到多个分词，将多个分词载入ngram模型，ngram模型对多个分词进行计算，计算多个分词中各分词的上下文出现概率，根据多个分词中各分词的上下文出现概率的乘积，输出待判断的语句的困惑度，根据待判断的语句的困惑度，判断待判断的语句的合理性，从而用户可以判断自己所说的语句是否合理，可快速改进自己的话术，旨在解决在现有的话术培训中，在没有专业老师在场的情况下，学员不能判断自己所说语句的合理性的问题。
如图2所示，本申请实施例提出一种基于语义解析的文本准确率计算装置1，装置1包括建立模块11、第一获取模块12、分词模块13、载入模块14、第二获取模块15、计算模块16和判断模块17。
建立模块11，用于建立ngram模型，其中所述ngram模型用于计算各分词的上下文出现概率。
在一些实施例中，装置1包括：
第一爬取模块，用于爬取领域内的语料；
第一过滤模块，用于对语料进行过滤，获得过滤后的语料；
第一中文分词模块，用于对过滤后的语料进行中文分词，获得分词后的语料；
第一纠正模块，用于对分词后的语料进行领域实体纠正，获得纠正后的语料；
第一训练模块，用于通过kenlm训练工具对纠正后的语料进行训练；
第一判断模块，用于根据训练结果判断是否达到预设目标；
第二判断模块，用于若达到预设目标，则判断ngram模型为合格模型。
第一获取模块12，用于获取用户输入的待判断的语句。
分词模块13，用于对所述待判断的语句进行分词处理，获得所述待判断的语句对应的多个分词。
载入模块14，用于将所述待判断的语句对应的多个分词载入所述ngram模型。
第二获取模块15，用于获取所述ngram模型输出的所述待判断的语句对应的多个分词中各分词的上下文出现概率。
在本实施例中，载入模块14包括：
第一子载入模块，用于将所述待判断的语句对应的多个分词载入所述ngram模型，同时将预设依赖数值发送给所述ngram模型，其中所述预设依赖数值是在计算分词的上下文出现概率时一个分词依赖前面分词的个数。
相应地，第二获取模块15包括：
第一子计算模块，用于通过所述ngram模型根据所述预设依赖数值计算所述待判断的语句对应的多个分词中各分词的上下文出现概率；
第一子接收模块，用于接收所述ngram模型输出的所述待判断的语句对应的多个分词中各分词的上下文出现概率。
在本实施例中，第一子载入模块包括：
第一子识别模块，用于识别所述待判断的语句对应的多个分词中分词的数量；
第一子判断模块，用于将所述待判断的语句对应的多个分词中分词的数量与预设数量进行比较；
第一子设定模块，用于若所述待判断的语句对应的多个分词中分词的数量大于所述预设数量，则将预设依赖数值设定为第一数值；
第二子设定模块，用于若所述待判断的语句对应的多个分词中分词的数量小于或者等于所述预设数量，则将预设依赖数值设定为第二数值，其中所述第二数值小于所述第一数值；
第二子载入模块，用于将所述待判断的语句对应的多个分词载入所述ngram模型，同时将所述预设依赖数值发送给所述ngram模型。
在一些实施例中，第一子载入模块包括：
第一子提示模块，用于提示用户输入预设依赖数值；
第三子载入模块，用于当接收到用户输入所述预设依赖数值时，将所述待判断的语句对应的多个分词载入所述ngram模型，同时将所述预设依赖数值发送给所述ngram模型。
计算模块16，用于将所述待判断的语句对应的多个分词中各分词的上下文出现概率进行乘积计算，输出所述待判断的语句的困惑度。
判断模块17，用于将所述待判断的语句的困惑度与预设困惑度阈值进行比较，根据比较结果，判断所述待判断的语句的合理性。
具体地，判断模块17包括：
第一子比较模块，用于将所述待判断的语句的困惑度与预设困惑度阈值进行比较；
第二子判断模块，用于若所述待判断的语句的困惑度小于所述预设困惑度阈值，则判断所述待判断的语句为合理的语句；
第三子判断模块，用于若所述待判断的语句的困惑度大于或者等于所述预设困惑度阈值，则判断所述待判断的语句为不合理的语句；
第一子提醒模块，用于当所述待判断的语句为不合理的语句时，对用户进行警告提醒。
在一些实施例中，预设困惑度阈值并非是用户输入一个困惑度阈值，而是通过计算所得，具体地，判断模块17包括：
第二子接收模块，用于接收用户输入多条合理的语句；
第二子分词模块，用于对所述多条合理的语句中每一条合理的语句进行分词处理，获得所述多条合理的语句中每一条合理的语句对应的多个分词；
第三子载入模块，用于将所述多条合理的语句中每一条合理的语句对应的多个分词分别载入所述ngram模型；
第二子获取模块，用于获取所述ngram模型输出的所述多条合理的语句中每一条合理的语句对应的多个分词中各分词的上下文出现概率；
第二子计算模块，用于将所述多条合理的语句中每一条合理的语句对应的多个分词中各分词的上下文出现概率分别进行乘积计算，输出所述多条合理的语句中每一条合理的语句的困惑度；
第三子计算模块，用于根据所述多条合理的语句中每一条合理的语句的困惑度，计算所述多条合理的语句的平均困惑度，获得标准困惑度；
第一子阈值设定模块，用于将所述标准困惑度的60%作为预设困惑度阈值；
第四子判断模块，用于将所述待判断的语句的困惑度与所述预设困惑度阈值进行比较，根据比较结果，判断所述待判断的语句的合理性。
如图3所示，本申请实施例中还提供一种计算机设备，该计算机设备可以是服务器，其内部结构可以如图3所示。该计算机设备包括通过系统总线连接的处理器、存储器、网络接口和数据库。其中，该计算机设计的处理器用于提供计算和控制能力。该计算机设备的存储器包括非易失性存储介质、内存储器。该非易失性存储介质存储有操作系统、计算机程序和数据库。该内存器为非易失性存储介质中的操作系统和计算机程序的运行提供环境。该计算机设备的数据库用于存储基于语义解析的语句合理性判断方法的模型等数据。该计算机设备的网络接口用于与外部的终端通过网络连接通信。该计算机程序被处理器执行时以实现一种基于语义解析的语句合理性判断方法。
本申请一实施例还提供一种计算机非易失性可读存储介质，其上存储有计算机程序，计算机程序被处理器执行时实现一种基于语义解析的语句合理性判断方法。

Claims (20)

  1. 一种基于语义解析的语句合理性判断方法，其特征在于，所述方法包括：
    建立ngram模型，其中所述ngram模型用于计算各分词的上下文出现概率；
    获取用户输入的待判断的语句；
    对所述待判断的语句进行分词处理，获得所述待判断的语句对应的多个分词；
    将所述待判断的语句对应的多个分词载入所述ngram模型；
    获取所述ngram模型输出的所述待判断的语句对应的多个分词中各分词的上下文出现概率；
    将所述待判断的语句对应的多个分词中各分词的上下文出现概率进行乘积计算，输出所述待判断的语句的困惑度；
    将所述待判断的语句的困惑度与预设困惑度阈值进行比较，根据比较结果，判断所述待判断的语句的合理性。
  2. 根据权利要求1所述的基于语义解析的语句合理性判断方法，其特征在于，在所述将所述待判断的语句的困惑度与预设困惑度阈值进行比较，根据比较结果，判断所述待判断的语句的合理性的步骤中，包括：
    接收用户输入多条合理的语句；
    对所述多条合理的语句中每一条合理的语句进行分词处理，获得所述多条合理的语句中每一条合理的语句对应的多个分词；
    将所述多条合理的语句中每一条合理的语句对应的多个分词分别载入所述ngram模型；
    获取所述ngram模型输出的所述多条合理的语句中每一条合理的语句对应的多个分词中各分词的上下文出现概率；
    将所述多条合理的语句中每一条合理的语句对应的多个分词中各分词的上下文出现概率分别进行乘积计算，输出所述多条合理的语句中每一条合理的语句的困惑度；
    根据所述多条合理的语句中每一条合理的语句的困惑度，计算所述多条合理的语句的平均困惑度，获得标准困惑度；
    将所述标准困惑度的60%作为预设困惑度阈值；
    将所述待判断的语句的困惑度与所述预设困惑度阈值进行比较，根据比较结果，判断所述待判断的语句的合理性。
  3. 根据权利要求1所述的基于语义解析的语句合理性判断方法，其特征在于，在所述将所述待判断的语句对应的多个分词载入所述ngram模型的步骤中，包括：
    将所述待判断的语句对应的多个分词载入所述ngram模型，同时将预设依赖数值发送给所述ngram模型，其中所述预设依赖数值是在计算分词的上下文出现概率时一个分词依赖前面分词的个数。
  4. 根据权利要求3所述的基于语义解析的语句合理性判断方法，其特征在于，在所述获取所述ngram模型输出的所述待判断的语句对应的多个分词中各分词的上下文出现概率的步骤中，包括：
    通过所述ngram模型根据所述预设依赖数值计算所述待判断的语句对应的多个分词中各分词的上下文出现概率；
    接收所述ngram模型输出的所述待判断的语句对应的多个分词中各分词的上下文出现概率。
  5. 根据权利要求3所述的基于语义解析的语句合理性判断方法，其特征在于，在所述将所述待判断的语句对应的多个分词载入所述ngram模型，同时将预设依赖数值发送给所述ngram模型的步骤中，包括：
    识别所述待判断的语句对应的多个分词中分词的数量；
    将所述待判断的语句对应的多个分词中分词的数量与预设数量进行比较；
    若所述待判断的语句对应的多个分词中分词的数量大于所述预设数量，则将预设依赖数值设定为第一数值；
    若所述待判断的语句对应的多个分词中分词的数量小于或者等于所述预设数量，则将预设依赖数值设定为第二数值，其中所述第二数值小于所述第一数值；
    将所述待判断的语句对应的多个分词载入所述ngram模型，同时将所述预设依赖数值发送给所述ngram模型。
  6. 根据权利要求3所述的基于语义解析的语句合理性判断方法，其特征在于，在所述将所述待判断的语句对应的多个分词载入所述ngram模型，同时将预设依赖数值发送给所述ngram模型的步骤中，包括：
    提示用户输入预设依赖数值；
    当接收到用户输入所述预设依赖数值时，将所述待判断的语句对应的多个分词载入所述ngram模型，同时将所述预设依赖数值发送给所述ngram模型。
  7. 根据权利要求1所述的基于语义解析的语句合理性判断方法，其特征在于，在所述将所述待判断的语句的困惑度与预设困惑度阈值进行比较，根据比较结果，判断所述待判断的语句的合理性的步骤中，包括：
    将所述待判断的语句的困惑度与预设困惑度阈值进行比较；
    若所述待判断的语句的困惑度小于所述预设困惑度阈值，则判断所述待判断的语句为合理的语句；
    若所述待判断的语句的困惑度大于或者等于所述预设困惑度阈值，则判断所述待判断的语句为不合理的语句；
    当所述待判断的语句为不合理的语句时，对用户进行警告提醒。
  8. 一种基于语义解析的语句合理性判断装置，其特征在于，所述装置包括：
    建立模块，用于建立ngram模型，其中所述ngram模型用于计算各分词的上下文出现概率；
    第一获取模块，用于获取用户输入的待判断的语句；
    分词模块，用于对所述待判断的语句进行分词处理，获得所述待判断的语句对应的多个分词；
    载入模块，用于将所述待判断的语句对应的多个分词载入所述ngram模型；
    第二获取模块，用于获取所述ngram模型输出的所述待判断的语句对应的多个分词中各分词的上下文出现概率；
    计算模块，用于将所述待判断的语句对应的多个分词中各分词的上下文出现概率进行乘积计算，输出所述待判断的语句的困惑度；
    判断模块，用于将所述待判断的语句的困惑度与预设困惑度阈值进行比较，根据比较结果，判断所述待判断的语句的合理性。
  9. 根据权利要求8所述的基于语义解析的语句合理性判断装置，其特征在于，所述判断模块包括：
    第二子接收模块，用于接收用户输入多条合理的语句；
    第二子分词模块，用于对所述多条合理的语句中每一条合理的语句进行分词处理，获得所述多条合理的语句中每一条合理的语句对应的多个分词；
    第三子载入模块，用于将所述多条合理的语句中每一条合理的语句对应的多个分词分别载入所述ngram模型；
    第二子获取模块，用于获取所述ngram模型输出的所述多条合理的语句中每一条合理的语句对应的多个分词中各分词的上下文出现概率；
    第二子计算模块，用于将所述多条合理的语句中每一条合理的语句对应的多个分词中各分词的上下文出现概率分别进行乘积计算，输出所述多条合理的语句中每一条合理的语句的困惑度；
    第三子计算模块，用于根据所述多条合理的语句中每一条合理的语句的困惑度，计算所述多条合理的语句的平均困惑度，获得标准困惑度；
    第一子阈值设定模块，用于将所述标准困惑度的60%作为预设困惑度阈值；
    第四子判断模块，用于将所述待判断的语句的困惑度与所述预设困惑度阈值进行比较，根据比较结果，判断所述待判断的语句的合理性。
  10. 根据权利要求8所述的基于语义解析的语句合理性判断装置，其特征在于，所述载入模块包括：
    第一子载入模块，用于将所述待判断的语句对应的多个分词载入所述ngram模型，同时将预设依赖数值发送给所述ngram模型，其中所述预设依赖数值是在计算分词的上下文出现概率时一个分词依赖前面分词的个数。
  11. 根据权利要求10所述的基于语义解析的语句合理性判断装置，其特征在于，所述第二获取模块包括：
    第一子计算模块，用于通过所述ngram模型根据所述预设依赖数值计算所述待判断的语句对应的多个分词中各分词的上下文出现概率；
    第一子接收模块，用于接收所述ngram模型输出的所述待判断的语句对应的多个分词中各分词的上下文出现概率。
  12. 根据权利要求10所述的基于语义解析的语句合理性判断装置，其特征在于，所述第一子载入模块包括：
    第一子识别模块，用于识别所述待判断的语句对应的多个分词中分词的数量；
    第一子判断模块，用于将所述待判断的语句对应的多个分词中分词的数量与预设数量进行比较；
    第一子设定模块，用于若所述待判断的语句对应的多个分词中分词的数量大于所述预设数量，则将预设依赖数值设定为第一数值；
    第二子设定模块，用于若所述待判断的语句对应的多个分词中分词的数量小于或者等于所述预设数量，则将预设依赖数值设定为第二数值，其中所述第二数值小于所述第一数值；
    第二子载入模块，用于将所述待判断的语句对应的多个分词载入所述ngram模型，同时将所述预设依赖数值发送给所述ngram模型。
  13. 根据权利要求10所述的基于语义解析的语句合理性判断装置，其特征在于，所述第一子载入模块包括：
    第一子提示模块，用于提示用户输入预设依赖数值；
    第三子载入模块，用于当接收到用户输入所述预设依赖数值时，将所述待判断的语句对应的多个分词载入所述ngram模型，同时将所述预设依赖数值发送给所述ngram模型。
  14. 根据权利要求8所述的基于语义解析的语句合理性判断装置，其特征在于，所述判断模块包括：
    第一子比较模块，用于将所述待判断的语句的困惑度与预设困惑度阈值进行比较；
    第二子判断模块，用于若所述待判断的语句的困惑度小于所述预设困惑度阈值，则判断所述待判断的语句为合理的语句；
    第三子判断模块，用于若所述待判断的语句的困惑度大于或者等于所述预设困惑度阈值，则判断所述待判断的语句为不合理的语句；
    第一子提醒模块，用于当所述待判断的语句为不合理的语句时，对用户进行警告提醒。
  15. 一种计算机设备，包括存储器和处理器，所述存储器存储有计算机程序，其特征在于，所述处理器执行所述计算机程序时实现基于语义解析的语句合理性判断方法，所述方法包括：
    建立ngram模型，其中所述ngram模型用于计算各分词的上下文出现概率；
    获取用户输入的待判断的语句；
    对所述待判断的语句进行分词处理，获得所述待判断的语句对应的多个分词；
    将所述待判断的语句对应的多个分词载入所述ngram模型；
    获取所述ngram模型输出的所述待判断的语句对应的多个分词中各分词的上下文出现概率；
    将所述待判断的语句对应的多个分词中各分词的上下文出现概率进行乘积计算，输出所述待判断的语句的困惑度；
    将所述待判断的语句的困惑度与预设困惑度阈值进行比较，根据比较结果，判断所述待判断的语句的合理性。
  16. 根据权利要求15所述的计算机设备，其特征在于，在所述将所述待判断的语句的困惑度与预设困惑度阈值进行比较，根据比较结果，判断所述待判断的语句的合理性的步骤中，包括：
    接收用户输入多条合理的语句；
    对所述多条合理的语句中每一条合理的语句进行分词处理，获得所述多条合理的语句中每一条合理的语句对应的多个分词；
    将所述多条合理的语句中每一条合理的语句对应的多个分词分别载入所述ngram模型；
    获取所述ngram模型输出的所述多条合理的语句中每一条合理的语句对应的多个分词中各分词的上下文出现概率；
    将所述多条合理的语句中每一条合理的语句对应的多个分词中各分词的上下文出现概率分别进行乘积计算，输出所述多条合理的语句中每一条合理的语句的困惑度；
    根据所述多条合理的语句中每一条合理的语句的困惑度，计算所述多条合理的语句的平均困惑度，获得标准困惑度；
    将所述标准困惑度的60%作为预设困惑度阈值；
    将所述待判断的语句的困惑度与所述预设困惑度阈值进行比较，根据比较结果，判断所述待判断的语句的合理性。
  17. 根据权利要求15所述的计算机设备，其特征在于，在所述将所述待判断的语句对应的多个分词载入所述ngram模型的步骤中，包括：
    将所述待判断的语句对应的多个分词载入所述ngram模型，同时将预设依赖数值发送给所述ngram模型，其中所述预设依赖数值是在计算分词的上下文出现概率时一个分词依赖前面分词的个数。
  18. 根据权利要求17所述的计算机设备，其特征在于，在所述获取所述ngram模型输出的所述待判断的语句对应的多个分词中各分词的上下文出现概率的步骤中，包括：
    通过所述ngram模型根据所述预设依赖数值计算所述待判断的语句对应的多个分词中各分词的上下文出现概率；
    接收所述ngram模型输出的所述待判断的语句对应的多个分词中各分词的上下文出现概率。
  19. 一种计算机非易失性可读存储介质，其上存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现基于语义解析的语句合理性判断方法，所述方法包括：
    建立ngram模型，其中所述ngram模型用于计算各分词的上下文出现概率；
    获取用户输入的待判断的语句；
    对所述待判断的语句进行分词处理，获得所述待判断的语句对应的多个分词；
    将所述待判断的语句对应的多个分词载入所述ngram模型；
    获取所述ngram模型输出的所述待判断的语句对应的多个分词中各分词的上下文出现概率；
    将所述待判断的语句对应的多个分词中各分词的上下文出现概率进行乘积计算，输出所述待判断的语句的困惑度；
    将所述待判断的语句的困惑度与预设困惑度阈值进行比较，根据比较结果，判断所述待判断的语句的合理性。
  20. 根据权利要求19所述的计算机非易失性可读存储介质，其特征在于，在所述将所述待判断的语句的困惑度与预设困惑度阈值进行比较，根据比较结果，判断所述待判断的语句的合理性的步骤中，包括：
    接收用户输入多条合理的语句；
    对所述多条合理的语句中每一条合理的语句进行分词处理，获得所述多条合理的语句中每一条合理的语句对应的多个分词；
    将所述多条合理的语句中每一条合理的语句对应的多个分词分别载入所述ngram模型；
    获取所述ngram模型输出的所述多条合理的语句中每一条合理的语句对应的多个分词中各分词的上下文出现概率；
    将所述多条合理的语句中每一条合理的语句对应的多个分词中各分词的上下文出现概率分别进行乘积计算，输出所述多条合理的语句中每一条合理的语句的困惑度；
    根据所述多条合理的语句中每一条合理的语句的困惑度，计算所述多条合理的语句的平均困惑度，获得标准困惑度；
    将所述标准困惑度的60%作为预设困惑度阈值；
    将所述待判断的语句的困惑度与所述预设困惑度阈值进行比较，根据比较结果，判断所述待判断的语句的合理性。
PCT/CN2019/089107 2018-12-06 2019-05-29 基于语义解析的语句合理性判断方法、装置、计算机设备 Ceased WO2020113918A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811489818.9A CN109992769A (zh) 2018-12-06 2018-12-06 基于语义解析的语句合理性判断方法、装置、计算机设备
CN201811489818.9 2018-12-06

Publications (1)

Publication Number Publication Date
WO2020113918A1 true WO2020113918A1 (zh) 2020-06-11

Family

ID=67128690

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/089107 Ceased WO2020113918A1 (zh) 2018-12-06 2019-05-29 基于语义解析的语句合理性判断方法、装置、计算机设备

Country Status (2)

Country Link
CN (1) CN109992769A (zh)
WO (1) WO2020113918A1 (zh)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111881293A (zh) * 2020-07-24 2020-11-03 腾讯音乐娱乐科技（深圳）有限公司 一种风险内容识别方法及装置、服务器、存储介质
CN113362824A (zh) * 2021-06-09 2021-09-07 深圳市同行者科技有限公司 一种语音识别方法、装置及终端设备
CN114282523A (zh) * 2021-11-22 2022-04-05 北京方寸无忧科技发展有限公司 一种基于bert模型与ngram模型的语句修正方法及装置
CN115587589A (zh) * 2022-09-16 2023-01-10 粤港澳大湾区数字经济研究院(福田) 针对多语种的语句困惑度获取方法、系统及相关设备
CN120353894A (zh) * 2025-04-09 2025-07-22 北京威和光数字科技有限公司 基于语义分析的智能客服交互应答方法及系统

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110852087B (zh) * 2019-09-23 2022-02-22 腾讯科技（深圳）有限公司 中文纠错方法和装置、存储介质及电子装置
CN111079410B (zh) * 2019-12-23 2023-12-22 五八有限公司 文本识别方法、装置、电子设备及存储介质
CN111144100B (zh) * 2019-12-24 2023-08-18 五八有限公司 一种问题文本识别方法、装置、电子设备及存储介质
CN111984845B (zh) * 2020-08-17 2023-10-31 江苏百达智慧网络科技有限公司 网站错别字识别方法和系统
CN112863499B (zh) * 2021-01-13 2023-01-24 北京小米松果电子有限公司 语音识别方法及装置、存储介质
CN113988047B (zh) * 2021-09-26 2024-12-24 北京捷通华声科技股份有限公司 一种语料筛选方法和装置
CN114298030B (zh) * 2021-12-14 2025-06-13 达闼机器人股份有限公司 语句提取方法、装置、电子设备和计算机可读存储介质
CN115062113A (zh) * 2022-06-27 2022-09-16 中国平安财产保险股份有限公司 快速销案方法、装置、设备及存储介质
CN116384388B (zh) * 2023-02-14 2024-02-02 上海熙瑾信息技术有限公司 反向识别ai智能写作的方法、装置、设备和介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120150532A1 (en) * 2010-12-08 2012-06-14 At&T Intellectual Property I, L.P. System and method for feature-rich continuous space language models
CN102681981A (zh) * 2011-03-11 2012-09-19 富士通株式会社 自然语言词法分析方法、装置及分析器训练方法
CN106294325A (zh) * 2016-08-11 2017-01-04 海信集团有限公司 自然语言生成语句的优化方法及装置
CN106844632A (zh) * 2017-01-20 2017-06-13 清华大学 基于改进支持向量机的产品评论情感分类方法及装置
CN107204184A (zh) * 2017-05-10 2017-09-26 平安科技（深圳）有限公司 语音识别方法及系统

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107122346B (zh) * 2016-12-28 2018-02-27 平安科技（深圳）有限公司 一种输入语句的纠错方法及装置
CN108255857B (zh) * 2016-12-29 2021-10-15 北京国双科技有限公司 一种语句检测方法及装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120150532A1 (en) * 2010-12-08 2012-06-14 At&T Intellectual Property I, L.P. System and method for feature-rich continuous space language models
CN102681981A (zh) * 2011-03-11 2012-09-19 富士通株式会社 自然语言词法分析方法、装置及分析器训练方法
CN106294325A (zh) * 2016-08-11 2017-01-04 海信集团有限公司 自然语言生成语句的优化方法及装置
CN106844632A (zh) * 2017-01-20 2017-06-13 清华大学 基于改进支持向量机的产品评论情感分类方法及装置
CN107204184A (zh) * 2017-05-10 2017-09-26 平安科技（深圳）有限公司 语音识别方法及系统

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111881293A (zh) * 2020-07-24 2020-11-03 腾讯音乐娱乐科技（深圳）有限公司 一种风险内容识别方法及装置、服务器、存储介质
CN111881293B (zh) * 2020-07-24 2023-11-07 腾讯音乐娱乐科技（深圳）有限公司 一种风险内容识别方法及装置、服务器、存储介质
CN113362824A (zh) * 2021-06-09 2021-09-07 深圳市同行者科技有限公司 一种语音识别方法、装置及终端设备
CN113362824B (zh) * 2021-06-09 2024-03-12 深圳市同行者科技有限公司 一种语音识别方法、装置及终端设备
CN114282523A (zh) * 2021-11-22 2022-04-05 北京方寸无忧科技发展有限公司 一种基于bert模型与ngram模型的语句修正方法及装置
CN115587589A (zh) * 2022-09-16 2023-01-10 粤港澳大湾区数字经济研究院(福田) 针对多语种的语句困惑度获取方法、系统及相关设备
CN115587589B (zh) * 2022-09-16 2023-07-18 粤港澳大湾区数字经济研究院(福田) 针对多语种的语句困惑度获取方法、系统及相关设备
CN120353894A (zh) * 2025-04-09 2025-07-22 北京威和光数字科技有限公司 基于语义分析的智能客服交互应答方法及系统

Also Published As

Publication number Publication date
CN109992769A (zh) 2019-07-09

Similar Documents

Publication Publication Date Title
WO2020113918A1 (zh) 基于语义解析的语句合理性判断方法、装置、计算机设备
CN102956126B (zh) 学习支持系统以及学习支持方法
US20150248898A1 (en) Computer-Implemented Systems and Methods for Determining an Intelligibility Score for Speech
US9443193B2 (en) Systems and methods for generating automated evaluation models
CN110135879B (zh) 基于自然语言处理的客服质量自动评分方法
CN105045778A (zh) 一种汉语同音词错误自动校对方法
JP6875819B2 (ja) 音響モデル入力データの正規化装置及び方法と、音声認識装置
CN105488142A (zh) 成绩信息录入方法及系统
CN103688254B (zh) 用于自动评估写作的基于示例的错误检测系统、方法及错误检测设备
CN114896382A (zh) 人工智能问答模型生成方法、问答方法、装置及存储介质
KR20210059995A (ko) 학습 기반의 외국어 말하기 평가 방법 및 그 시스템
CN115359799A (zh) 语音识别方法、训练方法、装置、电子设备及存储介质
CN111128181A (zh) 背诵题评测方法、装置以及设备
CN109346108B (zh) 一种作业检查方法及系统
US11049409B1 (en) Systems and methods for treatment of aberrant responses
CN119597921A (zh) 基于大语言模型的应急救援辅助方法以及计算机设备
CN106339366A (zh) 基于人工智能的需求识别的方法和装置
US11341961B2 (en) Multi-lingual speech recognition and theme-semanteme analysis method and device
CN111753062B (zh) 一种会话应答方案确定方法、装置、设备及介质
US20250298980A1 (en) Systems and methods for improved handling of out-of-vocabulary words in speech recognition systems
CN120373293A (zh) 一种基于大模型的文本纠错方法、装置、设备及介质
CN112309429A (zh) 一种失爆检测方法、装置、设备及计算机可读存储介质
CN117786105A (zh) 答题评分模型训练、答题评分方法、装置、设备和介质
JP2010092257A (ja) 敬語表現修正装置及びそれを用いた受付回答支援システム
WO2020022328A1 (ja) 文生成装置、モデル学習装置、文生成方法、モデル学習方法、及びプログラム

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 17.09.2021)

122 Ep: pct application non-entry in european phase

Ref document number: 19893129

Country of ref document: EP

Kind code of ref document: A1