CN115221294A

CN115221294A - Dialogue processing method, dialogue processing device, electronic equipment and storage medium

Info

Publication number: CN115221294A
Application number: CN202111671545.1A
Authority: CN
Inventors: 陆笛; 周霄
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2021-04-15
Filing date: 2021-12-31
Publication date: 2022-10-21

Abstract

The application provides a conversation processing method, a conversation processing device, electronic equipment and a computer readable storage medium; a natural language processing technology relating to artificial intelligence, the method comprises: acquiring a label corresponding to an input statement; acquiring a matched dialect template from a dialect template library based on the label, and acquiring matched structured data from a structured data set; filling the structured data into the talking template to obtain candidate objective talking questions, and acquiring candidate subjective topics matched with the labels from a talking question library; and sequencing the candidate objective topics and the candidate subjective topics, and determining reply sentences for responding to the input sentences according to sequencing results. By the method and the device, the information content and the topicality of the reply sentence can be increased.

Description

Dialogue processing method, dialogue processing device, electronic equipment and storage medium

Description of the priority

The application requires the application number of 202110405731.4, the application date of 2021, 04 and 15 days, and the name is: conversation processing method, device, electronic equipment and priority of storage medium.

Technical Field

The present application relates to natural language processing technologies, and in particular, to a dialog processing method and apparatus, an electronic device, and a computer-readable storage medium.

Background

Artificial Intelligence (AI) is a comprehensive technique in computer science, and by studying the design principles and implementation methods of various intelligent machines, the machines have the functions of perception, reasoning and decision making. The artificial intelligence technology is a comprehensive subject, and relates to a wide range of fields, for example, natural language processing technology and machine learning/deep learning and other directions.

Among them, the dialog system is a very important application in natural language processing technology, and has wide applications in intelligent sound boxes, intelligent voice assistants, vehicle-mounted intelligent voice devices, and the like. However, the dialog system provided by the related art depends on the quality of the search library or the question-and-answer training data, resulting in a smaller amount of information of the generated reply sentence, and at the same time, the ability to initiate the topic of interest to the user is also weak.

Disclosure of Invention

The embodiment of the application provides a conversation processing method and device, electronic equipment and a computer readable storage medium, which can increase the information amount and the topicality of reply sentences.

The technical scheme of the embodiment of the application is realized as follows:

the embodiment of the application provides a conversation processing method, which comprises the following steps:

acquiring a label corresponding to an input statement;

acquiring a matched dialect template from a dialect template library based on the label, and acquiring matched structured data from a structured data set;

filling the structured data into the talking template to obtain candidate objective talking questions, and acquiring candidate subjective topics matched with the labels from a talking question library;

and sequencing the candidate objective topics and the candidate subjective topics, and determining reply sentences for responding to the input sentences according to sequencing results.

In the above scheme, the sorting the candidate objective topics and the candidate subjective topics, and determining reply sentences for responding to the input sentences according to a sorting result includes: searching a reply sentence matched with the input sentence in a pre-constructed question-answer library to serve as a search type reply sentence; generating a reply sentence matched with the input sentence through a pre-trained language model to serve as a generating type reply sentence; taking the candidate objective questions and the candidate subjective questions as candidate topic reply sentences, and fusing the retrieval type reply sentences and the generation type reply sentences with the candidate topic reply sentences respectively to obtain a plurality of candidate reply sentences; and sequencing the candidate reply sentences according to the object characteristics, and selecting at least one candidate reply sentence sequenced at the front in the sequencing result as a reply sentence for responding to the input sentence.

An embodiment of the present application provides a dialog processing apparatus, including:

the acquisition module is used for acquiring a label corresponding to an input statement;

the acquisition module is further used for acquiring a matched dialect template from a dialect template library based on the label and acquiring matched structured data from a structured data set;

the filling module is used for filling the structured data into the conversational template to obtain candidate objective conversational questions;

the acquisition module is further used for acquiring candidate subjective topics matched with the tags from a topic library;

the sorting module is used for sorting the candidate objective topics and the candidate subjective topics;

and the determining module is used for determining a reply sentence for responding to the input sentence according to the sorting result.

In the foregoing solution, the obtaining module is further configured to perform at least one of the following operations: carrying out named entity identification processing on the input statement, and determining the identified named entity as a label corresponding to the input statement; performing word segmentation on the input sentence, acquiring a corresponding candidate entity from a knowledge base based on a reference word obtained after the word segmentation, and determining the candidate entity as a label corresponding to the input sentence; and performing intention recognition processing on the input statement, and determining an obtained intention recognition result as a label corresponding to the input statement.

In the above solution, the apparatus further includes a construction module, configured to construct the linguistic template library and the structured data set by: acquiring structured data and unstructured data, performing template extraction processing on the structured data to obtain a corresponding dialect template, and constructing a dialect template library based on the dialect template; and carrying out named entity identification processing on the unstructured data, converting the unstructured data into corresponding structured data based on the identified named entity, and constructing the structured data set based on the structured data and the structured data obtained by converting the unstructured data.

In the above scheme, the apparatus further includes a cleaning module, configured to perform data cleaning processing on the unstructured data to obtain cleaned unstructured data; the device also comprises an identification module used for carrying out sensitive information identification processing on the washed unstructured data and deleting the identified sensitive information from the washed unstructured data.

In the above solution, the constructing module is further configured to construct the topic library by the following method: acquiring unstructured data of different fields; carrying out data cleaning processing on the unstructured data to obtain cleaned unstructured data; conducting named entity extraction processing on the washed unstructured data to obtain extracted unstructured data; rewriting the extracted unstructured data to obtain corresponding subjective topics; and constructing the topic bank based on the subjective topics.

In the foregoing solution, the cleaning module is further configured to perform at least one of the following operations on the unstructured data to obtain cleaned unstructured data: deleting duplicate information in the unstructured data; completing the missing information in the unstructured data; and carrying out validity verification on the unstructured data, and deleting the information which does not pass the validity verification.

In the foregoing solution, the apparatus further includes a rewriting module, configured to perform at least one of the following operations: performing emotion analysis processing on the extracted unstructured data to obtain emotion categories, and rewriting the extracted unstructured data into subjective topics matched with the emotion categories; receiving a labeling operation aiming at the extracted unstructured data, rewriting the extracted unstructured data into a subjective topic when a labeling result represents that the extracted unstructured data belongs to legal data, and deleting the extracted unstructured data when the labeling result represents that the extracted unstructured data belongs to illegal data.

In the above scheme, the sorting module is further configured to add the candidate objective topic to a general speech queue, and add the candidate subjective topic to a non-general speech queue; determining an interested field according to object features, and sorting at least one candidate objective topic included by the general talking queue and at least one candidate subjective topic included by the non-general talking queue in a descending order according to the similarity between the interested field and the field to which the at least one candidate objective topic included by the general talking queue and the at least one candidate subjective topic included by the non-general talking queue belong; the determining module is further used for selecting at least one candidate subjective topic or candidate objective topic ranked in the top in the descending ranking result as a reply sentence for responding to the input sentence.

In the above scheme, when the corresponding interested area cannot be determined according to the object features, the ranking module is further configured to rank the candidate objective topics corresponding to at least one hot area included in the universal conversation queue in a forward manner; and ordering the candidate subjective topics corresponding to at least one hot field included in the non-universal conversation queue in a front position.

In the above scheme, the building module is further configured to obtain historical dialogue information; extracting the characteristics of the historical dialogue information to obtain historical dialogue characteristics; classifying the historical conversation features to obtain corresponding object labels; constructing the object features based on the object tags.

In the above scheme, the building module is further configured to call a multi-dimensional label classifier to classify the historical conversation features, so as to obtain label prediction probabilities of object account numbers corresponding to the input statements in N dimensions; the multi-dimension label classifier comprises N label classifiers which correspond to the object labels of the N dimensions one by one; sorting the label prediction probabilities on the N dimensions in a descending order, and selecting the first K object labels from the object labels of the N dimensions according to a sorting result; taking the first K object tags as object tags of the object account on the K dimensions; wherein K is less than or equal to N, and K is a positive integer.

In the above scheme, the sorting module is further configured to add the candidate objective topic to a general speech queue, and add the candidate subjective topic to a non-general speech queue; sorting at least one candidate objective topic included in the universal conversational queue and at least one candidate subjective topic included in the non-universal conversational queue in a descending order according to the number of the named entities included in the hit input sentence; the determining module is further used for selecting at least one candidate subjective topic or candidate objective topic ranked in the top from the ranking results as a reply sentence for responding to the input sentence.

In the above scheme, the sorting module is further configured to retrieve a reply sentence matched with the input sentence in a pre-constructed question-answer library as a retrieval type reply sentence; generating a reply sentence matched with the input sentence through a pre-trained language model to serve as a generating type reply sentence; taking the candidate objective questions and the candidate subjective questions as candidate theme reply sentences, and fusing the search type reply sentences and the generated type reply sentences with the candidate theme reply sentences respectively to obtain a plurality of candidate reply sentences; and sequencing the candidate reply sentences according to the object characteristics, and selecting at least one candidate reply sentence sequenced at the front in the sequencing result as a reply sentence for responding to the input sentence.

An embodiment of the present application provides an electronic device, including:

a memory for storing executable instructions;

and the processor is used for realizing the dialogue processing method provided by the embodiment of the application when executing the executable instructions stored in the memory.

The embodiment of the present application provides a computer-readable storage medium, which stores executable instructions for causing a processor to implement a dialog processing method provided by the embodiment of the present application when the processor executes the executable instructions.

The embodiment of the present application provides a computer program product, where the computer program product includes computer-executable instructions, and is used for implementing the dialog processing method provided in the embodiment of the present application when being executed by a processor.

The embodiment of the application has the following beneficial effects:

the matched dialect template and the structured data are obtained through the tags corresponding to the input sentences, the candidate objective topics are generated based on the obtained dialect template and the structured data, meanwhile, the matched candidate subjective topics are obtained from the topic library according to the tags, and the topics of different types are used as reply sentences responding to the input sentences, so that the information content and the topicality of the reply sentences are improved, the user can be guided to continue to talk, and the chat experience of the user is improved.

Drawings

Fig. 1 is a schematic diagram of a framework of a dialog system provided in an embodiment of the present application;

fig. 2 is a schematic architecture diagram of a dialog system 100 provided in an embodiment of the present application;

fig. 3 is a schematic structural diagram of a server 200 according to an embodiment of the present application;

fig. 4 is a schematic flowchart of a dialog processing method provided in an embodiment of the present application;

fig. 5A is a schematic flowchart of a dialog processing method according to an embodiment of the present application;

fig. 5B is a schematic flowchart of a dialog processing method according to an embodiment of the present application;

FIG. 6 is a block diagram of a dialog system according to an embodiment of the present application;

FIG. 7 is a schematic diagram of a dialog processing method according to an embodiment of the present application;

fig. 8 is a comparison diagram of average conversation turns corresponding to different conversation processing methods according to an embodiment of the present application.

Detailed Description

In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be described in further detail with reference to the attached drawings, the described embodiments should not be considered as limiting the present application, and all other embodiments obtained by a person of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

In the following description, reference is made to "some embodiments" which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of the present application only and is not intended to be limiting of the application.

Before further detailed description of the embodiments of the present application, terms and expressions referred to in the embodiments of the present application will be described, and the terms and expressions referred to in the embodiments of the present application will be used for the following explanation.

1) Named entity (named entity): referring to the name of a person, the name of an organization, the name of a place, and all other entities identified by name, the broader entities may also include numbers, dates, currency, addresses, etc., for example, the address "north X" is a named entity.

2) Natural Language Understanding (NLU, natural Language Understanding): make the computer understand natural language (i.e. human language words) and so on, and focus on understanding. More specifically, it is to understand the language, text, etc. and extract useful information for downstream tasks. It may be a structuring of natural language, such as word segmentation, part-of-speech tagging, syntactic analysis, etc.; or the method can be characterized learning, vector representation of characters, words and sentences and text classification of text representation; and the method can also be used for information extraction, such as information retrieval (including personalized search and semantic search, text matching and the like), and information extraction (such as named entity extraction, relation extraction, event extraction and the like).

3) Structuring data: the data stored in a fixed format in a record file, such as table data, depends on establishing a data model, which refers to how the data is stored, processed and logged, and in practical application, the data can be stored in a key-value pair manner, and a specific example of the structured data is as follows: { "director": "korean X", "movie name": "Flat XX way" }.

4) Unstructured data: meaning that the data does not have a predefined data model or is not organized in a predefined manner. Unstructured data is generally character type data, but there is much information in the data such as time, numbers, etc., which is more difficult to understand than structured data due to the non-specificity and ambiguity of unstructured data. Unstructured data generally includes all forms of documents, text, pictures, reports, audio/video data, and the like.

5) Intention recognition: the input sentence (Query) is classified into corresponding intention categories through a classification method. For example, when the Query input by the user is "i want to listen to a song of the week XX", the intention of this Query is a musical intention. The intention recognition method includes a rule method based on a dictionary and a template, a method based on a query click log, and a method of discriminating the intention of the user based on a classification model.

6) Emotion analysis: the method is a common scene in natural language processing, such as commodity evaluation and takeout evaluation in a shopping scene, and has a key role in guiding product update iteration. Through emotion analysis, the advantages and disadvantages of the product in all dimensions can be mined, and therefore how to improve the product is clear. For example, taking an emotion analysis method based on an emotion dictionary as an example, preprocessing such as word segmentation and word stop processing is performed on an input sentence, and then character string matching is performed on words obtained after word segmentation by using a pre-constructed emotion dictionary (including a positive word dictionary, a negative word dictionary, a degree adverb dictionary and the like), so that positive and negative information is mined.

7) Entity Linking (Entity Linking): given a knowledge base rich in a series of entities and a corpus that has been labeled with a term (fragment), the goal of the entity linking task is to match each fragment to its corresponding entity in the knowledge base. For example, for the corpus "zheng XX will be discharged from preschool college of university XX", the mentions "zheng XX", "XX university" etc. therein should be mapped to the corresponding entities of the knowledge base, respectively.

8) Subjective topics: similar to "subjective questions," there may be different replies to the same input question, as long as there is no deviation from the topic, and there is no standard reply. For example, for the "jones of Zhao XX teacher" input by the user, the corresponding subjective topic (i.e. the reply sentence corresponding to the input sentence) may be "the jones XX teacher has very deep host power, is the memory of the first generation of people, and i like him well".

9) Objective topics: similar to the objective question, there is a more or less standard reply for each input question, for example, for the user's input "the reminiscent Zhao XX teacher", the corresponding objective topic (i.e. the reply sentence corresponding to the input sentence) may be "the Zhao XX teacher is born in XX and is a famous television station host in China", i.e. the objective topic is mainly the sentence explaining the objective fact.

10 Object features): refers to a model of a target object built on top of a series of real data, where the object may be a real user or a computer program for simulating human dialogue behavior. Taking an object as an actual user as an example, the object feature may be a virtual representation of the actual user. It can be understood that the more dimensions of the user tag, the more complete the constructed object features, and the more accurate the real user can be represented.

11 Agent): a software or hardware entity capable of autonomous activity, also called an agent or agent, may be software that processes input statements and returns reply statements.

Referring to fig. 1, fig. 1 is a schematic diagram of a framework of a dialog system provided in an embodiment of the present application. As shown in fig. 1, after an arbitrary sentence (Query) input by an object (e.g. a real user) enters a dialog system, the arbitrary sentence is first preprocessed by an upstream preprocessing (permission) module, where the preprocessing includes: the method comprises the steps of performing processing such as Named Entity Recognition (NER), entity link, context history information retrieval, intention Recognition, multi-dimensional label acquisition and object feature acquisition on Query input by a user to obtain structured information for semantic understanding of Query input by the user, transmitting the structured information to downstream agents, generating a plurality of candidate reply sentences by the agents according to the input structured information (for example, the agents can predict reply sentences matched with the structured information through a language model (namely, a generating formula in figure 1), or obtain reply sentences matched with the structured information (namely, a retrieval formula in figure 1) through querying a retrieval library), sequencing the plurality of candidate reply sentences generated by the agents, and outputting the optimal reply sentences (for example, the candidate reply sentences ranked in the first place).

It can be seen that the quality of the reply sentence finally output by the dialog system depends on the quality of the candidate reply sentences output by the Agent and the sorting result. When the quality of a plurality of candidate reply sentences output by the Agent is poor, the final output reply sentence cannot be guaranteed to have higher quality even if the subsequent sequencing effect is good; when the quality of a plurality of candidate reply sentences output by the Agent is high, the finally output reply sentences can be ensured to have good effect even if the subsequent sequencing effect is not good. Therefore, improving the quality of the candidate reply sentences output by the Agent is the key for ensuring the quality of the finally output reply sentences.

However, the retrieval-based dialog has the problems of lack of timeliness, data obsolescence and the like due to data stored in a search library, and the result of candidate reply sentences generated by the retrieval-based dialog is single, so that the dialog experience of a user is poor. Meanwhile, the candidate reply sentences output by the generative dialog are more diverse than the retrievable dialog, but the candidate reply sentences output by the generative dialog have the problems of insufficient information quantity and poor controllability, and particularly can cause serious negative effects when generating negative and sensitive candidate reply sentences. That is, the dialog system provided by the related art depends on the quality of a search library (e.g., a question-Answer (QA) library) or a question-Answer training data, is not optimized in a targeted manner for a topic that is interested by an initiating user, and the reply sentence generated by the dialog system has a small information amount and a weak ability to initiate the topic that is interested by the initiating user, resulting in a poor chat experience for the user.

In view of this, embodiments of the present application provide a conversation processing method, a conversation processing apparatus, an electronic device, and a computer-readable storage medium, which can increase the information amount and the topicality of a reply sentence, so as to guide a conversation to continue, and improve the chat experience of a user.

The conversation processing method provided by the embodiment of the application can be independently realized by the terminal equipment or the server; the terminal device and the server may cooperate with each other, for example, the terminal device solely undertakes a dialog processing method described below, or the terminal device sends a sentence (e.g., a question sentence) input by the user to the server, and the server determines a corresponding reply sentence according to the received input sentence and returns the determined reply sentence to the terminal device.

The electronic device for executing the dialog processing method provided by the embodiment of the application may be various types of terminal devices or servers, wherein the server may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing cloud computing service; the terminal device may be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, a vehicle-mounted smart voice device, and the like, but is not limited thereto. The terminal device and the server may be directly or indirectly connected through wired or wireless communication, and the application is not limited herein.

Taking a server as an example, for example, the server cluster may be deployed in a cloud, and an artificial intelligence cloud Service (AI aas, AI as a Service) is opened to a user, the AIaaS platform splits several types of common AI services, and provides an independent or packaged Service in the cloud, this Service mode is similar to an AI theme mall, and all users may access one or more artificial intelligence services provided by the AIaaS platform by using an application programming interface.

For example, one of the artificial intelligence cloud services may be a conversation service, that is, a program of the conversation processing method provided by the embodiment of the present application is encapsulated in a cloud server. A user calls a dialogue service in a cloud service through a terminal device (a client is operated, such as an intelligent assistant, an online voice interaction system and the like), so that a server deployed at the cloud calls a program of a packaged dialogue processing method, a dialogue template matched with a label corresponding to an input statement is obtained from a dialogue template library, matched structured data is obtained from a structured data set, and then the obtained structured data is filled into the dialogue template to obtain candidate objective questions; meanwhile, candidate subjective topics matched with the labels are obtained from a topic library; and then, sequencing the acquired candidate objective topics and the candidate subjective topics, determining a reply sentence for responding to the input sentence according to a sequencing result, and finally returning the determined reply sentence to the terminal equipment of the user to be displayed in a human-computer interaction interface of the client.

The following describes a session processing method implemented by a terminal device and a server in cooperation.

Referring to fig. 2, fig. 2 is a schematic diagram of an architecture of the dialog system 100 provided in the embodiment of the present application, and a terminal device 400 is connected to a server 200 through a network 300, where the network 300 may be a wide area network or a local area network, or a combination of the two.

The terminal device 400 (running with the client 410, e.g., a chat robot) may be used to obtain an input sentence, e.g., a user may input an arbitrary sentence in the human-computer interaction interface of the client 410, e.g., input "memorial zhao XX teacher", the terminal device 400, after obtaining the input sentence, transmits the input sentence to the server 200 through the network 300, so that the server 200 obtains a tag (e.g., identified named entity, "zhao XX") corresponding to the input sentence, and returns a matched candidate subjective topic (composed of a conversational template and structured data) and a candidate objective topic based on the tag, and then sorts the recalled candidate subjective topic and objective topics according to object characteristics, takes the top ranked topic (e.g., the first ranked topic) as a topic for responding to the input sentence, and returns to the terminal device 400, e.g., the "zhao XX teacher" input by the user, the reply sentence correspondingly returned by the server may be a topic "zhao topic" the mr ", which is a topic hosted by the top ranked topic, is a generation of the teacher", thereby increasing the experience of the subjective information and the reply sentence, and further improving the user experience of the conversation.

In other embodiments, the terminal device may also separately implement the dialog processing method provided in the embodiment of the present application. For example, a dialog plug-in may be embedded in a client running on the terminal device, so as to implement the dialog processing method locally on the terminal device. For example, after acquiring any sentence input by the user, the terminal device 400 calls the dialog plug-in to acquire a tag corresponding to the input sentence, recalls the matched candidate subjective topic and candidate objective topic based on the tag, then sorts the recalled candidate subjective topic and candidate objective topic according to the object characteristics, and takes at least one topic in the top of the sorting as a reply sentence for responding to the input sentence.

In some embodiments, the terminal device 400 may further implement the dialog processing method provided in the embodiments of the present application by running a computer program, for example, the computer program may be a native program or a software module in an operating system; it may be a local (Native) APPlication (APP, APPlication), i.e. a program that needs to be installed in an operating system to run, such as a smart conversation client (smart assistant or chat robot); or may be an applet, i.e. a program that can be run only by downloading it to a browser environment; but also a chat applet that can be embedded into any APP. In general, the computer programs described above may be any form of application, module or plug-in.

In some embodiments, the dialog processing method provided by the embodiment of the present application may also be implemented in combination with a block chain technology.

The block chain is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, a consensus mechanism, an encryption algorithm and the like. A block chain (Blockchain), which is essentially a decentralized database, is a series of data blocks associated by using a cryptographic method, and each data block contains information of a batch of network transactions, so as to verify the validity (anti-counterfeiting) of the information and generate a next block. The blockchain may include a blockchain underlying platform, a platform product services layer, and an application services layer.

An exemplary application of the blockchain network is described below, taking as an example that the server uploads the obtained verbal template, the structured data and the topic to the blockchain network for storage.

First, after obtaining a phonetics template, structured data (including structured data converted from unstructured data) and a topic (a subjective topic obtained by rewriting unstructured data), a server sends the phonetics template, the structured data and the topic to a blockchain network and generates a corresponding transaction, wherein the transaction includes: the method comprises the steps of (1) calling a conversational template, structured data and topics for chaining, intelligent contracts needing to be called and parameters transferred to the intelligent contracts; the transaction may also include a digital certificate of the server, a signed digital signature, and broadcast the transaction to the consensus nodes in the blockchain network.

And then, when the common identification node in the block chain network receives the transaction, verifying the digital certificate and the digital signature carried by the transaction, and after the verification is successful, confirming whether the server has the transaction authority or not according to the identity of the server carried in the transaction, wherein the transaction failure is caused by any verification error in the digital signature and the authority verification. After verification is successful, the consensus node signs its own digital signature (e.g., by encrypting a digest of the transaction using the private key of the node) and continues to broadcast in the blockchain network.

And then, after the common identification node in the block chain network receives the transaction which is successfully verified, filling the transaction into a new block, and broadcasting. When the consensus node in the block chain network broadcasts a new block, the new block is verified, for example, whether a digital signature of a transaction in the new block is valid is verified, if the verification is successful, the new block is appended to the tail of the block chain stored by the consensus node, the state database is updated according to the transaction result, the transaction in the new block is executed, for example, for the submitted transaction for storing a conversation template, structured data and topics, the consensus node respectively creates a conversation template library, a structured data set and a topic library in the state database, adds the conversation template to the conversation template library, adds the structured data to the structured data set and adds the topic to the topic library.

Taking the example that the server sends the request for obtaining the language template, the structured data and the topic to the blockchain network, the exemplary application of the blockchain network is continuously described.

In some embodiments, after acquiring an input sentence sent by a terminal device, a server acquires a tag corresponding to the input sentence, then generates a transaction for querying a conversational template, structured data and topics, and submits the transaction to a block chain network, wherein the query request carries the tag, so that a common identification node in the block chain network executes the transaction to query the conversational template matched with the tag from a conversational template library included in a state database, query the structured data matched with the tag from a structured data set, and query candidate subjective questions matched with the tag from the topic library, then the block chain network sends the queried conversational template, structured data and topics to the server, so that the server fills the structured data into the conversational template to obtain candidate objective topics, ranks the candidate objective topics and subjective topics according to object characteristics, finally determines a reply sentence for responding to the input sentence according to a ranking result, and returns the determined reply sentence to the terminal device. In this way, by storing the obtained conversational template, structured data and topics into the blockchain network, the security of the data is ensured based on the characteristics of decentralized, distributed and non-falsification of the blockchain network.

The following describes a structure of an electronic device provided in an embodiment of the present application. Taking the server 200 in fig. 2 as an example, referring to fig. 3, fig. 3 is a schematic structural diagram of the server 200 provided in the embodiment of the present application, and the server 200 shown in fig. 3 includes: at least one processor 210, memory 240, at least one network interface 220. The various components in server 200 are coupled together by a bus system 230. It is understood that the bus system 230 is used to enable connected communication between these components. The bus system 230 includes a power bus, a control bus, and a status signal bus in addition to a data bus. For clarity of illustration, however, the various buses are labeled as bus system 230 in FIG. 3.

The Processor 210 may be an integrated circuit chip having Signal processing capabilities, such as a general purpose Processor, a Digital Signal Processor (DSP), or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, or the like, wherein the general purpose Processor may be a microprocessor or any conventional Processor, or the like.

The memory 240 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid state memory, hard disk drives, optical disk drives, and the like. Memory 240 optionally includes one or more storage devices physically located remote from processor 210.

The memory 240 comprises volatile memory or nonvolatile memory, and may also comprise both volatile and nonvolatile memory. The nonvolatile Memory may be a Read Only Memory (ROM), and the volatile Memory may be a Random Access Memory (RAM). The memory 240 described in embodiments herein is intended to comprise any suitable type of memory.

In some embodiments, memory 240 is capable of storing data, examples of which include programs, modules, and data structures, or subsets or supersets thereof, to support various operations, as exemplified below.

An operating system 241, including system programs for handling various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and handling hardware-based tasks;

a network communication module 242 for communicating to other computing devices via one or more (wired or wireless) network interfaces 220, exemplary network interfaces 220 including: bluetooth, wireless compatibility authentication (WiFi), and Universal Serial Bus (USB), etc.;

in some embodiments, the dialog processing apparatus provided in this embodiment may be implemented in software, and fig. 3 illustrates the dialog processing apparatus 243 stored in the memory 240, which may be software in the form of programs and plug-ins, and includes the following software modules: the fetch module 2431, the fill module 2432, the sort module 2433, the determine module 2434, the build module 2435, the purge module 2436, the identify module 2437, and the rewrite module 2438, which are logical and therefore arbitrarily combined or further separated depending on the functionality implemented. It is noted that all of the above modules are shown in fig. 2 at once for convenience of presentation, but should not be construed as excluding implementations that may include only the retrieving module 2431, the populating module 2432, the ordering module 2433 and the determining module 2434 at the dialog processing device 243, the functions of each of which will be described below.

In other embodiments, the dialog processing Device provided in the embodiments of the present Application may be implemented in hardware, and for example, the dialog processing Device provided in the embodiments of the present Application may be a processor in the form of a hardware decoding processor, which is programmed to execute the dialog processing method provided in the embodiments of the present Application, for example, the processor in the form of the hardware decoding processor may be implemented by one or more Application Specific Integrated Circuits (ASICs), DSPs, programmable Logic Devices (PLDs), complex Programmable Logic Devices (CPLDs), field Programmable Gate Arrays (FPGAs), or other electronic components.

The dialog processing method provided in the embodiment of the present application will be described below with reference to exemplary applications and implementations of the electronic device provided in the embodiment of the present application, and the dialog processing method provided in the embodiment of the present application may be independently completed by the above terminal device 400, may be independently completed by the above server 200, or may be cooperatively completed by the above terminal device 400 and the server 200. The following description will be given taking as an example that the server 200 performs the dialogue processing method provided in the embodiment of the present application alone.

Referring to fig. 4, fig. 4 is a schematic flowchart of a dialog processing method provided in an embodiment of the present application, which will be described with reference to the steps shown in fig. 4.

In step S101, a tag corresponding to the input sentence is acquired.

In some embodiments, after acquiring a sentence input by a user (for example, the sentence may be input by voice or by text), the terminal device sends the acquired input sentence to the server through the network, so that the server acquires a tag corresponding to the input sentence.

In an example, taking a terminal device as a smart phone as an example, a chat application is installed on the smart phone, and a user can input any words, such as "weather is true" in a chat page provided by the chat application, and then the smart phone sends a sentence input by the user to a server, so that the server obtains a tag corresponding to the input sentence.

In an example, taking the terminal device as an on-vehicle intelligent Speech device as an example, after the on-vehicle intelligent Speech device receives the Speech of the user, the Speech of the user is converted into a computer-readable input through a Speech Recognition technology (ASR), for example, into a corresponding character sequence, and the converted character sequence is sent to the server, so that the server performs subsequent processing.

In other embodiments, the input sentences may also be generated by a computer program for simulating human dialogue behavior, that is, the input sentences are generated by a program to test the dialogue capability of the dialogue processing method provided by the embodiment of the application.

In some embodiments, the tag corresponding to the input sentence may be obtained by: carrying out named entity identification processing on an input statement, and determining an identified named entity as a label corresponding to the input statement; performing word segmentation on an input sentence, acquiring a corresponding candidate entity in a knowledge base based on a reference word obtained after the word segmentation, and determining the candidate entity as a label corresponding to the input sentence; and performing intention recognition processing on the input sentence, and determining an obtained intention recognition result as a label corresponding to the input sentence.

For example, taking an input sentence as "the souvenir zhao XX teacher" as an example, conducting named entity recognition processing on the input sentence to obtain the name "zhao XX" of the person involved in the input sentence, and then taking the recognized name "zhao XX" as a label corresponding to the input sentence "the souvenir zhao XX teacher", so that the candidate subjective topic and the candidate objective topic related to "zhao XX" can be recalled subsequently.

For example, taking an input sentence as "i are reading" haxx te ", first performing word segmentation on the input sentence to obtain" i "," is reading "," haxx baud ", then mapping the reference word" haxx te "obtained after word segmentation onto an entity corresponding to the knowledge base, where the entity corresponding to the reference word" haxx te "shall be an entity of" haxx te (book) "instead of the entity of" hax baud (movie) ", and then taking the entity of" haxx baud (book) "as a tag corresponding to the input sentence" i are reading "haxx", so that candidate subjective and candidate objective titles related to the haxx book can be recalled later.

For example, taking an input sentence as "the quality of the domestic movie in a few years is true," an intention recognition process is performed on the input sentence (for example, an intention recognition process may be performed on the input sentence by a rule method based on a dictionary and a template, that is, a judgment is performed according to the matching degree or the overlapping degree of the intention and the dictionary, and the input sentence is distinguished to the field when the overlapping degree of the intention and the dictionary is high), and if the intention corresponding to the input sentence is a movie intention (that is, the user wants to exchange the movie), the movie intention may be used as a label corresponding to the input sentence "the quality of the domestic movie in a few years is true," and thus, a candidate subjective question and a candidate objective question related to the movie exchange may be recalled later.

In step S102, matching phonetics templates are obtained from the phonetics template library based on the tags, and matching structured data is obtained from the structured data set.

In some embodiments, before retrieving the matching linguistic template from the library of linguistic templates based on the tag, and retrieving the matching structured data from the structured dataset, the library of linguistic templates and the structured dataset may also be constructed by: acquiring structured data and unstructured data, performing template extraction processing on the structured data to obtain a corresponding dialect template, and constructing a dialect template library based on the dialect template; and carrying out named entity identification processing on the unstructured data, converting the unstructured data into corresponding structured data based on the identified named entity, and constructing a structured data set based on the structured data and the structured data obtained by converting the unstructured data.

For example, the structured data may include entity-related information in the fields of music, books, videos, and the like. For example, a web crawler (also called a web spider), which is a program or script that automatically captures internet information according to a certain rule, may be used to capture the structured data corresponding to different fields from the web page. Then, template extraction processing is performed on the captured structured data to obtain a corresponding conversational template, for example, taking the music field as an example, assuming that the structured data captured by the web crawler is: { "singer": "week XX", "song": "black X humor", "type": "popular music", then, the template extraction processing is performed on the structured data, so as to obtain the following dialectical template: { < PersonName > is an artist who compares that the named song is < Date >, this song belongs to < Type > }, and then, the resulting dialogies template is added to the dialogies template library.

By way of example, unstructured data may include office documents, text, pictures, various types of reports, audio/video information, and the like in various formats. For example, a web crawler mode may be used to capture unstructured data corresponding to different fields from a web page. Suppose that a certain unstructured data captured by the web crawler is: "the newest movie in korean X is true and good in the way of flat XX with a score as high as 9.0", the named entity recognition processing is first performed on the captured unstructured data, and the following named entities can be obtained: "hangeul X", "flat XX lane", "9.0", then, unstructured data "the latest movie flat XX lane for director hangeul X is really good, up to a score of 9.0" can be translated into corresponding structured data based on the named entities identified: { "director": "korean X", "movie name": "flat XX lane", "score": "9.0", then, structured data from unstructured data conversions, as well as structured data from web crawlers direct crawls, may be added to the structured dataset.

In some embodiments, before conducting the named entity recognition process on the unstructured data, the following operations may also be performed: carrying out data cleaning processing on the unstructured data to obtain cleaned unstructured data; and carrying out sensitive information identification processing on the cleaned unstructured data, and deleting the identified sensitive information.

For example, due to the fact that information on the web page is irregular, after the unstructured data is captured from the web page through the web crawler, data cleaning processing (for example, deleting repeated information in the unstructured data, completing missing information in the unstructured data, and performing validity verification on the unstructured data, and deleting information that the validity verification fails) may be performed on the captured unstructured data first, then, further sensitive information (including various information that is prohibited from being transmitted by laws and regulations) identification processing is performed on the cleaned unstructured data, and the identified sensitive information is deleted, so that it can be ensured that the subsequently output reply sentences are all positive and upward, and serious negative effects on the user caused by the output sensitive and negative reply sentences are avoided.

In step S103, the structured data is filled in the phonetics template to obtain candidate objective topics, and candidate subjective topics matched with the tags are obtained from the topic library.

In some embodiments, after obtaining matching conversational templates from a conversational template library based on the tags and obtaining matching structured data from the structured dataset, the obtained structured data may be populated into the obtained conversational templates to obtain candidate objective topics.

For example, if a sentence input by a user is "zhou XX singing with severe singing", first, named entity recognition processing is performed on the input sentence to obtain a name "zhou XX", then, a matching conversational template is obtained from a conversational template library by using the "zhou XX" as a label, and it is assumed that the obtained conversational template is: { < PersonName > is an artist who compares that the named song is < Date >, this song belongs to < Type > }, then, queries the structured data set based on the label "week XX", assuming that the acquired structured data is: { "singer": "week XX", "song": "black XX default", "type": "pop music", then, the value of the entity variable: < PersonName >. Value = weekly XX, < Date >. Value = black XX default, < Type >. Value = popular music is filled in the acquired dialogistic template, so that the following candidate objective topics can be obtained: "WeekXX is a singer who compares the named song with black XX default, which belongs to popular music".

In other embodiments, before obtaining the candidate subjective topics matching the label from the topic library, the topic library may be further constructed by: acquiring unstructured data of different fields; carrying out data cleaning processing on the unstructured data to obtain cleaned unstructured data; carrying out named entity extraction processing on the cleaned unstructured data to obtain extracted unstructured data; rewriting the extracted unstructured data to obtain corresponding subjective topics; and constructing a topic library based on the obtained subjective topics.

Illustratively, unstructured data in different fields can be captured from a webpage in a web crawler mode, and then the captured unstructured data is cleaned (for example, repeated information in the unstructured data is deleted; and finally, rewriting the extracted unstructured data to obtain a corresponding subjective topic. For example, assume that the unstructured data after washing is: "the newest movie in korean X is true and good in the way of flat XX with a score as high as 9.0", then named entity extraction processing is performed on the cleaned unstructured data, and the obtained extracted unstructured data is as follows: { hangeul X, passage of flat XX, 9.0}, and then, rewrite processing is performed on the extracted unstructured data, so that the following subjective topics can be obtained: "plain XX crossing stele of film by korean X director is true and true, score is as high as 9.0, strongly recommended".

It should be noted that different subjective topics can be generated for the same extracted unstructured data, as long as the content of the generated subjective topics does not deviate from the topic.

In other embodiments, the extracted unstructured data may be rewritten in the following manner to obtain corresponding subjective topics: performing emotion analysis processing on the extracted unstructured data to obtain emotion categories, and rewriting the extracted unstructured data into subjective topics matched with the emotion categories; and receiving a labeling operation aiming at the extracted unstructured data, rewriting the extracted unstructured data into a subjective topic when a labeling result represents that the extracted unstructured data belongs to legal data, and deleting the extracted unstructured data when the labeling result represents that the extracted unstructured data belongs to illegal data.

For example, when the extracted unstructured data is obtained, emotion analysis processing may be performed on the extracted unstructured data to obtain emotion categories (e.g., positive emotion, negative emotion, etc.), and then the extracted unstructured data may be rewritten into subjective topics matching the emotion categories, for example, when the identified emotion category is a negative emotion, the extracted unstructured data may be rewritten into subjective topics matching the negative emotion, for example, words with negative emotion such as "annoying" and "dismissal" may be added to the subjective topics.

For example, in order to ensure that the subjective topic in the topic database is positive, before rewriting the extracted unstructured data, a labeling operation for the extracted unstructured data may be received, for example, the extracted unstructured data is sent to a labeling person, so that the labeling person checks the extracted unstructured data. When the marking result represents that the extracted unstructured data belongs to legal data (namely, the marking personnel passes the examination), the extracted unstructured data is rewritten into subjective topics; and when the marking result represents that the extracted unstructured data belongs to illegal data (for example, sensitive information is found by a marking person during examination), deleting the extracted unstructured data, so that subjective topics stored in the topic library are ensured to be positive and positive, and the occurrence of negative topic conditions such as various information which is forbidden to be transmitted by output laws and regulations is avoided.

In some embodiments, after the tag corresponding to the input sentence is obtained, the matched candidate subjective topic may be obtained from the topic library based on the tag, for example, taking the tag as the name "zhao XX", and querying the topic library based on the name "zhao XX" to obtain the candidate subjective topic containing "zhao XX" from the topic library.

In other embodiments, the database of the conversation template, the structured data set, and the topic database may be updated, for example, a web crawler may periodically (for example, once every week) capture the latest external data (including the structured data and the unstructured data) from the web page, and update the database of the conversation template, the structured data set, and the topic database based on the captured latest external data, so that the timeliness of the topic can be ensured, the chat interest of the user can be stimulated, and the chat experience of the user can be improved.

In step S104, the candidate objective topics and the candidate subjective topics are sorted, and reply sentences for responding to the input sentences are determined according to the sorting result.

In some embodiments, step S104 shown in fig. 4 may be implemented by steps S1041A to S1043A shown in fig. 5A, which will be described in conjunction with the steps shown in fig. 5A.

In step S1041A, the candidate objective topic is added to the universal speech queue and the candidate subjective topic is added to the non-universal speech queue.

In some embodiments, a general speech queue and a non-general speech queue may be preset to store the obtained candidate subjective topics and candidate objective topics, respectively. For example, after the candidate objective topic and the candidate subjective topic are obtained in step S103, the candidate objective topic may be added to the general speech queue and the candidate subjective topic may be added to the non-general speech queue.

In step S1042A, the interest field is determined according to the object features, and the at least one candidate objective topic included in the general speech queue and the at least one candidate subjective topic included in the non-general speech queue are sorted in a descending order according to the similarity between the interest field and the field to which the at least one candidate objective topic included in the general speech queue and the at least one candidate subjective topic included in the non-general speech queue belong, respectively.

For example, taking an object as a real user as an example, assuming that when it is determined that the user is interested in a sports field according to the object characteristics, when the ranking is performed, the ranking of the candidate objective questions corresponding to the sports field in the universal conversational queue may be advanced (for example, when the ranking is performed, the candidate objective questions corresponding to the sports field may be adjusted to be the first order, or the candidate objective questions corresponding to the sports field may be moved forward by a fixed number of times), and the ranking of the candidate subjective questions corresponding to the sports field in the non-universal conversational queue is advanced, so that the finally output reply sentence is similar to the field interested by the user, which can stimulate the chat desire of the user, guide the conversation to continue, and improve the chat experience of the user.

In some embodiments, before the candidate objective topics and the candidate subjective topics are sorted according to the object features, the following operations may be further performed: acquiring historical dialogue information; extracting the characteristics of the historical dialogue information to obtain historical dialogue characteristics; classifying the historical conversation features to obtain corresponding object labels; object features are constructed based on the object tags.

For example, taking a user whose object is real as an example, the historical conversation information may be information generated by the user during a historical (a period of time of any previous duration, for example, the last week) chat session with a terminal device (for example, a smart speaker, a vehicle-mounted smart voice device), and the historical conversation information and a user account (i.e., a user account corresponding to an input sentence) may be stored locally or in a server in the form of a Key-Value pair (Key-Value) in the terminal device. After the historical dialogue information is obtained, the historical dialogue information may be subjected to feature extraction through a feature extraction model (e.g., a language model) to obtain historical dialogue features (e.g., analyzing the historical dialogue information, extracting entity features from the historical dialogue information and dependencies between entities). And then, classifying the historical conversation features to obtain corresponding user tags. The user tags refer to abstracting, classifying and summarizing certain characteristics of a user, and each user tag corresponds to a value, for example, when the user tags are sex tags, a male tag value can be set to be 0, and a female tag value can be set to be 1; finally, object features corresponding to the user account can be constructed based on the obtained plurality of user tags.

In other embodiments, the historical dialog features may be classified in the following manner to obtain corresponding object tags: calling a multi-dimensional label classifier to classify the historical conversation features to obtain label prediction probabilities of object account numbers corresponding to the input statements on N dimensions; the multi-dimensional label classifier comprises N label classifiers which correspond to the object labels with N dimensions one by one; sorting the label prediction probabilities in the N dimensions in a descending order, and selecting the first K object labels from the object labels in the N dimensions according to a sorting result; and taking the first K object labels as object labels of the object account in K dimensions, wherein K is less than or equal to N and is a positive integer.

For example, taking an object as a real user as an example, after obtaining the historical conversation features, the historical conversation features may be input into N tag classifiers to output tag prediction probabilities of the user account in N dimensions, where each tag classifier includes a different logistic regression function layer (Softmax) for outputting user tags in different dimensions. For example, assume that the multidimensional label classifier includes 5 label classifiers, where the predicted probability of the label output by each label classifier is (ordered according to the sequence of label classifier 1, label classifier 2, label classifier 3, label classifier 4, and label classifier 5): 0.2, 0.1, 0.05, 0.4, 0.5, then the 5 label prediction probabilities are sorted in descending order as: 0.5, 0.4, 0.2, 0.1, 0.05; meanwhile, assuming that the K value is 2, that is, selecting the user tags corresponding to the tag prediction probabilities ranked as top 2 from the user tags of 5 dimensions, that is, selecting the user tags corresponding to the tag prediction probabilities of 0.5 and 0.4, assuming that the tag classifier 5 corresponds to the user tags of the interest dimension (e.g., like playing games), and the tag classifier 4 corresponds to the user tags of the language style dimension (e.g., silent oligos), the user tags of the two dimensions are used as the user tags corresponding to the user account. Then, object features are constructed based on the user tags, and the fields in which the users are interested (such as the fields in which the games are more interested) are determined according to the object features, so that when the ranking is carried out, the ranking of candidate objective topics and candidate subjective topics related to the games in the general-use speech queue and the non-general-speech queue can be advanced.

It should be noted that, in practical applications, in addition to constructing the object feature according to the object tag, the object feature may also be constructed in combination with behavior information (e.g., social dynamic information, browsing information, and the like) and interest information of the object account, for example, the object feature may be constructed according to the behavior information, the interest information, and the object tag of the object account, so that the constructed object feature may better conform to the real situation of the user by combining information of multiple dimensions.

In other embodiments, when the interest field corresponding to the object cannot be determined according to the object features, the at least one candidate objective topic included in the universal conversation queue and the at least one candidate subjective topic included in the non-universal conversation queue may be sorted in the following sorting manner: ranking the top of at least one candidate objective topic corresponding to the popular topic included in the universal conversation queue (for example, statistics may be performed on the areas of interest of different users, and one or more areas of interest of the most users are determined as popular areas, for example, assuming statistics is performed on 100 areas of interest of users, where 90 users are interested in the music area and 50 users are interested in the sports area, and the music area is determined as popular areas); and ordering at least one candidate subjective topic corresponding to the hot field included by the non-universal conversation queue.

For example, when a user has a conversation with a chat robot for the first time, because there is no historical conversation information, object features cannot be constructed, and further, a field in which the user is interested cannot be determined, the ranking of candidate objective topics corresponding to popular fields (e.g., music, movies, etc.) included in the universal conversation queue may be advanced; and sequencing candidate objective topics corresponding to popular fields (such as fields of music, movies and the like) included in the non-universal conversation queue in a front position, so that the finally output reply sentences are topics related to the popular fields, and can arouse the chat interest of the user on a high probability, thereby guiding the conversation to continue.

In step S1043A, at least one candidate subjective topic or candidate objective topic ranked at the top is selected in the descending ranking result as a reply sentence for responding to the input sentence.

In some embodiments, after the descending order sorting result is obtained, the candidate topic (which may be a candidate subjective topic or a candidate objective topic) ranked first in the descending order sorting result may be used as a reply sentence for responding to the input sentence, and the candidate topic is returned to the terminal device.

In other embodiments, step S104 shown in fig. 4 can also be implemented by step S1041B to step S1043B shown in fig. 5B, which will be described with reference to the step shown in fig. 5B.

In step S1041B, the candidate objective topic is added to the universal talk queue and the candidate subjective topic is added to the non-universal talk queue.

In some embodiments, a universal speech queue and a non-universal speech queue may be preset, and the candidate objective topic obtained in step S103 is added to the universal speech queue, and the obtained candidate subjective topic is added to the non-universal speech queue.

In step S1042B, at least one candidate objective topic included in the universal dialog queue and at least one candidate subjective topic included in the non-universal dialog queue are sorted in descending order according to the number of named entities included in the hit input sentence.

In some embodiments, the input sentence may be subjected to a named entity identification process, named entities included in the input sentence are identified, then at least one candidate objective topic included in the universal dialog queue and at least one candidate subjective topic included in the non-universal dialog queue are matched with the identified named entities to determine the number of the identified named entities hit on each candidate topic (including candidate subjective topics and candidate objective topics), and then the at least one candidate objective topic included in the universal dialog queue and the at least one candidate subjective topic included in the non-universal dialog queue are sorted in descending order according to the number of the named entities included in the hit input sentence. That is, when sorting is performed, the candidate topics with the larger number of named entities included in the named input sentence have the higher corresponding positions.

In step S1043B, at least one candidate subjective topic or candidate objective topic ranked in the top is selected in the descending ranking result as a reply sentence for responding to the input sentence.

In some embodiments, after the descending order result is obtained, the candidate topic (which may be a candidate subjective topic or a candidate objective topic) ranked at the first or preceding multiple bits (i.e., at least two bits) in the descending order result may be used as a reply sentence for responding to the input sentence, and the candidate topic is returned to the terminal device.

In other embodiments, the above-mentioned ranking process for the candidate objective topics and the candidate subjective topics may be implemented as follows, and the reply sentences for responding to the input sentences are determined according to the ranking result: retrieving a reply sentence matched with the input sentence from a question-answer library constructed in advance as a retrieval type reply sentence; generating a reply sentence matched with the input sentence through a pre-trained language model as a generating reply sentence; taking the candidate objective questions and the candidate subjective questions as candidate theme reply sentences, and fusing the search type reply sentences and the generated type reply sentences with the candidate theme reply sentences respectively to obtain a plurality of candidate reply sentences; and sorting the plurality of candidate reply sentences according to the object characteristics, and selecting at least one candidate reply sentence which is sorted at the top in the sorting result as a reply sentence for responding to the input sentence.

For example, for the input sentence "how good you have seen no X-way", assuming that the retrievable reply sentence is "how you have seen this movie", the candidate subjective question "deep feeling" no X-way "is really the moment of the most brilliant near ten years of the movie world, and the drama, rhythm, and dubbing are all praised", the retrievable reply sentence "how you see this movie" and the candidate subjective topic "deep feeling" no X-way "are not really the moment of the most brilliant near ten years of the movie world, and the drama, rhythm, and dubbing are all praised" can be spliced, so that the final reply sentence "how you see this movie, deep feeling" no X-way "is the moment of the most brilliant near ten years of the movie world, and the drama rhythm, and dubbing" are all praised ", thus, the retrievable answer sentence can be well taken over, and the information of the retrievable topic can be increased through the subjective answer sentence and the topic reply topic experience of the user can be increased.

It should be noted that the process of sorting the multiple candidate reply sentences according to the object features may be similar to the process of sorting at least one candidate objective topic included in the general speech queue and at least one candidate subjective topic included in the non-general speech queue according to the object features in step S1042A, and specific technical details may refer to the description in step S1042A, which is not described herein again in this embodiment of the present application.

According to the conversation processing method provided by the embodiment of the application, the matched dialect template and the structured data are obtained through the label corresponding to the input sentence, the candidate objective topics are generated based on the obtained dialect template and the structured data, and meanwhile, the matched candidate subjective topics are obtained from the topic library according to the label, so that the subsequent reply sentences used for responding to the input sentence can be determined through two different types of topics, the information content and the topicality of the reply sentences are improved, the user can be guided to continue to carry out conversation, and the chat experience of the user is improved.

In the following, taking an object as an actual user as an example, an exemplary application of the embodiment of the present application in an actual application scenario will be described.

The embodiment of the application provides a conversation processing method (hereinafter also referred to as active conversation), which includes the steps of constructing a conversation template library, a structured data set and a conversation library by offline data construction (for example, supplementing data with timeliness), acquiring topics (namely candidate subjective topics) of natural languages matched with labels from the conversation library based on different strategies, generating candidate objective topics according to the conversation template and the structured data, sequencing the candidate subjective topics and the candidate objective topics according to object features, and selecting the optimal topics as reply sentences according to sequencing results, so that topics can be initiated to guide a user to continue conversation, the information content and the topicality of conversation are increased, and the chat experience of the user is improved.

For example, for the same sentence "yew of a question and XX teacher" input by the user, the answer sentence output by the generating type dialog provided by the related art is "yew XX teacher o", and the answer sentence output by the retrieving type dialog is "there are many nice answers worth the thoughts at all"; however, the reply sentence output by the active dialog (i.e., the dialog processing method provided in the embodiment of the present application) is "jones XX has a very good host base and is a memory of a generation", and it can be seen that compared with the retrieval dialog and the generation dialog, the reply sentence output by the active dialog has more information and stronger topicality, so that the user can be guided to continue the dialog, and the chat experience of the user is improved.

The dialogue processing method provided by the embodiment of the application has a very wide application scene, can be applied to intelligent dialogue products taking dialogue as a carrier, and can effectively improve the quality of reply sentences generated by a dialogue system, improve the experience of communication between a user and the dialogue system, and improve the turn of conversation.

The following specifically describes a dialog processing method provided in the embodiment of the present application.

The conversation processing method provided by the embodiment of the application relates to the processing of structured data/unstructured data, the construction of a conversation template library, a structured data set and a conversation library. After a Query input by a user enters a dialog system, firstly structured information for semantic Understanding is obtained through Natural Language Understanding (NLU), then, an Agent corresponding to active dialog invokes processed external data to generate a plurality of candidate topics (including candidate objective topics generated through a dialog template and the structured data and candidate subjective topics acquired from a topic library) according to the obtained structured information and a plurality of strategies, and then, the candidate topics are ranked according to the acquired object features, and the best topics (for example, the candidate topics ranked at the first place) are output as reply sentences for responding to the Query input by the user.

For example, referring to fig. 6, fig. 6 is a schematic diagram of a framework of a dialog system provided in an embodiment of the present application, as shown in fig. 6, after a Query input by a user enters the dialog system, the Query is first preprocessed, for example, real-name entity recognition processing is performed on the Query to obtain a name "korean X", and meanwhile, intent recognition processing may also be performed on the Query to obtain an intent of the user as a movie intent, and then, the name "korean X" and a field "movie" are transmitted as tags to a downstream active dialog Agent (i.e., an Agent that executes a dialog processing method provided in an embodiment of the present application), so that the active dialog Agent recalls a matching dialog template from a dialog template library and recalls matching structured data from a structured data set constructed based on external data, and then fills the structured data into a subjective dialog template to generate an objective topic question candidate, and besides, a topic candidate matching the tag may also be obtained from the topic library (not shown in the figure). And finally, sequencing the recalled candidate topics according to the object characteristics (for example, constructed according to the behavior tags, the interest tags and the basic information of the user), and determining a reply sentence for responding to the Query input by the user according to the sequencing result.

It should be noted that the active dialog and the generating dialog and the retrieval dialog may be in parallel relationship in the dialog system. In addition, the dialogue processing method provided by the embodiment of the application can splice the candidate replies generated by different agents in a fusion mode, so that the advantage complementation between the dialogue technical schemes of different agents is realized, and the finally output information amount of the reply statement is further increased.

For example, assuming that Query input by the user is "how you are good and you have not seen XX", and candidate reply output by the retrievable dialog Agent is "how you look at the movie", and candidate reply output by the active dialog Agent is "deep-feeling" wuxx "is indeed the moment of being brightest in the movie world for nearly ten years, and drama, rhythm, and dubbing are prosperized by people", the candidate reply output by the retrievable dialog Agent and the candidate reply output by the active dialog Agent can be spliced, that is, the finally output reply sentence can be "how you look at the movie, deep-feeling" wuxx "is disharmy in the movie world for nearly ten years, and drama, rhythm, and dubbing are prosperity by people", so that topics of the user can be well received, information amount of the reply sentence is increased, and chat experience of the user is improved.

For example, referring to fig. 7, fig. 7 is a schematic diagram illustrating a dialog processing method according to an embodiment of the present application, and as shown in fig. 7, the dialog processing method according to the embodiment of the present application mainly involves four parts, namely, external data construction, user input preprocessing, recall, and sorting, which are specifically described below.

1. External data construction: the external data can be divided into two parts, structured data and unstructured data. The structured data includes entity-related information in the fields of music, books, videos, and the like. Taking the video domain as an example, a specific example of the structured data is as follows:

{ "director": "week XX", "movie name": "mexx", "score": "9.0"}.

The unstructured data refers to unprocessed original raw corpus in an external data source, and structured data and subjective topics can be extracted from the unstructured data after preprocessing (including data cleaning, emotion analysis, named entity identification, template extraction, sensitive information (including various information prohibited from being spread by law and regulation), identification, manual marking, universality rewriting and the like).

That is, the external data construction aims to construct a library of conversational templates (including a plurality of conversational templates), a structured data set (including a plurality of structured data), and a conversational library (including a plurality of subjective topics). After the construction is completed, candidate objective topics (also called general dialects) can be generated through the dialect templates stored in the dialect template library and the structured data stored in the structured data set; and the natural language topics (i.e. candidate subjective topics) corresponding to the entities can be obtained by querying the topic library by means of key-value pairs (for example, taking the entities identified from the Query input by the user as keys).

It should be noted that the external data construction process may be performed offline.

2. User input preprocessing: the process of preprocessing the Query input by the user is a process of natural language understanding of the Query input by the user, wherein the process of natural language understanding includes named entity identification, entity linking, intention identification and the like, so as to obtain structured information (such as the identified named entity, the intention of the user and the like) for semantic understanding of the Query input by the user, and the structured information is transmitted to a downstream Agent, so that the Agent recalls a plurality of matched candidate topics. In addition, historical dialogue information corresponding to the user account can be acquired, and object features corresponding to the user account are constructed based on the historical dialogue information, wherein the object features are used for a subsequent sorting process.

3. Recalling: after step 2 is executed to obtain structural information (such as an identified named entity, the intention of a user, and the like) for semantically understanding Query input by the user, the dialog system may invoke external data (i.e., a constructed phonics template library, a structured data set, and a phonics library) processed in step 1 based on different strategies to generate a plurality of candidate topics, for example, a matched phonics template may be recalled from the phonics template library constructed in step 1 through tags such as an identified named entity, a user interested field, and the like, and matched structural data may be recalled from the structured data set, and then, the recalled structural data is filled into the recalled phonics template to generate a candidate objective topic, and the generated candidate objective topic is added to a universal phonics queue; meanwhile, the dialogue system can recall natural language topics (namely candidate subjective topics) matched with the identified named entities from the topic library constructed in the step 1 in a key-value pair mode, and add the recalled candidate subjective topics to the non-universal-dialogue queue.

In practical applications, the solution of recalling candidate topics in step 3 based on templates and policies may be replaced by recalling candidate topics in an end-to-end manner, that is, after obtaining semantically understood structured information for Query input by a user, recalling matched structured data and recalling matched topics, and outputting a final reply sentence.

4. Sorting: the method comprises a plurality of different sorting modes, such as entity sorting, domain sorting and policy & rule sorting, wherein the entity sorting refers to sorting candidate objective topics included in a universal conversation queue and candidate subjective topics included in a non-universal conversation queue according to the number of entities included in a Query input by a hit user, namely, the more the number of entities included in the Query input by the hit user is, the more the corresponding candidate topics are ranked; the domain sorting means that candidate objective topics included in the universal conversational queue and candidate subjective topics included in the non-universal conversational queue are sorted according to the object features, for example, when the user is determined to be interested in sports according to the object features, when the sorting is performed, the sorting of the candidate objective topics or the candidate objective topics related to sports in the universal conversational queue and the non-universal conversational queue is advanced, that is, the topics related to sports are preferentially output as reply sentences for responding to Query input by the user; the policy & rule is to prioritize the ranking of candidate topics corresponding to the hot domains in the general-use speech queue and the non-general-use speech queue (e.g. more neutral and safe domains such as movies, music, books, etc., and topics in these domains are more neutral and safe and data quality is more reliable than domains such as art and current affairs) when the domain in which the user is interested cannot be determined, for example, when the user is a new user, and ranking is performed, that is, preferentially outputting the topics corresponding to the hot domains as a reply sentence for responding to the Query input by the user.

After a dialog system (namely, a dialog system based on active dialog) is built, the workload of later-stage operation is small, maintenance and iteration are facilitated, and the labor amount is mainly embodied in the following two aspects: 1. iterative extrinsic data: acquiring the latest external data, and updating the structured data set, the phonetics template library and the phonetics library based on the acquired latest external data, wherein the steps can be carried out in a flow manner; 2. and the annotator audits the data to be added to the topic database (namely the subjective topics), so that the security of the topics is ensured.

The conversation processing method provided by the embodiment of the application has the following beneficial effects:

1. object features are established through historical dialogue information (context information in dialogue is indirectly utilized so that the interesting field of a user can be accurately determined), and candidate topics are sequenced according to the object features, so that the finally output reply sentences have topicality (namely the output reply sentences are the interesting topics of the user);

2. a language template library, a structured data set and a language database are constructed through external data, so that the information content and timeliness of the finally output reply statement are increased;

3. the annotating personnel audit topics to be added into the topic library, so that the finally output topics are ensured to be positive, and negative topics such as various information forbidden to be transmitted by laws and regulations are prevented from being output;

4. candidate objective topics are generated through the recalled conversational template and the structured data, matched candidate subjective topics are recalled from the topic library, the information amount and the topicality of the reply sentence are increased, and conversation can be guided to continue.

The following will further describe the beneficial effects of the dialogue processing method provided by the embodiment of the present application with reference to experimental data.

On-line contrast experiments prove that the conversation processing method provided by the embodiment of the application obtains better conversation effect and better promotion on indexes such as contrast tests (namely A/B tests) and user average conversation turns.

For example, referring to table 1, when an agent of an active conversation (i.e., a conversation processing method provided in the embodiment of the present application) is online alone, G with statistical frequency before and after online: s: b (Good: same: bad) indicates a gain of 66% (G: B = 1.66.

TABLE 1

In addition, the average turn of dialog for a user is also an important indicator reflecting the quality of the reply statements generated by the dialog system and the sustainability of the dialog.

For example, referring to fig. 8, fig. 8 is a comparison diagram of average conversation turns corresponding to different conversation processing methods provided in this embodiment, and as shown in fig. 8, the average conversation turn per day of a user who triggered an active conversation (i.e., the conversation processing method provided in this embodiment) is significantly improved compared to the average conversation turn per day of a user who did not trigger an active conversation, where the average conversation turn increases from 3.22 to 5.55, the average conversation turn increases by 2.33 turns, and the increase is as high as 72%, which reflects that the active conversation has a significant effect on guiding the user to continue the conversation.

The dialog processing method provided by the embodiment of the present application has been described so far with reference to exemplary applications and implementations of the server provided by the embodiment of the present application. In practical applications, each functional module in the dialog processing apparatus may be cooperatively implemented by hardware resources of an electronic device (such as a terminal device, a server, or a server cluster), such as computing resources like a processor, communication resources (such as for supporting communications in various manners like optical cables and cellular), and a memory. Fig. 2 shows a dialog processing device 243 stored in the memory 240, which may be software in the form of programs and plug-ins, for example, software C/C + +, software modules designed by a programming language such as Java, application software designed by a programming language such as C/C + +, java, or dedicated software modules, application program interfaces, plug-ins, cloud services, etc. in a large software system, and the following examples illustrate different implementations.

Example one, the dialogue processing device is a mobile terminal application program and a module

The session processing device 243 in the embodiment of the present application may be provided as a software module designed using a programming language such as software C/C + +, java, or the like, and may be embedded in various mobile applications based on systems such as Android or iOS (stored as executable instructions in a storage medium of the mobile and executed by a processor of the mobile), so as to directly use computing resources of the mobile to complete related box structure detection tasks, and periodically or aperiodically transmit processing results to a remote server through various network communication methods, or locally store the processing results in the mobile.

Example two, the conversation processing device is a server application and platform

The session processing device 243 in this embodiment may be provided as application software designed using a programming language such as C/C + +, java, or the like, or a dedicated software module in a large-scale software system, and run on the server side (stored in a storage medium of the server side in the form of executable instructions and run by a processor of the server side), and the server uses its own computing resources to complete the relevant information recommendation task.

The embodiment of the application can also provide an information recommendation platform and the like for individuals, groups or units to use by carrying a customized and easily interactive network (Web) Interface or other User Interfaces (UI) on a distributed and parallel computing platform consisting of a plurality of servers.

Third, the dialogue processing device is a server side Application Program Interface (API) and a plug-in

The dialog processing device 243 in the embodiment of the present application may be provided as an API or a plug-in on the server side, so that the user can call the dialog processing device to execute the dialog processing method provided in the embodiment of the present application, and the dialog processing device is embedded in various application programs.

Example four, the dialog processing device is a Mobile device client API and plug-in

The dialog processing device 243 in the embodiment of the present application may be provided as an API or a plug-in on the mobile device side, so that the user may call the dialog processing device to execute the question-answering processing method in the embodiment of the present application.

Example five, the conversation processing device is a cloud open service

The session processing device 243 in the embodiment of the present application may provide a target detection cloud service developed for a user, so that an individual, a group, or an entity may obtain a recommendation list.

Continuing with the exemplary structure of the dialog processing device 243 provided by the embodiment of the present application implemented as a software module, in some embodiments, as shown in fig. 2, the software module stored in the dialog processing device 243 of the memory 240 may include: an obtaining module 2431, a padding module 2432, a ranking module 2433, and a determining module 2434.

An obtaining module 2431, configured to obtain a tag corresponding to an input statement; an obtaining module 2431, further configured to obtain a matching conversational template from a conversational template library based on the tag, and obtain matched structured data from the structured data set; a filling module 2432, configured to fill the structured data into the conversational template to obtain candidate objective topics; the obtaining module 2431 is further configured to obtain candidate subjective topics matched with the tags from the topic library; the sorting module 2433 is configured to sort the candidate objective topics and the candidate subjective topics; a determining module 2434, configured to determine a reply sentence for responding to the input sentence according to the sorting result.

In some embodiments, the obtaining module 2431 is further configured to perform at least one of the following: carrying out named entity identification processing on an input statement, and determining an identified named entity as a label corresponding to the input statement; performing word segmentation on an input sentence, acquiring a corresponding candidate entity from a knowledge base based on a reference word obtained after the word segmentation, and determining the candidate entity as a label corresponding to the input sentence; and performing intention recognition processing on the input sentence, and determining the obtained intention recognition result as a label corresponding to the input sentence.

In some embodiments, the dialog processing device 243 further includes a construction module 2435 for constructing the library of dialog templates, and the structured dataset by: acquiring structured data and unstructured data, performing template extraction processing on the structured data to obtain a corresponding dialect template, and constructing a dialect template library based on the dialect template; and carrying out named entity identification processing on the unstructured data, converting the unstructured data into corresponding structured data based on the identified named entity, and constructing a structured data set based on the structured data and the structured data obtained by converting the unstructured data.

In some embodiments, the dialog processing device 243 further includes a cleaning module 2436, configured to perform a data cleaning process on the unstructured data, to obtain cleaned unstructured data; the device also comprises an identification module used for identifying the sensitive information of the washed unstructured data and deleting the identified sensitive information from the washed unstructured data.

In some embodiments, construction module 2435 is further configured to construct the topic library by: acquiring unstructured data of different fields; carrying out data cleaning processing on the unstructured data to obtain cleaned unstructured data; carrying out named entity extraction processing on the cleaned unstructured data to obtain extracted unstructured data; rewriting the extracted unstructured data to obtain a corresponding subjective topic; and constructing a topic library based on the subjective topics.

In some embodiments, the cleaning module 2436 is further configured to perform at least one of the following operations on the unstructured data to obtain cleaned unstructured data: deleting information that is not repeated in the unstructured data; completing missing information in the unstructured data; and carrying out validity verification on the unstructured data, and deleting the information which does not pass the validity verification.

In some embodiments, dialog processing device 243 further includes a rewrite module 2438 for performing at least one of: performing emotion analysis processing on the extracted unstructured data to obtain emotion categories, and rewriting the extracted unstructured data into subjective topics matched with the emotion categories; and receiving a labeling operation aiming at the extracted unstructured data, rewriting the extracted unstructured data into a subjective topic when a labeling result represents that the extracted unstructured data belongs to legal data, and deleting the extracted unstructured data when the labeling result represents that the extracted unstructured data belongs to illegal data.

In some embodiments, the ranking module 2433 is further configured to add the candidate objective topics to a general speech queue and the candidate subjective topics to a non-general speech queue; determining an interested field according to the object characteristics, and sorting at least one candidate objective topic included in a general speech queue and at least one candidate subjective topic included in a non-general speech queue in a descending order according to the similarity between the field to which the at least one candidate objective topic included in the general speech queue and the interested field belong and the field to which the at least one candidate subjective topic included in the non-general speech queue belongs; the determining module 2434 is further configured to select at least one candidate subjective topic or candidate objective topic ranked at the top in the descending ranking result as a reply sentence for responding to the input sentence.

In some embodiments, when the corresponding interested field cannot be determined according to the object features, the ranking module 2433 is further configured to rank the candidate objective topics corresponding to the at least one popular field included in the universal dialect queue in front; and ordering the candidate subjective topics corresponding to the at least one hot field included in the non-universal conversation queue in a front position.

In some embodiments, the building module 2435 is further configured to obtain historical dialog information; extracting the characteristics of the historical dialogue information to obtain historical dialogue characteristics; classifying the historical conversation features to obtain corresponding object labels; object features are constructed based on the object tags.

In some embodiments, the building module 2435 is further configured to invoke a multi-dimensional label classifier to classify the historical conversation features, so as to obtain label prediction probabilities of object account numbers corresponding to the input statements in N dimensions; the multi-dimension label classifier comprises N label classifiers which correspond to the object labels with N dimensions one by one; sorting the label prediction probabilities in the N dimensions in a descending order, and selecting the first K object labels from the object labels in the N dimensions according to a sorting result; taking the first K object labels as object labels of the object account in K dimensions; wherein K is less than or equal to N, and K is a positive integer.

In some embodiments, the ranking module 2433 is further configured to add the candidate objective topics to a general speech queue and the candidate subjective topics to a non-general speech queue; according to the number of named entities included in the hit input statement, at least one candidate objective topic included in the universal language queue and at least one candidate subjective topic included in the non-universal language queue are sorted in a descending order; and the determining module is further used for selecting at least one candidate subjective topic or candidate objective topic ranked at the top in the ranking result as a reply sentence for responding the input sentence.

In some embodiments, the sorting module 2433 is further configured to retrieve, in a pre-constructed question-and-answer library, a reply sentence matching the input sentence as a retrievable reply sentence; generating a reply sentence matched with the input sentence through the pre-trained language model to serve as a generating type reply sentence; taking the candidate objective questions and the candidate subjective questions as candidate topic reply sentences, and fusing the retrieval type reply sentences and the generation type reply sentences with the candidate topic reply sentences respectively to obtain a plurality of candidate reply sentences; and sorting the plurality of candidate reply sentences according to the object characteristics, and selecting at least one candidate reply sentence which is sorted at the top in the sorting result as a reply sentence for responding to the input sentence.

It should be noted that the description of the apparatus in the embodiment of the present application is similar to the description of the method embodiment, and has similar beneficial effects to the method embodiment, and therefore, the description is not repeated. The inexhaustible technical details in the dialog processing device provided in the embodiment of the present application can be understood from the description of any one of fig. 4, fig. 5A, or fig. 5B.

Embodiments of the present application provide a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and executes the computer instructions, so that the computer device executes the conversation processing method described in the embodiment of the present application.

Embodiments of the present application provide a computer-readable storage medium having stored therein executable instructions that, when executed by a processor, cause the processor to perform a method provided by embodiments of the present application, for example, a dialog processing method as illustrated in fig. 4, fig. 5A, or fig. 5B.

In some embodiments, the computer-readable storage medium may be memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash, magnetic surface memory, optical disk, or CD-ROM; or may be various devices including one or any combination of the above memories.

In some embodiments, executable instructions may be written in any form of programming language (including compiled or interpreted languages), in the form of programs, software modules, scripts or code, and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

By way of example, executable instructions may correspond, but do not necessarily have to correspond, to files in a file system, and may be stored in a portion of a file that holds other programs or data, such as in one or more scripts in a hypertext Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub-programs, or portions of code).

By way of example, executable instructions may be deployed to be executed on one computing device or on multiple computing devices at one site or distributed across multiple sites and interconnected by a communication network.

To sum up, the matching tactical template and the structured data are obtained through the tag corresponding to the input sentence, the candidate objective topic is generated based on the obtained tactical template and the structured data, and meanwhile, the matched candidate subjective topic is obtained from the topic library according to the tag, so that the follow-up reply sentence for responding to the input sentence can be determined through two different types of topics, the information content and the topicality of the reply sentence are improved, the user can be guided to continue to carry out conversation, and the chat experience of the user is improved.

The above description is only an example of the present application, and is not intended to limit the scope of the present application. Any modification, equivalent replacement, and improvement made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. A method of dialog processing, the method comprising:

acquiring a label corresponding to an input statement;

2. The method of claim 1, wherein the obtaining the label corresponding to the input sentence comprises:

performing at least one of the following operations:

carrying out named entity identification processing on the input statement, and determining the identified named entity as a label corresponding to the input statement;

performing word segmentation on the input sentence, acquiring a corresponding candidate entity from a knowledge base based on a reference word obtained after the word segmentation, and determining the candidate entity as a label corresponding to the input sentence;

and performing intention recognition processing on the input statement, and determining an obtained intention recognition result as a label corresponding to the input statement.

3. The method of claim 1, wherein prior to retrieving a matching linguistic template from a library of linguistic templates based on the tag, and retrieving matching structured data from a structured dataset, the method further comprises:

constructing the library of verbal templates, and the structured dataset, by:

acquiring structured data and unstructured data, performing template extraction processing on the structured data to obtain a corresponding dialect template, and constructing a dialect template library based on the dialect template;

and carrying out named entity identification processing on the unstructured data, converting the unstructured data into corresponding structured data based on the identified named entities, and constructing the structured data set based on the structured data and the structured data obtained by converting the unstructured data.

4. The method of claim 3, wherein prior to performing named entity recognition processing on the unstructured data, the method further comprises:

carrying out data cleaning processing on the unstructured data to obtain cleaned unstructured data;

and carrying out sensitive information identification processing on the washed unstructured data, and deleting the identified sensitive information from the washed unstructured data.

5. The method of claim 1, wherein prior to obtaining the candidate subjective topics matching the label from a topic library, the method further comprises:

constructing the topic library by:

acquiring unstructured data of different fields;

conducting named entity extraction processing on the washed unstructured data to obtain extracted unstructured data;

rewriting the extracted unstructured data to obtain a corresponding subjective topic;

and constructing the topic library based on the subjective topics.

6. The method according to claim 5, wherein the performing a data cleaning process on the unstructured data to obtain cleaned unstructured data comprises:

performing at least one of the following operations on the unstructured data to obtain cleaned unstructured data:

deleting duplicate information in the unstructured data;

completing the missing information in the unstructured data;

and carrying out validity verification on the unstructured data, and deleting the information which does not pass the validity verification.

7. The method according to claim 5, wherein the rewriting the extracted unstructured data to obtain a corresponding subjective topic comprises:

performing at least one of the following operations:

performing emotion analysis processing on the extracted unstructured data to obtain emotion categories, and rewriting the extracted unstructured data into subjective topics matched with the emotion categories;

and receiving a labeling operation aiming at the extracted unstructured data, rewriting the extracted unstructured data into a subjective topic when a labeling result indicates that the extracted unstructured data belongs to legal data, and deleting the extracted unstructured data when the labeling result indicates that the extracted unstructured data belongs to illegal data.

8. The method according to claim 1, wherein the sorting the candidate objective topics and the candidate subjective topics and determining reply sentences responding to the input sentences according to the sorting result comprises:

adding the candidate objective topics to a universal conversation queue and the candidate subjective topics to a non-universal conversation queue;

determining an interested field according to object features, and sequencing at least one candidate objective topic included by the general conversational queue and at least one candidate subjective topic included by the non-general conversational queue in a descending order according to the similarity between the interested field and the field to which the at least one candidate objective topic included by the general conversational queue and the at least one candidate subjective topic included by the non-general conversational queue belong;

and selecting at least one candidate subjective topic or candidate objective topic ranked at the top in the descending ranking result as a reply sentence for responding to the input sentence.

9. The method of claim 8, wherein when a corresponding area of interest cannot be determined from the object features, the method further comprises:

the sorting of candidate objective topics corresponding to at least one hot field included in the general conversational queue is advanced;

and ordering the candidate subjective topics corresponding to at least one hot field included in the non-universal conversation queue in a front position.

10. The method according to claim 8, wherein before the ranking the candidate objective topics and the candidate subjective topics according to subject characteristics, the method further comprises:

acquiring historical dialogue information;

extracting the characteristics of the historical dialogue information to obtain historical dialogue characteristics;

classifying the historical conversation features to obtain corresponding object labels;

constructing the object features based on the object tags.

11. The method of claim 10, wherein the classifying the historical dialog features to obtain corresponding object tags comprises:

calling a multi-dimensional label classifier to classify the historical conversation features to obtain label prediction probabilities of the object account corresponding to the input statement on N dimensions;

the multi-dimension label classifier comprises N label classifiers which correspond to the object labels with the N dimensions one by one;

sorting the label prediction probabilities on the N dimensions in a descending order, and selecting the first K object labels from the object labels of the N dimensions according to a sorting result;

taking the first K object tags as object tags of the object account on the K dimensions;

wherein K is less than or equal to N, and K is a positive integer.

12. The method according to claim 1, wherein the ranking the candidate objective topics and the candidate subjective topics and determining reply sentences for responding to the input sentences according to the ranking result comprises:

sorting at least one candidate objective topic included in the universal conversational queue and at least one candidate subjective topic included in the non-universal conversational queue in a descending order according to the number of the named entities included in the hit input sentence;

and selecting at least one candidate subjective topic or candidate objective topic ranked at the top in the ranking result as a reply sentence for responding to the input sentence.

13. A conversation processing apparatus, characterized in that the apparatus comprises:

the acquisition module is further used for acquiring a matched conversational template from a conversational template library based on the label and acquiring matched structured data from a structured data set;

the filling module is used for filling the structured data into the conversational template to obtain candidate objective topics;

and the determining module is used for determining a reply statement for responding to the input statement according to the sorting result.

14. An electronic device, characterized in that the electronic device comprises:

a memory for storing executable instructions;

a processor for implementing the dialog processing method of any of claims 1 to 12 when executing executable instructions stored in the memory.

15. A computer-readable storage medium having stored thereon executable instructions for implementing the dialog processing method of any one of claims 1 to 12 when executed.